I hope you are keeping well. We are getting random responses without chatGPT set up. We have implemented Function calling for our setup. However, chatGPT sometimes calls the function for user input. And sometimes, with the same user input, it responds directly without calling the function. It is working quite randomly. How we can reduce this randomness and consistently work whenever user input relates to function calls (It should actually go to function however, it just responds directly without calling the function)?
ChatGPT is OpenAI’s own chatbot, so let’s talk about “API models”.
I suggested just yesterday how some API parameter could be provided so we can affect the likelihood that a function is called. They made an API parameter to make calling a function mandatory, but no other control.
The calling of functions is completely out of the API developer’s control. Even if you use “logit_bias” to increase the probability of the AI saying “Sure” greatly as if to a user, that doesn’t affect the unseen signal generated by AI of whether it wants to use a function (Only being able to write Sure can break the function though, giving a 500 error).
I’ll have to review if top_p or temperature are affecting this with some carefully-considered tests, but you can start with reduced temperature and top_p of 0.5 to constrain the choices of output token to only those more likely.
Thus, you only have the instructions and description with the function specification itself. The function’s purpose, when to invoke it, what will be performed or obtained, and what is returned, should be extremely clear to the AI. The top-level description you write can be multi-line and long to ensure the AI knows all about the tool, and the instructions should be just as easy for a human to understand everything about how your tool works by just reading them.
Hopefully by improving the AI’s understanding, the function tool will only be invoked when it is useful for satisfying user input.