If you use the Function Calling feature of chat completion API, you may notice that the API may request to call unexciting functions. For me and other developers, we got requests to call the “python” function, which is not in our functions list.
I like to share how we handled this case and would like to hear from you how did you handle it so we can learn from each other.
First, I noticed that I need to keep the history of function calling in the conversation, especially when the function needs to be called multiple times, otherwise, the API will not request the function again, if it is not present in the conversation history where it should be.
Now, when it comes to invalid function names (not in the list of the available functions), I respond with “none” result, and I omit the invalid function call and response (hallucinatedFunctionMessages) from the conversation history:
In this way, the conversation will contain only the user messages and responses, and the valid function calls.
Please let me know how did you handle the invalid function call issue, for example, did to find a preventive solution instead of the mitigative one?
If you read my post on the function calling support I added to AlphaWave you’ll see the feedback I’m sending the model for hallucinations and invalid JSON:
I use a JSON schema validator and have the model not only repairing hallucinated function names but also fixing malformed JSON and schema errors like missing required properties.
It’s important that instead of telling the model the error it made you tell it the correction you want it to perform. It also helps if you temporarily fork the conversation history while you’re repairing the response. You don’t want to leave the hallucinated response in your conversation history as that increases the likelihood of future hallucinations.