This is how I handle the "python" Function Calling in chat API. How do you handle it?

API
,
1

If you use the Function Calling feature of chat completion API, you may notice that the API may request to call unexciting functions. For me and other developers, we got requests to call the “python” function, which is not in our functions list.

I like to share how we handled this case and would like to hear from you how did you handle it so we can learn from each other.

First, I noticed that I need to keep the history of function calling in the conversation, especially when the function needs to be called multiple times, otherwise, the API will not request the function again, if it is not present in the conversation history where it should be.

Omitting Function Call Messages From History
Omitting Function Call Messages From History791×671 37.9 KB

Function Call Messages In History
Function Call Messages In History481×813 40.5 KB

Now, when it comes to invalid function names (not in the list of the available functions), I respond with “none” result, and I omit the invalid function call and response (hallucinatedFunctionMessages) from the conversation history:

In this way, the conversation will contain only the user messages and responses, and the valid function calls.

Please let me know how did you handle the invalid function call issue, for example, did to find a preventive solution instead of the mitigative one?

Thanks

5 Likes
2

Awesome, ameramayreh!

Thank you very much for sharing your code and thoughts on this, I know it’s a hot topic and it’s great to see a potential solution. :smiley:

1 Like
3

This is very similar to how I’m handling things.

Please let me know how did you handle the invalid function call issue, for example, did to find a preventive solution instead of the mitigative one?

I’m assuming there is not a way to prevent malformed JSON in the returned function_call? I get these from time-to-time.

I do basically the same as you, adding a blank message. Internally we also store the function_call OpenAI returned.

Then, we retry the same API request up to 3 times.

1 Like
4

If you read my post on the function calling support I added to AlphaWave you’ll see the feedback I’m sending the model for hallucinations and invalid JSON:

I use a JSON schema validator and have the model not only repairing hallucinated function names but also fixing malformed JSON and schema errors like missing required properties.

It’s important that instead of telling the model the error it made you tell it the correction you want it to perform. It also helps if you temporarily fork the conversation history while you’re repairing the response. You don’t want to leave the hallucinated response in your conversation history as that increases the likelihood of future hallucinations.

3 Likes