I am able to get the Batch API To accept functions to perform text classification, but get different results compared to running it through the normal chat completion API. Can anyone confirm whether the Batch API officially supports function calls or if they have had success in doing so? I can’t seem to find it in the documentation.
2 Likes
How’s your finding? real curious
I am fairly sure that the batch API calls exactly the same models, just in their “free time.” However I was experiencing the same issue and the problem was that I was writing my requests to a JSONB table to be picked up by some job and submitted to openai. JSONB sorts the keys in the json, so my function was re-ordered and order matters a lot for performance. Changing the data type to JSON in my table fixed the issue.