How do we submit evals for function calls?

API
1

I am noticing quite a few cases where evals for function calls can help

  1. While testing I made a searchbot and asked it to search for “404 in user profiles.” no matter what I did I only got “profiles404 user” in my function args.

  2. While testing I noticed that no matter what I do I can not get the model to run multiple functions, “search for car and then bar, what is common?”

Reading through eval source code I can see that the completions only look at the returned message:

I tried searching through the repo for function_call and can not find anything.

Can OpenAI add a sample eval for function calls, then we can all together help hone the functionality! :hugs:

4 Likes
2

I faced this problem myself today. Is it even possible to include the functions in the first place?

1 Like