Hi, I’m switching a task from leveraging the functions parameter to leveraging the tools parameter for chat completions.
Keeping all other parameters the same (temperature, seed, etc.), I’m seeing a noticeable difference in behavior with around 10% of completions. A very small portion of that is actually calling multiple tools, so I’m wondering what the root cause of the difference is.
What is the difference in the implementation of Tools v. Functions on OpenAI’s end? Why would a completion that messages the user with functions now call a tool with tools?
OAI staff here. The key difference between functions and tools is that tools supports parallel function calling which allows the model to output multiple tool calls in a list which the user can execute in parallel.
In terms of the model’s choice on whether or not to call a function/tool, there shouldn’t be any meaningful difference between the two params. What you observed could be due to the non-deterministic nature of the model behavior. But if this is consistently happening to you and you are willing to share your example, I’m happy to take a look.
3 Likes
Makes sense, thanks for the explanation! I ran a baseline eval to see what deviation normally occurs and that helped in benchmarking performance. Now I’m only seeing ~5% increase above the baseline deviation which seems a bit more reasonable.
Appreciate the help!