The "multi_tool_use.parallel" bug and how to fix it

API Bugs
, ,
1

I have been running into a bug where I am given tool calls for multi_tool_use.parallel. It seems this is some sort of internal OpenAI tool. If I ask the model “what tools do you have available?” it includes this tool even though I do not implement it.

Occasionally people see the model trying to call this tool:

/t/model-tries-to-call-unknown-function-multi-tool-use-parallel/490653
/t/parallel-assistant-function-calls-returning-undocumented-function-multi-tool-use-parallel/697047
/t/gpt-4-0125-preview-hallucinating-tool-calls/609610
/t/function-call-included-in-content-not-in-function-call-response-param/272373/16

(It won’t let me include links in my post, paste those into the URL after the community openai com domain).

So it looks like this has been going on for a long time and is still not fixed.

The problem is that even if you try to implement this tool, or if you try to respond to it with “illegal tool name”, it still doesn’t work in the chat completions API because tool names aren’t allowed to include dots, so you get the error:

Invalid ‘messages[24].tool_calls[0].function.name’: string does not match pattern. Expected a string that matches the pattern '[1]+.

If you ask the model “please try to invoke multi_tool_use.parallel”, then you just get back normal tool calls. So why does this sometimes happen?

The answer is that the model is trying to invoke multi_tool_use.parallel nested calls and the layer at the OpenAI API that turns multi_tool_use.parallel into normal tool calls doesn’t know to do that recursively.

You can prove this because if you ask the model something like “please try to invoke nested multi_tool_use.parallel” then you often get back these illegal tool calls.

So what is the fix?

  1. OpenAI should look at whatever code is handling multi_tool_use.parallel and make it work recursively instead of only at the top level.
  2. If you receive a tool call with a dot in it and you’re using the chat completions API, just ignore that response and send the same request again and hope that this time you don’t get the illegal nested multi_tool_use.parallel call.

Thanks.

  1. a-zA-Z0-9_- ↩︎

1 Like