Infinite loop with "tool_choice": "required" or type: "function"

API
1

My completion function calls itself recursively when tool_calls is true in the response message. With tool_choice: "auto" everything runs as expected. I append assistant’s tool calls and tool’s response to the chat, call the model again with updated messages and get completion back properly.

But with tool_choice: "required" or tool_choice: { type: 'function', function: { name: 'getContext' } } the model keeps returning the same tool calls with different ids resulting in my function being stuck in infinite loop.

Example:

  1. First call to API with messages:
[
    {
      role: 'system',
      content: 'You are a customer support agent'
    },
    { role: 'assistant', content: 'Hello, how may I help you?' },
    { role: 'user', content: 'hi' }
 ],
  1. API response:
{
  finishReason: 'stop',
  message: {
    role: 'assistant',
    content: null,
    tool_calls: [
      {
        id: 'call_2LuhMirCrpHwWrDrYeHpbseJ',
        type: 'function',
        function: { name: 'getContext', arguments: '{"query": "greeting"}' }
      }
    ]
  }
}
  1. Calling API again with updated messages:
[
    {
      role: 'system',
      content: 'You are a customer support agent'
    },
    { role: 'assistant', content: 'Hello, how may I help you?' },
    { role: 'user', content: 'hi' },
    {
      role: 'assistant',
      content: null,
      tool_calls: [
        {
          id: 'call_2LuhMirCrpHwWrDrYeHpbseJ',
          type: 'function',
          function: { name: 'getContext', arguments: '{"query": "greeting"}' }
        }
      ]
    },
    {
      tool_call_id: 'call_2LuhMirCrpHwWrDrYeHpbseJ',
      role: 'tool',
      name: 'getContext',
      content: 'content provided by getContext function'
    }
  ],
  1. API responds with another tool call to the same function:
{
  finishReason: 'stop',
  message: {
    role: 'assistant',
    content: null,
    tool_calls: [
      {
        id: 'call_pzU4fsFxQPzaeyn5xq0SkWsq',
        type: 'function',
        function: { name: 'getContext', arguments: '{"query":"greeting"}' }
      }
    ]
  }
}
2

Don’t call the model again with a forced tool choice

1 Like
3

Ah, of course it’s something simple. Appreciate you brother.

1 Like