ChatGPT occasionally reuses tool IDs in the same session

API Bugs
, ,
1

I can’t easily reproduce this, but this happens maybe 1 in 100 times so it is happening enough to be a problem.

I am building an autogpt-like experience where ChatGPT would call functions successively and execute on some task. I’m using the tool API, so if there is a tool ID referenced in an assistant’s function call, there must be an associated tool response with the same ID. As a toy example, if I use these kwargs:

kwargs = {
  "stream": False,
  "model": "gpt-3.5-turbo-1106",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant"
    },
    {
      "role": "user",
      "content": "Send so-and-so@gmail.com an email"
    },
    {
      "role": "assistant",
      "content": None,
      "tool_calls": [
        {
          "id": "call_gO7MBZKAevgoMoTpdmowmTcx",
          "function": {
            "name": "perform_action",
            "arguments": "some arguments"
          },
          "type": "function"
        }
      ]
    } #, Uncommenting this will fix the below error
    # {
    #   "role": "tool",
    #   "tool_call_id": "call_gO7MBZKAevgoMoTpdmowmTcx",
    #   "content": "some outputs"
    # }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "perform_action",
        "description": "Some action"
      }
    }
  ]
}

I will get the following error:

BadRequestError: Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_gO7MBZKAevgoMoTpdmowmTcx", 'type': 'invalid_request_error', 'param': 'messages', 'code': None}}

The problem I’m running into is that ChatGPT will occasionally reuse a tool ID in a tool call, and that causes a different error:

openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid parameter: messages with role 'tool' must be a response to a preceeding message with 'tool_calls'.", 'type': 'invalid_request_error', 'param': 'messages.[10].role', 'code': None}}

I can reproduce the error by repeating a function call & output with a duplicate Tool ID.

kwargs = {
  "stream": False,
  "model": "gpt-3.5-turbo-1106",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant"
    },
    {
      "role": "user",
      "content": "Send so-and-so@gmail.com an email"
    },
    {
      "role": "assistant",
      "content": None,
      "tool_calls": [
        {
          "id": "call_gO7MBZKAevgoMoTpdmowmTcx",
          "function": {
            "name": "perform_action",
            "arguments": "some arguments"
          },
          "type": "function"
        },
        {
          "id": "call_gO7MBZKAevgoMoTpdmowmTcx",
          "function": {
            "name": "perform_action",
            "arguments": "some arguments"
          },
          "type": "function"
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_gO7MBZKAevgoMoTpdmowmTcx",
      "content": "some outputs"
    },
    {
      "role": "tool",
      "tool_call_id": "call_gO7MBZKAevgoMoTpdmowmTcx",
      "content": "some outputs"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "perform_action",
        "description": "Some action"
      }
    }
  ]
}

As a developer, it is annoying that the tool call IDs are not unique across the same session, and it seems like there may be bug that’s causing this to happen. I know that there are workarounds like issuing my own IDs, but this seems like it should be fixed on the API side.

5 Likes
2

Just discovered this same issue – though I would say not ChatGPT, but rather in models gpt-3.5-turbo and gpt-4-1106-preview through API calls (chat completions). Re-used call-identifiers across tool calls are proving to be problematic, and these are across sessions (different days, different tools, etc.).

I guess I assumed these would be unique.

4 Likes
3

We are experiencing same issue with Gpt-4o.
Have you found any workaround to this?

4

i just ended up issuing my own uuids instead of using the tool ID that the API provides

2 Likes
5

I’m having the same problem, but I’m using autogen.

6

The issue isn’t that the tool id’s are being recycled but rather a glitch or bug that incorrectly flags valid sequences as erroneous.

7

I am also experiencing this with gpt-4o. did any of you manage to get support to look at this?

8

I had this issue a long time ago, one walkaround is to replace it with your own id, here is some sample code, you can adjust the length based on your usage case.

    def replace_tool_call_ids(self, response_data: ChatCompletion, model: str) -> None:
        """
        Replace tool call IDs in the response to:
        1) Ensure uniqueness by generating new IDs from the server if duplicates exist.
        2) Shorten IDs to save tokens (length optimization may be adjusted).
        """
        for choice in response_data.choices:
            message = choice.message
            tool_calls = message.tool_calls
            if tool_calls:
                for tool_call in tool_calls:
                    tool_call.id = self.generate_tool_id()

    def generate_tool_id(self) -> str:
        """Generate a short tool call ID for session-scoped uniqueness.

        Uses 4-char random suffix since unique IDs only needed within
        a single session's context window

        Returns:
            String like: 'call_a1b2'
        """
        tool_call_id = generate_id(prefix="call_", length=4)
        return tool_call_id

here is the full example: cue/src/cue/llm/openai_client.py at main · zhaolongzhong/cue · GitHub

1 Like
9

yes, im already doing that, but we should not have to do that workaround. It is pretty a absurd bug to have duplicate IDs, since those are not generated by the LLM.

2 Likes
10

I have this problem too. Recently the frequency has increased, I would say. We are currently using gpt-4o-mini and gpt4-turbo

11

Hi and welcome!

Could you setup some logging such that everything you send to the model and the full response object gets logged?

I think it would be valuable for you to be able to check those logs after you get an error like this to see exactly what was sent and what the error actually is.

It would be super helpful to be able to track this down if it is indeed an error on the backend, but without detailed logs of the issue, it’s very hard to catch.

1 Like