Function call not resulting in correct JSON object

API Bugs
, ,
1

When using a function call on the gpt-3.5-turbo-0613 model I get an incorrect response format in the arguments property of the function. I recently added a “maxItems” property to the JSONSchema used to limit the amount of objects in the array which could be causing the issue but according to the docs this should be no issue.

I’ve also used the gpt-3.5-turbo-0125 model and here it seems that sometimes the function isn’t even used and a response is inserted into the content property of the message. Other times the array is way longer than 5 items.

Here is my request:

{
  "messages": [
    {
      "role": "user",
      "content": "Instruction ..."
    }
  ],
  "tool_choice": "auto",
  "model": "gpt-3.5-turbo-0613",
  "tools": [
    {
      "function": {
        "name": "Get",
        "description": "Get something",
        "parameters": {
          "type": "object",
          "properties": {
            "actions": {
              "maxItems": 5,
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "id": {
                    "type": "string"
                  },
                  "input": {
                    "type": "string"
                  },
                  "type": {
                    "type": "string",
                    "enum": [
                      "a",
                      "b",
                      "c"
                    ]
                  }
                },
                "required": [
                  "id",
                  "type",
                  "input"
                ]
              }
            }
          }
        }
      },
      "type": "function"
    }
  ]
}

The response from the 0125 model with too many items :

{
  "created": 1708946219,
  "usage": {
    "completion_tokens": 1296,
    "prompt_tokens": 6972,
    "total_tokens": 8268
  },
  "model": "gpt-3.5-turbo-0125",
  "id": "chatcmpl-xxxx",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "role": "assistant",
        "tool_calls": [
          {
            "function": {
              "name": "Get",
              "arguments": "{\"actions\":[{\"id\":\"0.0.0\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.1\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.8\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.9\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.10\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.11\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.12\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.13\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.14\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.15\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.16\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.17\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.18\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.19\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.20\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.21\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.22\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.0.23\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.0.0\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.1\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.1.0\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.1.1\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.1.2\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.1.3\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.2.0\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.2.1\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.2.2\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.2.3\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.3.0\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.3.1\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.3.2\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.3.3\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.4.0\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.4.1\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.4.2\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.4.3\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.5.0\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.5.1\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.5.2\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.5.3\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.6.0\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.6.1\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.6.2\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.6.3\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0.0.7.0\",\"input\":\"a\",\"type\":\"a\"},{\"id\":\"0.1.0.0.4.0.0."
            },
            "id": "call_xxx",
            "type": "function"
          }
        ],
        "content": null
      },
      "logprobs": null
    }
  ],
  "system_fingerprint": "xx",
  "object": "chat.completion"
}

Is anybody experiencing the same issue? Any help is greatly appreciated.

2

With an identical request but removing the maxItems works 80% out of the times and then suddenly returns a response without a tool_calls property:

{
  "created": 1708952494,
  "usage": {
    "completion_tokens": 268,
    "prompt_tokens": 1609,
    "total_tokens": 1877
  },
  "model": "gpt-3.5-turbo-0613",
  "id": "chatcmpl-xxx",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Based on the available ...."
      },
      "logprobs": null
    }
  ],
  "system_fingerprint": null,
  "object": "chat.completion"
}
1 Like
3

From the docs:
If you want to force the model to call a specific function you can do so by setting tool_choice with a specific function name.

Also, trying putting something about your 5 item limit, in the dunction fdescription, not jus t rely on the max items property,

4

Thanks, I will try this.
I had a look at this parameter but I misinterpreted the text :sweat_smile: