Assistants API - Sometimes the Required_Action portion of the Run Object is in a format that is undocumented

API Bugs
1

The program is a simple example of using the assistants API with function calling. There are three functions: get_lat, get_long and get_display_name. The first two functions use an API call to get latitude and longitude respectively, the third function provides a name of the of the location.

To satisfy the prompt: Get the coordinates of the location 55 King Street, Cobourg, Ontario, all three functions should be called.

Most times the required_action portion of the Run object looks like the one below. Some times it breaks the work into multiple steps. Either way is fine.

'required_action': {'submit_tool_outputs': {'tool_calls': [{'id': 'call_FLf8qdNeIh7YC3tauBfTkQ1d',
     'function': {'arguments': '{"location": "Cobourg, Ontario"}',
      'name': 'get_lat'},
     'type': 'function'},
    {'id': 'call_3HrT4gh7oX8Yh2gMirC6Lq2y',
     'function': {'arguments': '{"location": "Cobourg, Ontario"}',
      'name': 'get_long'},
     'type': 'function'},
    {'id': 'call_Z6lrL2GDeCoXzBKkQ23IFodR',
     'function': {'arguments': '{"location": "Cobourg, Ontario"}',
      'name': 'get_display_name'},
     'type': 'function'}]},
  'type': 'submit_tool_outputs'},
 'started_at': 1701577250,
 'status': 'requires_action',

The problem arises when the required_action portion of the run object looks like the one below. Notice the function name becomes ‘paralell’. To the best of my knowledge this format is undocumented.

'required_action': {'submit_tool_outputs': {'tool_calls': [{'id': 'call_O5hqVnwak0UWOEGqWRUQEKt7',
     'function': {'arguments': '{\n  "tool_uses": [\n    {\n      "recipient_name": "functions.get_lat",\n      "parameters": {\n        "location": “55 King St, Cobourg, Ontario"\n      }\n    },\n    {\n      "recipient_name": "functions.get_long",\n      "parameters": {\n        "location": “55 King St, Cobourg, Ontario"\n      }\n    }\n  ]\n}',
      'name': 'parallel'},
     'type': 'function'}]},
  'type': 'submit_tool_outputs'},
 'started_at': 1701548209,
 'status': 'requires_action',

The question is am I missing something ? I can run this program over and over again, and most times a correctly formatted (as documented) run object is created, other times I get the format shown above. When this does occur the format is always the same, so if it is a hallucination, it is a consistent one.

2

Are you using an 1106 AI model? They are the only ones trained to emit parallel by tool specification.

What we expect from API docs:

Untitled
Untitled445×518 40 KB

{
  "required_action": {
    "submit_tool_outputs": {
      "tool_calls": [
        {
          "id": "call_O5hqVnwak0UWOxxxxxxxxx",
          "function": {
            "arguments": "{\n  \"tool_uses\": [\n    {\n      \"recipient_name\": \"functions.get_lat\",\n      \"parameters\": {\n        \"location\": “55 King St, Cobourg, Ontario\"\n      }\n    },\n    {\n      \"recipient_name\": \"functions.get_long\",\n      \"parameters\": {\n        \"location\": “55 King St, Cobourg, Ontario\"\n      }\n    }\n  ]\n}",
            "name": "parallel"
          },
          "type": "function"
        }
      ]
    },
    "type": "submit_tool_outputs"
  },
  "started_at": 1701548209,
  "status": "requires_action"
}

and the readable content of the arguments:

{
  "tool_uses": [
    {
      "recipient_name": "functions.get_lat",
      "parameters": {
        "location": “55 King St, Cobourg, Ontario"
      }
    },
    {
      "recipient_name": "functions.get_long",
      "parameters": {
        "location": “55 King St, Cobourg, Ontario"
      }
    }
  ]
}

So it seems like tool_uses is supposed to map to the multiple function objects, but the AI encoder of the non-1106 understands the specification as a single function, or the 1106 model produced an unexpected output that was not parsed properly.

If it is “sometimes”, it can be that sometimes wrong tokens are selected when the AI produces function, and a parameter like top_p = 0.5 is needed to constrain to only the token 50% or more likely.

1 Like
3

Yes I am using the 1106 model and thanks, this helps !

I have consistently used the 1106 model and the same prompt and have encountered three different kinds of run objects.

  1. Three ‘requires_action’ steps, one right after another, each with a single function call.
  2. One ‘requires_action’ step with a list of the functions to be called.
    3)One require action step, where the function name is set to ‘parallel’ and the function names be called are in the form of functions.xxxxxx, where xxxx is my function name to be called.
4

@dlaytonj2

Is it OK if a moderator closes this topic?

5

:+1:Yes, this can be closed.