Formatting Assistant Messages After Tool Function Calls in GPT Conversations

API
1

If the assistant called a tool function in a previous step, how to format the subsequent assistant message such that it will match the expected format built into the gpt?

When the function calls in previous messages are formatted in say, markdown or some other non-structured way, GPT-4 stops calling the function and instead starts following the custom format that was used. This is obviously annoying because it stops calling the tool / function.

I’m going to try something like

json.dumps({
            "function": "ask_user_clarifying_question",
            "arguments": {
                "question": clarification.question,
                "choices": [choice.name for choice in clarification.choices]
            }
        })

but wanted to check in case there was a better format to use that wouldn’t confuse the model so much.

EDIT: Formatting the function like above still results in the model printing out a string like the format above instead of calling the tools after 2 or 3 responses. So there must be some official / API documented way to report back function calls in previous steps to agent.

EDIT 2: While I see that there is a type ChatCompletionMessage type inside of the response, I can’t reuse this object directly and must instead reconstruct it from scratch.

Something like this but converted back into the correct format that can be passed back to the model

message = ChatCompletionMessage(
    content=None,
    role='assistant',
    function_call=None,
    tool_calls=[
        ChatCompletionMessageToolCall(
            id='call_W6C7NV3fSVomN4zgw1EBNlNI',
            function=Function(
                arguments=(
                    '{"question":"What type of device are you experiencing slow Wi-Fi on?",'
                    '"choices":["Laptop","Smartphone","Tablet","Desktop PC","Smart TV","Other"]}'
                ),
                name='ask_user_clarifying_question'
            ),
            type='function'
        )
    ]
)

EDIT 3:

Ok, getting the format right and reusing the same tool call id wasn’t so difficult. However this still leads to the issue

openai.BadRequestError: Error code: 400 - {
    'error': {
        'message': (
            "An assistant message with 'tool_calls' must be followed by tool messages "
            "responding to each 'tool_call_id'. The following tool_call_ids did not have "
            "response messages: call_auTHoCmlzOHNdwQuO1NXtOZT"
        ),
        'type': 'invalid_request_error',
        'param': 'messages.[3].role',
        'code': None
    }
}

So it seems like tools really aren’t supposed to be used in the way I’m using them, which is namely to format text for some external function call. It seems like using the json formatter is probably the way to go, except that with json formatter it’s not as clear or easy to prompt the model to follow a certain function format.

4 Likes
2

Having similar issue with this https://llamahub.ai/l/tools-google_search
The error occur when invoking agent.chat(....) then gives error as follows:

BadRequestError: Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_BgScdFLK2sY1tz3VaP4ExrFc", 'type': 'invalid_request_error', 'param': 'messages.[2].role', 'code': None}}
3

https://platform.openai.com/docs/guides/function-calling

 messages.append(
      {
          "tool_call_id": tool_call.id,
          "role": "tool",
          "name": function_name,
          "content": function_response,
      }
  )  # extend conversation with function response

Seemingly this is the right way to construct a response to a tool call.

I’m going to try to code this solution and see how it goes.

UPDATE: Responding with the role “tool” and the tool call id as above is the right way to respond to tool calls and will allow you to keep a context with GPT such that it won’t go off the rails and start producing invalid tool calls. For reference, here’s the way I construct the assistant’s tool call from scratch in python.

function_response = {
    "name": "ask_user_clarifying_question",
    "arguments": json.dumps({
        "question": clarification.clarification_question.question,
        "choices": [choice.name for choice in clarification.clarification_question.choices]
    })
}
assistant_message = {
    "role": "assistant",
    "tool_calls": [{"id": clarification.clarification_question.tool_call_id, "function": function_response, "type": "function"}]
}
messages.append(assistant_message)
3 Likes