Missing required parameter: message[3].content[0].type

API
1

When I pass the result of a locally invoked tool to the chat completion endpoint as an element in messages list, I found that if the content is a list of JSON objects, each object must have a key-value pair {"type": "text"} . Otherwise, an error will occur.

I only found in the documentation that content must be a string or a list, but there’s no specific requirement for the objects within the list.

Now I have two options:

  1. Manually add {"type": "text"} to each object in the content list.
  2. Directly convert the list into a string and send it.

I’m wondering if there will be any differences in how the model processes the List itself and the List as String.

2 Likes
2

The messages doesn’t accept objects as a model context input, only a string. You don’t return a JSON as a nested python data object, the return from a tool running is plain text in whatever format the AI will best understand.

Here’s an example adding the return strings right to the messages list within the API parameters sent, in this case, two tool returns for the parallel tool call previously emitted. (which also has to be accompanied by the assistant message previously output translated back into input).

# Return values: what the tool_call with multiple functions gives
# rename xparams to params here also for the 2nd run
params['messages'].extend(
[
  {
    "role": "tool", "tool_call_id": "call_rygjilssMBx8JQGUgEo7QqeY", "content":
        "Seattle 2022-12-15 forecast: high 62, low 42, partly cloudy\n"  
  },
  {
    "role": "tool", "tool_call_id": "call_pI6vxWtSMU5puVBHNm5nJhw3", "content":
        "Miami 2022-12-15 forecast: high 77, low 66, sunny\n"  
  }
]
)

There is an alternate case you have stumbled up on which has two keys: type, and “text” if type indicates text. This format is newer and is specifically for multimodal input, also allowing image_url as a type.

You can see the “content” of a user message with images being constructed here into a list of blocks.

content = [
        {"type": "text", "text": "Classify this image using computer vision skill."}
    ]
    
    for path in image_paths:
        base64_image = encode_image_to_base64(path)
        content.append({
            "type": "image_url",
            "image_url": {
                "url": f"data:image/png;base64,{base64_image}"
            }
        })

You can just send the “text” part thusly in the list even if there are no “type”: “image_url” parts. But as images cannot be returned through a tool or function, there’s no need to presently complicate the tool return with anything more than just the string itself as content.

1 Like
3

Got it! Thank you so much! I was confused about the “content” part of a tool message. The docs said it could be a string or an array, so I was like, “When would I ever need an array?” But now I get it - I can just stick with strings for now.