A structured schema specification can be used on either the function writing or on the final response to the user (response output JSON is probably used by code instead of directly shown). – or both.
Therefore, it is completely documented and permissible to specify strict:true on a function specification, with the other requirements: of no properties being optional, all properties being specified in “required” at every object nest level, and additionalProperties: false at every object level.
A function is not for getting an output from the AI - it is for using your code’s tool utilities when needed.
I crafted a little (big, actually) chatbot-as-utility for sending and parsing, using streaming, using whatever features and formats I might “turn on”, and spent a good part of the day fuzzing the API. Here’s what I find:
Structured Output
Completely working with strict function AND strict response_format
This is what actually causes issues, that others have not overcome.
This requires valid schema and valid tool specification of course. Also a requirement is to use the message content object format with blocks of “type” (as if sending images also). On everything sent. Not untyped strings.
A complete “chat session” is shown here (read along) that supplies a tool call back and a tool return to the API, first a single tool invocation seen in the history, and then parallel tool call.
{
"model": "gpt-4o-2024-08-06",
"messages": [
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are a helpful assistant."
}
]
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "Check the weather in london"
}
]
},
{
"role": "assistant",
"tool_calls": [
{
"index": 0,
"id": "call_bJbU2WrFTr8r2dEXROGsw5Pi",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"location\":\"London\",\"unit\":\"celsius\"}"
}
}
]
},
{
"role": "tool",
"name": "get_current_weather",
"content": [
{
"type": "text",
"text": "London, UK: 19C, sunny"
}
],
"tool_call_id": "call_bJbU2WrFTr8r2dEXROGsw5Pi"
},
{
"role": "assistant",
"content": [
{
"type": "text",
"text": "{\"response_to_user\":\"The current weather in London is 19\u00b0C and sunny.\"}"
}
]
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "How about Portland and Seattle?"
}
]
},
{
"role": "assistant",
"tool_calls": [
{
"index": 0,
"id": "call_bJbU2WrFTr8r2dEXROGsw5Pi",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"location\": \"Portland\", \"unit\": \"fahrenheit\"}"
}
},
{
"index": 1,
"id": "call_Hhhlczyjfqlb4Pc3o5Qyow2N",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"location\": \"Seattle\", \"unit\": \"fahrenheit\"}"
}
}
]
},
{
"role": "tool",
"name": "get_current_weather",
"content": [
{
"type": "text",
"text": "Portland, OR: 72F, Sunny"
}
],
"tool_call_id": "call_bJbU2WrFTr8r2dEXROGsw5Pi"
},
{
"role": "tool",
"name": "get_current_weather",
"content": [
{
"type": "text",
"text": "Seattle, WA: 66F, Overcast"
}
],
"tool_call_id": "call_Hhhlczyjfqlb4Pc3o5Qyow2N"
}
],
"stream": true,
"stream_options": {
"include_usage": true
},
"tools": [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city to get the weather of"
},
"unit": {
"type": "string",
"enum": [
"celsius",
"fahrenheit"
],
"description": "The unit of temperature, F for USA"
}
},
"required": [
"location",
"unit"
],
"additionalProperties": false
},
"strict": true
}
}
],
"tool_choice": "auto",
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "structured_response",
"schema": {
"type": "object",
"properties": {
"response_to_user": {
"type": "string",
"description": "The assistant's response to the user"
}
},
"required": [
"response_to_user"
],
"additionalProperties": false
},
"strict": true
}
}
}
(it was sent without the white space)
That gives our response, AI talking about the weather from two cities of two parallel tool calls - again the output being placed in my basic response schema as it was earlier in the chat history:
{
"response_to_user": "In Portland, OR, the current weather is 72\u00b0F and sunny.\nIn Seattle, WA, the current weather is 66\u00b0F and overcast."
}
Unpredictable performance
The AI can write to the user before calling a function, in the same output. If using a response schema, AI can write this response to a user in JSON, and also call the tool with structured response being strict.
What fails is sending the assistant message back with both “content” and “tool_call” - even though that’s what the AI emitted. OpenAI failed to account for this and does not provide a good error message why (such as message rejection from validation error).
"messages": [
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are a helpful assistant."
}
]
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "Tell me what tools you have for getting weather. Then get the weather for Miami."
}
]
},
{
"role": "assistant",
"content": [
{
"type": "text",
"text": "{\"response_to_user\":\"I can get the current weather for a given location. Let me check the weather for Miami for you.\",\"response_topic\":\"Get current weather\"}"
}
],
"tool_calls": [
{
"index": 0,
"id": "call_PhUTQUo00ds0WwuisCICzWwz",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"location\":\"Miami\",\"unit\":\"fahrenheit\"}"
}
}
]
},
{
"role": "tool",
"name": "get_current_weather",
"content": [
{
"type": "text",
"text": "Miami, FL: 88F, Sunny and clear"
}
],
"tool_call_id": "call_PhUTQUo00ds0WwuisCICzWwz"
}
]
The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID…
The solution there seems to be a workaround where you send an assistant message with the content just for the AI’s understanding of its pre-discussion, and then send a second assistant message with the tool_call that was emitted, immediately followed by the tool return.
Successful chat session with combined user response and tool call, all structured
By splitting assistant feedback into two chat history messages
Note: I manually provide what a tool function would return.
User: Tell me what tools you have for getting weather. Then get the weather for Miami.
{“response_to_user”:“I have a tool called get_current_weather which allows me to fetch the current weather for a given location. Here’s the weather for Miami:”,“response_topic”:“Weather in Miami”}
Usage: prompt_tokens=155, completion_tokens=67, total_tokens=222
prompt_tokens_details: cached_tokens: 0
completion_tokens_details: reasoning_tokens: 0
*** Tool call detected ***
get_current_weather with arguments:
{“location”:“Miami”,“unit”:“fahrenheit”}
Please provide the result for get_current_weather: Miami's weather today: 77F and sunny!
{“response_to_user”:“The current weather in Miami is 77°F, and it’s sunny!”,“response_topic”:“Weather in Miami”}
Usage: prompt_tokens=245, completion_tokens=32, total_tokens=277
prompt_tokens_details: cached_tokens: 0
completion_tokens_details: reasoning_tokens: 0
Latest gpt-4o or mini required…