Hi everyone,
I’m testing a voice agent using the OpenAI Realtime API via SIP.
I have a production flow that works +/- correctly with gpt-realtime-mini: the assistant collects a delivery order by phone, confirms the order with the caller, and then calls a function tool to register/send the order to my system.
However, when I change only the model to gpt-realtime-2.1-mini, using the same prompt, same SIP accept payload, same tools, and same backend handler, the model does not call the function tool. Instead, after collecting and confirming the order, it answers something like:
“I understand, but I can’t register the order here. You should contact human support or use WhatsApp.”
This does not happen with gpt-realtime-mini. With the older model, the function is called as expected.
My SIP accept payload is basically:
return [
'type' => 'realtime',
'model' => gpt-realtime-2.1-mini,
'instructions' => PROMPT,
'output_modalities' => ['audio'],
'audio' => [
'input' => [
'format' => [
'type' => 'audio/pcm',
'rate' => 24000
],
'noise_reduction' => [
'type' => 'far_field'
],
'transcription' => [
'model' => 'gpt-4o-transcribe',
'language' => 'pt',
'prompt' => 'Transcribe with maximum fidelity. Proper names are critical. Do not correct names. If you’re unsure, keep exactly what you heard.'
],
'turn_detection' => [
'type' => 'server_vad',
'threshold' => 0.6,
'prefix_padding_ms' => 500,
'silence_duration_ms' => 1100,
'idle_timeout_ms' => 12000,
'create_response' => true,
'interrupt_response' => true,
],
],
'output' => [
'format' => [
'type' => 'audio/pcm',
'rate' => 24000
],
'voice' => $voice,
],
],
'tools' => $tools,
'tool_choice' => 'auto'
];
The main function tool is similar to this:
{
"type": "function",
"name": "send_notification_2",
"description": "Registers and sends a confirmed delivery order to the system. Use only when all required fields are complete and confirmed.",
"parameters": {
"type": "object",
"properties": {
"resp_endereco": {
"type": "string",
"description": "Delivery address"
},
"resp_quantidade": {
"type": "string",
"description": "Product and quantity"
},
"resp_observacao": {
"type": "string",
"description": "Optional delivery notes"
}
},
"required": [
"resp_endereco",
"resp_quantidade"
]
}
}
Expected behavior:
-
Caller asks for a gas delivery.
-
Assistant collects product/quantity and address.
-
Assistant confirms the full order.
-
Caller confirms.
-
Assistant calls
send_notification_2. -
Backend executes the function and returns the result.
Actual behavior with gpt-realtime-2.1-mini:
-
Caller asks for a gas delivery.
-
Assistant collects product/quantity and address.
-
Assistant confirms the full order.
-
Caller confirms.
-
Assistant does not emit a function call.
-
Assistant replies that it cannot register/send the order directly and suggests human support or WhatsApp.
Actual behavior with gpt-realtime-mini:
The same flow successfully emits the function call and the order is registered.
I also tried making the prompt more explicit, with instructions such as:
-
“The assistant is allowed to register orders during the call.”
-
“Registering an order means executing the function tool.”
-
“Never say you cannot register the order if the required fields are complete.”
-
“After order confirmation, call the function.”
But gpt-realtime-2.1-mini still avoids the function call.
I also tested adding text to output_modalities, but in the SIP accept flow this caused the WebSocket connection to fail with:
Unexpected server response: 404
So I reverted to:
"output_modalities": ["audio"]
Questions:
-
Is there any known difference in tool calling behavior between
gpt-realtime-miniandgpt-realtime-2.1-mini? -
Is
gpt-realtime-2.1-miniexpected to support function calling in Realtime SIP sessions the same way asgpt-realtime-mini? -
Should function tools be configured differently for
gpt-realtime-2.1-mini? -
Is there a recommended way to force or strongly bias function calling after confirmation in SIP Realtime?
-
Could
create_response: truewith server VAD affect tool calling decisions in this model?
Any suggestions on what to log or test would be appreciated. I can share sanitized event logs if helpful.
As I know, the 2.1-mini it’s working better then tha old mini model to listen and answer voice calls.
The voice call and the prompt is totally in Portuguese, because it I don’t share details.