Recently, I noticed unexpected behavior in OpenAI’s Realtime API with WebRTC. Previously, setting turn_detection to null when creating or updating a session allowed manual control over sending audio to the server. However, after a recent change, this configuration no longer works.
Now, if turn_detection is set to null, the AI does not generate a response, or in some cases, the connection fails to establish correctly. This change affects the ability to manually manage when to send audio to the server, which is crucial for implementations requiring custom flow control.
For reference, here is a typical configuration where turn_detection used to work correctly:
{
"event_id": "event_123",
"type": "session.update",
"session": {
"modalities": ["text", "audio"],
"instructions": "You are a helpful assistant.",
"voice": "sage",
"input_audio_format": "pcm16",
"output_audio_format": "pcm16",
"input_audio_transcription": {
"model": "whisper-1"
},
"turn_detection": null,
"tools": [
{
"type": "function",
"name": "get_weather",
"description": "Get the current weather...",
"parameters": {
"type": "object",
"properties": {
"location": { "type": "string" }
},
"required": ["location"]
}
}
],
"tool_choice": "auto",
"temperature": 0.8,
"max_response_output_tokens": "inf"
}
}
Has this change been officially documented? Is there any recommended alternative to maintain manual audio control without relying on turn_detection?
I would appreciate any information or clarification on this behavior and whether there is a viable solution to retain the previously available manual workflow.