Realtime Audio API Connection Drops After 20-30 Seconds

API
1

After about 20-30 seconds of interaction with the Realtime Audio API, the connection drops. My audio is still physically transmitted from my side, but I stop receiving responses from the server at that point. It’s like the connection breaks mid-sentence from their end, cutting off the reply. This issue is consistently reproducible. Please help troubleshoot this.

1 Like
2

You are probably hitting the max tokens. When this happens I will see the full text in transcription but the model stops sending. .. no other warning, it just stops!

be sure MAX_OUTPUT_TOKENS is set high enough… I need a few thousand for a friendly conversation about news, family, weather, etc. My application is a companion for senior citizens.

{
            "type": "response.create",
            "response": {
                "conversation":"auto",
                "instructions": greet_user,
                "max_output_tokens": MAX_OUTPUT_TOKENS,
                "output_modalities": ["audio"],
                "audio": {
                    "output": {
                        "format": {
                            "type": "audio/pcm",
                            "rate": NATIVE_OAI_SAMPLE_RATE_HZ
                        },
                        "voice": master_state.conman.get_config("VOICE"),
                    }
                }
            }
2 Likes
3

Hey, thanks a lot for the suggestion - I actually checked that too, but it turned out not to be a token issue.
My logs showed that audio was still streaming normally while the DataChannel suddenly closed in the middle of the session.

Once I fixed the DataChannel handling and reconnection logic, everything started working perfectly - no more 20–30s dropouts. :smiley:

Appreciate the help though, it really pointed me in the right direction :raising_hands:

2 Likes