I’m encountering an issue with the OpenAI Realtime API. Our WebRTC setup and connections used to work fine, but since Saturday it has stopped connecting. The ICE process stays in “checking” and eventually fails. I’m not using any ICE servers (iceServers: ), so I’m only getting host candidates. This issue is reproducible even with the official WebRTC example. My network isn’t blocked, so I’m wondering if something changed on OpenAI’s side or if there’s a configuration adjustment needed.
This is exactly what happened to us. We are suddenly not able to establish a connection. It returns status failed when trying to connect to an ice candidate
I’m using web sockets rather than webrtc, but it’s suddenly not working properly, either (first noticed yesterday). It will connect, but then disconnects immediately.
It turned out the issue was that we were passing null for the turn_detection property during session creation instead of undefined (despite the client documentation says null can be passed)
Our code worked prior to Saturday this week and this suggests that OpenAI recently introduced a silent and breaking change allowing invalid sessions to be created that later upon connection will fail
You are right, passing undefined does indeed allow the connection to be established. Our problem now is that this keeps the turn detection on, which we don’t want. How did you manage to keep it off now that “null” is not allowed to be set?
Passing undefined establishes the connection without issues. However, this disables manual control of audio events since it keeps turn detection enabled, which we don’t want.
@chsotelogut we were able to at least prevent the events with this configuration:
‘type’: ‘server_vad’,
‘threshold’: 1.0,
‘create_response’: false,
‘interrupt_response’: false,
basically the threshold is so high it prevents the api to engage vad. Its a bit hacky tho
This issue is still not solved if you’re building a “push to talk” feature.
Setting “turn_detection” to undefined enables turn detection, and setting the “threshold” to 1.0 disables the turn detection but also prevents initiating an audio input.
Has anyone found a solution for this?
I was looking at what the playground does, and even with turn detection turned off prior to starting the session, they still start the session with server_vad and its default settings. Then they send a session update to set turn_detection to null (among other things). I tried setting turn detection up that way, and it seems like it works to me.