When initiating a WebRTC session with the GPT realtime API, it’s possible to connect to the same “call” using a websocket connection. When initiating the websocket connection, there appears to be no response event sent from the call/Realtime API. That means, if the WebRTC session is quiet for e.g more than a minute (or whatever the specified timeout for ping_response events in the websocket connection), the websocket connection drops and needs to be re-established. Is it possible/in future scope, to also relay response events to the side-band websocket connection?
Hi @nw79
What language/platform are you using? Nothing on the server side enforces a timeout. I will investigate what we can do so it doesn’t drop. Maybe we aren’t implementing stuff on our side properly.
For response events you want to get the audio from the model down the websocket? Are you also interested in what the user is saying?
thanks
Hi @Sean-Der thanks for the follow up.
Using Node.js express server to initiate the WebRTC session through the browser (client-side) and using a websocket side-band connection to the same WebRTC session (server-side) using the call_id returned when initiating the session.
Here is a small code snippet showing how the python client connects using websockets to create the side-band connection:
`self.ws = await websockets.connect(`
url,
ssl=self.ssl_context,
extra_headers=headers,
ping_interval=10,
ping_timeout=ping_timeout, # configurable timeout
)
The ping timeout is what determines when the connection drops. So if I set this to e.g 60 seconds, after 60 seconds of silence, the websocket connection will drop from the call, presumably because no response/pong is sent back to ping messages from the websocket library.
The python client already implements event handler logic for the response events returned by the realtime API, so that is not a concern. It’s maintaining the side-band connection.
Let me know if I can provide more info!
Thanks!