Realtime session automatically closed after session update

API Bugs
,
1

Hi everyone, I’ve been dealing with this issue for several days but can’t find where’s the problem, please help me out if possible, thank you so much!

I was using Realtime API for my product, everything worked fine on local machine.

I built a WebSocket client, created a realtime session with OpenAI, then sent session update and started my business logic. But when I deploy it to a remote machine (I’ve tried on Render and Heroku), once I sent the session update request, the connection was closed with code 1002.

I’ve tried the same code on my local machine and it still worked fine, but not on any remote instance.

I first though it had something to do with Render or Heroku, but then I upgraded my instance and still couldn’t make it work. I even pinged the server every second to keep the connection alive, but as long as I sent session update request, it just magically closed.

Here’s my code for reproduction:

export async function oaiConnectionHandler(wss: WebSocket, Queue: Queue, botId: string, language: string[  ]) {
   wss.on('open', () => {
        console.log('OpenAI connection established');

        const event = {
            "type": "session.update",
            "session": {
                "modalities": ["text"],
                "instructions": "My instruction",
                "model": "gpt-4o-realtime-preview-2024-12-17",
                "input_audio_transcription": {
                    "model": "whisper-1",
                },
                "tools": [
                    {
                        "type": "function",
                        "name": "translate",
                        "description": "Description here",
                        "parameters": {
                            "type": "object",
                            "properties": Object.fromEntries(
                              language.map((lang: string) => [lang, { type: "string" }])
                            ),
                            "required": language,
                        }
                    }
                ],
                "tool_choice": "required",
                "turn_detection": {
                    "type": "server_vad",
                    "threshold": 0.4,
                    "prefix_padding_ms": 50,
                    "silence_duration_ms": 200,
                    "create_response": false
                },
            }
        } 
        wss.send(JSON.stringify(event));
        console.log("Session update request sent");

        const pingInterval = setInterval(() => {
            if (wss.readyState === WebSocket.OPEN) {
                wss.ping();
            } else {
                clearInterval(pingInterval);
            }
        }, 15000);

        wss.on('close', function(data)  {
            clearInterval(pingInterval);
            const closeEvent = data.toString();
            const event: OaiConnectionCloseEvent = {
                botId,
                type: 'close',
                timestamp: Date.now()
            };
            sessionEvents.emit('close', event);
            console.log('OpenAI Connection closed: ', closeEvent);
        });
    });

    wss.on('pong', () => {
        console.log('Pong received');
    });

    wss.on('ping', () => {
        console.log('Ping received');
        wss.pong();
    });

    wss.on('error', (error) => {
        console.error('OpenAI connection error:', error);
        throw error;
    });

Please let me know if there’s problem in my code that causes this issue. Any help would be much appreciated!

Edit: I also noticed that when the connection was established first, the first session update request won’t get response from OpenAI and the second one would cause the session to close.