Realtime WebRTC doesn't work with ephemeral token

API
1

I am attempting to connect via WebRTC to the Realtime API for live transcription. If I use the OpenAI API key, it works fine! But, if I add a fetch request to get an ephemeral token, and then pass that token, it does not work. But I think I am getting the ephemeral token correctly?

Here is a minimal example. It is just an HTML file that logs everything. I am just trying to get this working in the browser before building on top of it.

<!DOCTYPE html>
<html>
<head>
    <title>Minimal OpenAI WebRTC Example</title>
</head>
<body>
    <button id="startButton">Start</button>
    <script>
        // Your API Key
        const API_KEY = "<your_api_key_here>";

        // Model & endpoints
        const MODEL = "gpt-4o-transcribe";
        const SESSION_ENDPOINT = "https://api.openai.com/v1/realtime/transcription_sessions";
        const REALTIME_ENDPOINT = "https://api.openai.com/v1/realtime";

        document.getElementById('startButton').addEventListener('click', async () => {
            try {
                console.log("Starting connection...");

                // Get ephemeral key
                console.log("Getting ephemeral key...");
                const tokenResponse = await fetch(SESSION_ENDPOINT, {
                    method: "POST",
                    headers: {
                        "Authorization": `Bearer ${API_KEY}`,
                        "Content-Type": "application/json"
                    },
                    body: JSON.stringify({
                        "input_audio_transcription": {
                            "model": MODEL,
                            "language": "en"
                        }
                    })
                });

                if (!tokenResponse.ok) {
                    throw new Error(`Token error: ${await tokenResponse.text()}`);
                }

                const data = await tokenResponse.json();
                console.log("Session data:", data);

                const ephemeralKey = data.client_secret?.value;
                if (!ephemeralKey) {
                    throw new Error("No ephemeral key in response");
                }
                console.log("Ephemeral key:", ephemeralKey);

                // Create WebRTC connection
                console.log("Setting up WebRTC...");
                const pc = new RTCPeerConnection();

                // Get microphone access
                const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
                pc.addTrack(stream.getTracks()[0], stream);

                // Create data channel for messages
                const dc = pc.createDataChannel("oai-events");
                dc.onmessage = (e) => {
                    console.log("Message from OpenAI:", JSON.parse(e.data));
                };

                // Create and send offer
                const offer = await pc.createOffer();
                await pc.setLocalDescription(offer);

                // Send SDP to OpenAI
                console.log("Sending SDP to OpenAI...");
                const sdpResponse = await fetch(`${REALTIME_ENDPOINT}?model=${MODEL}`, {
                    method: "POST",
                    body: offer.sdp,
                    headers: {
                        "Authorization": `Bearer ${ephemeralKey}`,
                        "Content-Type": "application/sdp"
                    }
                });

                if (!sdpResponse.ok) {
                    throw new Error(`SDP error: ${await sdpResponse.text()}`);
                }

                // Apply the answer
                const sdpText = await sdpResponse.text();
                console.log("Got SDP response");

                await pc.setRemoteDescription({
                    type: "answer",
                    sdp: sdpText
                });

                console.log("Connection established! Check console for messages.");

                // Update button to allow stopping
                const button = document.getElementById('startButton');
                button.textContent = "Stop";
                button.onclick = () => {
                    stream.getTracks().forEach(track => track.stop());
                    pc.close();
                    button.textContent = "Start";
                    button.onclick = async () => { location.reload(); };
                    console.log("Connection closed");
                };

            } catch (error) {
                console.error("Error:", error);
                alert(error.message);
            }
        });
    </script>
</body>
</html>

This code gives 400 Bad Request.

Instead, if I replace ephemeralKey with API_KEY in the line where we make sdpResponse, then it works!

Is there a problem with how I am creating or using the ephemeral token? Iā€™d greatly appreciate help on this.

2

Think I figured out the problem. If you get the ephemeral token from https://api.openai.com/v1/realtime/sessions it works. Just have to be mindful of the different configuration for the body of the request, e.g., you now have to pass a model parameter.

1 Like