Realtime API Beta -> Realtime API GA - Receiving type error with session.audio.input.format

API
, ,
1

Hello all! Has anyone ran into the following error while migrating their realtime API code to the latest updates for the session.update client event? Currently connecting over WebSocket:

Invalid type for ‘session.audio.input.format’: expected an object, but got a string instead.

The API docs show format as a string type but I’m seeing the error above mentioning an object is expected instead:

image
image612×185 5.9 KB

I also checked the playground to see if it was using the latest API updates, but the logs show its still using the previous session.input_audio_format and session.output_audio_format properties. Has anyone noticed this or know if I may be missing something?

2

Have you tried something like this?

{
  audio: {
    input: {
      format: { type: "audio/pcmu" },
    },
    output: {
      format: { type: "audio/pcmu" },
    },
  },
}
1 Like
3

I happened to try something like that yesterday and ended up with the below (after receiving an error about missing rate property as well) for input.format and output.format:

format: {
 type: 'audio/pcm',
 rate: 24000,
},

Oddly this doesn’t match up with what their API Reference docs show, just assuming it hasn’t been updated yet. Now running into an issue where response.audio.delta events aren’t coming through to the client, though haven’t determined the cause just yet. This only occurs when removing the "OpenAI-Beta": "realtime=v1" header, so current assumption is there’s just something I’m missing :grinning_face_with_smiling_eyes:

4

Took me a while to realise that the response.audio.delta event has been changed to response.output_audio.delta: https://platform.openai.com/docs/api-reference/realtime_server_events/response/output_audio/delta

2 Likes
5

Ah, thank you for pointing that out, that was the last missing piece! :grinning_face_with_smiling_eyes: Really hoping for a migration guide just to ensure I’m not missing anything, but so far so good! :tada: