Realtime api has high latency occasionally

API
,
1

I noticed this starting yesterday Feb 24th 2025. Occasionally the Realtime Api server will take around 10 seconds to send the rate_limits.updated before continuing to send other server events until response.done.

My session config is quite simple

.updateSession({
      instructions: config.instructions,
      voice:  "shimmer",
      tool_choice: "none",
      max_response_output_tokens: "inf",
    });

Here’s a list of timestamped received events:

[Websocket/2025-02-25T19:00:24.841Z] received: session.updated 
[Websocket/2025-02-25T19:00:24.842Z] received: response.created 
[Websocket/2025-02-25T19:00:36.641Z] received: rate_limits.updated 
[Websocket/2025-02-25T19:00:36.647Z] received: response.output_item.added 
[Websocket/2025-02-25T19:00:36.647Z] received: conversation.item.created 
[Websocket/2025-02-25T19:00:36.652Z] received: response.content_part.added 
[Websocket/2025-02-25T19:00:36.654Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.727Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.727Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.728Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.728Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.777Z] received: response.audio.delta 
[Websocket/2025-02-25T19:00:36.797Z] received: response.audio.delta 
[Websocket/2025-02-25T19:00:36.798Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.904Z] received: response.audio.delta 
[Websocket/2025-02-25T19:00:36.907Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.907Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.926Z] received: response.audio.delta 
[Websocket/2025-02-25T19:00:36.928Z] received: response.audio.delta 
[Websocket/2025-02-25T19:00:36.930Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.930Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.930Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.930Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.930Z] received: response.audio_transcript.delta 
[Websocket/2025-02-25T19:00:36.972Z] received: response.audio.delta 
[Websocket/2025-02-25T19:00:37.045Z] received: response.audio.delta 
[Websocket/2025-02-25T19:00:37.110Z] received: response.audio.delta 
[Websocket/2025-02-25T19:00:37.116Z] received: response.audio.done 
[Websocket/2025-02-25T19:00:37.116Z] received: response.audio_transcript.done 
[Websocket/2025-02-25T19:00:37.116Z] received: response.content_part.done 
[Websocket/2025-02-25T19:00:37.117Z] received: response.output_item.done 
[Websocket/2025-02-25T19:00:37.117Z] received: response.done

Note the time diff between response.created event and the rate_limits.updated 12 seconds.

UPDATE: seems to be only with 4o-mini