[Realtime API] Audio is randomly cutting off at the end

API Bugs
13

EDIT: IN FACT THIS DIDNT SOLVE THE PROBLEM. AUDIO LAST 0.5-1 SECONS IS MISSING MOST OF THE TIMES NOW. PLEASE FIX THIS OPENAI.

In fact I think I found the problem. I really don’t have time to dig in code lol. It’s funny some still think we are heading in 2-3 years to a future of total abundance just like magic. Without a massive human effort…

In the rtclient the RTAudioContent class:

async def audio_chunks(self) -> AsyncGenerator[bytes]:
    while True:
        message = await self.__content_queue.receive(
            lambda m: m.type in ["response.audio.delta", "response.audio.done"]
        )
        if message is None:
            break
        if message.type == "response.content_part.done":  # <-- This might be coming too early
            self._part = message.part
            break
        if message.type == "error":
            raise RealtimeException(message.error)
        if message.type == "response.audio.delta":
            yield base64.b64decode(message.delta)
        elif message.type == "response.audio.done":
            # We are skipping this as it's information is already provided by 'response.content_part.done'
            continue

in your code, when you receive the item message you need to wait for both the content_part.done AND audio.done messages. I guess they arrive asynch so if you get the audio.done first maybe drop the last chunks or so.

I would assume for others using jscript the issue is similar.

have a good week.