Realtime audio getting abruptly terminated after getting audio transcript since 17th Jan

I have created a small POC to integrate functions(tool) with Realtime API using webRTC using Realtime Console sample. The normal conversation is working fine. However, if a function response being passed to realtime api, it provides the proper transcript but audio gets truncated abruptly. After that, I need to restart the session to converse further. Till last week ie 16Jan or 17th Jan , it was working perfectly fine.

3 Likes

Is any one else experiencing the issue. I have implemented functions with external api call and for this case audio is getting terminated abruptly? Was working perfectly fine a week ago and there was no change in my code.

What is the exact timing of events?
i.e.
User talks
Server detects silence
Function/Tool call response received
API Called
Function/Tool result supplied to OpenAI
AI Speaks
User Speaks
etc, etc.

The event timing are as follow:
Session Starts
AI speak
User talk
AI speak
User ask question based on Function
Function/Tool call response received
API Called (external)
Function/Tool result supplied to OpenAI
AI start Speaking ( Here is where where audio abruptly stops)
AI transcript received

This stopping of audio started on 16-17 Jan. Before that, it was working flawlessly. Attached screen recording for reference.

Issue Screen Recording

Do you need any further information? I still see the same issue.

We’re experiencing the same issue. Sometimes the bot doesn’t play the response after a function call, even though it’s generated in the background, or it gets cut off and stays silent. This forces us to ask, “Did you get the result?”

After each function call, we send conversation.item.create and response.created events to create a new conversation item and trigger a response from the model. But even with this, the bot sometimes remains silent.

We are using:

turn_detection: {      type: "server_vad",      threshold: 0.8  }   

Without function calling, it works fine

@sanja.stojkova This is something you can fix by sending function response to message of type: “conversation.item.create”. I was facing the same during initial dev and fixed it by this.

@Foxalabs Any findings on the issue raised?