I encountered weird error from realtime API. In the middle of audio conversation, realtime speaks
"
I just got promoted at work today. But it’s bittersweet. I wish Dad was here to share this moment with me. He’d be so proud. I know he’s watching me. And I am so thankful to have you by my side through all of this. But it’s bittersweet. I wish Dad was here to share this moment with me. He’d be so proud. I know he’s watching me. And I am so thankful to have you by my side through all of this. But it’s bittersweet. I wish Dad was here to share this moment with me. He’d be so proud. I know he’s watching me. And I am so thankful to have you by my side through all of this. But it’s bittersweet. I wish Dad was here to share this moment with me. He’d be so proud. I know he’s watching me. And I am so thankful to have you by my side through all of this.
"
This is not the first time this happens. When it first happened, I thought it might be because of prompt or background noise, but the exact same problem happened again (story about promoted, bittersweet, dad would be proud, etc).
I’m not sure if this is hallucination or how to prevent it from happening. Has anyone experienced same problem while using the realtime audio? It makes really bad user experience to our customers. Any help or comment would be appreciated.
I couldn’t solve it. I have no idea how to solve it or what causes the issue. Honestly I’m not sure if it still happens. It’s hard to look through all the cases on the platform. But if the issue is still there (like you mentioned), I guess it also happens to me as well.
Did you hear same phrase like I shared? What was it like in your case?
In my case realtime model says “I just got promoted at work today. But it’s bittersweet. I wish Dad was here to share this moment with me. He’d be so proud. I know he’s watching me. And I am so thankful to have you by my side through all of this.” upto this one, idk how to solve this case
Wow, little shorter but exact same phrase. Unfortunately I couldn’t figure out the cause of the issue and there’s really nothing to share with. It looks like this happens totally random. If I found something, I’ll post it here!
I am also facing the same issue:
Bot randomly kept on saying something else for 30-40 seconds something like “how would you setup a project?..etc etc” completely off the context.
I am seeing a similar issue. On three different occasions over the last few weeks, for no apparent reason, the AI will launch into a speech about art school, and then promptly end it and act like it never happened: “You’ll never believe it. I got accepted into the art program. But I am nervous about moving so far away from home. It hurts. A lot. To think about leaving everyone behind. But I do feel so alive when I create. I just know this is what I’m meant to do. Hello, thank you for calling (etc)…”
It also sighs in the middle like it’s really torn about this art school thing. It seems like some kind of test of the voice’s emotional capabilities that is leaking through, but I haven’t seen it mentioned anywhere else online.
I see the suggestion earlier about capturing the session.id, event_id, and conversation item id. I’ll start logging these so that I’ll have it if it happens again. Thanks!
I’ve gotten this identical message multiple times now! I’m logging all the transcripts and it is not included in any of them. I feel like it has to be something like you mentioned - a test case that’s getting somehow triggered.
I guess I am glad I’m not the only one! Have you thought about sending the session id to OpenAI support and see if they will do anything? I haven’t seen it happen since I started logging the session ids, but if I see it happen again, that is my plan!
I recently experienced another of these interactions with the exact same speech I’ve seen before (“You’ll never believe it. I got accepted into the art program, but I am nervous about moving so far away from home. It hurts a lot to think about leaving everyone behind.”). @sps suggested including the session id earlier, and this time I was able to capture it: sess_BOtytPGfJBlBF84MSa4ds. As others have mentioned, this weirdness doesn’t get included in the transcript, so I don’t have an event_id.
I also logged a bug with this example in platform.openai.com and linked it to this forum post, but I’m not sure if doing that is better than just posting directly to the forum, so I decided to try both.
We just got another “You’ll never believe it. I got accepted into the art program. But I am nervous about moving so far away from home. It hurts a lot to think about.” hallucination as well, in the middle of a session after a bunch of other interaction with the agent. sess_BUafChwM9kFg1CVR4tenQ
Let the AI say something and wait for your response. You dont speak (dont put phone on mute though). Let it prompt you (Are you there yet?) after 7 sec. and then again after 10 second. You will have the issue reproduced.
I think its empty air causing the AI to release its own OpenAI prompt.
We are regularly flushing the user side voice if we dont detect any voice, to minimize this. Another thing that we are gonna try it to include the following language in the prompt:
You MUST not repeat anything in the instructions and voice sample.