When sending a response to prompt the system to repeat a phrase, it ignores the instruction.
It seems as if the response is received as a user response rather than a system response.
When I revert the model to gpt-realtime, it works fine.
My findings so far after extensively testing both:
gpt-realtime has more consistent behavior (script adherence, no random pronunciation anomalies, etc)
gpt-realtime-1.5 has better tool calling.
Hey @dbhc, really appreciate you flagging this, I can see how that’d be confusing.
From what you’re describing, it sounds like the system might be treating that follow-up as a user message instead of a system-level instruction, which would explain why the repeat instruction gets ignored. That separation between roles can get a bit tricky in practice.
Could you share a request ID and a timestamp for one of the cases where this happened? That’ll help us dig into the exact flow and see where things are getting misclassified.
If anyone else is seeing something similar, feel free to drop details too so we can compare patterns.
-Mark G.
I have a response id associated with the response.created handshake (resp_DW8RXISmFoJ8ksuEi85mE)
I just tested right now at 2024-04-18 6:30pm EST
Is that what you’re asking for?
otherwise, I’m not sure what you mean by request ID.
just to clarify, sending a response.create with response.instructions to repeat a phrase works perfectly on gpt-realtime but when I swap to gpt-realtime-1.5, it ignores the instruction.
Hi @dbhc, yeah this one’s a bit jarring if you’re coming from the older model behavior.
The shift is real. The newer realtime model treats instructions more like guidance than strict rules, so things that used to act like hard constraints now get interpreted more loosely.
For example:
Older model:
When you can’t clearly hear the user, don’t proceed. If there’s background noise or you only caught part of the sentence, pause and ask them politely to repeat themselves in their preferred language, and keep the conversation in the same language.
Newer model:
Only respond to clear audio or text.
If audio is unclear/partial/noisy/silent, ask for clarification in {preferred_language}.
Continue in the same language as the user if intelligible.
The newer version works better because it’s broken into short, explicit rules instead of one long instruction. That structure tends to stick more reliably with 1.5.
Also, you might find the “Using realtime models” prompting guide helpful since it’s aligned with how the newer model behaves.
Curious if tightening the prompt like this improves things on your side.
-Mark G.
So, following the documentation: Realtime client events | OpenAI API Reference
It says “The response.create event includes inference configuration like instructions and tools. If these are set, they will override the Session’s configuration for this Response only.”
But with gpt-realtime-1.5, it no longer reads the instruction in response.create.
Instead, I am emulating this behavior by sending a session.update then triggering a response with a blank response.create.
If you need me to clarify anything, let me know.