Hi OpenAI Developer Community,
I’m working on a real-time chatbot app using gpt-4o-realtime , and I’ve run into a strange issue. I tell the model in my prompt to always respond in the user’s language, but it sometimes “sticks” to a previous language and ignores the one the user is currently speaking.
Here’s what I mean:
User: Hello my chatbot, how are you doing today?
Chatbot: I’m fine, what about you?
User: Ich bin sehr gut, danke! Kannst du Deutsch sprechen?
Chatbot: Ja, ich kann.
User: What is the capital of France?
Chatbot: Die Hauptstadt von Frankreich ist Paris.
In that last turn, the question is in English, but the model keeps answering in German because it picked up the language from earlier in the conversation.
I have prompted my model like this but still did not work for me. Also I have tried to change temperature but still same issue… Thanks!
Please respond in the same language as the user's current message.
Always detect the language based only on the current user message.
Do not rely on any previous message to guess the language.
Even if the user spoke a different language before, only use the language of their most recent message to generate your response.
2 Likes
Can you try this:
Always reply in the language used in the user’s message. If the user switches to a different language, respond in that new language immediately.
EDIT:
I tried on Playground:
User: Hello my chatbot, how are you doing today?
Chatbot: Hey there! I’m doing great, thanks for asking. How about you? How’s your day going?
User: Ich bin sehr gut, danke! Kannst du Deutsch sprechen?
Chatbot: Ja, natürlich! Ich kann Deutsch sprechen. Wie kann ich dir denn weiterhelfen?
User: Karibu. Nimetoka Kenya.
Chatbot: Asante sana! Inafurahisha kukutana na mtu kutoka Kenya. Ungependa tuzungumze kuhusu nini leo?
User: Gözünün yağını yediğim, nasılsın?
Chatbot:Çok teşekkür ederim, ben gayet iyiyim. Sen nasılsın, her şey yolunda mı?
1 Like
Thank you for your answer, @polepole It seems like it is working better then my previous version but I see the issue and it still happens. Is there anything that you can suggest to me about parameters or any logic that I can try to solve the issue ? I am also able to reproduce this issue on playground.
For example,
user: how are you
assistant: I’m doing great, thanks for asking! How about you?
user: bana bir şaka yapar mısın
assistant: Tabii! Adamın biri dişçiye gitmiş. Dişçi sormuş: “Sizi daha önce görmüş müydüm?” Adam cevap vermiş: “Sanmıyorum, dişlerimle ilk kez…”
user: it is not funny
assistant: Haklısın, belki espri yeteneğim biraz daha gelişmeli! Bir başka şaka deneyeyim mi?
I also seem to be experiencing this. What I have noticed is it tends to struggle specifically for me with swapping back to English. For my use case We need it to be able to switch between Spanish and English mid conversation. If the user says something like “Hey can you please respond in English” it will. But it almost never auto swaps back to English from the user just speaking English despite it being in the system command. Strangely enough if it is speaking English and the user starts speaking Spanish it will 100% of the time swap to Spanish responses.
I do at the very start have a response.create which creates a greeting in Spanish for the user and I assume this is why it believes the user was speaking Spanish and tends to lean towards it. (I tried setting “conversation” to “none” to make it an “out of band” response but it didn’t change anything.
Regardless of how detailed the system command is in detailing to always use the language the user is currently responding with and that it may change mid conversation it doesn’t seem to have any effect.
I was so excited when i saw they released gpt-realtime and it specifically says that it is able to mid conversation switch languages seamlessly but It still seems to have the same problem and also the old problem of cutting off conversation responses early.
It’s so close to being perfect but end user’s having to ask it to speak their language vs it auto switching to their language seems like something it should 100% be able to do. Strangely also the input_audio_transcription is always None/null Not sure if this plays a part in it.
If anyone has any ideas on a solution to this I would love to chat about it.
1 Like