Hi !
I’m currently working on Hoomano’s digital assistant conversational part based on GPT-3.
My prompt is composed of a context that is always in english but the conversation language I want can be different and specified by a language code. I use Davinci-instruct to perform this task and most of the time, it answers in the right language ! But sometimes it does not and this is really annoying…
Here is my context:
« This is an AI that aims to serve humans. The AI is clever, understanding and very smart. The AI speaks {locale}. The whole conversation is in {locale}. The AI and the user already know each other. The AI wants to discover user’s interests.»
Where {locale} is replaced by a language code (‘fr’, ‘ja’,…).
I also noticed that when adding some english examples following this context, the language specified is no more taken into account and my bot always answers in english.
Do you have any tips about this issue ? Should I redesign my prompt ? Would fine-tuning help solving ?
2 Likes
This conversation is a very good illustration of my problem! Even trying to change the language after the conversation begins within the conversation seems hard !
1 Like
I get your point, it’s a pleasant way to get around the problem ! The condition is to catch early enough that AI is not using the expected language, before the user is lost. Some confusion detected on the user side could raise awareness about something going wrong in the discussion. Yet this fallback should be kept to the minimum: we don’t want our users to be confused 
That’s why I would like to fine the best tuning / prompt to avoid this language problem that is too often happening for now, in my opinion 
1 Like
Hey Kelly!
Keep in mind that GPT-3 was trained primarily on English, so the fact that it can even do other languages is mostly an accident. It’s true that models trained on multiple languages tend to do better - this is for similar reasons that polyglot humans do better with communication - it forces the brain to build better models!
I would break this problem down into multiple steps. Since GPT-3 is strongest in English, I would do most of the work in English, and then simply translate to French or other languages with a second model/prompt.
In order to handle multiple languages with a single prompt, you are most certainly going to benefit from fine-tuning, where you can give it examples of respecting language codes. But still, the task of “thinking” and “chatting” is a very different neural task from translating. This is why I recommend splitting it up into multiple phases.
2 Likes
Hi David !
I get your point. But I am afraid that each separated step brings uncertainty in the process. Moreover, when GPT-3 follows the instruction and answers french, it is good at it !
I’ll see what fine-tuning brings and I will also try to have one prompt by language in the desired language: it might be a worthwhile effort.
2 Likes
I am about to incorporate my Core Objective Functions into a dedicated chatbot (something that should be much simpler and robust than Raven/NLCA) but can also demonstrate the power of fine-tuning. I will send you all the results when it is done. I think it will show how fine-tuning can streamline chatbots to give powerful and consistent output
1 Like
Hi Kelly,
Assuming you re building an UI around your prompt, you may consider
translating your text
« This is an AI that aims to serve humans. … discover user’s interests.» statically to all locales first.
Then call the API with the translated text and let the conversation flow.
In your UI, you may just show the user the English text at beginning, but the rest of the conversation in the local language.
That way you d provide more text in your desired language already to the engine.
Hi, Thank you for sharing those ideas !
It’s exactly what I did at end : writing the context and examples in all language available in the final application to provide it to GPT-3 directly in the desired language. It’s working pretty good !
If you want to have a look at the whole UI and global experience, we have juste released our first app one the Apple Store. It is called MeetMojo: https://apps.apple.com/app/meetmojo/id1590320992 Mojo is a news presenter that discuss about news with the user thanks to GTP-3. He also analyses user’s facial expressions in real time to interact with him using our perception tool and including this additional information in GPT-3 prompt. Hope you’ll enjoy it 
1 Like