I am trying to understand an observed behaviour of gpt-3.5-turbo (gpt)
I have a prompt, where I ask gpt to formulate an announcement based on some user input.
My prompt text is in English and I have observed that when users input a non-English description, the response from gpt is equally as good as when the user enters a description in English.
This is positive, but I am surprised about this.
Does gpt translate any non-English text to English before it actually response to the full prompt or is there something else at work here?
It seems that LLM’s, when sufficiently trained on all of human language, begin to understand the rules that human brains have that underpin language, so specific languages tend to have the same things encoded into them, the only difference is the specific sequence of characters used for that encoding and some minor rule adjustments in terms of placement and ordering.
This makes the model a polyglot and gives it an ability to seamlessly transcribe and transpose from language to language.
3 Likes
It actually brings up an important potential for Language Models that I have spoken to others members on here about, namely @elmstedt and a few of the other regulars, it seems that a large portion of the models parameters are given over to this multilingual understanding, the proposition is to train a model purely on one language, in this case English, but it could be any large dataset language. Now if only one language is used and an additional translation model is placed atop it for such transposition of languages, it could be argued that the underlying “logic” model, having only a single set of language rules to follow, would be able to gain greater depth of understanding and therefor be more performant in inferencing tasks… if that would work in practice is not know.
1 Like
Thank you for explaining it and putting it into perspective.
It’s very positive for us. Makes it so easy for users.