I am using the GPT-4 API, using this model: gpt-4-0125-preview. Sadly, the response of this model is being generated in another language, in this case, Spanish.
First issue: is it detecting my IP. Privacy issues.
Second issue: Why?
Even when the system prompt is:
Analyze the provided content using topic modeling techniques. Your objective is
to extract the main themes or topics from the content, giving a clear
understanding of the subject matters discussed within.
Additionally, identify prominent bigrams (two-word phrases) that can offer
insight into recurring patterns or significant points of discussion.
Structure your response as a basic report, highlighting:
1. The top themes or topics you've identified.
2. Noteworthy bigrams and their relevance.
3. Any observations or narrative patterns that can help understand the content.
4. Always respond in the language in which the user made the request.
Emphasis on point 4: ‘Always respond in the language in which the user made the request.’
The user prompt and context are in English. Why is the response being generated in Spanish? Even GPT-3.5 performed better.
Has anyone else experienced unexpected behavior in language-related responses?
I can confirm that.
However, it seems that the language of the response is influenced more by the bigrams (phrases of two words) extracted here rather than by determining the language from the IP address.
For example, when the names of companies or organizations specific to Spain are included, it tends to be influenced by the main language of the location of those companies or organizations.
This is particularly noticeable when proper nouns from countries where Romance languages, such as Italian, French, Spanish, and Portuguese, are spoken, appear at the beginning.
The AI is powered solely by the input language. Unlike Microsoft Copilot, which creeps on you by putting your location and user name into AI context, the API AI only knows what you provide it.
If you make the AI “adaptive” by system instruction that makes it no longer a chat bot, what is it supposed to adapt to? It may see the only thing that would change the responding language (if it is a preprogrammed backend entity) would be the varying data and not instruction.
Is the user message actually provided by an unreliable user, or also part of the script?
If this AI is completely an automated task (and yet the language of instructions would change??), the data content to be analyzed should be framed by an instruction wrapper like “here’s the content to be analyzed:” and then the data wrapped in a container like triple-quotes or multiple brackets.
Encoding is unlikely to be the problem.
Even when dealing with text that consists solely of ASCII codes, attempting to determine the language from a single system message can be error-prone.
So it’s better to handle language detection in a separate layer, assign the detected language to a string variable, and then specify the language to be used in subsequent processes as a placeholder.
I believe GPT-3.5-turbo is sufficient for language detection.
When specifying the language in downstream processes through system messages, I also recommend specifying it at the beginning or end of the message.