Hello,
I am using model gpt-4o with the associated python API.
I am trying to translate texts. But GPT sometimes answer with non sense translations whith very explicit/simple text ??
Am I giving to much instructions ?
Is there any way of improving it ? making it clearer for gpt ?
Is it good to split developer instructions ?
I would be happy to hear your opinion on my prompt.
instructions = [
{
"role": "developer",
"content": "You are a professional translator specializing in smart cards and smart chips technology, and you are aware of cryptographic terms."
},
{
"role": "developer",
"content": "You will be provided a json as input, which is an array of objects that contains texts to be translated."
"Each object is composed of following keys:"
" - 'id': don't translate it"
" - 'text': actual text to be translated. Translate EVERY 'text' value, and ONLY these one."
"In addition, each object can have following:"
" - 'instruction': contains customized translation instruction"
" - 'glossary': it is a typical glossary"
" - 'context': it is an object composed of key/value. The key should be translated as its use in the sentence given in value."
"You must use these additional parameters ONLY for the translation of the associated object."
},
{
"role": "developer",
"content": f"Use following global glossary rules for the translations of all the objects inside the json array in the input json (do not use cashed glossary rules): {global_glossary}."
},
{
"role": "developer",
"content": "Only answer in a json format, that contains an array of objects like: { 'id': 'given id in the input json', 'text': 'translated text' }"
},
{
"role": "developer",
"content": f"Translate texts into '{language_name}({lang_code})'"
}
]
1 Like
I think overall it seems good but it could all fit into a single prompt.
Providing an example of a correct answer may also improve the response.
You can use response_format to enforce you want a json response or structured output to make it even more specific.
Sorry I omitted that I was already using a response_format.
My text is quite long (50k+), and I want to translate it in multiple languages (over 30). GPT is providing multiple bad answers, and these change each time I re-ask for a translation. It’s hard for me to guess which text it will fail to translate. In addition, I only speak french.
I have seen some benchmarks on translation tools (GPT, Google translation, Microsoft translation), and GPT was doing quite good. I think I might be doing something wrong
Sometimes LLMs do hallucinate with no particular explanation, including issues from moments of server instability.
You could also add a complimentary prompt to check if the translation output makes sense, asking to compare it with the original text and run it again automatically if necessary.
What about the temperature ? May setting it to 0.1 (or lower) limit the issues I am seeing ?
1 Like
I run a translation business.
My solution was to get the texts translated by GPT + Claude + DeepL.
I then give to GPT the text to translate + the DeepL translation + Claude translation and ask him to do the best translation.
I do the same with the GPT translation that I send to with the DeepL to Claude.
And finally, I get a diff compared document using convertAPI .
And since the text we translate are of high value, I still have humans in the loop, choosing the best from both corrected versions originating from 3 translations.
Here is a graph illustrating it.
But yes, for sure, unless it is a very creative text you want to translate, a poem, marketing slogan, (which it doesn’t seem to be according to the info you shared), I would use a temperature of 0,2. There might still be some inconstancies.
Always worth to read it once, or at least, send it in another prompt asking if there is some inconsistencies, possible translation mistakes, etc… receive a warning only if there is.
Below is an example of a refined prompt incorporating some improvements:
const instructions = [
{
"role": "developer",
"content": "You are a professional translator specializing in smart cards and smart chips technology, and you are familiar with cryptographic terms."
},
{
"role": "developer",
"content": "Input will be provided as a JSON array of objects. Each object includes: \n1. 'id': a unique identifier (do not translate), \n2. 'text': the text to be translated. \nOptionally, an object may also include:\n - 'instruction': custom translation guidelines,\n - 'glossary': a specific glossary for that text,\n - 'context': an object with key/value pairs where the key should be translated as it is used in the sentence given by the value.\nUse these additional parameters ONLY for that object."
},
{
"role": "developer",
"content": "Apply the following global glossary rules for all objects in the input JSON (do not use cached glossary rules): {global_glossary}."
},
{
"role": "developer",
"content": "Respond ONLY in JSON format. The output should be a JSON array of objects, each in the format: { 'id': 'original id', 'text': 'translated text' }."
},
{
"role": "developer",
"content": "Translate the texts into '{language_name} ({lang_code})'."
},
{
"role": "developer",
"content": "API Recommendation: For consistent translations, consider setting the temperature to a low value (e.g., 0.1-0.2) to reduce variations and potential hallucinations."
}
];
I guess this version could make your instructions more organized and clear, ensuring that the model focuses solely on translating the required field, adheres to the JSON format, and benefits from a lower temperature setting to enhance translation consistency. If that makes sense.
