I don’t know if it’s just me but I’m getting too many repeated responses when I use the chat completion api with gpt3.5 and 4.0. I’ve tested ‘gpt-4-0613’ and ‘gpt-3.5-turbo-0613’.
I’m attaching the chats that I’ve had and as you can see when I ask something related to the previous response I almost always get a repeated answer.
I double checked that my messages array it’s adding correctly each message: For example this is one of them:
[{‘role’: ‘user’, ‘content’: ‘What’s hormesis?’}, {‘role’: ‘assistant’, ‘content’: ‘Hormesis is a biological phenomenon whereby a beneficial effect (improved health, stress tolerance, growth or longevity) results from exposure to low doses of an agent that is otherwise toxic or lethal when given at higher doses. It's essentially the concept that “what doesn't kill you makes you stronger.”’}, {‘role’: ‘user’, ‘content’: ‘Give me an example’}, {‘role’: ‘assistant’, ‘content’: ‘Hormesis is a biological phenomenon whereby a beneficial effect (improved health, stress tolerance, growth or longevity) results from exposure to low doses of an agent that is otherwise toxic or lethal when given at higher doses. It's essentially the concept that “what doesn't kill you makes you stronger.”’}, {‘role’: ‘user’, ‘content’: ‘Give me an example’}, {‘role’: ‘assistant’, ‘content’: ‘A common example of hormesis is the health benefits of moderate alcohol consumption. While high levels of alcohol are harmful and can lead to various health problems such as liver disease and cancer, moderate consumption (particularly of red wine) has been associated with a reduced risk of heart disease. This is an example of hormesis, where a substance that is harmful in large amounts can have beneficial effects in small amounts.’}, {‘role’: ‘user’, ‘content’: ‘Give me another’}, {‘role’: ‘assistant’, ‘content’: ‘A common example of hormesis is the health benefits of moderate alcohol consumption. While high levels of alcohol are harmful and can lead to various health problems such as liver disease and cancer, moderate consumption (particularly of red wine) has been associated with a reduced risk of heart disease. This is an example of hormesis, where a substance that is harmful in large amounts can have beneficial effects in small amounts.’}]
Hopefully somebody knows how to solve this, it happens too frequently.
Thank you. I tried different temperatures, the examples I’m showing used a temperature of 0.1, the only other option I’m using is max tokens of 1024, nothing else.
Not that I see. The user only asked for an example and he said again “Hormesis is a biological phenomenon whereby…”.
The user first asked “what is hormesis” and got the right answer. Then the user asked for an example and got the same answer and not an example. Then the user asked again for an example and finally it got an example, then the user asked for another example and it got the same example than before.
After this the user asked why is repeating itself and finally got a second example, this is not included in the logs posted but here it is:
I don’t think so. It showed a repeated answer before the repeated question and its the only chat with a repeated question. The question was repeated because it didn’t answer the question. And the other examples have the same dynamic, the first answer is fine and the next one is wrong.
{‘role’: ‘assistant’, ‘content’: ‘Hormesis is a biological phenomenon whereby a beneficial effect (improved health, stress tolerance, growth or longevity) results from exposure to low doses of an agent that is otherwise toxic or lethal when given at higher doses. It’s essentially the concept that “what doesn’t kill you makes you stronger.”’},
{‘role’: ‘user’, ‘content’: ‘Give me an example’},
{‘role’: ‘assistant’, ‘content’: ‘Hormesis is a biological phenomenon whereby a beneficial effect (improved health, stress tolerance, growth or longevity) results from exposure to low doses of an agent that is otherwise toxic or lethal when given at higher doses. It’s essentially the concept that “what doesn’t kill you makes you stronger.”’},
Again, in your example, you’re repeating an assistant message which is likely leading to more repititions when you call it again. I’d recommend looking at what messages you send back to the LLM.
I showed many examples, this is the only one with a repeated question. And the question was repeated because it didn’t answer the question, instead received a repeated answer. After that repeated answer the question was repeated.
And it happens in almost every chat, here it is another with the exact same problem withou repeating any question.
[{‘role’: ‘user’, ‘content’: ‘What’s the smallest mammal there is?’}, {‘role’: ‘assistant’, ‘content’: “The smallest mammal is the bumblebee bat (Craseonycteris thonglongyai), also known as Kitti’s hog-nosed bat. It weighs just 2 grams and is 29 to 33mm in length.”}, {‘role’: ‘user’, ‘content’: ‘And the biggest one is?’}, {‘role’: ‘assistant’, ‘content’: “The smallest mammal is the bumblebee bat (Craseonycteris thonglongyai), also known as Kitti’s hog-nosed bat. It weighs just 2 grams and is 29 to 33mm in length.”}, {‘role’: ‘user’, ‘content’: ‘But I asked you for the biggest’}, {‘role’: ‘assistant’, ‘content’: ‘The biggest mammal is the blue whale (Balaenoptera musculus). It can grow up to 100 feet long and weigh as much as 200 tons.’}]
I’m not sending any prompt. This is how my request looks, and messages is the array I’ve been posting:
url = 'https://api.openai.com/v1/chat/completions'
headers = {
'Content-Type': 'application/json',
'Authorization': f'Bearer {api_key}', # Use an environment variable or AWS Secrets Manager
}
data = {
"model": version,
"messages": messages,
"max_tokens": 1024,
"temperature": temperature
}
data = json.dumps(data).encode('utf-8') # Convert dictionary to bytes
req = urllib.request.Request(url, data=data, headers=headers, method='POST')
try:
print("Making request to OpenAI...")
with urllib.request.urlopen(req) as response:
response_data = response.read()
parsed_response = json.loads(response_data)
print(f"Received response: {parsed_response}")
Another example:
[{‘role’: ‘user’, ‘content’: ‘What’s the oldest country in the world?’}, {‘role’: ‘assistant’, ‘content’: ‘The oldest country in the world is a matter of debate and interpretation. However, one of the contenders for the oldest continuously existing country is considered to be San Marino. San Marino, officially known as the Republic of San Marino, is a microstate located within Italy. It claims to be the oldest surviving sovereign state and constitutional republic, with its founding dating back to September 3, 301 AD.’}, {‘role’: ‘user’, ‘content’: ‘And who founded San Marino?’}, {‘role’: ‘assistant’, ‘content’: ‘Another contender for the oldest country is Egypt. Ancient Egypt, with its rich history and civilization, is often considered one of the oldest continuous civilizations in the world. The country of Egypt as we know it today has roots dating back to the unification of Upper and Lower Egypt around 3100 BC, making it one of the oldest countries in terms of cultural and historical continuity.’}]
Try it again with a simple system prompt. “You are an assistant that truthfully answers all questions without repeating yourself.” …or something similar…
You mean adding a system message right? I’ll try that but if this solves it, the docs would need an update as the say:
However note that the system message is optional and the model’s behavior without a system message is likely to be similar to using a generic message such as “You are a helpful assistant.”
And I don’t think you actually need to ask it to tell truths and not repeat itself, but I’ll try.
Thank you for your help. That indicated me that it’s a problem of the front end and I found the issue, the state variable that had the messages array was not updating fast enough to be included in the API request. Now it’s finally working perfectly =)
