We use the API for our AI for language learning. Since gpt4o was introduced, we have noticed frequent typos where the end of words are cut off. We’ve primarily noticed this in Spanish and French but it may be happening in other languages too. Examples of errors:
‘neces’ instead of ‘necesarias’
‘prev’ instead of ‘previas’
‘euthanas’ instead of ‘euthanasia’
As you can see, there is a pattern here. It happens irrespective of the GPT4 model used. This urgently needs to be fixed because these errors are likely confusing thousands of language learners across the world.
1 Like
Hello,
I have regularly been using GPT-4 and GPT-4o in the past few months in both English and French and have not had this issue at all.
My first guess is that this may be an issue with the temperature setting? I noticed that setting it to 1.0 or above can make it do weird stuff with words sometimes, which looks similar to what you’re describing. I personally set mine between 0 and 0.7.
Or perhaps an issue with tokens not being streamed to your app correctly? Does this also happen if you disable streaming the chat/assistant response?
In both cases, the output text seems to be missing/skipping the word’s last token, as the words you provided happen to be tokenized (truncated into tokens) in a way that cuts them off exactly like you described:
2 Likes
It’s not just Spanish and French. @turbolucius I think you are correct.
GPT4 makes mistakes like changing FullPageLoader into FullPageJumper. It seems like random spelling mistakes, but actually they are token mistakes.
Whereas the default in the chat app GPT4o does not make these mistakes.
We need access to the temperature setting in the app.
1 Like
Thank you very much, appreciated! It was indeed the tokens getting cut off due to an issue with our ruby gem.
2 Likes