Now Azure has also encountered this issue, constantly generating “\ \ n \ \ n \ \ n”. Microsoft engineers told me that their model is about 10 days later than OpenAI’s, so when OpenAI discovers this issue, Azure’s will also appear 10 days later
I am still having this issue which makes gpt-3.5-turbo-1106 and gpt-4-1106-preview unusable for my Use case as it is in German. Would be great to see a fix coming soon!
4 Likes
For anyone struggling with this, here’s a workaround based on the fact that this bug only affects function calling and not json_object response format:
- Make a request only to determine which function(s) to call
- For each function that you need to call, make another request, but this time do not use function calling and instead request
'response_format': {"type": "json_object"},. If you supply the same arguments object as in function calling in your prompt, the response json will adhere.
This sounds promising, could you share a code example? Many thanks in advance
4 Likes
Still having the issue with non-english characters. Current solution is doing an extra API call when we detect non-english characters in the prompt but it’s definitely not optimal.
4 Likes
Having the same issue with special characters. Is there any update from development. This is a CRITICAL bug for any language apart from English apparently. Also I have seen the same issue for specific mathematical characters.
2 Likes
Fully agree! It’s a real blocker for us to use this model at the moment… Also, as we’re using it via langchain, none of the workarounds above are really applicable.
thank you!!
we will use this until the model is fixed.
OpenAI update models last weekend? I suddenly find the issue disappears with the same input I used last week. (Trying to make a workaround today)
edit: looks like that they only fix 3.5-turbo-1106, 4-1106-preview is as bad as before
I’m waiting for a fix for this too. There is any update on a model update?
The update last Thursday was that it is not forgotten about.
Working on new AI models is kind of what OpenAI does, after all.
A potential workaround I propose that OpenAI could expose for experimentation without app-breaking would be a new chat completions API parameter:
tools_json_mode: boolean (default: true)
falsedisables the enforced response_format of json for tool and function AI outputs. Disabling the token-run detection of json compliance can reduce some unintended effects presently seen, where international characters are replaced by looping linefeed or tab tokens.
1 Like
This is a very big problem. When will we see an update?
I have the same problem in Korean.
2 Likes
Just wanted to give my solution which works well (in my case with few non-latin characters):
createTool(
CHANGE_ADDRESS_ORDER, "Change address of order, use AA for å, OE for ø and AE for æ.
So basically just ask it to map the special character to a capitalised sequence of letters that would not appear in the real text, and then doing a simple replace.
Hi @enoch.
We are now end of January: do you plan to share an update on the model, a fix, or a new ETA?
This issue is absolutely critical, and some of us (including my company) are spending thousands of dollars monthly for a service that is not correctly fulfilled.
We would at least hope for a commercial gesture, and I think the whole community here wants to be updated on what is going on here.
Thanks in advance
9 Likes
ist there any update, any fix??? Is there sombebody who do anything, hello are you alive or is there only AI alone at home, after CHRIST-mas?
In my tests, the GPT 4 Turbo version gpt-4-0125-preview resolves this problem.
OpenAI is also announcing the arrival of gpt-3.5-turbo-0125 next week, which should also resolve this issue.
Very, very good news!
6 Likes
It does seem so yes!
Today, we are releasing an updated GPT-4 Turbo preview model, gpt-4-0125-preview. This model completes tasks like code generation more thoroughly than the previous preview model and is intended to reduce cases of “laziness” where the model doesn’t complete a task. The new model also includes the fix for the bug impacting non-English UTF-8 generations.
Will try today!
2 Likes
It works!! (Still up for the commercial gesture though @enoch lol)
1 Like
Hi, thank you for the ping 
The newly released gpt-4-0125-preview, as well as the upcoming gpt-3.5-turbo-0125, fixes these issues! Thank you for your patience while we addressed this.
We will keep this thread open until the release of gpt-3.5-turbo-0125.
Enoch
4 Likes
Would be great to read a post about what happened and how it was solved 