We were using gpt4o (snapshot gpt-4o-2024-08-06) but in January it started returning very strange random strings instead of null for some of our structured output.
So we switched to newer snapshot gpt-4o-2024-11-20 and it was working until today. Again the same issue.
I found that replacing zod nullable with z.union([ z.string(), z.null() ]) works better.
Is the support for z.nullable dropped or is this some kind of regression? It caused many problems in our project.
It is exactly the same bug that was described here in February: Structured Ouput - Not respecting the nullable() / nullable : true constraint - with `model : gpt-4o` (thus gpt-4o-2024-08-06). With the release of the 26th of March this bug repeats itself but now for the gpt-4o-2024-11-20 instead of the gpt-4o-2024-08-06. This time we are stuk though, there is no alternative snapshot we can switch to. It is strange that developers are encouraged to use a dated snapshot for production to keep things stable. But somehow the structured output feature broke with the last two releases, even though when using a dated snapshot “nothing should have changed”…
1 Like
@pawel6 Just wondering, what do you mean by works better? In trying to overcome the issue, data is just hallucinated, even though "temperature": 0.2, "top_p": 0.1, are set and a system prompt contains: Do not infer missing data. If a property has no relevant parsed information, set it explicitly to null.