Does anyone often get a wrong malformed JSON array? Quite often I’m getting something like this:
[foo": "bar"}]
Instead of
[{"foo": "bar"}]
In other words, it misses the opening curly bracket of the first object in the array
Hi and welcome to the Developer Forum!
In response to what? Your post seems be missing the bulk of it’s content, what API endpoint are you calling? what model? what does your calling code look like?
In order for reliable data structures to be produced, the temperature and top_p must be considerably reduced, to reduce the “tail” of other token generation probabilities.
If the AI is 85% certain that { will be produced, but still gives a chance 10% of foo, then 10% of your generated outputs will be corrupt.
You can see that in the first case, the AI had a good idea of the intention, by continuing with enclosure within quotes and brackets, but it can’t control the sampling parameters itself (unless you program it to!).
1 Like
Hi @mikeborozdin
Even that is not a valid json array.
In my case I have noticed that simply mentioning the term “valid JSON” instead of “JSON” improves model performance on this task.