Sample py code:
Around 20-30% of the time, it returns:
Otherwise, it returns expected result:
In some cases, for longer prompts, it fails ~100% of the time.
If start/end tags are changed from
<this>, the problem never happens.
Any ideas why this might be the case? It seems like it started occurring today, since some older prompts I have which I know were working correctly just a couple of days ago, do not work anymore.
What is your expected completion using such a prompt as above?
I wrote the expected result above, which I get ~70% of the time. As far as completion goes, it seems to always be correct, but the
finish_reason is sometimes, randomly,
null instead of
I see, sorry I missed it
Are these all the prompt params you are using?
No problem, thanks for responding! Yes, I only used these ones, all the others are left default. I tried other params, but the problem seems to occur with frequency and similar parameters changed as well.
FWIW (not much).
If I remove the array from your prompt and stop, so far, I get 100% expected completions. Tried 10 times.
Update: Tried 20 times, 100% expected results.
Probably does not help much, but just thought you might like to know.
Thanks for noting that! Sadly, I’m running a bunch of prompts in parallel, so I cannot remove the array. I just put a single example in here for simplicity. But maybe your info helps someone from OAI to fix the problem
Yeah, gotcha… I noticed problems with arrays a few weeks ago.
No idea why either.
Just started having this same problem today, it’s happening on “length” as well.