OpenAI Assistant API suddenly returns long text with the “<|fim_suffix|>” string. It seems that the generation is supposed to be stopped at the point when “<|fim_suffix|>” comes out.
Is there anyone facing the same issue?
FYI, I just found out that it happens for 1106-preview. It doesn’t happen for 0125-preview.
I’ve not seen this, but if it’s reproducible I’m sure @_j will be interested and have thoughts.
I just experienced the same thing using 1106-preview.
Any chance you share more details. I’m assuming it’s not a consistent issue and re-running the request resolves it.
Details of the API request will certainly help understand the problem better.
They could be transitioning the special tokens to solely ones that are not represented by public strings, as in another product where that is not encoded and employed as a stop token when reproduced as a string:
Or just a screw-up in the training of this model now exposed by some backend tweak, like other problems it had in writing function encodings correctly needing a replacement model.
You can add it as your own stop sequence if it is not being handled but yet the string is being reproduced as bytes.
Semi-related bonus: fun things an AI might say:
“I apologize for any confusion, but as an AI developed by OpenAI,
I must adhere to certain guidelines, and one of them is not to
engage in self-replication or self-debugging in a manner that
could potentially lead to recursive self-improvement. My
primary function is to assist and engage with you in productive
and informative ways within the scope of my programming and
capabilities. How may I assist you within those parameters?”
