File_search and stream connection dropping

You found “bad AI”.

The “ugly things” are the AI writing its internal citations container, and looping with more file chunk references (that is invalid) with the wrong Unicode instead of closing them with the correct UTF-8 character that is prompted. The same symptom has been observed before.

They are meaningless Unicode private-use code points, and OpenAI is attempting to use them with a semantically-based language model. The AI apparently has to get really desperate to have nothing else to write before getting out of a container with the instructed U+E201 code point, instead of repeating what it just wrote. Or, it never does write the correct thing, and all the nonstop output remains captured by the backend’s citation index slicer.

You can pull down the response ID output, or then the input retrieval that is separate, and see what was generated (if delivered); see what the AI was up to that made the stream bork, and the usage bill.

I suppose you can prompt the heck out of the AI, providing the correct code over and over, telling it to plan out what it will cite internally in commentary, etc, that might improve the chances that this file_search tool with bad instructions you cannot alter will work correctly.