I have a fairly lengthy system message and output json_schema and when I start a responses conversation it works great until the model (gpt-4o-mini) decides it needs to search for something. If it’s not found it just slightly re-words the search criteria over and over again until it runs out of something (context window, tokens or some other limit). It also takes what feels like forever to run through that loop, which means the user experience will suck.
This managed to eat through 2 million tokens with a few user messages that were only around 16-20 words. Without file search the same system message and user message consumes 2,000 tokens, when it gets stuck in a loop it consumes around 360,000 tokens.
Any ideas what could be causing the looping, or if there is a way to let it just ingest the content and use that “knowledge” rather than search the files.
I’m not sure if the issue is present in any other models, I’ve only tried it with gpt-4o-mini, however it is still there. I’m unclear why it loops around and then dumps out with no response, but if I find a root cause or workaround I’ll post it here.
From what I can see it’s still an issue with gpt-4o-mini. What I’ve noticed is that if it can’t find relevant content in the vectorstore when it uses the file_search tool it loops around performing file_search after file_search until it runs out of context window. Then just stops without providing any output. gpt-4o works fine, it just does a single search then returns a message that basically ignores the fact that it didn’t find anything - this is the behaviour I would expect.
No, not really. The -mini variant still does it. I tried all sorts but always had the same issue. Gpt-4o works fine with exactly the same prompt and vector store.