When interacting with multiple prompts in the same conversation, we don’t really know how ChatGPT manages the information. It’s certainly not the entire conversation that’s being passed to the model. Perhaps some RAG-like technique determines what’s relevant so the model doesn’t ‘forget’ important details throughout the conversation.
Up until the release of the o3 model, this worked relatively well. When the topic was complex or the model made too many mistakes, staying in the same conversation was more practical.
But since the o3 model was launched, there’s been a drastic drop in quality in this regard. The model forgets even basic aspects of the initial prompt by the third or fourth exchange. It becomes lazy about what needs to be done (starts replying quickly instead of reasoning through things). It’s extremely frustrating.
To use o3 for anything meaningful that requires multiple interactions, I’m forced to restart the conversation with a freshly refactored prompt that includes all the relevant context.
This isn’t a limitation of the model itself. Clearly, it was an internal change in how dialogue information is passed to the model. I imagine that, since o3 is more expensive, OpenAI did something to cut costs, but this update turned out terribly.
Am I the only one noticing this? I haven’t seen anyone talking about it anywhere.