I run a platform that uses a variety of OpenAI models. We’ve had them all set up for a while now, and they were working well. In the last few days, it seems GPT-4o specifically is having a pretty catastrophic issue where it can answer the first exchange in a conversation correctly, but thereafter it gets stuck basically repeating it’s first response.
We’re not facing this issue with other OpenAI models, so it doesn’t seem like a problem with our implementation. I wanted to see if anyone else has started experiencing this strange issue over the last few days. If it’s truly just us, maybe there’s a problem we’re not seeing.
We are building a chatbot conversational interface on top of the API. There is a system message that guides the conversation often with a structure (do this, then this, then ask this, then use that information to write this, etc.)
Often towards the end of the sequence it will just start the whole process over again. We’ve been building hundreds of things like this for a while now without any problems. In the last week or so, I started getting reports of this problem.
I’ve accounted for context window and all the appropriate token limits. At a loss here as well.