Today has been a very tough day.
Our agent that runs on GPT 5.4 started acting out so badly that we had to literally shut down our agent. Being completely offline was better than whatever was happening.
Been running evals all day, massive regression with no apparent explanation.
UPDATE:
- After using 5.4 since launch, we just changed it to 5.2 and everything is back to normal.
- One interesting behaviour to note: we make every call using flex : true. It usually takes ages, errors many times. However, today these have been running as fast as if we weren’t using flex at all.
Hi @soviero
Thanks for sharing this, that sounds like a pretty severe regression, especially if it forced you to take your agent offline.
The fact that switching from 5.4 back to 5.2 resolved things is a really strong signal. The change in flex: true behavior is also interesting and may point to something on the serving side rather than just model quality.
To help narrow this down, could you share a bit more detail?
- Approximate time this started
- Whether the issues were consistent or intermittent
- A few request IDs from failing 5.4 calls
Also, have you continued to see the faster-than-expected behavior with flex: true, or was that limited to earlier today?
~Smith