Anyone else experiencing degraded API today for GPT4o?
Typically for our requests we average ~8s per response. Today it’s 16s, with many many requests taking longer than 30 seconds. I’ve been monitoring the outage page, but no mention, so just curious if others are noticing this?
This issue has continued for us today, though it’s slightly better than yesterday. I’ve been trying to communicate this with OpenAI support, but it’s taken a full day to get to the point where they’re asking us for data.
On normal days (looking at our data going back 3 months against the same model, gpt-4o-2024-08-06), we see < 1% requests taking over 30 seconds. Yesterday, 25% of our requests took longer than 30 seconds.
Today it’s improved to around 10%, but still way above normal.
Typical days we see avg response times of 7s. Yesterday it was around 17s average.
This is for ~4K token prompts (1.2K cached) and response sizes of ~350 tokens. Nothing insane.
Are we really the only ones experiencing this issue?
I didn’t notice a particularly longer delay, but it may vary depending on the time and location.
But perhaps this can give you some insights:
Thank you, that is indeed interesting, hadn’t ever run across that site. Though from our data, it doesn’t seem to line up with the high latency we’re seeing from requests made from us-east-1 AWS data center. Would be nice if OpenAI was more… Open about the live latency of their product and whether they’re working to fix any issues.
The latest response from OpenAI support seems to acknowledge a problem finally, but no timeline on a fix:
At this point, there’s no further action needed on your end. I’ve added your case to our internal investigation and will continue to monitor your organization’s usage as part of this effort. I’ll follow up with you directly as soon as a fix is deployed or any meaningful updates become available.
We’re still seeing 6% of our requests taking 30s or longer (compared to normal times when it’s < 1%).
Meanwhile:
Another update that is more explicit:
As mentioned, this issue is part of a known and ongoing degradation affecting GPT-4o performance. Our engineering team is actively working to address it, and your case has been added to our internal investigation.
Yet status.openai.com says nothing about it. This is the problem.
Well it was resolved a month ago finally, and then high latency returned on August 15 (15-20% of our requests take longer than 30 seconds, compared to the usual rate of ~1%).
I am once again dealing with an OpenAI support agent that appears to be clueless. I’m sure after 4-5 more back-and-forths they will again finally admit that yes there is a latency issue they’re working on, but for some reason no one at OpenAI thinks this should be reflected on the status page.
Only took 27 back-and-forths with OpenAI support, but finally got them to commit to highlighting known high latency issues on the status page!
Thank you for your persistence and for highlighting this critical issue. We understand your experience and sincerely apologize for the lack of clarity and transparency regarding latency issues on our status page.
To address your concern directly: OpenAI is actively working to improve how we surface and communicate latency issues, including ensuring that such issues are reflected on the status page in real time whenever they meet the threshold for public acknowledgment. Your feedback has been shared with the relevant teams, and it has reinforced the need for more proactive and transparent communication about performance degradations, especially those impacting production environments.
While we cannot guarantee that every latency issue will immediately appear on the status page, we are committed to refining our processes to better identify and communicate these issues as they arise. This includes improving internal monitoring and escalation workflows to ensure users like you are informed promptly.
High latency on GPT4o has once again resumed this week, we are on day 3 of high latency.
As usual they are not reporting the issue on the status.openai.com page. And they are not responding with any clarity via support emails.
And the new wrinkle is they’ve clearly integrated their LLMs into the email support system, because I now occasionally get a very fast, detailed, and INCORRECT response to my questions. You now can never tell if you’re talking to a human or a bot, other than response times.
No one else experiencing high latency this week? Or noticing it?
