502 on GPT-4 for the past 18 hours and 10% on GPT-3.5-Turbo

API
,
1

Our software uses gpt-3.5-turbo to summarize a transcription from our transcription service company and then transforms the summarization with gpt-3.5-turbo. We have gigabit internet and all firewalls and VPNs have been turned off for this.

GPT-4 hasn’t had a single successful response in 18 hours. GPT-3.5-Turbo throws a 502 every 10 responses or so. The software is completely broken and we’re getting charged for each transcription from our transcription service every time it runs, and we’re also getting charged by OpenAI (luckily we have $2500 in credits) for these completions that don’t actually happen.

However, using the playground works for some reason, although it fails often as well.

It’s clearly not a connection issue on our end or a VPN/Firewall blocking it, because A. we turned them off, and B. gpt-3.5-turbo goes through 90% of the time.

The status page says all operations functional however, so I’m just wanted to see if anyone else it having problems.

It’s either a 502 or an openai.error.Timeout.

2

Yes, OpenAI are buckling under their success.

And it’s annoying that you get charged even for failures. I think the way that works, is that the front-end gateway has a timeout, that ends up timing out while the model is still inferring, and because your query used model resources, billing sees it as “use.”

You may be able to switch to streaming responses, which will reduce the risk of idle timeouts.