I’ve been using the API for a while. I’m using the gpt-3.5-turbo-0613 model, and I have the ‘Pay-as-you-go users (after 48 hours)’ rate limits: 3,500 RPM and 90,000 TPM.
However, I’m having some problems. Whilst I only occasionally hit the rate limit, I’m getting the following error very frequently:
openai.error.ServiceUnavailableError: The server is overloaded or not ready yet.
Is this an issue on my end, or does it just mean that a lot of people are using GPT-3.5?
Is there any way to avoid this? Will OpenAI fix this issue? I haven’t just been experiencing this issue today. I use a function to retry the AI response if it fails – but it’s set to raise an error after eight retries, and it retries every second. Should I increase the time it takes to retry?
Even if I do retry many times and ultimately get the response, it still takes a lot longer if it keeps getting this error. I don’t understand why this is happening. Any help would be much appreciated!
Try to use exponential backoff in your retry. It does not make it shorter but if there are too many requests at the same time, it can help to delay the request for some time.
October 7th and getting this error for 5% of my requests today. You’d think with OpenAI being all advanced programmers these guys could figure out how to handle server overload on their end instead of making developers code hacky solutions for their API “to empower developers” that actually just has 100x the downtime of their own commercial APP, ChatGippity. I will be testing Azure OpenAI and if that does not solve the issue, we are stringing up a LLAMA 2 server boys, reliability is key for UX and the OpenAI API seems like pay to play and not pay to actually work.
It has been a few days since the responses of OpenAI started taking too much time until getting the overload error “openai.error.ServiceUnavailableError: The server is overloaded or not ready yet.”. My server was affected by this issue causing high CPU usage. I believe this issue is severe enough to be looked into because there have been several topics posted about it for a pretty long time.
I am also working with the chat-gpt API and encounter the same error:
The server is overloaded or not ready yet
AND
raise self.handle_error_response(
openai.error.APIError: Bad gateway.
I’ve been getting the same error for over a week now, is there anyone who has actually solved it and can make a number of calls close to the advertised number ?
Thanks for the support
