API Error : That model is currently overloaded with other requests. You can retry your request

API
1

openai.error.APIError: That model is currently overloaded with other requests. You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID ***************** in your message.) (Error occurred while streaming.)

I’ve been encountering this error a lot lately, especially when I use stream=True, I feel like it has become more frequent. Is there any solution you can suggest? It affects the users of my app very negatively. This error happens before the RateLimit is exceeded.

2

Catch openai API errors by return type (there about eight of them), and handle each uniquely. You can’t fix a bad API key by retrying, but you can retry the “overloaded” after a pause and also status bar the user.

3

I got this error from my server logs, it throws the error like this, so without error code. What should I do to catch it with the error code? Also, do you have a suggestion method or an example for this pause and reprocessing?

4

An example from a script I have handy:

def retry_openai(answernum, idtok, messages, model=MODEL_GPT3, temperature=0.0, n=1, stop=None, retry=5, use_functions=False):
    answer = None
    fns = None
    fcs = 'none'
    if use_functions:
        fns = functions
        fcs = 'auto'
    for i in range(retry):
        if i > 0:
            time.sleep(0.25*(2**i)+0.5)
        try:
            answer = openai.ChatCompletion.create(
                model = model,
                temperature = temperature,                                                                                                              messages = messages,
                n = n,
                stop = stop,
                functions = fns,                                                                                                                        function_call = fcs
            )
            return answer
        except openai.error.RateLimitError as rle:
            print(datetime.datetime.now(), json.dumps({'message':'rate limit error', 'error':str(rle), 'idtok':idtok, 'answernum':answernum, 'iteration':i}))
        except openai.error.APIError as apie:
            print(datetime.datetime.now(), json.dumps({'message':'api error', 'error':str(apie), 'idtok':idtok, 'answernum':answernum, 'iteration':i}))
        except openai.error.ServiceUnavailableError as sue:
            print(datetime.datetime.now(), json.dumps({'message':'service unavailable error', 'error':str(sue), 'idtok':idtok, 'answernum':answernum, 'iteration':i}))
        except Exception as x:
            print(datetime.datetime.now(), json.dumps({'message':'exception in openai help', 'error':str(x), 'idtok':idtok, 'answernum':answernum, 'iteration':i}))
            traceback.print_exc()
    raise Exception("The help service is currently overloaded. Please try again in a few seconds.")