GPT 3.5-Turbo API call randomly hangs indefinitely

Using a standard python request to call the API typically performs quite well, however randomly the request will hang indefinitely (been >2 hours and no response) - have anyone experienced a similar problem and if so how did you solve it?

Example below (This json data included a function call, but it both hangs randomly with and without functions included)

gpt_response = requests.post(
    'https://api.openai.com/v1/chat/completions',
    headers=headers,
    json=json_data,
)
1 Like

not sure if it is related to your case but seems some problem in gpt-3.5-turbo today.

Elevated API error rates on GPT-3.5-turbo models

1 Like

Yep, ChatGPT has sat at outputting nothing for long times before producing today, and also stalled mid-output with a blinking cursor.

One needs to write a threaded function that can be killed with a timer and retried if it doesn’t stream.

It should return some kind of error though. You can listen for that error to prompt it again, but you might want to follow these guidelines: OpenAI Platform

There is no “error” for a SSE subscription that returns one token per minute.

1 Like

I’m also having this issue. It seems like if you wait long enough you get output. Is there anyway to get a faster response by requesting a very small amount of data.

1 Like

I too am on this page (however many months later) cause requests to gpt-3.5-turbo-1106 frequently seem to stall today. Worked lightning fast earlier yesterday, but began having issues during the evening. The 4-preview API still seems to work fine.

Same! gpt-3.5-turbo-1106 is hanging where as gpt-4-1106-preview is working fine?

I had the same issue and solved it with the optional parameter request_timeout and incremental timeout times. The approach is like this. This pattern is quite common for REST requests as timeouts may often occur because of overloads.

import openai

# preparations here

for timeout_secs in (5, 10, 20, 40):
    try:
        response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo",
            messages=[<here go your messages>],
            request_timeout=timeout_secs
        )
    except Exception as exc:
       print(f"Timed out after {timeout_secs} secs with exception, retrying...")
    else:
        if response:
            break
        print(f"Timed out after {timeout_secs} secs without response, retrying...")