GPT 3.5-Turbo API call randomly hangs indefinitely

API
1

Using a standard python request to call the API typically performs quite well, however randomly the request will hang indefinitely (been >2 hours and no response) - have anyone experienced a similar problem and if so how did you solve it?

Example below (This json data included a function call, but it both hangs randomly with and without functions included)

gpt_response = requests.post(
    'https://api.openai.com/v1/chat/completions',
    headers=headers,
    json=json_data,
)
1 Like
2

not sure if it is related to your case but seems some problem in gpt-3.5-turbo today.

Elevated API error rates on GPT-3.5-turbo models

1 Like
3

Yep, ChatGPT has sat at outputting nothing for long times before producing today, and also stalled mid-output with a blinking cursor.

One needs to write a threaded function that can be killed with a timer and retried if it doesn’t stream.

4

It should return some kind of error though. You can listen for that error to prompt it again, but you might want to follow these guidelines: OpenAI Platform

5

There is no “error” for a SSE subscription that returns one token per minute.

1 Like
6

I’m also having this issue. It seems like if you wait long enough you get output. Is there anyway to get a faster response by requesting a very small amount of data.

1 Like
7

I too am on this page (however many months later) cause requests to gpt-3.5-turbo-1106 frequently seem to stall today. Worked lightning fast earlier yesterday, but began having issues during the evening. The 4-preview API still seems to work fine.

8

Same! gpt-3.5-turbo-1106 is hanging where as gpt-4-1106-preview is working fine?

9

I had the same issue and solved it with the optional parameter request_timeout and incremental timeout times. The approach is like this. This pattern is quite common for REST requests as timeouts may often occur because of overloads.

import openai

# preparations here

for timeout_secs in (5, 10, 20, 40):
    try:
        response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo",
            messages=[<here go your messages>],
            request_timeout=timeout_secs
        )
    except Exception as exc:
       print(f"Timed out after {timeout_secs} secs with exception, retrying...")
    else:
        if response:
            break
        print(f"Timed out after {timeout_secs} secs without response, retrying...")
10

I have same issue with all models. Python API. await client.chat.completions.create - call was hang there.

11

It keeps happening in past 4 hours, calls got random timeouts.