Using a standard python request to call the API typically performs quite well, however randomly the request will hang indefinitely (been >2 hours and no response) - have anyone experienced a similar problem and if so how did you solve it?
Example below (This json data included a function call, but it both hangs randomly with and without functions included)
gpt_response = requests.post(
'https://api.openai.com/v1/chat/completions',
headers=headers,
json=json_data,
)
not sure if it is related to your case but seems some problem in gpt-3.5-turbo today.
Elevated API error rates on GPT-3.5-turbo models
Yep, ChatGPT has sat at outputting nothing for long times before producing today, and also stalled mid-output with a blinking cursor.
One needs to write a threaded function that can be killed with a timer and retried if it doesn’t stream.
It should return some kind of error though. You can listen for that error to prompt it again, but you might want to follow these guidelines: OpenAI Platform
There is no “error” for a SSE subscription that returns one token per minute.