GPT-3.5-turbo stalling out and not responding

API
1

Hi!

I’ve been playing around with the new GPT-3.5 turbo API in python, and it has been working great. I have some code that fires off a couple of ChatCompletions, and they usually respond in a couple of seconds, but today, I’ve been running into some problems.

The first couple of responses have been great, but usually, the third or fourth will stall out… no response and continue loading forever. I can get it to skip through, but the issue just ends up popping up again. All of the completions are relatively similar, and it doesn’t happen consistently on the same ones.

Has anyone run into something like this before? Any advice on what might be going wrong? Its worked so great for me in the past, and today, these issues seem to be popping up out of nowhere.

Thanks so much in advance.

3 Likes
2

I have run into the same issue. While other chat rooms suggest doing retries after a certain time with exponential backoff, it smells like an ugly hack to deal with an error that can’t be resolved.