having 3 + days of failed requests because of timeouts is pretty bad for a business
4 Likes
same error to me, what is the solution? 
It does. Sometimes, outages do not work the same way for all the models/endpoints (sometimes they do).
@ljia31523 All the strategies are pretty much explained in this link that @ruby_coder already shared. Feel free to reach out if you still have doubts or do not know how to implement them.
1 Like
Same problem with davinci-003 and 3.5-turbo
Experiencing the same issue with both davinci-003 and davinci-002.
It’s almost been 3 days with the same issue. It’s making it really hard to work.
1 Like
what is most annoying to me is:
I try to set a timeout at openai.ChatCompletion.create(..., timeout=10) hoping that that after 10seconds, if it is still “hunging”, please stop.
I am using @retry(wait = wait_random_exponential(min =1, max = 60), stop = stop_after_attempt(6)) def chatcompletion_with_backoff(**kwargs): to hope to handle this.
HOWEVER, there are no “timeout”. I have been monitoring the process. There are mulititimes that the the request being going on >minutes, and yet no Timeout is thrown out.
See this for the similar complaints…
I mean, basic coding principle, things should “do what they say, say what they do”… it’s clearly not the case here.
any guidance pls? Just keep “trying” and wait for things to clear, unfortunately is not an option
1 Like
Exactly, does not give me the confidence to roll my app functionality out to further users at all, unless things get a bit more stable.
2 Likes
I’ve faced the same timeout issue. In the Python API, if you pass the parameter request_timeout to openai.ChatCompletion.create instead of timeout then it will throw a Timeout exception after n seconds. Hope it helps
But yes, I’ve been facing these issues as well…
We have exactly the same problem with timeouts.
thank you for pointing out therequest_timeout
1 Like
same here! I plan to use claude to see if it is more stable
thanks, I have try catch and add timeout in my code. The api may be stable today.
1 Like
Probably is intermittent. If it times out continuously more than a certain amount of time it would change to something else.
Hi @georgei
Yeah, I hear you.
I’m been working for about an hour and it TIMEOUT about 70% of the time or so…
1 Like
Timeouts started increasing around 1am Sunday and continued through Monday. Back to normal today. Here’s a graph of the increase. The blue line is timeout errors.
Is it working with retry decorator?
Call result = openai.ChatCompletion.create within a ThreadPoolExecutor. Try and catch future.result(10). When ThreadPoolTimeoutError is raised, use tenacity to retry on this exception
Just use plain http with retry in python. Looks like the read timeout is set to 600 sec for each connection internally by the openai.ChatCompletion library which is too bad. So I used regular http connection and it worked great. But I set connection time out, read timeout to 20,60 sec.
@retry(Exception, tries=5, delay=1, backoff=2, max_delay=120)
def call_openai_api(chat_gpt_command,max_tokens,presence_penalty):
OPENAI_API_KEY = os.getenv(“OPENAI_API_KEY”)
url = “https://api.openai.com/v1/completions”
headers = {
“Content-Type”: “application/json”,
“Authorization”: f"Bearer {OPENAI_API_KEY}"
}
data = {
“model”: “text-davinci-003”,
“prompt”: chat_gpt_command,
“temperature”: 0.7,
“max_tokens”: max_tokens,
“top_p”: 1,
“frequency_penalty”: 0,
“presence_penalty”: presence_penalty,
“n”: 1
}
response = requests.post(url, headers=headers, json=data, timeout=(60, 120))
response.raise_for_status()
return response.json()
1 Like
You can add parameter request_timeout, it will be pass to requests.post(timeout=xxx)
eg:
openai.ChatCompletion.create(
model=“gpt-3.5-turbo”,
messages=[
{
“role”: “user”,
“content”: prompt,
}
],
request_timeout=60,
)
I also keep getting errors, even-though I set the timeout to 10 minutes “request_timeout=600” anything that takes longer than 5 minutes times out.