Frequent API timeout errors recently

API
21

thanks, I have try catch and add timeout in my code. The api may be stable today.

1 Like
22

Probably is intermittent. If it times out continuously more than a certain amount of time it would change to something else.

23

Hi @georgei

Yeah, I hear you.

I’m been working for about an hour and it TIMEOUT about 70% of the time or so…

:slight_smile:

3 Likes
24

Timeouts started increasing around 1am Sunday and continued through Monday. Back to normal today. Here’s a graph of the increase. The blue line is timeout errors.

image
image1220×844 54.2 KB

25

Is it working with retry decorator?

26

Call result = openai.ChatCompletion.create within a ThreadPoolExecutor. Try and catch future.result(10). When ThreadPoolTimeoutError is raised, use tenacity to retry on this exception

27

Just use plain http with retry in python. Looks like the read timeout is set to 600 sec for each connection internally by the openai.ChatCompletion library which is too bad. So I used regular http connection and it worked great. But I set connection time out, read timeout to 20,60 sec.

@retry(Exception, tries=5, delay=1, backoff=2, max_delay=120)
def call_openai_api(chat_gpt_command,max_tokens,presence_penalty):
OPENAI_API_KEY = os.getenv(“OPENAI_API_KEY”)
url = “https://api.openai.com/v1/completions”
headers = {
“Content-Type”: “application/json”,
“Authorization”: f"Bearer {OPENAI_API_KEY}"
}
data = {
“model”: “text-davinci-003”,
“prompt”: chat_gpt_command,
“temperature”: 0.7,
“max_tokens”: max_tokens,
“top_p”: 1,
“frequency_penalty”: 0,
“presence_penalty”: presence_penalty,
“n”: 1
}
response = requests.post(url, headers=headers, json=data, timeout=(60, 120))
response.raise_for_status()
return response.json()

2 Likes
28

You can add parameter request_timeout, it will be pass to requests.post(timeout=xxx)

eg:

openai.ChatCompletion.create(
model=“gpt-3.5-turbo”,
messages=[
{
“role”: “user”,
“content”: prompt,
}
],
request_timeout=60,
)

29

I also keep getting errors, even-though I set the timeout to 10 minutes “request_timeout=600” anything that takes longer than 5 minutes times out.

1 Like
30

They are playing dirty header tricks, and thats all I"ll say…

31

Hey Cyntranic,

Would you mind elaborating please?

32

Hi I have some question
I only set the max tokens is 80
however the response is time out
I don’t know what the reason of the problem because it didn’t happen two weeks age
I use the python

33

the same question could be answer two weeks ago, and max tokens was 256 at that time
And when I use the same max tokens , it is time out so I lower my max tokens

34

Hi I built a library to deal with the problems mentioned in this thread. It handles timeout, rate limit, api keys errors

cc @crowdreactor @robeatz20

35

Use a user agent changer, proxy, API is fine.

37

Can you please share info about the library that you are talking about?

38

I’m seeing occasional timeouts. That isn’t a problem in itself, but the 600 second wait is too long. It would be nice to have a parameter for the timeout in the library so we can build better error handling around it.

2 Likes
39

I am handling it with the following:

except (openai.APIError,  # andling errors (for some of them I have not a clue of what they are! )
                requests.exceptions.Timeout, 
                APIConnectionError,
                openai.error.APIError, 
                openai.error.APIConnectionError, 
                openai.error.RateLimitError, 
                openai.error.ServiceUnavailableError, 
                openai.error.Timeout):

But it does not capture all the issues!

40

Look up in this post you’ll see the request_timeout parameter is passed to requests(timeout=request_timeout) and then except openai.error.timeout

1 Like