Is "request_timeout" parameter real?

API
1

I have occasional, but repeating problems with the requests to OpenAI timing out.

Request timed out: HTTPSConnectionPool(host='api.openai.com', port=443): Read timed out. (read timeout=600).

I have gone through the forum and have noticed that some users are recommending using request_timeout parameter, to shorten that time, and repeat the request after. However, others are recommending creating own function in Python to do that. Also, the parameter is not mentioned in the OpenAI documentation.

  1. Does this parameter exist, have someone managed to make it work?
  2. If not and if I need to do it by myself, I am not really clear the logic behind that. I have implemented the exponential backoff strategy, as suggested by OpenAI - but, if the timeout is 600 seconds, this means that the next retry will come after 10 minutes? Should I somehow cut the request if it takes more than, let’s say 30 seconds?
2

Yes, it works. Set it to 4 seconds, get an error in four seconds.

2 Likes
3

Hi,

So looking around the forum, I found this comment

phuib

Mar 28

Setting request_timeout worked for me.
e.g.

openai.Completion.create(request_timeout=1)

I was able to get this exception message.

Request timed out: HTTPSConnectionPool(host=‘api.openai.com ’, port=443): Read timed out. (read timeout=1)

I can’t find reference to it in the OpenAI docs, but it is mentioned in the git repo, I think the folks that are managing it themselves are doing so by .create(ing) the object and then if a reply is not received within a specific period they are calling the .close method and generating their own exceptions.

1 Like
4

It’s not working with the JS library for me.

chatCompletion = await openai.chat.completions.create({
  ...
  request_timeout: 4,
});

Error: 500 Internal Server Error (400 Unrecognized request argument supplied: request_timeout)

5

Also getting an error on the parameter in Python:

TypeError: Completions.create() got an unexpected keyword argument 'request_timeout'

completion = client.chat.completions.create(
                model="gpt-3.5-turbo",
                messages=[
                    {"role": "system", "content": "You are an assistant, skilled in marketing and discernment."},
                    {"role": "user", "content": "What's 2 + 2"
                       }
                ],
                request_timeout=10,  
            )
            return completion.choices[0].message
1 Like
6

You can set a timeout for the new python library when setting up your OpenAI object.

client = OpenAI(timeout=60)

Or for streaming, you can get really specific and set the parameters on the httpx library as exposed through the openai parameters.
client = OpenAI(timeout=httpx.Timeout(15.0, read=5.0, write=10.0, connect=3.0))

Then instead of a parameter error, you’ll get a timeout error to handle after the time elapses.

3 Likes