How to set a timeout on an API function call using using the Python library

Prompting
1

Hi, I like using the Python library but I need to set a timeout on responses. Has anyone found a way that’s easier than this:

I know I could call the REST API directly using requests and set the timeout there, but I would rather keep using the Python library if possible, so I don’t have to change too much code.

3 Likes
2

What issue is requiring you to change the timeout?

1 Like
3

@daveshapautomator I’m firing off 100 requests in parallel, 99 of them come back within a few seconds, 1 takes over 30 seconds. I want my app to gracefully kill long-running requests so i can return the 99 successful responses without waiting for the long-running thread.

3 Likes
4

Ah makes sense. I usually just set a pause and retry.

1 Like
5

doh just realized that there is a timeout parameter in the Python binding

Signature: openai.Completion.create(*args, timeout=None, **kwargs)
Docstring:
Create a new instance of this model.

Parameters:
timeout (float): the number of seconds to wait on the promise returned by the API, where 0 means wait forever.
File:      /opt/anaconda3/lib/python3.7/site-packages/openai/api_resources/completion.py
Type:      method
4 Likes
6

Tried using the timeout parameter. It doesn’t seem to have any effect. No Error received after timeout interval is passed.

7

Setting request_timeout worked for me.
e.g.

openai.Completion.create(request_timeout=1)

I was able to get this exception message.

Request timed out: HTTPSConnectionPool(host=‘api.openai.com’, port=443): Read timed out. (read timeout=1)

4 Likes