Is it still supported to set a request timeout for request?

API
1

Hi all,

It seems supported to specify the timeout. However, that does not work
I found this “timeout” and “request_timeout” but the income is unsupported values.

Anyone have a reference about how to set the timeout?

Thanks in advance

1 Like
3

The request_timeout parameter is obsolete. It is from OpenAI SDK libraries before November 2023, which were revamped with an upgrade to “version 1.0.0”.

When using OpenAI SDK “client” methods, you have a timeout parameter in seconds that you can set upon instantiation. Python:

from openai import OpenAI
client = OpenAI(timeout=120)
...

A timeout used as an API parameter in an OpenAI SDK API call after that is not sent to the API, but rather, overrides that default timeout with the same mechanism being used, but just for that call.

For Python, since the underlying client timeout is being set on the httpx library module, you can also use a more granular httpx spec timeout.

The default is ten minutes.

You do not have control over OpenAI’s server-side closing of API connections by their own timeout (if that were to ever happen, like the 15 minute life of a realtime websocket).

2 Likes
4

Thankfully, the information in this particular post can be safely ignored thanks to the post below :point_down: :slight_smile:

I was gonna add that you can use the SSEs to manually manage your clientside timeouts, but it looks like that might no longer be a thing.

I was just looking through the docs again, because I firmly believe that it was documented that if you terminate the SSE connection, generation would end on the server side as well. But I can’t find a reference to that anymore.

I hope I’m just blind and they didn’t just ninja deprecate-and-retire that too.

2 Likes
5

By default the library closes underlying HTTP connections whenever the client is garbage collected. You can manually close the client using the .close() method if desired, or with a context manager that closes when exiting.

A streaming connection that is generating can stop processing shortly after the network connection then fails. A non-stream generation being closed will not stop the internal consumption of tokens, however.

2 Likes