I often got a timeout 600s error when using the Chat completions API. So I found out that if you put a request_timeout as a parameter it would stop the request after a certain amount of time. And this worked great until I had to update the SDK version from 0.28.1 to 1.2.2. Now the request_timeout pa…

See the first 10 lines or so of the script I posted for different purpose here: [image] Gpt-3.5-turbo-1106 is very slow API Oh noes Llama that can make 80 tokens a second? “I was the first one on the server and it was fast!” “So angry that this is only twice…

This indeed fixed it. Thank you! But now I am interested in what are the other values in the Timeout,except the default one, are about: client = OpenAI(timeout=httpx.Timeout(15.0, read=5.0, write=10.0, connect=3.0)) Like the read, write and connect parameters?

You can read the httpx library docs. read should achieve the goal of timing out if there is a lack of SSE activity, but that’s a bit hard to test without doing your own simulator.

Setting request_timeout in openai v1.2.2

API

_j November 10, 2023, 1:56pm 2

See the first 10 lines or so of the script I posted for different purpose here:

Topic		Replies	Views
Is "request_timeout" parameter real? API api	5	10373	November 14, 2023
Is it still supported to set a request timeout for request? API api	4	11554	January 31, 2025
Configuring timeout for ChatCompletion Python API	26	43691	September 6, 2023
Frequent API timeout errors recently API	38	50995	September 29, 2023
Recommended way to limit the amount of time a Python ChatCompletion.create() runs API gpt-4	8	2879	September 15, 2023

Setting request_timeout in openai v1.2.2

Related topics