Setting request_timeout in openai v1.2.2

API
1

I often got a timeout 600s error when using the Chat completions API. So I found out that if you put a request_timeout as a parameter it would stop the request after a certain amount of time. And this worked great until I had to update the SDK version from 0.28.1 to 1.2.2. Now the request_timeout parameter is gone and there is only the timeout parameter. But from my experience using timeout doesn’t stop the request even after it goes past the custom time set. This is what my previous request looked like:

response = openai.ChatCompletion.create(
                model='gpt-3.5-turbo-1106',
                messages=[
                    {"role": "system",
                     "content":
                     "I will give you a sentence. I want you to summarize the topic of the sentence in 3 words."
                     "These 3 words will be used to search for an image to fit the sentence so make sure the words make sense to the meaning of the sentence."
                     f"Answer by writing the 3 words only."
                     },
                    {"role": "user",
                     "content": f"This is the sentence: '{sentence}'."},
                ],
                # temperature=0.3,
                request_timeout=90,
            )

            message = response.choices[0]['message']
            # Get the content from the message
            content = message['content']

And this is what it looks like now:

 try:
            completion = client.chat.completions.create(
                model='gpt-3.5-turbo-1106',
                messages=[
                    {"role": "system",
                     "content":
                         "I will give you a sentence. I want you to summarize the topic of the sentence in 3 words."
                         "These 3 words will be used to search for an image to fit the sentence so make sure the words make sense to the meaning of the sentence."
                         f"Answer by writing the 3 words only."
                     },
                    {"role": "user",
                     "content": f"This is the sentence: '{sentence}'."},
                ],
                # temperature=0.3,
                timeout=90,
            )

            message = completion.choices[0].message
            # Get the content from the message
            content = message.content
1 Like
2

See the first 10 lines or so of the script I posted for different purpose here:

1 Like
3

This indeed fixed it. Thank you! But now I am interested in what are the other values in the Timeout,except the default one, are about:

client = OpenAI(timeout=httpx.Timeout(15.0, read=5.0, write=10.0, connect=3.0))

Like the read, write and connect parameters?

4

You can read the httpx library docs. read should achieve the goal of timing out if there is a lack of SSE activity, but that’s a bit hard to test without doing your own simulator.