Timeout not honored in Python API?

General API discussion
1

Python API supposed to take timeout parameter for API calls. But it appears the timeout parameter is not honored. Possible a bug?

Signature: openai.Completion.create(*args, timeout=None, **kwargs)
Docstring:
Create a new instance of this model.

Parameters:
timeout (float): the number of seconds to wait on the promise returned by the API, where 0 means wait forever.
File: /opt/anaconda3/lib/python3.7/site-packages/openai/api_resources/completion.py
Type: method

2

Hi @kumarM

Can you share the code that you’re using to make the API call?

3

import openai

api_key = “your_openai_api_key”
openai.api_key = api_key

prompt = “Translate the following English text to French: ‘Hello, how are you?’”

try:
response = openai.Completion.create(
engine=“davinci-codex”,
prompt=prompt,
max_tokens=50,
n=1,
stop=None,
temperature=0.5,
timeout=10, # Set the timeout to 10 seconds
)
print(response.choices[0].text.strip())
except openai.error.ApiError as e:
print(f"Error occurred: {e}")

Timeout here is set to 10 seconds. Last 24 hrs there were too many timeouts with API calls but timeout error never occurred.

4

Unless it’s a hidden parameter, there is no timeout. It wouldn’t even make sense.
If a server cannot handle your request, how can it know to time you out?
Also, you are trying to use a model which is purposed for typing code. You either want Davinci, or GPTx in ChatCompletions

Use your own library to manage timeouts. retry ¡ PyPI

5

100% aligned with @RonaldGRuckus

1 Like
6

Ronald,

See attached screenshot of code for API.

Screenshot 2023-03-20 at 8.57.40 AM
Screenshot 2023-03-20 at 8.57.40 AM1536×810 72.5 KB

7

This is very neat. Nice catch.

Although I would still recommend using a client timeout and not relying on the server. It’s a very simple module to use and will be much more robust. Using hidden parameters is not recommended.

Looking a bit further, it could be possible to use the timeout currently - although again, it’s not really a good idea to use hidden parameters. There’s one other issue: You’re using Codex.

1 Like