Configuring timeout for ChatCompletion Python

General API discussion
22

You can use this client implemented with asyncio and httpx.
It supports fine grained connect/read timeout setting and connection reuse.

from httpx import Timeout
from openai_async_client import AsyncCreate, Message, ChatCompletionRequest, SystemMessage, OpenAIParams

create = AsyncCreate(api_key=os.environ["OPENAI_API_KEY"])
messages = [
    Message(
        role="user",
        content=f"ChatGPT, Give a brief overview of the Pride and Prejudice by Jane Austen.",
    )
]
response = create.completion(ChatCompletionRequest(prompt=messages),client_timeout=Timeout(1.0,read=10.0),retries=3)


create = AsyncCreate()
response = create.completion(TextCompletionRequest(prompt=f"DaVinci, Give a brief overview of Moby Dick by  Herman Melville."))
23

I completely recommend granular timeouts, but in a general use-case it makes absolutely no sense. Especially in the context of an OpenAI request.

I’d say it’s like recommending sport car parts to someone who just wants to fix their Honda Civic (fantastic car btw).