Currently I stream openai response and I use this settings
client = OpenAI(api_key = os.getenv(“OPENAI_API_KEY”),timeout=65.0, max_retries=1)
Problem with this is sometime I get openai.APIConnectionError when it gets too long and reaches timeout.
Is their anyway which I can use to decrease timeout and retry more till I get first response for streaming? I tried setting timeout=20.0, max_retries=6 but it resulted in lot of timeout errors.
Also I am confused according to documentation:
When the library is unable to connect to the API (for example, due to network connection problems or a timeout), a subclass of openai.APIConnectionError is raised.
When the API returns a non-success status code (that is, 4xx or 5xx response), a subclass of openai.APIStatusError is raised, containing status_code and response properties.
so suppose I have max retries of 2 will APIConnectionError error be raised after 2 retries or on first retries itself?