Has anyone seeing this issues? It seems like all my requests results in timeout, and my timeout duration is set to 15s. It started to happen in the past 24h. Thanks!
Not I’m started to see this error:
**That model is currently overloaded with other requests. You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID 009edcc977aa0911240eaa3d7fa56902 in your message.**
We’ve seen it from time to time. Haven’t seen it tonight. I don’t think it has to do with your timeout. You are connecting to the service. It’s just overloaded. In your code you should have a retry backoff strategy to handle this case.(If you don’t already) After ‘x’ number of retries show an error to your users.
There are two reasons: 1. your token count. Simple as that, if you send maxTokens in one call it could take 20-40 sec. 2… The traffic on the api or model.
Get used to it and you should check this behavior by streaming the response and stop the time needed.
Yes, my same script, after switching to gpt3.5-turbo-16k, keeps getting the below error.
Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 1.0 seconds as it raised APIError: Invalid response object from API: 'Error: stream was reset: CANCEL' (HTTP response code was 500).
Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 2.0 seconds as it raised APIError: Invalid response object from API: 'Error: Request timeout has expired [url=https://api.openai.com/v1/chat/completions, request_timeout=600000 ms]' (HTTP response code was 500).
Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIError: Bad gateway. {"error":{"code":502,"message":"Bad gateway.","param":null,"type":"cf_bad_gateway"}} 502 {'error': {'code': 502, 'message': 'Bad gateway.', 'param': None, 'type': 'cf_bad_gateway'}} {'alt-svc': 'h3=":443"; ma=86400', 'cache-control': 'private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0', 'cf-ray': '7d9ca44b59c84cc5-SIN', 'date': 'Mon, 19 Jun 2023 15:05:28 GMT', 'expires': 'Thu, 01 Jan 1970 00:00:01 GMT', 'referrer-policy': 'same-origin', 'server': 'istio-envoy', 'x-frame-options': 'SAMEORIGIN', 'content-length': '84', 'content-type': 'application/json; charset=UTF-8', 'x-envoy-upstream-service-time': '300955'}.
Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 8.0 seconds as it raised APIError: Bad gateway. {"error":{"code":502,"message":"Bad gateway.","param":null,"type":"cf_bad_gateway"}} 502 {'error': {'code': 502, 'message': 'Bad gateway.', 'param': None, 'type': 'cf_bad_gateway'}} {'alt-svc': 'h3=":443"; ma=86400', 'cache-control': 'private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0', 'cf-ray': '7d9cabbd6eda3f69-SIN', 'date': 'Mon, 19 Jun 2023 15:10:36 GMT', 'expires': 'Thu, 01 Jan 1970 00:00:01 GMT', 'referrer-policy': 'same-origin', 'server': 'istio-envoy', 'x-frame-options': 'SAMEORIGIN', 'content-length': '84', 'content-type': 'application/json; charset=UTF-8', 'x-envoy-upstream-service-time': '304047'}.
And sometimes it can last for hours. please fix this new model endpoint.