Are you using that for fine tunning?
I am trying to fine tune the model. but seems like its getting discounted every time.
The status says there’s no issues currently, yet everyone using 3.5 or 4 API is getting “timeout” when sending a request.
1 Like
API timing out for me too, even with a retry policy with 5 tries.
Can you share further details? What is the timeout per retry? Do you fallback to other models? Do you have backoff?
It is 5 retries with exponential backoff (1, 2, 4, 8, 16), no I do not fallback to other models, but that’s a good idea. However not sure if DaVinci is also timing out and it might require different prompts than 3.5 Turbo.
Ideally 3.5 just works consistently.
And what is the custom timeout per API call?
The timeout is currently at 20 seconds.
@crowdreactor thanks a lot for sharing the details about the implementation. This is the only way we developers can help other debug their problems. So, for your case, I’d say:
- 5 retries is probably too much, especially if you’re using the same model all the time.
- 20s timeout is probably too short, especially if you’re asking for huge completions and you’re not streaming.
If your base model is gpt-3.5-turbo, I’d say to experiment with something like:
- 1 call to turbo with
timeout = 30s.
Wait for 4s. - 1 call to turbo with
timeout = 30s.
Wait for 8s. - 1 call to davinci-003 with
timeout = 30s.
And yeah, the output would obviously depend on the model. You can try to optimize your prompt for your model. Even if you don’t, it’s usually better to return something rather than nothing. Anyways, the actual implementation totally depends on your use case. You might want to set up even longer initial timeouts (1min or more), especially if your customers do not need online interaction with your app.
2 Likes
Alright thanks I will try that out. Unfortunately this is a web app and customers need to interact with it.
For what it’s worth, I’m using the same prompts I’ve always used, and they have been lightning fast before yesterday. It’s only since yesterday since these issues have started, and it doesn’t look like I’m the only one.
1 Like
You’re not. But this is not the first time that this has happened. And it won’t be the last one. Scaling up technology and predicting demand is not as easy as some people seem to believe. So our apps need to be ready to deal with partial or global outages. Because they will eventually happen.
3 Likes
The actual chat has gone down for me as well.
Reporting 429. Hopefully they get their scaling figured out.
Must be an incredible task considering how fast everything is growing.
1 Like
just clear the browser cache and everything goes well 
According to statuspage there’s an outage related with text-curie-001 and has been resolved at 04:21 PDT. And chat.openai.com outage was marked as resolved at 03:17 PDT
I don’t know if there’s another chat.openai.com downtime related to the previous hours, but at the moment (23:47 GMT+7), all my previous chat are gone.
It is still down in ChatGPT even for Plus memers - I hope everything is well.
3 Likes
Am a plus member and it is down for me. 1:51 EST
It is a known issue, please follow the status here: https://status.openai.com/. Closing for now to avoid extra noise, please refer to the status page.
2 Likes