Error {'code': 524, 'message': 'Gateway timeout.', 'param': None, 'type': 'cf_gateway_timeout'}}

API
1

Very often receiving error {‘code’: 524, ‘message’: ‘Gateway timeout.’, ‘param’: None, ‘type’: ‘cf_gateway_timeout’}}

I think this is server side error. im calling the api from a long running aws lambda with 15 minutes timeout, but receive this error at the 10 minutes mark.
Anyone can help?

1 Like
2

I’m getting the same kind of errors all day today:
current token count: 4388
OpenAI API returned an APIError:
Gateway timeout. {“error”:{“code”:524,“message”:“Gateway timeout.”,“param”:null,“type”:“cf_gateway_timeout”}} 524 {‘error’: {‘code’: 524, ‘message’: ‘Gateway timeout.’, ‘param’: None, ‘type’: ‘cf_gateway_timeout’}} {‘Date’: ‘Mon, 22 May 2023 06:22:31 GMT’, ‘Content-Type’: ‘application/json’, ‘Content-Length’: ‘92’, ‘Connection’: ‘keep-alive’, ‘X-Frame-Options’: ‘SAMEORIGIN’, ‘Referrer-Policy’: ‘same-origin’, ‘Cache-Control’: ‘private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0’, ‘Expires’: ‘Thu, 01 Jan 1970 00:00:01 GMT’, ‘Server’: ‘cloudflare’, ‘CF-RAY’: ‘7cb2e86deacf10d5-ORD’} Retrying after error: Gateway timeout. {“error”:{“code”:524,“message”:“Gateway timeout.”,“param”:null,“type”:“cf_gateway_timeout”}} 524 {‘error’: {‘code’: 524, ‘message’: ‘Gateway timeout.’, ‘param’: None, ‘type’: ‘cf_gateway_timeout’}} {‘Date’: ‘Mon, 22 May 2023 06:22:31 GMT’, ‘Content-Type’: ‘application/json’, ‘Content-Length’: ‘92’, ‘Connection’: ‘keep-alive’, ‘X-Frame-Options’: ‘SAMEORIGIN’, ‘Referrer-Policy’: ‘same-origin’, ‘Cache-Control’: ‘private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0’, ‘Expires’: ‘Thu, 01 Jan 1970 00:00:01 GMT’, ‘Server’: ‘cloudflare’, ‘CF-RAY’: ‘7cb2e86deacf10d5-ORD’}. Retry attempt 1 of 3

3

I am getting the 500 errors a lot in the last 48 hours. Is this currently being addressed by the OpenAI team?

The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID xxxxxxxx in your email.

1 Like
4

when the token count is under 4k for GPT-4 it seems to work, but I get this error a lot when token count is over 4k… What happened to the 8k window?

5

another day and the error continues (and we are billed for it…). Ive opened a ticket to openai but didnt get a response or ack…

[ERROR] APIError: Gateway timeout.
{
“error”: {
“code”: 524,
“message”: “Gateway timeout.”,
“param”: null,
“type”: “cf_gateway_timeout”
}
}
524 {‘error’: {‘code’: 524, ‘message’: ‘Gateway timeout.’, ‘param’: None, ‘type’: ‘cf_gateway_timeout’}} {‘Date’: ‘Thu, 25 May 2023 17:03:32 GMT’, ‘Content-Type’: ‘application/json’,

6

I have been facing this exact error for the past 24 hrs.I have a long prompt and request (that will produce atleast 4096 tokens). Is stream=True, the only solution? And is anyone else facing the same issue. Can I change some settings on my side?

1 Like
7

Yes also facing this error, any advice would be great since this wasn’t happening previously, but starting happening today.

8

Yea we are getting this type of error quite a bit recently too. We honestly don’t really know what else to do at this stage.

9

We are getting this error too from yesterday. We didn’t change our code. What is up?

10

CloudFlare has a 100-second limit on HTTP requests.

So, it is necessary to keep your requests under this 100-second limit.

Depending on current server load, tool use, etc it’s possible for some requests to take more time than this, which is causing this error.

The solution is to use streaming which chunks the response. Since each chunk takes far fewer than 100-seconds to complete the call does not get interrupted and you won’t get the 524 errors.

1 Like
11

@anon22939549 - I’ve had requests longer than 3 minutes that worked fine, so I’m not sure why this 100-second limitation all of a sudden.

12

The 100-second limit is the CloudFlare default. It’s possible OpenAI has set a different limit. The more important point is that some limit exists and the way to ensure you do not run up against it is to use streaming when making long requests.

13

Thanks for the explanation @anon22939549 , but can you please clarify (if you know) why is the limitat arbitrary in the sense that sometimes calls over 100 seconds are completed successfully and some don’t?

14

We’re having the same 524 cf_gateway_timeout issue - started around 12 July 2024 but seems to be exacerbated in the last 48 hours. Retrying the requests sometimes works. Using gpt-4-turbo on tier 5.

15

For long running requests, is there a way to do callbacks so the http connection doesn’t get timed out by CloudFlare?

16

Streaming.

1 Like
17

Seeing elevated cf_gateway_timeout errors for Whisper endpoint. Started a few days ago.

18

This issue should be fixed on their side. Streaming was introduced to improve the UX for front-end applications, there is no real reason to use streaming for machine-to-machine communication.

Also, requests with large input prompts or attached media will often be slower than 100 seconds so this change is really detrimental to the their API usage.

1 Like
19

I totally agree. Streaming should NOT be used as a way to overcome hard-setted limits by CloudFlare…

20

The issue is fixed on OpenAI’s side—with streaming.

Streaming is just sending chunked responses as server-side events. This not only side-steps the CloudFlare timeout issue but also brings several other benefits with it as well.

  1. It allows you to stop a process if you determine the response is of poor quality midstream.
  2. It allows for early error detection, so if there is some interrupt in your connection the model won’t keep generating a response no one will ever get for the next 10-minutes.
  3. It allows for more dynamic resource management for servers and memory.
  4. Along with allowing you to stop a process midstream, it also enables the ability to reconnect and resume if there is a connection hiccup as the client has already received some completed chunks.

Streaming is the way to overcome these limits, simply increasing the timeout is an inelegant hack and very poor design.

The problem is solved—just use streaming.