Rate limits not aligned? Am i missing something?

API
1

The first image is of my limits shown in the general setting limits section, which claims I can do 500RPM’s for o1. (client is tier 3) – the second image shows the documentation which presents up to 5000RPM’s for o1.

I have a scenario where I may have to make bulk request to o1 to process some images at 1000’s of images per second, and I’m sure that will not be an issue as my clients tier automatically upgrades, but I guess I just need some clarity on which one it is that I need to reference to stay compliant with my clients current limits?

Am i goofy and missing something, or is there a mismatch here?

The image shows a table listing usage limits for various models, displaying tokens per minute, requests per minute, and tokens per day for each model at a tier 3 usage level. (Captioned by AI)
The image shows a table listing usage limits for various models, displaying tokens per minute, requests per minute, and tokens per day for each model at a tier 3 usage level. (Captioned by AI)1459×922 58.9 KB

image
image1861×1527 165 KB

2 Likes
2

It looks like the information in the docs (https://platform.openai.com/docs/guides/rate-limits) is a bit buggered at the moment.

Here’s a similar thread decrying more or less the same issue: Why do I have a limit of 7 images for DALLE 3?

What’s the actual source of truth here? :grimacing: Hard to say. This is probably the closest to being accurate: https://platform.openai.com/settings/organization/limits

3 Likes
3

Passed it along with the other report. Thanks.

3 Likes
5

The source of truth is what is reported about your account and its models.

Not the documentation with every tier.

Which you can verify in the headers returned when making an API request to o1, looking like:
"x-ratelimit-limit-requests": "500",
"x-ratelimit-remaining-requests": "499",

The token per minute rate will also be a significant impediment at tier-3, since reasoning used on the smallest of input consumes about 1000 tokens.

1 Like
6

Ah shoot, well I’ll just get them to fund the account to tier 5, hopefully that solves that problem.

We also are showing the the RPM’s go to 10,000 in tier 4 but show that they got back down to 1,000 in tier 5, is there something I am missing there?

A screenshot shows rate limits for Tier 4 models, including RPM, TPM, and Batch Queue Limits, with details for models like "gpt-4.5-preview" and "o1." (Captioned by AI)
A screenshot shows rate limits for Tier 4 models, including RPM, TPM, and Batch Queue Limits, with details for models like "gpt-4.5-preview" and "o1." (Captioned by AI)1033×681 42.1 KB

This image shows the Tier 5 rate limits for different models, detailing their RPM, TPM, and batch queue limits, with specific highlights on the "o1" model's RPM limit. (Captioned by AI)
This image shows the Tier 5 rate limits for different models, detailing their RPM, TPM, and batch queue limits, with specific highlights on the "o1" model's RPM limit. (Captioned by AI)1039×706 47.2 KB

1 Like
7

What you are missing?

The part where I said “the source of truth is…not the documentation”.

Tier 5 indeed maxes out at 1000 TPM. O1 is not even widely deployed to all API users, so the rate limit for those who do have access seems understandable. The only thing odd is the comparison to o1-preview, with access to all at 10x higher limits, just doesn’t make sense.

8

Yes, I wanted to post a similar finding earlier but the 10,000 RPM for Tier 4 threw me off.

If you are not in a big hurry it’s an option to wait for the reply to @PaulBellow 's inquiry and find out if the numbers for Tier 3 and 5 or Tier 4 are off.

Otherwise you can look into the Batch API or back-off methods also detailed in the documentation.

https://platform.openai.com/docs/guides/rate-limits#error-mitigation

9

Got it - so you’re saying that the documentation is unreliable and to ignore it completely and only rely on the limits returned in the headers for my tier?

I would just like to know what to expect for the different tiers so i know how to position my methods for sending over my api requests.

1 Like
10

Whelp… that needs to be fixed!
I use that info all the time. and now I’m seeing its incorrect.
I literally just used it about 30 min ago to explain to GPT what it was doing wrong.
Nice catch ssavancvic!

11

Sorry for all the confusion here. (@PaulBellow and I have been coordinating on this the past couple days.) We’re looking into this and will be back with an update shortly.

3 Likes