Increase Batch API request limit beyond 1,000,000 requests queued

Batch API has an undocumented rate limit - only 1,000,000 requests can be queued at a time.

Since the queue takes like 8 hours to clear nowadays, this means 3,000,000 requests per day. Assuming 200 input tokens per request, that’s 600M input tokens per day. This is much lower than the 4B input tokens per day promised to usage tier 5 users using text-embedding-3-small.

Please allow a way to increase this undocumented rate limit.

2 Likes

explore the platform, i think there’s a way to request a bigger rate limit

1 Like

Nope can’t find. Your reply does not add anything useful, sorry!