[BUG REPORT] batch error: input is larger than the 209,715,200 maximum

I’m using the batch API to send 5k requests to the GPT-4o-mini model, with each prompt being approximately 12k tokens.

However, the batch job fails immediately with the following error:
The batch input file is larger than the 209715200 maximum for the gpt-4o-mini model. Please try again with a smaller batch.

I’m confused—209 million what?
I don’t understand why this is happening. I ran a very similar batch a few weeks ago, and it worked without any issues.

Any insights or suggestions would be greatly appreciated!

1 Like

I’m hitting the same thing. The constraint is that you can’t have more than 209715200 tokens enqueued at any one time.

This is an awkward API. Why impose the limit like this? Why not just take as long as it takes, to complete the work?

At first I thought the limit was tokens per batch, but it turns out that the limit is “tokens enqueued”. What this means in practice, is that in order to submit a big chunk of work, you need to start a batch, and then poll the API to wait until that batch is finished. Only then can you submit your next batch. This is awkward. Why not just allow one to submit N batches, and whenever they’re done, they’re done. There’s already no practical limit on the files uploaded, so they’re already waiting there in storage. All you (OpenAI) need to do, is add a queue!

Hey! I contacted support through the platform and the problem is with the file size being to big.

Try splitting the .JSONL into smaller files.

Hmmm… I don’t think it’s that simple. Splitting into smaller files was the first thing I tried, and I STILL hit this error. Try splitting your files into smaller chunks, and then submitting them all within a few minutes of each other. I’m getting that error when I do precisely that. I’ve even tried serializing my chunks - i.e. I submit one batch, wait for it to finish, then submit my next batch, etc. And even in that scenario, I’m hitting this error. But I’m curious to hear if this works for you. Maybe I was hitting up against some kind of 24h limit or something like that.