No batches in progress but get " Enqueued token limit reached"

[shreyas.10.singh]

I am creating an app using the batch API, but irrespective of the number of tokens, I get the token_limit_exceeded error, with a message

Enqueued token limit reached for gpt-4o in organization org-IlvxxTdYJquYkdT6ofcrGQuW. Limit: 1,350,000 enqueued tokens. Please try again once some in_progress batches have been completed.

A single line in the jsonl file looks like this:

{"custom_id": "66447121891ca023aa67b36c", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o", "messages": [{"role": "system", "content": 2150 tokens(8817 chars)}, {"role": "user", "content": [{"type": "text", "text": around 50 tokens(244 chars)}, {"type": "image_url", "image_url": {"url": image url 316x316}}]}], "max_tokens": 50}}

I tried it with a single datapoint and it still gave me the same error, I have no other files being processed.

Any help regarding this would be appreciated,
Thanks !

[eiral]

I’m using the batch API (Python) and encountering the following error:

code='token_limit_exceeded', line=None, message='Enqueued token limit reached for gpt-4o in organization XXX. Limit: 90,000 enqueued tokens. Please try again once some in_progress batches have been completed.', param=None)

I currently don’t have any in_progress batches (all previous batches are completed or failed), and it’s been over 24 hours since I first received this error.

1. When can I expect access to the batch API again (i.e., when will the TPD limit reset)?
2. Is there a way to retrieve the remaining tokens for the TPD limit via the API (e.g., x-ratelimit-remaining-tokens)? If so, how?

[astromatt07]

Hi,

I have the exact same issue, did you manage to solve it ?

Thanks a lot,

[adicco]

Same issue here. Hitting the rate limit while I am sending a payload well below our enqueued limit (90,000) and there are no other batches in progress.

[pedro.schuller95]

Same issue with gpt-4o-mini, no batches in the queue and I am unable to start new batches. I already ran two batches successfully but starting another is proving impossible even with one request in the batch.

[yuerout22]

Same issue here – I have no other batches in progress but when I tried to run another one it says I’ve reached the limit for gpt-4o

[jiazhang8024]

Same issue, no response from the support team yet. Seems the token limit doesn’t reset after the completion of previous batches, different from what they wrote in the documentation: https://platform.openai.com/docs/guides/rate-limits#:~:text=Once%20a%20batch%20job%20is%20completed%2C%20its%20tokens%20are%20no%20longer%20counted%20against%20that%20model’s%20limit.

[kovart.dev]

I have been experiencing this issue for at least two weeks.
Canceling all batches doesn’t help.

[bl4m3_the_bots]

Same situation here. I can confirm.

[shreyas.10.singh]

This was quite a long back, so I don’t exactly remember what I did to solve the issue, however I’ll share the steps in getting the batch transform to work, and you could check if there’s anything you missed.

1. Create a batch input file

   batch_input_file = client.files.create(
     file=open("output.jsonl", "rb"),
     purpose="batch"
   )
   

2. Get the batch file id
   batch_input_file_id = batch_input_file.id

3. Create a batch

client.batches.create(
    input_file_id=batch_input_file_id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
    metadata={
      "description": <desc if you've got one>
    }
)

running this would give a batchId in response

4. you can get the status of your code
   client.batches.retrieve(f"{batchId}")

[appleprivate]

I’m using batch and got this error. I am very sure of no in progress batches. Why do I got this error?

Enqueued token limit reached for ft:gpt-4o-mini-2024-07-18:personal:f*** in organization org-a***hlh. Limit: 2,000,000 enqueued tokens. Please try again once some in_progress batches have been completed.

[appleprivate]

Same issue. The limitation seems to be released once a day. That means we can only run one batch job a day. I have two account. The other one don’t have that error

[appleprivate]

One possible solution: set the max_completion_tokens to as small as possible.
After setting it to 128, although I still encounter the same error, I don’t need to wait for a whole day for it. Just an hour later I can submit another batch to run.

Let’s hope it will be entirely solved when I am in Tire 3 level of API

[int_stealar]

I got the same problem. it happened after I tried to send in a batch that was too big. It failed with the error “This batch contains more than the 50,000 maximum number of requests.” so I tried sending in one that had only 20,000. But now I get the error “Limit: 2,000,000 enqueued tokens. Please try again once some in_progress batches have been completed.”, even though there are no in_progress batches, there is only the failed one.

[appleprivate]

Wait for 24h then try again by adding “max_completion_tokens” in the batch json.

[swarajgaikwad34]

I was using the GPT-4i mini-batch API (Tier 1), and it was working fine until yesterday. However, suddenly, regardless of the file size I upload, I’m encountering the error: “Enqueued token limit reached for GPT-4o mini.” I have 3,000 batch requests, and the token size is well under 2 million. Additionally, the file size is only around 15 MB.

[VedantParikh]

I’m experiencing a similar issue. My organization is on Tier 5, yet occasionally, even a single batch fails due to exceeding the 1,000,000 enqueued token limit. Interestingly, when I retrigger the same batch a few hours later, it processes without any problems and doesn’t encounter any rate-limiting constraints. This error consistently seems to occur around midnight.

[jaime.valenzuelasoto]

@appleprivate @int_stealar

were you able to solve this? I’m having the same issue when submitting batch jobs using the openai python client. All my batch jobs are either completed or with status failed but I still get the messages that I have reached the 2,000,000 limit.

I’m analyzing some text documents with an structured output, so I do not know if the max_completion_tokens set to a low value would help me with this.

Do you know any way to check the current enqueued tokens before submitting a new batch job? I was doing that calculating the tokens that I’m sending vs. the sum of all tokens that I have submitted and are still in progress but since the limit is TPD (tokens per day, I assume) I do not know how to check this.

[jaime.valenzuelasoto]

Same issue here. I’m programmatically tracking and calculating the number of tokens that I have submitted through my batch jobs so I check before submitting a new one, but even though my running batches do not exceed the 2,000,000 limit, I cannot submit new ones because I get the error that I have already exceeded the 2,000,000 enqueued tokens.

Is there a way to fetch this information directly from OpenAI so I can actually check before submitting new batch jobs? or I would need more precise information about how this is working.

Does anybody has more information regarding this?

[_j]

@jaime.valenzuelasoto

Do you know any way to check the current enqueued tokens

OpenAI does not have a “tokens in queue” feature to retrieve the current consumption awaiting processing.

You will need to accurately count how many tokens are being sent in a job with a good message-counting processor and the right encoder, and see if enough jobs have been completed to start the next batch job size.

The token counting may or may not be accurate, I’m not sure. The API rate limiter for normal calls just uses an estimator that can be 25% off or more depending on the language sent. So you’d leave margin.

OpenAI has several times said that “enqueued” should not act as a “per day” limit, but a “maximum that can be awaiting” limit.

You can just break the jobs up into smaller portions. If one batch being done doesn’t let you submit another, you just have to automate submitting the automation when more complete.