Batches don't work at all

The Chat GPT batch is not working at all. It used to work fine with the same code, but now it’s not functioning at all and has been down for three days now.

I thought there might be an issue with my code, so I tried uploading previously validated data directly on the Open API website, but that isn’t working either.

After waiting 24 hours, I get an “Expired” error, with the message: "error": {"code": "batch_expired", "message": "This request could not be executed before the completion window expired."}}.

I requested assistance through messages, but they haven’t responded. A few other people seem to be experiencing similar issues, but there’s no solution in sight. What should I do?

model: gpt-4o-mini

3 Likes

After conducting multiple tests, it seems that this issue only occurs with the “gpt-4o-mini” model. The more expensive models work fine. However, my business is still in its early stages, and I can’t afford the cost of the pricier models, so operations are currently on hold.

1 Like

I, too, am having issues with batch API with gpt-4o-mini. Normal chat completion works. gpt-4o batch API works. Just not batch API with gpt-4o-mini

1 Like

Yes, it’s stopped about 2 or 3 days ago until now, I have to switch to the completion until OpenAI fix it

Same for us too. gpt-4o-mini batch not working at all. We tried a test batch with gpt-3.5-turbo-0125 and that processed fine so it does indeed seem to be model-specific.

All I see on the incidents page is a mention of “Increased latency in the API on GPT-4o mini” which was apparently resolved on 17th October.

It’s disappointing when we have valuable business processes dependent on this. Due to this (plus a previous batch taking >24hrs and being cancelled) we are now starting to investigate using Gemini as an alternative or back up. They offer a batch service through their API.

Based on this line in the documentation:

Batches that do not complete in time eventually move to an expired state;

It seems implied that there’s no guarantee that the batch will complete. The only guarantee given is that your batched items will be held in a queue for a total of 24-hours.

In all LLM batch systems you are in some sense gambling for an open window of low-demand for that sweet 50% discount.

If there is no low-demand in a 24-hour period then you have to try again.


If batching is a big part of your process it may be useful to batch single items throughout the day to gauge a rough understanding of what to expect.

1 Like

Thanks @RonaldGRuckus - and yes I have the same understanding as you that the 24hr ‘SLA’ has no guarantee. It was just unexpected for us.

I like your idea about periodically testing the batch API for response measurement. Clever.

1 Like

I’m having the same issue, support doesn’t seem to care.

I just started using GPT API, is that common or it’s just bad luck?

I understand the nature of the cheap/not-granted service, but I need some reliability

2 Likes

I have same problem for gpt-4o since last 2 days. The batches get expired after 24 hour window.

1 Like

Hey folks, I had encountered the issue as well, then I realized that it is still working, but only for some requests with short prompts.

A batch like the following would be able to complete within a couple minutes:

{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "What's the capital of Japan?"}],"max_tokens": 1000}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "system", "content": "You are a sarcastic assistant."},{"role": "user", "content": "What's the capital of US"}],"max_tokens": 1000}}

However, the major type of prompts that I am working with right now are much longer the ones appeared in the above example (and my batches tends to have 100-200 entries as well)

1 Like

Maybe I’m spoiled in North America, but I send my batches in the evenings and weekends, usually after 8pm on weekdays, west coast. I’ve never had a job take even one hour to complete. Most are 10-20 minutes tops. My batches are usually 500-5000 calls each, perhaps I’m not hitting it as hard.

One thing I will recommend is always use full model strings w/ Batches API. I’ve encountered some model-string bugs recently, and you want to get the right models completing your calls. Although, given that there is only one snapshot model of 4o-mini, that one shouldn’t cause problems, theoretically…

1 Like

You’re absolutely right—I tried your requests, and they finished within a few minutes. I’ve already reached out to support, but I haven’t received any help yet. It’s reassuring to know that shorter requests still work, but I really need a solution for longer ones. Everything was working fine just a week ago.

1 Like

Hey team, thanks for flagging this. I’m investigating this now, will update here once I have more info.