The Chat GPT batch is not working at all. It used to work fine with the same code, but now it’s not functioning at all and has been down for three days now.
I thought there might be an issue with my code, so I tried uploading previously validated data directly on the Open API website, but that isn’t working either.
After waiting 24 hours, I get an “Expired” error, with the message: "error": {"code": "batch_expired", "message": "This request could not be executed before the completion window expired."}}.
I requested assistance through messages, but they haven’t responded. A few other people seem to be experiencing similar issues, but there’s no solution in sight. What should I do?
After conducting multiple tests, it seems that this issue only occurs with the “gpt-4o-mini” model. The more expensive models work fine. However, my business is still in its early stages, and I can’t afford the cost of the pricier models, so operations are currently on hold.
Same for us too. gpt-4o-mini batch not working at all. We tried a test batch with gpt-3.5-turbo-0125 and that processed fine so it does indeed seem to be model-specific.
All I see on the incidents page is a mention of “Increased latency in the API on GPT-4o mini” which was apparently resolved on 17th October.
It’s disappointing when we have valuable business processes dependent on this. Due to this (plus a previous batch taking >24hrs and being cancelled) we are now starting to investigate using Gemini as an alternative or back up. They offer a batch service through their API.
Batches that do not complete in time eventually move to an expired state;
It seems implied that there’s no guarantee that the batch will complete. The only guarantee given is that your batched items will be held in a queue for a total of 24-hours.
In all LLM batch systems you are in some sense gambling for an open window of low-demand for that sweet 50% discount.
If there is no low-demand in a 24-hour period then you have to try again.
If batching is a big part of your process it may be useful to batch single items throughout the day to gauge a rough understanding of what to expect.
Hey folks, I had encountered the issue as well, then I realized that it is still working, but only for some requests with short prompts.
A batch like the following would be able to complete within a couple minutes:
{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "What's the capital of Japan?"}],"max_tokens": 1000}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "system", "content": "You are a sarcastic assistant."},{"role": "user", "content": "What's the capital of US"}],"max_tokens": 1000}}
However, the major type of prompts that I am working with right now are much longer the ones appeared in the above example (and my batches tends to have 100-200 entries as well)
Maybe I’m spoiled in North America, but I send my batches in the evenings and weekends, usually after 8pm on weekdays, west coast. I’ve never had a job take even one hour to complete. Most are 10-20 minutes tops. My batches are usually 500-5000 calls each, perhaps I’m not hitting it as hard.
One thing I will recommend is always use full model strings w/ Batches API. I’ve encountered some model-string bugs recently, and you want to get the right models completing your calls. Although, given that there is only one snapshot model of 4o-mini, that one shouldn’t cause problems, theoretically…
You’re absolutely right—I tried your requests, and they finished within a few minutes. I’ve already reached out to support, but I haven’t received any help yet. It’s reassuring to know that shorter requests still work, but I really need a solution for longer ones. Everything was working fine just a week ago.