Gpt-4o Batch Processing Jobs Response Time Increased Significantly, Causing Job Timeouts/failures

Hi All - we have batching jobs that run nightly, last 30 days these jobs have taken 2 hours and always completed. Starting on last Friday (which coincides with the gpt-3o release) we have seen 1 success that took 16 hours and 2 full failures (jobs took longer than 24 hours to complete). Has anyone else seen a big slow down in batch processing times? If you are running batch jobs on gpt-3o, can you confirm if jobs are taking that long?

I have a hypothesis that inference resources were shifted to gpt-3o given the launch, and we should move our jobs to that model for faster processing, but Iā€™d be curious to know if anyone else has seen similar issues.

3 Likes

Yes, exact the same issue!

This started on last Friday as you mentioned, Iā€™m affected by it as well,
Response time has increased up to 12-20 hours, before it was blazing fast (within 10-30 min), and Timeouts/Failures happens a lot which is a disaster and causes loss of $ in return,

This is happening only with gpt-4o, meanwhile gpt-3o is very quick with work being processed at Batch API,

I do hope they fix it as it is impacting our work and it is a disaster right now,
For the moment Iā€™m not doing of any of work and waiting OpenAI to fix this, since it is causing loss of $ and without any work getting done.

I investigated substituting gpt-3o, but I noticed some batch parameters are not present there:
Unsupported parameter: ā€˜max_tokensā€™ is not supported with this model. Use ā€˜max_completion_tokensā€™ instead.
And Unsupported parameter: ā€˜presence_penaltyā€™ is not supported with this model.", ā€œtypeā€: ā€œinvalid_request_errorā€,

I couldnā€™t find any updated information about that. Iā€™ve opened a support ticket and will update this thread if/when I have more info.