Hi All - we have batching jobs that run nightly, last 30 days these jobs have taken 2 hours and always completed. Starting on last Friday (which coincides with the gpt-3o release) we have seen 1 success that took 16 hours and 2 full failures (jobs took longer than 24 hours to complete). Has anyone else seen a big slow down in batch processing times? If you are running batch jobs on gpt-3o, can you confirm if jobs are taking that long?
I have a hypothesis that inference resources were shifted to gpt-3o given the launch, and we should move our jobs to that model for faster processing, but Iād be curious to know if anyone else has seen similar issues.
This started on last Friday as you mentioned, Iām affected by it as well,
Response time has increased up to 12-20 hours, before it was blazing fast (within 10-30 min), and Timeouts/Failures happens a lot which is a disaster and causes loss of $ in return,
This is happening only with gpt-4o, meanwhile gpt-3o is very quick with work being processed at Batch API,
I do hope they fix it as it is impacting our work and it is a disaster right now,
For the moment Iām not doing of any of work and waiting OpenAI to fix this, since it is causing loss of $ and without any work getting done.
I investigated substituting gpt-3o, but I noticed some batch parameters are not present there:
Unsupported parameter: āmax_tokensā is not supported with this model. Use āmax_completion_tokensā instead.
And Unsupported parameter: āpresence_penaltyā is not supported with this model.", ātypeā: āinvalid_request_errorā,
I couldnāt find any updated information about that. Iāve opened a support ticket and will update this thread if/when I have more info.