Use GPT-4o-mini and GPT-3.5-turbo-0125 each to answer the same query. Sample 10 times. GPT-3.5-turbo speed is consistent (about 5 seconds for 500 tokens). GPT-4o speed is mostly slightly slower than GPT-3.5, but about 30% of the time, it somehow takes an insanely long time (19 seconds for 500 toke…

GPT-4o-mini randomly much slower than GPT-3.5-turbo

gokulraya September 4, 2024, 8:48pm 2

Thanks for flagging this. Do you have an example request_id that we can take a look to debug?

Topic		Replies	Views
GPT-4o mini slow inference API gpt-4o , gpt-4o-mini	6	209	April 9, 2025
Gpt-4o-mini is really slow API gpt-4o-mini	6	1888	March 18, 2025
Gpt-4-0125-preview INCREDIBLY slower than 3.5 turbo API	12	9553	July 22, 2024
Gpt-4-0125-preview is slower than gpt-4-0613? Feedback gpt-4 , api	5	5554	January 30, 2024
GPT-4o-2024–08–06 slower then previous version API gpt-4o	9	954	January 7, 2025