Slow response time with GPT-4

Hi, in comparation with GPT 3.5 turbo, GPT-4 is much better, but response times are sometimes crazy from 30-100 seconds. With GPT 3.5 turbo I got 3-20 max ±.
Is there any way how to improve it from my side? I think its OpenAI related.
Thanks a lot

1 Like

Hi @ondrassssek,

I think with the switch in infrastructure to azure and the increase in computing power and the fact that this is a new model this is normal. I have the hope that this will improve further on :slight_smile:

Problem with API request (long answer time) - General API discussion - OpenAI API Community Forum

Maybe you can build some more resilience to fall back to another model if response times surpass a for your case acceptable duration. I think what @Jeffers did is a great thing:

1 Like

yeah, hopefully in the future. that is great idea, thanks a lot for link!