Slow response time with GPT-4

[linus]

Hi @ondrassssek,

I think with the switch in infrastructure to azure and the increase in computing power and the fact that this is a new model this is normal. I have the hope that this will improve further on

Problem with API request (long answer time) - General API discussion - OpenAI API Community Forum

Maybe you can build some more resilience to fall back to another model if response times surpass a for your case acceptable duration. I think what @Jeffers did is a great thing:

Problem with API request (long answer time)

I’m going to start monitoring the 3.5 Turbo response times across the server and when there are too many “curl_setopt($curl, CURLOPT_TIMEOUT, 30);” timeouts, I’ll revert to Davinci 003. or GPT4. It will be interesting to see how I can switch/convert the Turbo chat style resprequests into the completion style.