Hi there! I’m migrating from 3.5 to 4, but it’s extremely slow. I’m putting all my responses through a relay service, which has a timeout of 30 seconds. Normally this wouldn’t be a problem, but with GPT4 it often happens even small requests under 1000 tokens will take longer than that.
Any idea if this is a currently known issue? Is there anything we can do?
We can only hope it will get faster. Experiencing the same problem
Yeah, it seems to be a kind of indirect view into how poor the Microsoft Azure infrastructure services supporting OpenAI actually are.
From a business perspective it’s a tough sell when people can experience GPT4 through ChatGPT blazingly fast.
GPT3.5 was significantly faster than 3.0, so I really hoped GPT4 would be even better. So it’s not only 29x more expensive to use compared to 3.5, but at least 2x slower as well.
I really hope they’ll fix this soon and make it at least fast enough so requests will fit within the 30second window. ChatGPT can do it in 1-3 seconds already. So looks-like it’s a priority choice they made.
Might be a current problem with whole infrastructure overload, as on my side 3.5turbo is almost unresponsive as well …
Kind of seems not surprising, as I can imagine the challenge behind tackling scalability in such a short time frame with this popularity and number of users of both chat and API. But then again yeah, with Azure behind it should be better… lets hope they manage it and fix it …