Yeah, I’m not privy into the OpenAI infrastructure, but my best guess is that adding 100 million users since December 2023 has really hurt the infrastructure.
Also, OpenAI quickly rolled out the “paid” versions, and I got the feeling that the work flow and internal processes for rate limiting in both the OpenAI cloud and in Cloudflare are fragile, at best.
So, like many companies, OpenAI is in a firefight right now trying to make money from users so they can invest more. Even the recently announcements of secured funding will not immediately solve these problems.
Also, it may be possible that their are “per country” quotas set up behind the scene (I once set this up for an app), and so since I am developing and testing from a supported “small country in Asia, with a relative small tech base” my interactions (API or direct) may be different than in large countries with a lot more developers, etc.
For example, when a open ChatGPT session expires and I get the silly chatbot message the system is overloaded to try later, I simply reload once or twice an “I’m in”.
On the API side, I never seen any errors.
This leads me to believe their many be some “geo-based” throttling possible, but of course, I have no way to confirm this, and I don’t access OpenAI via a VPN, TOR, etc. and I’m in a “supported” country.
On my end, all I have had to do was add a little sleep to my loops, and have never had an error since (which “someone” has totally berated me about, haha)
Sleep is good 