Why changing organization in the header affects the Open AI api response speed?

I use Open AI api chat completions
I have two organizations X and Y. X was created around year ago and Y is created recently. When I use the key Sk-* with OpenAI-Organization header set to X, I get the response in 1 second in average. But when I use the same key but with OpenAI-Organization header set to Y, I get response in 4-5 seconds in average.
Does anyone know how to make the key faster when used under organization Y. And why does the organization affect the speed of the response

Hi and welcome to the Developer Forum!

IS Organisation X using the API more regularly than Y? If Org Y is in a lower usage tier than X, then that would explain it.

Hello, thank you for mentioning this.
Yes, Org Y just started using API, so the usage is much less than Org X..
I did not see any mention of that before, but I suppose that the more of API calls the Organization has made, the faster the API response, is this right?
Is there any solution of how to speed up the API response for the Organization Y if it has small usage for now?

OpenAI has put everyone that hasn’t ponied up and prepaid big money in unrefundable credits that expire after one year into the rubbish bin of AI models.

Look at your rate limits in your account. If you are under 80000 TPM for gpt-3.5-turbo you are also on their “useless freeloader” list.

Click on the “about rate limits” to see new guidance about newly-invented tiers to punish those who haven’t transferred money to OpenAI for future services (that may continue to be lower quality than today).

Answer: You need to hand over $50+. Then you have to wait to see if “they may” move you to back to the “lower latency” faster models you were accessing.

1 Like

I see, so, if I get high TPM like 1.000.000 I will get fast responses.
Thank you for your answer!
I wonder why there is no mention of this anywhere, because I could not find it.

Why was there no email sent to every API user about their account degradation, a system that wants money for services not yet rendered transferred into OpenAI’s bank account to the tune of $1M per 20000 users, one built for breakage, a “LOL you’re banned when it was most profitable for us, SOL” system? Because it’s shady as hell.

They finally implemented a system to show our tiers. Look at the bottom of the limit page. It has become clearer now. Become tier 2 to get the normal speed of gpt 3.5.

3 Likes

Good spot!

The presentation is less obfuscated, but still doesn’t say “you are slowed”. You have to click-through more documentation for “As your usage tier increases, we may also move your account onto lower latency models behind the scenes.” to explain the slowdowns you were hit with, and now there are more tiers listed to potentially bin to lower performance. As tier 5 doesn’t show different rate limits, we assume it has other anticipated differences that can be assigned to accounts.

The hard limits page within billing overview also has been linked back to the enhanced main rate limits page.

They bumped up my monthly account’s limit, and also improved the disparity between 5000 RPM gpt-3.5 and 200 RPM gpt-4 I had.