I’m building a BYOK application and do not have a way to determine which rate limits apply to a given user. In other words, I have no way to distinguish if a user’s API key is on the free-trial tier, or pre vs. post 48 hours pay-as-you-go tier).
I suppose it’s possible to write some code that infers the rate limits based on which rate of sending requests is successful or not, but that seems awfully complicated and it would be much simpler if OpenAI could expose an endpoint that takes an API key and returns the RPM/TPM rate limits that apply to that user for the various endpoints.
Having an API endpoint like this would also allow OpenAI to change their rate limits going forward without breaking end-users’ applications since we would then be able to dynamically adjust our rate limiting code instead of hardcoding the rate limits.
My application will be sending large amounts of data to the embeddings endpoint in batch processing jobs, and so optimizing for throughput is paramount for me.
Here’s OpenAI’s rate limit table for reference:
