Why don’t you build an API gateway to connect/proxy all applications and devs on and get rid of multiple keys?
I mean it has many advantages other than just control in case of stolen keys. You also have a centralized log, retry logic and you can also see which application / developer does what.
And how long would it take to build? A day?
Plus: I have build a queue to never have to worry about max tokens per minute.
I simply don’t send more requests per minute.
I need another job done on that API? It runs through my API Gateway.
Centralizing the API connection does make alot of sense.
And it also has another advantage. I have set the connections to openai into a standardized class with an interface. And I have made the same with my deployed models on azure.
And there is a strategy pattern upfront that redirects the requests to different models.
Just in case anyone wonders why you should use depency injection.
At least when I am not mistaken in thinking TPM counts per account and not per key, right?