What is the recommended approach for managing API keys when implementing a service that utilizes the ChatGPT OpenAI API with various users? The service I offer is usually to institutions, and each institution has a wide array of users.
Should API keys be assigned per-user or per-institution basis, considering the likelihood of multiple users accessing the API simultaneously? What’s the most cost-efficient option? Is there another option I’m not aware?
That’s tricky, but I wouldn’t manage it that way at all.
You want to keep track of usage, but that can be done at your layer and you can still use only one key to talk to Open AI. If it’s possible, you can then bill your users or groups of users against your metrics without having to have a bunch of keys.
I wrote a Chatbot plugin for Discourse and one of the features I added very early on was a quota system to prevent individual users spamming the bot excessively. The plugin has a trust level system so that users can be one of three levels, the most trusted of which gets the biggest quota (and access to the best model).
This is open source and the expectation is that administrators that use the plugin can monetise their membership so that they can recoup costs on an aggregate basis. The plugin uses just one key.
Your mileage may vary, but I’m sure a similar solution might be appropriate.
That’s interesting. Having only one key might make more sense as long as I have a system that can control and bill each user (=institution). It’s also straightforward. I’ll look into it; thanks.
I created a project that lets you create “proxy API keys” that come with monitoring and a quota system like what @merefield did.
You’d keep your real API key to yourself and give out “proxy API keys” to your users. That gives you monitoring + access control per user. Let me know if it helps!