We are about to launch a B2B solution, where we plan to count the tokens, show them to clients in real-time so they are aware what they are paying for, and then pass the cost as it is, without any surcharge at the end of the month.
I am now inclined to move to what you have proposed for two reasons: 1) every client will pay for its own usage and decrease the risk for us in case of non-payments. 2) Limits imposed by OpenAI; I am afraid we might hit those - would we need to open additional OpenAI accounts to handle these?
I guess this is easier for B2B products, but would be more difficult for the smaller services aimed to general public. How to set up API key? (and before that - what is the API key) What am I paying and am I overpaying it? etc.
Do you have some kind of experience with this model already?
The other risk is that somebody might abuse your app and it’d be your key that gets banned by OpenAi.
Ridunkuklous!
Of course I’m totally biased because I’m working on a BYOK solution. The truth is I don’t have the skills to make some convoluted SAAS solution that isn’t even conducive to the customer experience I want to offer my users. Nay I won’t do it!
I fully support every single point you made in your original post, OpenAI already supports the BYOK solution but I think there’s some issues with the current implementation.
Inexperienced users tend to forget that their API key is tied to their credit card details, meaning that they have no problem pasting their key into shady looking websites and unknown repos they’ve cloned of GitHub. They will then blame OpenAI or your business when they get scammed.
Personal use, and business use are very different API use case’s, right now OpenAI is selling the same product to both groups, the forum is full of users asking “why was I charged 5$” at the same time as business users asking for “usage limit increase”
I think this problem could be solved by just separating users automatically based on whether or not they’ve added tax & vat information to their account.
Business users need increased usage & rate limits, personal users need the exact opposite.
I thought BYOK was prohibited? But I guess in a way it is “supoorted”. It’s just a matter of OpenAI actually making their policies more clear on the topic to say “you may ask your users to provide their own key, as long as you do X” or some other form.
I think we’re all capable of asking users for their API key and give them a warning about the potential implications of that, we’re just worried that it goes against an “unwritten rule” that might get the devs kicked off the platform
To be clear, while I would like OpenAI to acknowledge and accept BYOK solutions, what I mostly want is for them to make it explicitly clear in their policies whether or not this is permitted, because I have read their policy from beginning to end and cannot come to a conclusion.
It could be a case of “if its not written, then it must be allowed”. But I’m not willing to chance it
Ah yes, I’ve stumbled upon this one. Seems like they moved it when the plugins came out. It’s too bad because it really seems like they actually adressed this, but erased it (probably be accident). Even tried looking it up in Wayback Machine
Indeed,
There might be some legal reasons why they won’t tell you, I assume it’s a question of liability, if OpenAI specifically tells you that you can create BYOK applications, they may be liable if a user gets scammed and decides to sue them.
Hmm I’m confused - BYOK (Bring your own Key) stands for bringing you own encryption Key, no you license Key!
Wonder if OpenAI plans to integrate BYOK to encrypt customer related training data (Would be mitigate a lot of security concerns and helps to fulfill GDPR).
When you have questions, please feel free to contact me.
you are right… it’s a term in cryptography. I haven’t thought about that…
I think the solution would be some sort of OAuth
the user authenticates to OpenAI and authorizes the 3rd party developer’s application to use the API on their behalf. OpenAI should then bill per usage.
That way the developer does not have to pay the bill to OpenAI and the end user does not have to pay 20$ here 20$ there…
Actually, I created a website that allows you to track users’ tokens through an API, there’s no need for counting tokens yourself or for BYOK. The API will be a layer between your app and OpenAI, it’ll forward your requests over to their Chat API and track token usage for you automatically in the same request. You can then get each person’s usage through our Usage API. Check it out and lmk what y’all think: https://www.tiktokenizer.dev/
