Supporting the BYOK (Bring Your Own Key) Model for Fair and Sustainable AI Integration

API
1

Dear Respected Software Developers,

As devoted users and enthusiasts of AI-powered applications that leverage OpenAI’s GPT technology, we understand the challenges that both users and developers encounter concerning API usage costs. To establish a more equitable, transparent, and sustainable ecosystem, we advocate for the implementation of a Bring Your Own Key (BYOK) model across all GPT-based products.

The practice of developers covering the costs of OpenAI API usage and passing them on to users can result in imbalanced pricing models. Overcharging may seem like a favorable option, but it leads to an unsustainable market, prevents AI applications from being accessible, and ultimately hurts everyone. On the other hand, undercharging can lead to developers losing money and facing financial difficulties.

If each developer takes the safe bet and overcharges to avoid undercharging, users may end up paying excessive costs, such as $2 here and $2 there per day, which can accumulate to hundreds of dollars per month across numerous AI apps. This situation not only limits the adoption of AI applications but also stifles innovation and growth in the industry.

By adopting a BYOK model, in which users bring their own OpenAI Secret Key and are billed directly by OpenAI for their usage, we can create a more balanced and beneficial system:

  1. Cost Transparency: Users will be aware of their exact usage, ensuring they pay only for what they consume.
  2. Fair Pricing: With direct billing from OpenAI, developers will no longer need to overcharge or undercharge to cover API costs, promoting a more competitive market.
  3. User Control: The BYOK model allows users to manage their budgets and scale their usage according to their individual needs.
  4. Market Expansion: As AI-powered applications become more accessible and affordable, OpenAI accounts may achieve widespread adoption similar to AppleID or Google accounts.
  5. Encouraging Innovation: Lower barriers to entry and competitive pricing will enable users to explore various AI-powered products, fostering innovation and driving the industry forward.

We urge you to consider implementing the BYOK model in your GPT-based products. This approach benefits users, developers, and the industry as a whole, paving the way for a future where AI-powered applications are accessible, innovative, and fairly priced for everyone.

Sincerely,

Tamas Simon (Sic) - Founder and CEO at Warmer Sun Education

If you agree please sign the petition in the comments at https://store.warmersun.com/p/supporting-the-byok-bring-your-own-key-model-for-fair-and-sustainable-ai-integration

2 Likes
2

Hi Tamas, reading this post I understood you are trying to make cost attribution easier through BYOK. But can this not be implemented without BYOK also. If I store the total tokens after each prompt along with the userid, we can charge the user based on the token usage.

3

Yes you are right… Isn’t it more complicated though?

4

The user might still be surprised at the costs, because internally you can change the models that are used. Suddenly the user is charged at 10x more and doesn’t know why. Also, if you use chatgpt4 and the user still doesn’t have API access that will break the product.

5

About the choice of model … only OpenAI could create a bulletproof solution, associating the key to the model. You can be transparent about which model you use. Valid point. At least the user can set soft and hard spending limits.
I need to check what you are saying about the chatgpt4
I thought about letting the user pick the model … but models behave differently…

6

Counting tokens yourself is not very reliable. In the past, I used OpenAI’s recommended tool to count them (gpt3-tokenizer) and found some minor offsets in what I counted and what I was billed. I believe I saw a line somewhere on their website saying that OpenAI may count more tokens for some other misc processing… I’ll edit my response if I find it again.

However, as thomas pointed out, this adds complexity and has it’s own load of burden. How can the user trust my count? How can you detect miscalculated counts? How do you handle overcharges/undercharges?

At the very least, there should be clear documentation in OpenAI’s usage policies, terms of service and API reference on the matter. I actually didn’t find any clear line in any policies which would suggest that it would be a violation for a third-party application to use other users’ API keys on their behalf. Maybe it’s worded in some lawyerly way I missed… I stumbled on multiple BYOK posts with replies from staff that still weren’t clear, pointing to links that no longer exist or talking about an “app review process” that was abolished in favor of a manual, per-case basis.

Nevertheless, I shouldn’t be finding such sensitive policies in a forum. I want to develop with OpenAI’s products but don’t want to risk losing my access to the platform because I wasn’t able to find a clear “green light”.

7

We are about to launch a B2B solution, where we plan to count the tokens, show them to clients in real-time so they are aware what they are paying for, and then pass the cost as it is, without any surcharge at the end of the month.

I am now inclined to move to what you have proposed for two reasons: 1) every client will pay for its own usage and decrease the risk for us in case of non-payments. 2) Limits imposed by OpenAI; I am afraid we might hit those - would we need to open additional OpenAI accounts to handle these?

I guess this is easier for B2B products, but would be more difficult for the smaller services aimed to general public. How to set up API key? (and before that - what is the API key) What am I paying and am I overpaying it? etc.

Do you have some kind of experience with this model already?

8

Still there remains a question. How does the developer generate revenue from this?

9

From what? Overcharging for the API usage or from building your own AI-powered useful services for individuals or companies?

10

What do you mean by “overcharging”?

11

devs can still make money… build something your customers value and pay for (same as always)

2 Likes
12

So you mean you build a software and charge a monthly fee and let them add their key so they also have to pay for api usage?

Well, I guess that’s what you meant by the post in the first place.

13

Yes, exactly . (You could also charge a fix one time price.)

14

This makes so much sense.

The other risk is that somebody might abuse your app and it’d be your key that gets banned by OpenAi.

Ridunkuklous!

Of course I’m totally biased because I’m working on a BYOK solution. The truth is I don’t have the skills to make some convoluted SAAS solution that isn’t even conducive to the customer experience I want to offer my users. Nay I won’t do it!

15

I see now you can give the keys a name.
I hope soon it will show up in the usage report

16

Is it even allowed to share your key with platforms?

17

No, you are not supposed to … so what we really need is some sort of OAuth

1 Like
18

I fully support every single point you made in your original post, OpenAI already supports the BYOK solution but I think there’s some issues with the current implementation.

Inexperienced users tend to forget that their API key is tied to their credit card details, meaning that they have no problem pasting their key into shady looking websites and unknown repos they’ve cloned of GitHub. They will then blame OpenAI or your business when they get scammed.

Personal use, and business use are very different API use case’s, right now OpenAI is selling the same product to both groups, the forum is full of users asking “why was I charged 5$” at the same time as business users asking for “usage limit increase”

I think this problem could be solved by just separating users automatically based on whether or not they’ve added tax & vat information to their account.

Business users need increased usage & rate limits, personal users need the exact opposite.

19

What do you mean by this?

I thought BYOK was prohibited? But I guess in a way it is “supoorted”. It’s just a matter of OpenAI actually making their policies more clear on the topic to say “you may ask your users to provide their own key, as long as you do X” or some other form.

I think we’re all capable of asking users for their API key and give them a warning about the potential implications of that, we’re just worried that it goes against an “unwritten rule” that might get the devs kicked off the platform :sweat_smile:

To be clear, while I would like OpenAI to acknowledge and accept BYOK solutions, what I mostly want is for them to make it explicitly clear in their policies whether or not this is permitted, because I have read their policy from beginning to end and cannot come to a conclusion.

It could be a case of “if its not written, then it must be allowed”. But I’m not willing to chance it :sweat_smile:

20

The information may have moved, but there used to be a section about BYOK in the usage policies: