Surely you meant 40 messages per hour right?
Nope. 40 messages a month. On gpt-4. On gpt-3.5-turbo-16k, 384 messages a month.
I’m gearing my system to let the end-user make the choice. I’m staying out of it. My audience does not consist of computer geeks who are going to be banging on the bot to outsmart it or get weird answers to even weirder questions. They aren’t developers using the AI as constant assistants. They are going to be people who occasionally have a tough question they need answered quickly. Or students who want to clarification on their practice exam answers. Until the price for gpt-4 comes down, I’m not going to spin my wheels trying to make access to it for as cheap as possible. Like someone else mentioned before, without at least a 100% markup, it’s a losing proposition.
Some of those guys on that X list are probably funded by venture capital and can afford to lose money for months. I’m not and I can’t.
If it turns out that people in this industry aren’t willing to pay for this, then I move on to the next industry.
I am telling you guys let the user deal directly with OpenAI. Don’t be a middle man.
Charge for your platform a monthly fee and let them pay their own OpenAI bill
Apps have to be a sort of “middle man” because the API key has to be kept secret.
However I agree with your premise, @jahzwolf1955 ,which is that I think doing a “pass thru” of the OpenAI expense onto the customer’s balance is best, because in that way they only pay for what they consume, in terms of their OpenAI costs. Doing this in no way precludes the app from being able to tack on the normal $4/mo (or whatever) for access to the app itself.
EDIT: I think lots of apps want to sort of shroud from the user the knowledge that there’s actually OpenAI doing the real work on the back end, so once that decision has been made you can no longer really do what I just said regarding pricing tho.
- BYOK platforms are not officially supported.
- The only official language from OpenAI has been that they are not allowed.
- The API (and hence API keys) are for developers not for consumers.
OpenAI could, at any time, do any number of things which would kill—dead in the water—any service operating as a BYOK service.
Unlikely it may be that OpenAI would simply pull the rug out from under you, it would be the pinnacle of recklessness for anyone to try to build a business on top of an unsupported service model.
Look at what happened at janitor.ai if you require convincing.
Unless you are providing a service to other devs in a B2B arrangement, there is zero reason for any business to expect their customers to go to OpenAI, create a developer account, create an API key, and share it with that business.
It’s a bad idea for the user and it’s a bad idea for the business.
No business should be jumping at the opportunity to take on so much risk! No business should want to hold (or even have temporary access to) an end user’s API key for any service.
Is there a legitimate place for services which provide a platform for users to amalgamate their own services with their own credentials? Sure, in very limited circumstances. This isn’t one of them—especially because it isn’t a business model officially supported by OpenAI.
Until such time as OpenAI officially green-lights this model, it is unethical to recommend anyone adopt such practices.
1 Like
So I love you guys but it is amazing how daft some of you are acting…
You guys - OpenAI charges you .00003 (average) cents per token ( this is what i calculated it to, calculate it for your own business, just take all of your messages and find the average .
If OpenAI charges you for every token, and you cannot know how much a user will use your service, AND you cannot know how OpenAI will tokenize your message because theres almost 0 transparency on how to predict that, than that means you should charge your user (going off the hypothetical average) .00006 cents per token.
It is that simple. That is it. There is no other math.
If you want your Gold Bronze Purple Rainbow tiers to be signifcant, add features for those tiers. OR mark your tokenage up even more than 100% and come down some for higher tiers to appear like its a better deal. I dont know
But this whole idea that you will maintain profitability without profiting on every single token, without knowng how much a user will use it and what for, is erroneous.
So lets say you DO have some user who is a poop mouth that starts using your bot as a chat bot, somehow breaks it, and even though its meant for real estate - now you’re getting charged out the ass and have to go find the user and delete him.
Theres so much extra work with all these tiers.
Just charge them on every god damn token and you never lose money, its that simple.
How is that not what ALLLL of you want? to eliminate all bleeding… riiiiight? If theres no bleeding, theres only profit…
1 + 1 does indeed = 2
Edit: If you have a freemium model like me - you’ll have to navigate how much to let your freemium tier use, so there will be bleeding from that, but thats just sort of a given with SaaS
1 Like
How about you just profit on every token, and never have to worry about it…ever again?
So if you calculate your average message tokenage - a message would count as the payload from your API, it counts as both the user and bot prompt and is returned once the bot responds to the user in this JSON object…
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1677858242,
"model": "gpt-3.5-turbo-0613",
"usage": {
"prompt_tokens": 13,
"completion_tokens": 7,
"total_tokens": 20
},
"choices": [
{
"message": {
"role": "assistant",
"content": "\n\nThis is a test!"
},
"finish_reason": "stop",
"index": 0
}
]
}
store the usage.total_tokens for each users uid in your database. I use Firefoo a firebase GUI to track the usage and query it by user. At the end of each month either manually bill a automatic payment invoice through your payment processor or have logic set up that tracks and bills their tokenage through your payment processor. Stripe offers usage tracked billing but I was too stupid to set it up.
For each tier, mark up the tokens as it makes sense. So maybe the gold tier is marked up only 100% , but the bronze tier is marked up 300%, and it only appears cheaper because the monthly fee is lower. I’m not sure - there’s some math there, but the point is you should never not be profiting on a token.
I too went through this just the other week and the only way to profit is through what I’m telling you. And you’re convoluting it. It’s not your problem if a person whom is over 18 (which is a disclaimer you should have, or in your TOS) overuses your product and amasses a bill they can’t pay. In the same way it’s no one else’s fault when you overuse your credit card.
And also, it’s totally up to YOU what you do if they can’t pay - you can be as harsh or as lenient as you want. They can’t pay but are telling you they got you in two weeks? Be lenient if you want. Their card declines? Shut off premium access and tell them when they pay it will be reactivated. It’s all up to you. But just because its set up like a credit card payment doesn’t mean you have to be as harsh as a credit card company
Not sure how the above squares with the below
So, I am assuming from this:
You mean markup the token cost to end user ?
I have an extensive background in database development, so I’ve been tracking user activity, token usage per user per model since day one. In addition, each message involves at least 3 api calls (concept, standalone and completion), and all of those tokens are added up. Billing won’t be the problem. Pricing is the issue for me right now.
Honestly, my hope is to license the whole thing to another entity and let them worry about tokens so I can move on to the next project. But, if I have to go into the consumer marketplace myself, I’m just trying to be as educated and prepared as I can be.
So, what I have so far is: mark up tokens 100% to 300%. Keep basic tier as simple as possible so it doesn’t take more than 5 seconds to understand. Be as upfront and transparent as possible (I’ve always done that). Offer additional service for the price.
Yes.
Maybe its 1000% - How much do you want to profit? you’ll have to run those numbers. I was saying at a minimum I think 100% mark up
Wait until you have enough data to see how much the average message is (so for you as you stated this would be the 3 API calls) and find a way to tell them roughly how much it would be. If your not using it as a chatbot and messages aren’t compounding that should be pretty easy actually. But I see your point now. If you want to limit to 40 messages a month you can do that but all the tokenage still has to be billed.
1 Like
Been unit testing now for a little over 2 months, plus feeding it real world exam questions, so I’ve got a very good idea of what each message, query, round trip around the model, is going to cost.
Thank you for the comments. Very helpful.
Yes, I do want to profit. But, I also want to make it affordable for most people who need / can use it. Guess we’ll see.
Something to consider: users will be attracted to the service that gives them the most value for their dollar.
So put in place two different systems on your site:
- pay by question
- pay by data
You’ll find you’ve auto selected for minimum margin, or vs competitors, attract the customers that your tiering appeals to most.
Making it optional perhaps
My focus has always been encouraging to build their own bot
But if it were a App Store app I can
See offering a way to as you say
Pass thru or bring your own key
But that being discouraged by policy
Makes it harder to not be a middle man
Offering a way to enter your OpenAI
Key in the app I think is the simplest
Solution
Without being able to code that
And no authentication API available
What choice does the App Store app have ?
My option lets them build their own bot
And put their key in what would be the difference with a App Store app offering it?
So far I haven’t heard any noise about it
And I suspect I won’t either because I offer
Free 3.5 online and 4 password protection
I encourage building your own server
I discourage ongoing casual use of my prototype for demonstrating a new bot
So far it has worked and I see people building it. Mission accomplished keeping robots out have been a issue.
App Store apps I wonder about their usage numbers
quite is twisty little passage
The forum discussion revolves around determining an optimal pricing model for a GPT-4 chatbot. Users shared various methods used:
tventura94 reduces the prompt from 1400 to 500 and considers a usage-based pricing model with a limit. They’re currently using a subscription model via Stripe’s webhook and Firebase.
wclayf operates a prepayment system, deducting users’ credit as the chatbot responds, using token counts from the API returned JSON to calculate costs. This model prominently displays user’s remaining credit on the site, preventing credit extension, and shares “current average” input/output costs. Payment processor details are not specified.
tventura94 also contemplates a ‘pay-as-you-go’ approach with unlimited margin that allows customers to set their own limits, billing them monthly but frequently updating them about their usage. They compare the model to a cell phone service, having a regular monthly charge along with a separate fee for usage.
elmstedt suggests charging 2-3 times the average usage per user to account for unexpected cost increases as usage grows.
RonaldGRuckus recommends a basic tier that covers the worst-case scenario of typical usage with rate-limits to prevent loss.
SomebodySysop utilizes a basic tier at $20/month for 40 messages at a gold level, with a fine print that gives details for the same amount at lesser levels.
Across the board, there’s agreement on maintaining transparency about charges with customers and ensuring the pricing model is not overly complex.
Summarized with gpt-4-32k on Sep 14
(interesting that from 54 replies, AI summarized participants and not more overall sentiment.)
1 Like
Totaling month to date usage per user per model. I’m not separating prompt and completion tokens.
I’m also adding in tokens used in all api calls associated with one message.
Anything I’ve overlooked here?
Array
(
[model1] => Array
(
[cost_per_1k] => 0.1
[total_tokens] => 164388
[total_queries] => 58
[total_cost] => 16.44
)
[model2] => Array
(
[cost_per_1k] => 0.01
[total_tokens] => 115578
[total_queries] => 52
[total_cost] => 1.16
)
[model3] => Array
(
[cost_per_1k] => 0.06
[total_tokens] => 267022
[total_queries] => 119
[total_cost] => 16.02
)
[model4] => Array
(
[cost_per_1k] => 0.04
[total_tokens] => 55892
[total_queries] => 25
[total_cost] => 2.24
)
[model5] => Array
(
[cost_per_1k] => 0.02
[total_tokens] => 41542
[total_queries] => 13
[total_cost] => 0.83
)
[Grand Total] => Array
(
[total_tokens] => 644422
[total_cost] => 36.69
)
)
I say let the user deal with OpenAI charge for your app a one time fee to own the code.
1 Like
not a bad idea actually, i think maybe might incorporate this in the enterprise model
3 Likes
I’ve been working on a pricing model where the normal subscription is supplemented with a smart throttling mechanism. The throttling system basically applies an incremental throttling to the fewest number of users, only at peak usage times, to reach the spending limit. The user has the option to remove the throttling for 24 hours by paying a fee.
I spent a lot of time thinking about different ways to make the pricing work, but this was the only way I could think of to guard against a few power users making the costs very unpredictable. I also wrote a whole LangChain based SDK around it to to make its integration simpler.
curious what everyone thinks about it?
1 Like
This seems the way to go I would even offer an uncap subscription if a credit card is saved and options like charge every 24 hrs, add usage limit
I am not sure if you re willing, but can you share how you built your credit based payment system?
I’m trying to do the same and have been running around in circles for about 20 hours now.
I’ve got a Python langchain open-air APIs based chatbot and Al wondering if it’s just langchain screwing me up at this point.