How can I split the usage of my subscription for different bots?

Community
1

So I want to create different bots, and track usage (tokens/money spent) per each bot.

So right now my subscription under “personal”… I think there is another type of subscription?

Anyway… is there a way to track usage among different bots under the same subscription?

I think I can generate more than one API key… if I create 2,3, 10 different API keys, would I be able to track usage per each API key?

2

Best way to think of this is electricity.

You are an electrical distribution supplier and you get your power from the power station, the power station does not monitor what you do with your power, they just monitor how much you use and bill you accordingly. It is up to you to build and maintain a billing system for your customers.

To this end, I personally find having an API relay server to be invaluable, my clients authenticate with that server and that server then uses tiktoken to count tokens in and out and store that usage to my company database, I then run my monthly accounts code to count up what each customer has used and produce bills for them/do nothing as they pay me a fixed monthly fee, and create analytics.

So don’t rely on the API billing system to monitor your customers usage, your customers are yours and you should monitor their usage through your infrastructure, it gives you infinite flexibility. You should also build some accounting system that detects discrepancies between customer usage and your total API bill to track issues and usage in general.

1 Like
3

that is true, although it would be interesting to see how the community generally handles this. it feels like there should be an integrated solution for this on the market somewhere.

for us, for now, we use firebase (firestore/auth/functions) as middleware to log and monitor user incurred costs.

4

I think llamaindex and langchain have this functionality along with a few other usage managers, but it’s not a terribly complex thing to implement.

5

well, that’s kind of true, but also not. there’s a lot of considerations when you start monetizing, you can’t just have user.moneys+=tokens*0.003.

i mean you could, and can, of course, I guess. but I could imagine that it would suck if you can no longer reconcile your balance sheets. obviously depends on what’s at stake.

6

so long as you make an entry in a ledger saying at {unix_time} {client_number} {input token count} {output token count} and maybe the message content … you can build any level of complex accounting on top of that that you wish, that raw data will allow you to reconstruct everything.

1 Like
7

Ask for that. That OpenAI keeps track of your customers. That OpenAI keeps track of their conversations. That OpenAI provides you a python sandbox to run their code.

Pretty soon you are just asking “how can you guys run ChatGPT but I get some of your money?”

1 Like
8

I love your usual snark @_j, but I was thinking of a general api management solution on the market somewhere, not provided by openai. I hear that mulesoft/hypercurrent can do stuff like this, but it doesn’t seem to be geared towards small business.

Still interested to hear how everybody else manages this.

9

I mean… openai is already selling the tokens, and allows you to create so many api keys… I would think they might already have that information… I wouldnt be surprise if they start offering it under the settings… that is why i was asking…

10

You can go right to the way OpenAI bills you:

Per-API-call addition of charges to a user’s bill might seem excessive, but based on how credits take time to work on API if there is a payment delay, and how overages are allowed sometimes, I think that might well describe the level of API interaction between OpenAI and Stripe.

(not the way I would do it though)

11

what you can do is add users to your org, and then under billing you can see a daily usage breakdown by user. but it’s not super fun to use.

1 Like
12

how would you do it? 25char

13

Local customer and transaction DB, and all the other data support you’d need.

Check the user is authorized or has balance before submission,
Record API call with token usage, etc in transactions
Update runtime customer balance, Record old/new balances in transactions.
Chat history, security logs, moderation logs, ban system, etc

1 Like