Billing Management System

API
, ,
1

Hello,

We are in the process of creating a product that utilizes the OpenAI API. This product will be offered to end customers who will use it. Our goal is to keep track of how our product is being used, and we intend to charge end customers based on their usage of the product. To achieve this, we are looking for a solution to effectively monitor and bill for the usage of our product integrated with the OpenAI API. If you have any recommendations or insights on how we can implement this, please feel free to share your thoughts.

2

The API sends back as part of each “reply” the input and output token counts, and you can convert those to USD using their pricing per token page.

2 Likes
3

However, when streaming - displaying words as they are generated and how you’d want many customer-facing products to operate - you don’t receive that token count.

You can calculate tokens exactly with an encoding library, tiktoken.

Then you can determine if it is more practical to charge by your exact billing, or whether instead to present per-query costs, per character, per month with quota and purchase of boosters, etc.

4

Yes API sends input and ouput tokens that pricning will be charged by Open AI for the tokens which i consumed but here i want sytem /application where i need to monitor the input/output token consumed by my cusomter through our product

5

In reading this thread, I realized that its safe just to count the tokens used by the client from each response and then sending them up to your database for this user’s “used tokens count”. For those who just use the full completions. I’ve never used streaming so I cannot comment on that.

But for anyone is looking for billing management. Just count the tokens and sync them with your billingDB. I suppose its possible that some token counts don’t get added, but right now this seems to be the only way.

6

Late to this thread, but - are you looking to only manage the accuracy of your users’ usage, or also the timing of payment? i.e are you planning on billing them monthly on their pay-as-you-go bill, or is it more of a prepaid credits/tokens charge? (or even a more straight forward account-to-account setup without waiting till end of month)

7

I’ve not actually implemented this yet, but here is my plan:

  1. My system keeps track of total tokens used per query per user.
  2. I plan to use Stripe for recurring monthly billing which will consist of a monthly fee + $$ credit. This credit is applied user’s running balance.
  3. Based upon the model(s) token rate, the system will deduct appropriate $$ for each query from user’s balance.
  4. If the user’s balance reaches 0 before end of month, he has option of “refill”.
  5. If the user has a remaining balance at the end of the month, it is rolled over into the next month with additional $$ credits.

Seems pretty simple. Everything is in place to track credits and usage, just haven’t implemented the recurring billing yet.

1 Like