How do I track and limit API token usage for users on my website?

API
1

Im creating a website that uses API but I want to monitor the number of tokens users use on the chats so that they have to purchase different subscription tiers in order to get more usage. How do I track how many tokens the users on my website are using? Can I make them members on my project without them having access to the API account? How does it work?

2

I’m really confused by the question.

You would presumably be tracking this internally and just counting tokens in and out with each message the user sends and receives.

3

I believe that the use case is that @esl has developed a service that the users would like to use on her website. She would like to be compensated for the use of this service by her users.

However her users have different usage patterns of her service; so one size does not fit all for her users compensating her for the token consumption( hence the need for different subscription tiers).

At least thats what i understood. @esl please clarify

1 Like
4

Yes. Thanks I didn’t really know how to ask the question lol

5

I’m confused on asking the question myself tbh. I had chatgpt write it let me know if this clarifies my question.
I am developing a website that uses the OpenAI API, and I want to allow users to sign up and use this service. I need to track the number of tokens each user consumes during their interactions and have these tokens deducted automatically. Additionally, I want to offer different subscription tiers to accommodate various usage patterns and ensure users compensate for their token consumption appropriately. How can I implement this functionality so that each user’s token usage is monitored and managed through the OpenAI API, without inviting them as a “member” and giving them direct access to my OpenAI account?

6

Hi @esl

Every chat completion request response has a usage object that you can use to check input and output token usage.
For example:

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "gpt-3.5-turbo-0613",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "\n\nHello there, how may I assist you today?",
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 12,
    "total_tokens": 21
  }
}
2 Likes
7

While the following steps may seem a little overwhelming at first, you will see that they are logical.

  1. Obviously each user must “login” into your website.

  2. Each “session” of the logged in user must be monitored for token count. Let’s simplify for the time being to be a simple word count =token count. Let’s say you want the average of a week to be benchmark for each user…and you bucket by 100k tokens (S), 1M(M), 10M(L) / week.

  3. Each user starts out with a S subscription.

  4. You then need to monitor each user’s total token count per week by summing up all sessions per user and ensure that they do not exceed S subscription. If they do, the user is blocked from using service till either they upgrade or next week.

Obviously all of above needs to be built…but that’s the simple part. The payment integration and login remains the most complex part. Have you thought about how folks are going to pay you?

1 Like
8

You cannot do this, OpenAI is not an accounting firm, they do not offer a service which tracks the accounts payable of your customers.

1 Like