Any API end point to display token usage for the query

I am working on a tool to make API call to display response on the wordpress post. I want to know if there is any API endpoint which is available (token counter?) which I can use in my script show that it displays number of tokens used in that query on the same page.

Also, is there any option that I put max tokens usage for each API key so as soon as it hits that cap, it stops working so that I can ask my customers to upgrade to a different or higher plan. Thanks

1 Like

Yeah the current API displays the total of tokens used in the returned response.

"usage": {"prompt_tokens": 23, "completion_tokens": 1, "total_tokens": 24}}

As far as I can tell, it is across fine-tuned models and normal models as well.

1 Like

That is fantastic. I will definitely try to use these endpoints and see how I go. Thanks a lot.

Still looking for a second question… is there a way to put a cap on each API for my customers?


Yes, you can cap it for each user when you send the request. Set max_tokens to the max allowable total tokens, which is the sum of input + output.

Thank you so much. Sorry if I didn’t specify my question properly. I wanted to know if there is anything I can do so that a particular API stop working as soon as they use their total tokens (whatever I allocate them when they sign up on my website). For example, if I have allocated 20K tokens a particular API key, as soon as that key consumes that total token (e.g. by searching for multiple prompts in a week or so), they get an error that you have consumed your monthly quota and you need to upgrade your account to continue using the tool. Thanks

That is something you will have to track with your own database. Not aware of any such tracking built-in to the current API.

1 Like