Calculating token usage with streaming?

API
1

I’ve written an OpenAI integration for WordPress, for my client. They have a large number of users using it, and they’d like me to develop a way of showing who is using the most tokens, so they can come up with a pricing model for using this tool.

If I knew the token usage for a given API call, I’d stash it in the database and be in position to offer some nice usage on reporting.

However, 100% of my work uses the “streaming” option for the OpenAI API. Given that, I’m having trouble seeing how I can conveniently keep track of the token usage for a given request.

Am I missing something obvious here?

2

Hi and welcome to the Developer Forum!

Yes, you are missing the Tiktoken library and running that against the returned deltas, i.e. every time you get a delta packet returned, it is typically going to be a single token, but from time you time you may get several so just run each one against toktoken and keep a running total, then write that total to a db when you are sure the call has ended with a finish reason.

1 Like
3

We have now launched support for this! See: Usage stats now available when using streaming with the Chat Completions API or Completions API