OpenAi API - get usage tokens in response when set stream=True

The workaround I used was to get Chat GPT to create a paragraph in each language and then use the online tokenizer to calculate the number of tokens for each paragraph then divide by the number of characters in each language paragraph.

That gives me an average character to token ratio for reach language that I can store in a table.

I did this on a per language basis because the average number of characters per token can vary dramatically by language.

I’m then able to guestimate how many tokens would be used for any arbitrary number of characters. It’s not 100% accurate but it is very fast and it’s close enough (90%-ish).

Of course this does require you to know the language of the text up front and it wont work if the text has multiple languages.

In my case I’m only using it to roughly guesstimate how much each conversation costs.

1 Like

guys, unable to measure tokens usage is a special feature because we then can focus on the code, instead of the bill :rofl:

2 Likes

Yeah, why just not include “usage” section as described with non streamed response in final chunk just before [DONE]

1 Like

+1, really unfortunate that we are not getting that information.

1 Like