How to determine the token usage for a session when using stream:true?

API
1

I am using the Node.js API. Previously, when stream:false, the API would return prompt_token and completion_token (maybe these are the field names). But after using streams, I cannot find these two fields. How can I accurately retrieve them? --Translated from ChatGPT

1 Like
2

Welcome to the forum. Thanks for translating to english!

As far as I know, when stream=true, you don’t get that info and have to count it up on your own.

Hope this helps.

3

@PaulBellow is correct. You don’t get token counts when streaming. But each packet you get back is one token. Count the packets and you have the tokens for the response

There is no way to get the count for the prompt part without using a tokenizer yourself.