I agree I would like to have the usage in the response to stream requests (at least in one of the events in the stream), similar to usage for non stream requests.
However, a solution for the meanwhile -
- The number of prompt tokens can be calculated offline using tiktoken in python for example (This is a guide I used)
- The number of the events in the stream should represent the number of the tokens in the response, so just count them while iterating,
- By adding 1 and 2 together you get the total number of tokens for the request.
I want to stress out that I would still like to receive the usage in the response to the request and not have to compute it myself.