how to get totaltokens from a stream of CompletionCreateRequests
To see how many tokens are used by an API call, check the usage field in the API response (e.g., response['usage']['total_tokens'] ).
response[‘usage’] is null
how to get total_tokens prompt_tokens completion_tokens ?
Each response in the stream is one token. I have found that you also need to add one extra token as overhead.
There is no easy way to get the token count for the prompt part. You will have to use a tokenizer library. There are also overhead tokens for the start and end of each user, system, and assistant record.
Streams don’t pass the token count.