OpenAi API - get usage tokens in response when set stream=True

API
21

The workaround I used was to get Chat GPT to create a paragraph in each language and then use the online tokenizer to calculate the number of tokens for each paragraph then divide by the number of characters in each language paragraph.

That gives me an average character to token ratio for reach language that I can store in a table.

I did this on a per language basis because the average number of characters per token can vary dramatically by language.

I’m then able to guestimate how many tokens would be used for any arbitrary number of characters. It’s not 100% accurate but it is very fast and it’s close enough (90%-ish).

Of course this does require you to know the language of the text up front and it wont work if the text has multiple languages.

In my case I’m only using it to roughly guesstimate how much each conversation costs.

22

guys, unable to measure tokens usage is a special feature because we then can focus on the code, instead of the bill :rofl:

24

Yeah, why just not include “usage” section as described with non streamed response in final chunk just before [DONE]

25

+1, really unfortunate that we are not getting that information.

26

+1 to this - it’s not clear whether the number of streaming chunks corresponds to number of tokens and I don’t want to estimate how much was used per request, ideally. Please include this in the final chunk!

27

+1 - tiktoken workaround does not work when you include images in the messages. Include usage in the streaming response.

28

+1. Has anyone found a solution for this? Is the answer just to calculate token usage offline with tiktoken? Seems pretty counter-intuitive to not include the tokens used for the streaming api.

29

I finally found a solution I’m happy with, after hours of scouring documentation. Unfortunately, they do not give an option to query for usage information by ID, or even just returning usage somehow; that would’ve been the easier solution. Instead, here’s my implementation. It involves:

  • Counting tokens for images with the new gpt-4-turbo/vision models
  • The scuffed and varied additional tokens that get added in with openai’s api
  • Wrapping the returned Stream generator, appending any tokens to a list before yielding, and finally processing the list as the output message

Implementation of the CountStreamTokens class (types are slightly scuffed):

Code:


def add_token_count(self, prompt_tokens: int, completion_tokens: int, model: str) -> None:
        # I append the tokens to a running total here. This will be called after the calculation is finished, as a callback. 
        # You can choose to do anything here with the numbers.
        self.detailed_usage.append({
            "model": model,
            "usage": {"prompt_tokens": prompt_tokens, "completion_tokens": completion_tokens},
            "time": datetime.now()
        })

completion = openai.chat.completions.create(messages=messages, stream=True, **params)

# completion is now a generator, or a 'stream' object. 
# CountStreamTokens is a custom class that is initialized with the model you use, and the messages you want to query with. 
# These are saved as class attributes for use in the .wrap_stream_and_count() function.
# The .wrap_stream_and_count() returns another generator, yielding all the same tokens as OpenAI provides, 
# but simultaneously collecting the output tokens.
# When the generator detects a None (ending) token in the stream, 
# it yields the final token and begins counting tokens (as to keep the stream running)

return CountStreamTokens(model, messages).wrap_stream_and_count(completion, add_token_count)
30

Please implement this in the final chunk. I don’t want to estimate something and introduce inaccuracies.

PS. (ahem Claude already tells me the usage in the final chunk of a stream ahem)

31

The OpenAI API compatible endpoint of the llama.cpp server does exactly this:

data: {“choices”:[{“finish_reason”:“stop”,“index”:0,“delta”:{}}],“created”:1713786803,“id”:“chatcmpl-pmnNZxAQ2B1JYX9AZfJXCe0uwII0Aejx”,“model”:“openchat-3.5-0106.Q5_K_M.gguf”,“object”:“chat.completion.chunk”,“usage”:{“completion_tokens”:18,“prompt_tokens”:28,“total_tokens”:46} }

32

(OpenAI here!)

We have added support for this, so no more workarounds needed! See Usage stats now available when using streaming with the Chat Completions API or Completions API

33

the same is happening to me, only while streaming.

The final chunk does not contain the usage

34

With the new Responses API, this doesn’t seem too well documented, so I made a gist explaining token tracking there (very similar to this), in case anyone is confused.
I cant send a link here, but search this on github gist:
Tracking token usage while streaming OpenAI Responses API in python

35

Capturing the streaming token usage of Responses is quite easy, not needing more than a passing understanding of events, not needing off-site tutorials, not needing a specific request parameter for it.

Streaming in Responses is sending you named events in the SSE subscription.

The final event you’ll receive is response.completed

It looks very like the output object of non-streaming, and in fact, repeats what has been streamed as content in deltas. RESTful:

event: response.completed
data: {"type":"response.completed","sequence_number":42,"response":{"id":"resp_348921","object":"response","created_at":1755446846,"status":"completed","background":false,"error":null,"incomplete_details":null,"instructions":null,"max_output_tokens":1500,"max_tool_calls":null,"model":"gpt-4.1-nano-2025-04-14","output":[{"id":"msg_902134","type":"message","status":"completed","content":[{"type":"output_text","annotations":[],"logprobs":[],"text":"Yarr, treasure ahead!"}],"role":"assistant"}],"parallel_tool_calls":true,"previous_response_id":null,"prompt_cache_key":null,"reasoning":{"effort":null,"summary":null},"safety_identifier":null,"service_tier":"default","store":true,"temperature":1.0,"text":{"format":{"type":"text"}},"tool_choice":"auto","tools":[],"top_logprobs":0,"top_p":1.0,"truncation":"disabled","usage":{"input_tokens":42,"input_tokens_details":{"cached_tokens":0},"output_tokens":7,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":49},"user":null,"metadata":{}}}

Let’s talk Python, then:

data = r"""{"type":"response.completed","sequence_number":42,"response":{"id":"resp_348921","object":"response","created_at":1755446846,"status":"completed","background":false,"error":null,"incomplete_details":null,"instructions":null,"max_output_tokens":1500,"max_tool_calls":null,"model":"gpt-4.1-nano-2025-04-14","output":[{"id":"msg_902134","type":"message","status":"completed","content":[{"type":"output_text","annotations":[],"logprobs":[],"text":"Yarr, treasure ahead!"}],"role":"assistant"}],"parallel_tool_calls":true,"previous_response_id":null,"prompt_cache_key":null,"reasoning":{"effort":null,"summary":null},"safety_identifier":null,"service_tier":"default","store":true,"temperature":1.0,"text":{"format":{"type":"text"}},"tool_choice":"auto","tools":[],"top_logprobs":0,"top_p":1.0,"truncation":"disabled","usage":{"input_tokens":42,"input_tokens_details":{"cached_tokens":0},"output_tokens":7,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":49},"user":null,"metadata":{}}}"""
payload = json.loads(data)
usage = payload["response"]["usage"]
print(usage["input_tokens"])

Voila - the answer is 42

With one of OpenAI’s API SDK library modules, you’ll be iterating over the generated events and just receive the data itself in your language’s native format, or rather, in OpenAI’s Pydantic-based class objects with attribute methods.

Capturing, parsing, and taking appropriate recursive action for 25 other event types is the challenge on this endpoint.

36

I guess you’re right about the offsite tutorials. I prefer using the pydantic output of the openAI API.

dotenv.load_dotenv(dotenv.find_dotenv('secrets.env'))
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

response = client.responses.create(
    model="gpt-4.1",
    input="Write a small story.",
    stream=True,
)

all_content = []
all_chunks = []
usage: ResponseUsage | None = None
complete_content = None
for chunk in response:
    all_chunks.append(chunk)
    if chunk.type != "response.output_text.delta":
        if chunk.type == "response.completed":
            usage = chunk.response.usage
            complete_content = chunk.response.output_text  # This is only available in the official API packages (SDK-Only)
            break
        continue
    content = chunk.delta or ''
    print(content, end='')
    # use the content here

if usage is not None and complete_content is not None:
    total_tokens = usage.total_tokens
    input_tokens = usage.input_tokens
    output_tokens = usage.output_tokens
    # Now you could calculate the exact price of the tokens
    print('\n---')
    print(f"response content was:\n{complete_content}")
    print('\n---')
    print(f"total tokens used: {total_tokens} of which {input_tokens} were input and {output_tokens} were output tokens")