When streaming with the Chat Completions or Completions APIs you can now request an additional chunk to be streamed at the end that will contain the “usage stats” like the number of tokens generated in the entire completion. Previously this usage data was not available when using streaming. Just se…

usage_dict = chunk.usage This will still be “pydantic”. You can use dict(chunk.usage) to get a “clean” dict, where you then can get values by key: output_cost = usage_dict['completion_tokens'] * model cost / 1000 for example. [image] (you’ll have to trust me that the stream made pokey emoji on…

Azure typically lags behind a bit on releases. You can see in the latest Azure OpenAI API spec that the stream_options field is not supported yet. We’ll have to wait for them to make a new release and update the api_version param on our side.

@nimobeeren I suppose it is the same for the Assistants API v2 on Azure OpenAI. Don’t they tell its community when they plan to support the latest OpenAI features?

There doesn’t really seem to be a lot of communication on roadmaps coming from MS. It’s basically take what you get.

If I cancel streaming data halfway through, how are the costs calculated, only for tokens sent and 50% of tokens received?

If you close a streaming connection on Chat Completions with actual network client close, the AI generation and billing of output should stop shortly thereafter. You of course would not get a stream chunk that says the cost. The usage page used to have per 5-minute-window information for cost disco…

The stream_options param doesn’t seem to be available on the Assistants createRun API ( https://platform.openai.com/docs/api-reference/runs/createRun ). Am I missing something? If I add that, the API returns 400 bad request. Could you please add the same option to the Assistants API? Is that on the r…

[image] fedme: The stream_options param doesn’t seem to be available on the Assistants createRun API Discerning when the methods described in this announcement would be useful to you, and on what endpoints, takes careful reading of the first sentence of the first post: [image] owencmoore: …

Thanks, you’re right, I confused the 2 endpoints. If anybody makes the same mistake, for the Assistant API you will receive the usage of the Run as part of the thread.run.completed and thread.run.step.completed streaming events.

This doesn’t seem to be supported in the Javascript library yet. Any ideas? I’m using the 4.50.0 version.

Usage stats now available when using streaming with the Chat Completions API or Completions API

API

conciergeai August 14, 2024, 12:55am 21

The same is happening to me:

Topic		Replies	Views
Why there is no USAGE object returned with Streaming Api Call? API api , chat-completion , completions	20	6152	February 20, 2025
Issue with Token Usage in Streaming Responses Bugs api	17	1719	February 21, 2025
Azure .net OpenAI Client: Usage disabled for streamed completion API azure , azure-openai	1	944	September 3, 2024
Stream_options not working for image_url contents Bugs api , streaming	4	415	December 24, 2024
OpenAi API - get usage tokens in response when set stream=True API	34	41812	August 17, 2025

Usage stats now available when using streaming with the Chat Completions API or Completions API

Related topics