When streaming with the Chat Completions or Completions APIs you can now request an additional chunk to be streamed at the end that will contain the “usage stats” like the number of tokens generated in the entire completion. Previously this usage data was not available when using streaming. Just se…

Usage stats now available when using streaming with the Chat Completions API or Completions API

ritchie.karl May 7, 2024, 2:44pm 5

Nice feature.

Looks like it’s not working for AsyncAzureOpenAI.
Is this intended and support for Azure will be added later or is it possibly a bug ?

2 Likes

Topic		Replies	Views
Why there is no USAGE object returned with Streaming Api Call? API api , chat-completion , completions	20	5562	February 20, 2025
Issue with Token Usage in Streaming Responses Bugs api	17	1307	February 21, 2025
Azure .net OpenAI Client: Usage disabled for streamed completion API azure , azure-openai	1	727	September 3, 2024
KeyError: "usage" for gpt-3.5-turbo-16k Bugs gpt-35-turbo , chatgpt	6	840	February 9, 2024
Usage Info in API Responses Announcements	20	12268	September 27, 2023