Azure .net OpenAI Client: Usage disabled for streamed completion

API
,
1

Hello, I’ve been trying to retrieve the Usage information when I stream chat completions using the Azure OpenAI client. I used this post from the OpenAI changelog for guidance:

May 6th, 2024

https://platform.openai.com/docs/changelog/may-6th-2024)

In the Azure OpenAI Client, StreamOptions is an internal field in the ChatCompletionOptions object. By default, StreamOptions.IncludeUsage is set to true (see pic below).

Here is the issue I am having:

Issue: When calling the CompleteChatStreamingAsync() function of the ChatClient class, the StreamOptions field of the passed in ChatCompletionOptions object reference, which by default, has the StreamOptions.IncludeUsage field set to true, is unexpectedly cleared out, and is set to null.
The pictures below illustrate the value of the StreamOptions field before and after the function call (see red stars in pics):

Before:

oaibug_beforeCal
oaibug_beforeCal1575×483 120 KB

After:

oaibug_afterCal
oaibug_afterCal1827×523 151 KB

Here is my initialization of the ChatCompletionOptions object:

        var chatCompletionOptions = new ChatCompletionOptions()
        {              
          Temperature = (float)0.8,
          FrequencyPenalty = 0,
          PresencePenalty = 0
        };

I do not touch this object between the time of initialization and the time I pass it to the above function. I have also checked to make sure I am indeed not receiving any Usage info from the StreamingChatCompletion object that is returned from CompleteChatStreamingAsync() . According to the documentation, the last chunk of the stream should contain usage info, but that is not case for me.

Am I missing something (is this intended behavior?)?

Additional Info:
Azure.AI.OpenAI nuget package version: 2.0.0-beta.2
.NET version: .NET 7.0

Let me know if you need any more info

Thanks,
Seb

2

Azure OpenAI does not support stream_options yet.

If you will try to pass this option even on latest api version you will receive response:

{
    "error": {
        "message": "Unknown parameter: 'stream_options'.",
        "type": "invalid_request_error",
        "param": "stream_options",
        "code": "unknown_parameter"
    }
}