Invalid json in assistants streaming with annotations

Using the latest .net beta-10 SDK and also trying the older beta-3 SDK, I get the following serialisation error from the Assistants API when trying to stream the response. A “plain text” response is fine, but a response that references content in the vector store fails.

The calling code looks like this:

        var streamingRun = openAiClient
            .GetAssistantClient()
            .CreateRunStreamingAsync(threadId, selectedAssistantId, new RunCreationOptions()
            {
            });
        await foreach (var streamingUpdate in streamingRun)

The exception is thrown from the “foreach” loop, when the SDK tries to parse the “next streaming run”.

The requested operation requires an element of type 'String', but the target element has type 'Object'.
   at System.Text.Json.ThrowHelper.ThrowJsonElementWrongTypeException(JsonTokenType expectedType, JsonTokenType actualType)
   at System.Text.Json.JsonDocument.GetString(Int32 index, JsonTokenType expectedType)
   at OpenAI.Assistants.InternalRunStepDeltaStepDetailsToolCallsFileSearchObject.DeserializeInternalRunStepDeltaStepDetailsToolCallsFileSearchObject(JsonElement element, ModelReaderWriterOptions options)
   at OpenAI.Assistants.InternalRunStepDeltaStepDetailsToolCallsObjectToolCallsObject.DeserializeInternalRunStepDeltaStepDetailsToolCallsObjectToolCallsObject(JsonElement element, ModelReaderWriterOptions options)
   at OpenAI.Assistants.InternalRunStepDeltaStepDetailsToolCallsObject.DeserializeInternalRunStepDeltaStepDetailsToolCallsObject(JsonElement element, ModelReaderWriterOptions options)
   at OpenAI.Assistants.InternalRunStepDeltaStepDetails.DeserializeInternalRunStepDeltaStepDetails(JsonElement element, ModelReaderWriterOptions options)
   at OpenAI.Assistants.InternalRunStepDeltaObjectDelta.DeserializeInternalRunStepDeltaObjectDelta(JsonElement element, ModelReaderWriterOptions options)
   at OpenAI.Assistants.InternalRunStepDelta.DeserializeInternalRunStepDelta(JsonElement element, ModelReaderWriterOptions options)
   at OpenAI.Assistants.RunStepDetailsUpdate.DeserializeRunStepDetailsUpdates(JsonElement element, StreamingUpdateReason updateKind, ModelReaderWriterOptions options)
   at OpenAI.Assistants.StreamingUpdate.FromEvent(SseItem`1 sseItem)
   at OpenAI.Assistants.AsyncStreamingUpdateCollection.AsyncStreamingUpdateEnumerator.System.Collections.Generic.IAsyncEnumerator<OpenAI.Assistants.StreamingUpdate>.MoveNextAsync()
   at MyOrg.Ai.Maia.Chat.Services.Mai.MaiService.RunAndWaitAsyncStream(String threadId, String prompt, String selectedAssistantId)+MoveNext() in /Users/MyName/MyOrg/repos/MyOrg.AI/MyOrg.AI.ChatAssistant/MyOrg.Ai.Maia.Chat/Services/Mai/MaiService.cs:line 231
   at MyOrg.Ai.Maia.Chat.Services.Mai.MaiService.RunAndWaitAsyncStream(String threadId, String prompt, String selectedAssistantId)+MoveNext() in /Users/MyName/MyOrg/repos/MyOrg.AI/MyOrg.AI.ChatAssistant/MyOrg.Ai.Maia.Chat/Services/Mai/MaiService.cs:line 231
   at MyOrg.Ai.Maia.Chat.Services.Mai.MaiService.RunAndWaitAsyncStream(String threadId, String prompt, String selectedAssistantId)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at MyOrg.Ai.Maia.Chat.Components.Pages.Chat.SendPrompt(String prompt) in /Users/MyName/MyOrg/repos/MyOrg.AI/MyOrg.AI.ChatAssistant/MyOrg.Ai.Maia.Chat/Components/Pages/Chat.razor:line 216
   at MyOrg.Ai.Maia.Chat.Components.Pages.Chat.SendPrompt(String prompt) in /Users/MyName/MyOrg/repos/MyOrg.AI/MyOrg.AI.ChatAssistant/MyOrg.Ai.Maia.Chat/Components/Pages/Chat.razor:line 216
   at MyOrg.Ai.Maia.Chat.Components.Pages.Chat.ProcessPrompt() in /Users/MyName/MyOrg/repos/MyOrg.AI/MyOrg.AI.ChatAssistant/MyOrg.Ai.Maia.Chat/Components/Pages/Chat.razor:line 274

I see nothing about this on the status page, nor in the SDK issue queue. Since both SDK beta3 and beta10 have the issue. I suspect the API.

Anyone else seen this or know of a fix?

2 Likes

I’m having the same issue since yesterday! It used to work fine for weeks.

1 Like

It has been reported as SDK issue that came to light due to underlying changes in the API. See issue #200 in the openai-dotnet project.

EDIT: Fixed now in beta.11

Is this version on Nuget? I only see version 1.0.0-beta.4 for Azure.AI.OpenAI.Assistants. Is there another package we should be using? It seems like the Microsoft / OpenAI team might have two different SDKs?

Yes, it is on nuget: NuGet Gallery | Azure.AI.OpenAI 1.0.0-beta.11