Unable to make openai to return a specific object using its new library for dotnet

API
,
1

Hi,
Im using the new Open-AI library for dotnet,
Im trying to make a service that analyzed an image and return a specific object with values, however, its turns out to return an ugly object with missing properties, and it seems like I can’t make it to return a consist output.
What am I doing wrong ?

The service:

using System.Text.Json;
using backend.DTOs;
using OpenAI.Chat;

namespace backend.Services;

public class OpenAiService
{
    private readonly ChatClient _chatClient;
    private readonly ChatCompletionOptions _options;

    public OpenAiService(IConfiguration config)
    {
        var apiKey = config.GetValue<string>("OpenAI:Key");
        _chatClient = new ChatClient("gpt-4o", apiKey);
        _options = new ChatCompletionOptions()
        {
            MaxTokens = 300,
            ResponseFormat = ChatResponseFormat.JsonObject
        };
    }

    public async Task<ReceiptDto> ExtreactListOfItems(Stream imageStream)
    {
        var imageBytes = await BinaryData.FromStreamAsync(imageStream);

        var messages = new List<ChatMessage>
        {
            new UserChatMessage(
                new List<ChatMessageContentPart>
            {
                ChatMessageContentPart.CreateTextMessageContentPart(
                    @"From the following image, extract a list of items. Ensure the output is a well-formed JSON object with correctly formatted Hebrew text (right-to-left). The JSON structure should be: 
                    { 
                        storeName: 'x', 
                        purchase: { 
                            purchaseDate: 'yyyy-mm-dd', 
                            items: [{ 
                                itemName: '', 
                                itemPrice: 10, 
                                quantity: 2, 
                                totalItemPrice: 20 
                            }] 
                        } 
                    }. 
                    Make sure all strings are properly closed, all objects and arrays are correctly terminated, and the JSON is valid,
                    Make sure each items has the the properties itemName, itemPrice, quantity totalItemPrice"
                    ),
                ChatMessageContentPart.CreateImageMessageContentPart(imageBytes, "image/png")
            })
        };
        
        ChatCompletion completion = await _chatClient.CompleteChatAsync(messages, _options);

        var resp = completion.Content[0].ToString().Trim();
        
        //resp = resp.Replace("```json", "").Replace("```", "").Trim();
        
        Console.WriteLine("Raw Response: " + resp); 
        
        using var document = JsonDocument.Parse(resp);

        
        var receiptDto = JsonSerializer.Deserialize<ReceiptDto>(document.RootElement.GetRawText());
        
        Console.WriteLine("Deserialized Receipt: " + JsonSerializer.Serialize(receiptDto));

        return receiptDto;
    }

}

Sometimes the properties:
quantity, totalItemPrice

are missing, or the object form is not fully completed. what am I doing wrong ?