Using GPT-4o via assistants API vs ChatGPT Issue

API
1

I’m trying to use the assistants API to reach an image ( a customer order ) and output into a specific JSON format.

Using chat GPT, this works nearly perfectly, however, using the same instructions and image in the assistants playground, the results are always different and nearly always wrong.

It can see the image as it will pull information from it but the output is just very bad.

For example the assistant give me this:

  "lineItems": [
    {
      "SKU": "1202424010",
      "quantity": 50
    }
  ],
  "customerNotes": "Please confirm receipt of this order.",

Please confirm receipt of this order. and the number 1202424010 doesn’t exist in the image i have provided.

Where as chatGPT gives me;

"lineItems": [
    {
      "SKU": "10304244",
      "quantity": 50
    }
  ],
  "customerNotes": "Invoices will not be paid until a POD has been provided.",

These fields are all correct.

Why could the output be so different when i am using the same model, the same prompt / instruction and the same image?

2

You have your choice of several AI models to use with Assistants, multiple choices that support computer vision. Each have different quality.

In ChatGPT, we have no idea what they use for model. They say GPT-4 when clearly it is a turbo variant. You may even get different provisional model changes tried out on you. We do know what kind of system prompt it uses, though, and you must establish the user’s rights to receive verbatim transcripts out of images.

Finally, you have both a detail option, and the ability to resize. You can even send an image multiple ways in a message.