Gpt-4-turbo-2024-04-09 not accepting images for Node js?

API Bugs
1

BadRequestError: 400 Invalid type for ‘messages[0].content[1].image_url’: expected an object, but got a string instead.
at APIError.generate

1 Like
2

This sounds like the exact error you need to fix.

Here’s what ChatGPT has to say about it:

3

Yeah, if you’re uploading a base64-encoded image:
https://platform.openai.com/docs/guides/vision/uploading-base-64-encoded-images

{
  "type": "image_url",
  "image_url": {
    "url": f"data:image/jpeg;base64,{base64_image}"
  }
}

And if you’re submitting a URL:
https://platform.openai.com/docs/guides/vision/quick-start

{
  type: "image_url",
  image_url: {
    "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
  },
},
1 Like
4

Its working perfect with gpt-4-vision-preview with base64, so definitely a bug. Check out this please:

async function imageFileToBase64(file) {
    const data = await fsPromises.readFile(file);
    return data.toString('base64');
}

async function generateContent(imagePaths, textOutput, bookName) {
    try {
        if (imagePaths === 0) {
            console.log("No images found.");
            return '';
        }

        let imageContents = [];

        for (const imagePath of imagePaths) {
            const imageContent = await imageFileToBase64(imagePath);

            imageContents.push(imageContent);
        }

     
   
        const response = await openai.chat.completions.create({
            model: `gpt-4-vision-preview`,
            max_tokens: 4096,
            temperature: 1.0,
            messages: [
                {
                    role: 'user',
                    content: [
                        {
                            type: 'text',
                            text: `
                            Extract all topics
                           `
                        },
                        ...imageContents.map(image => ({
                            type: 'image_url',
                            image_url: `data:image/png;base64,${image}`
                        }))
                    ]
                }
            ]
        })

        const textContent = response.choices[0].message.content;

        console.log('Text Content', textContent);

        // Create Text file
        await fs.promises.writeFile(textOutput, textContent, 'utf-8');

        console.log(`Successfully created the Text Content to ${textOutput}` );

        return textContent;


        
    } catch (error) {
        console.error(error);
    }
}
5

Error: Invalid type for ‘messages[0].content[1].image_url’: expected an object, but got a string instead.

Your user message

Still looks like you’re trying to send a string instead of an object.

@awmartin above shows the object to be sent for image (in the documented, validated, format), being an object because it accepts both url and detail properties.

OpenAI yaml schema to validate against:

ChatCompletionRequestMessageContentPartImage:
      type: object
      title: Image content part
      properties:
        type:
          type: string
          enum: ["image_url"]
          description: The type of the content part.
        image_url:
          type: object
          properties:
            url:
              type: string
              description: Either a URL of the image or the base64 encoded image data.
              format: uri
            detail:
              type: string
              description: Specifies the detail level of the image. Learn more in the [Vision guide](/docs/guides/vision/low-or-high-fidelity-image-understanding).
              enum: ["auto", "low", "high"]
              default: "auto"
          required:
            - url
      required:
        - type
        - image_url
6

I get the same error. It also works fine in the previous vision preview model…

7

Hi,

please check an example on the OpenAI vision example page.

OpenAI has slightly changed the specification for the parameter image_url, now it looks like:

“image_url”: {
“url”: “your URL”,
}

I have changed it this way and it works now …

Best Regards,
Kirill

1 Like
8

Same here. Thank you very much for the “workaround”

9

Can you see that I am using base64 images? Not url

10

Thanks!

I was reading the documentation here which is outdated (i.e. it’s schema is {image_url: string}, not {image_url: {url: string}}: https://platform.openai.com/docs/api-reference/chat/create