How to upload image to GPT-4o using Response API

API
1

Hi
I want to extract details from an image (PNG) encoded in base64 string using Response API:

const response = await client.responses.create({
  model: "gpt-4o-2024-11-20",
  instructions: "...",
  input: [
    {
      role: "user",
      content: [
        {
          type: "input_file",
          filename: "page.png",
          file_data: `data:image/png;base64,${images[0]}`,
        },
      ],
    },
  ],
  temperature: 0,
  text: {
    format: zodTextFormat(AccountSchema, "Account"),
  },
});

The following error is thrown when run,

Invalid file data: 'input[0].content[0].file_data'. Expected a base64-encoded data URL with a valid file MIME type (e.g. 'data:text/plain;base64,SGVsbG8sIFdvcmxkIQ=='), but got unsupported MIME type 'image/png'. Please see https://platform.openai.com/docs/assistants/tools/file-search#supported-files for supported file types.

It looks like we cannot upload the image file. May I know how to upload the an image to GPT-4o to analayze and extract the details?

2

You’re almost there—just a tiny tweak needed! When you want to send an image to the AI model, use input_image instead of input_file (which is only for PDFs) in your user message.

Also, don’t forget: the API reference shows you can use a file ID that you’ve uploaded to the storage endpoint. There’s even a code snippet right next to the “Responses - completion” section that demonstrates how to include an image as input.

You’re on the right track! Just need to read the documentation with a careful eye to details (and expanding the input section of the Responses API reference many times to finally see what contents of messages can be sent)

3

Now I see it at https://platform.openai.com/docs/guides/images-vision?api-mode=responses&format=base64-encoded

Thank you!