Use base64 encoded images or urls within prompts?

API
1

Hey,

I don’t find a real best practices example regarding images for the openAI API.

Would you recommend to encode images in base64 or use just the image url?

    const messages = [
      { role: "system", content: systemContent },
      {
        role: "user",
        content: [
          {
            type: "text",
            text: "Analyze this nutrition plan and suggest meals.",
          },
          imageUrl
            ? {
                type: "image_url",
                image_url: {
                  url: imageUrl,
                },
              }
            : { type: "text", text: "No image provided." },
        ],
      },
    ];

    const messages = [
      { role: "system", content: systemContent },
      {
        role: "user",
        content: [
          {
            type: "text",
            text: "Analyze this nutrition plan and suggest meals.",
          },
          imageData
            ? {
                type: "image_url",
                image_url: {
                  url: `data:${imageData.contentType};base64,${imageData.base64Data}`,
                },
              }
            : { type: "text", text: "No image provided." },
        ],
      },
    ];

  const { text } = await generateText({
        model: openai("gpt-4o"),
        messages,
        max_tokens: 10000,
      });

I mean for me base64 would be “better” but I always get some error like

Error in chat API: 400 Invalid MIME type. Only image types are supported.

Even its a image file…

 "contentType": "image/jpeg"

So I’m interested how you are solving this.

Thank you in advance

2

See https://platform.openai.com/docs/guides/vision/managing-images where it states the following:

For long running conversations, we suggest passing images via URL’s instead of base64.

What’s your usecase? I’d use image URLs for most projects since they’re easier for me to work with and if users provides URLs instead of files, I wouldn’t need to upload the files to my server, which would save me bandwidth.

2 Likes
3

You can find similar content at this video or this video, where it discusses the following: Use base64 encoded images or urls within prompts.