GPT-4-turbo vision API recognizes image_url as base64 encoded image data

API Bugs
1

Provided is the section of my typescript code. s3ImageURL and resolution variables have valid values.
s3ImageURL looks like https://s3.ap-northeast-2.amazonaws.com/bucketName/fileName
resolution - “auto” | “low” | “high”

    const response = await openai.chat.completions.create({
      model: 'gpt-4-turbo',
      messages: [
        {
          role: 'user',
          content: [
            {
              type: 'text',
              text: 'What’s in this image?'
            },
            {
              type: 'image_url',
              image_url: {
                url: s3ImageURL,
                detail: resolution
              }
            }
          ]
        }
      ],
      max_tokens: maxTokens
    });

I am using gpt-4-turbo to describe an image stored in AWS s3 bucket (public). I pass the correct s3 URL but the API returns an error saying

Error: 400 Invalid image URL: 'messages[0].content[1].image_url.url'. Expected a base64-encoded data URL with an image MIME type (e.g. 'data:image/png;base64,aW1nIGJ5dGVzIGhlcmU='), but got a value without the 'data:' prefix.

I don’t know why it keeps recognizing the image URL as encoded data. Is there any character in the URL that I need to escape? I do not want to download the image locally and convert it to base64 data.

Additional context:
When I run the above section of code in a function from a cmd, it works fine. It successfully describes the image from that imageURL. However, when I call the same function in the express router function, it keeps recognizing imageURL as base64-encoded data URL for some reason and fails.

3 Likes
2

did you figure this out? I am running into the same problem, passing an image_url keeps giving an error saying it’s expecting a base64 image

1 Like
3

In my case the max_tokens property value was a string and not an integer. And this was the reason why “Error: 400 Invalid image URL: 'messages[0].content[1].image_u…”. :expressionless:

1 Like
4

you mean you were passing “max_tokens” : “300” instead of “max_tokens”: 300? because that’s what i am doing. (i ended up base64 encoding to bypass my issue)

1 Like
5

Sorry for the late response. I changed my function to use base64 encoding as I could not figure out why. Later, I tried strictly typing my parameters(resolution, maxTokens), and it resolved the issue. Make sure all your parameter values are the correct type if you are using typescript.

1 Like
6

this solution save my day using GPT-4o , converting max_tokens to integer. Thank you

1 Like
7

for those who come across this question, this might have nothing to do with the image_url.
check that no unexpected values are passed to the api. for example, check that you are not passing null where it is not necessary.