400 while processing image file: Expected file type to be a supported format: .jpeg, .jpg, .png, .gif, .webp but got none

API Bugs
, ,
1

I am trying to use an assistant to identify things in an image. Below code works conceptually when I use it in a separate tool call however the api docs led me to believe i could/should add the image as ‘content’ to the message.

Why not just use the tool call if it’s working? Well several reasons

  1. We have a lot of tool calls and I’m trying to minimize the amount of tool calls
  2. What I’m asking it to do in the tool call requires that I use 4o-mini model and That is the same model I am using in my assistant so I should be able to do the same thing in the assistant based on the api docs without all the added overhead of using a tool call.
  3. I went to all the pain of digging through all these different permutations so seems a waste to not post this; it might help the community and hopefully help improve the api a little tiny bit??
  4. I love to type.

Here’s my code:

/**Post  base 64 data URI  to openai file store (this works fine)**/
             imageUrl = `data:image/png;base64,${base64PngImageDataString}`;

              const formData = new FormData();
              formData.append("purpose", "vision");
              formData.append("file", dataURItoBlob(imageUrl), {
                filename: tempFileName,
                contentType: "image/png",
              });

              const response = await fetch("https://api.openai.com/v1/files", {
                method: "POST",
                headers: {
                  Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
                },
                body: formData,
              });

              if (!response.ok) {
                throw new Error(`Error uploading file: ${response.statusText}`);
              }

              const result = await response.json();
              logger.debug("File uploaded successfully", result);
              file = result;


...

//Then add resulting file data to message content
       /**Attach image files to payload.content  on openai message
          -This works but I'm trying to figure out a better way to do it*/
          payload.content.push(
            {
                type: "image_file",
                image_file: { file_id: file.id }, //This is the file we saved above
               //Default detail to auto
             })

/**Create new message with payload and prompt - this fails with below error**/
 await openai.beta.threads.messages
        .create(threadId, payload)

Here’s the error:

Error: 400 Invalid message content: Expected file type to be a supported format: .jpeg, .jpg, .png, .gif, .webp but got none.
    at Function.generate (c:\code\eO\cli\MVPs\AIAssistant\api\node_modules\openai\src\error.ts:70:14)
    at OpenAI.makeStatusError (c:\code\eO\cli\MVPs\AIAssistant\api\node_modules\openai\src\core.ts:431:21)
    at OpenAI.makeRequest (c:\code\eO\cli\MVPs\AIAssistant\api\node_modules\openai\src\core.ts:495:24)

Based on the error message It seems the api is trying to decide what the string type is but that shouldn’t be necessary if I’ve given it the string type in the data URI format this should be easy for someone with access to the code to debug it since I have the line numbers above :slight_smile: