Vision training files has 0 examples

API
,
1

Hello everyone,

I’m currently trying to fine-tune gpt-4.1 for a vision task. My training files are JSONLs with the image URLs being base64-encoded jpegs. These images are on my computer and I do not intend to host them just for the finetuning.

My problem: The JSONLs are present in the storage section of the openai platform website but when I start a finetuning job it fails and returns the error:

Training file file-<file_id> contains 100 examples with images that were skipped for the following reasons: invalid image format, expected one of (JPEG, PNG, WEBP). These examples will not be used for training.

The part of the JSONL entry with the image url looks like this:

{“role”: “user”, “content”: [{“type”: “image_url”, “image_url”: {“url”: "data:image/jpeg;base64,

Why is this not working?

Thanks in advance.

2

Hi and welcome to the community!

It’s not needed to upload the image itself for a successful fine-tuning job. But your approach to use base64 encoded data URLs appears to lack the actual image data.

It’s also not possible to reference images uploaded to the file storage for fine-tuning at the moment.

Here is an example from the relevant cookbook:

{
    "messages": 
    [
        {
            "role": "system",
            "content": "Use the image to answer the question."
        },
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is the title of this book?"},
                {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,<encoded_image>"}}
            ]
        }
    ]
}
3 Likes