Issue: gpt-4.1-nano fine-tuned model cannot analyze images - blocked by endpoint validation

[vunhan.contact]

Summary
I have created a fine-tuned model based on gpt-4.1-nano for vision tasks, but when I try to analyze a user-provided image using that custom model, I get a BadRequestError (400) error. The base gpt-4.1-nano model without fine-tuning works as expected. According to the Vision Fine-Tuning guide, image fine-tuning should be supported, but I’m unsure how to invoke my custom model correctly for image analysis.

---

1. Background

• Base model: gpt-4.1-nano (multimodal, supports image input)
• Custom model: ft:gpt-4.1-nano:ORG:custom-vision-model (finetuned with image→caption pairs)
• Official docs:
  • Vision Fine-Tuning overview
  • “Introducing vision to the fine-tuning API”

---

2. Expected Behavior

When I send an image payload to the fine-tuned model, I expect it to return a caption or analysis, similar to how the base model handles image inputs. Example input:

[
  { role: 'user', content: [ {
    image_url: 'https://...',
    type: 'input_image',
    detail: 'low'
  }] },
  { role: 'assistant', content: 'What can I do for you with this image?' },       
  { role: 'user', content: 'Describe this image' },
]

3. Actual Behavior

Instead of a valid response, I receive:

BadRequestError
Error code: 400
error.message: There was an issue with your request. Please check your inputs and try again

• The same payload works successfully with the base gpt-4.1-nano model (no fine-tuning).
• My custom model name and deployment appear correctly in my account.

---

4. Code Snippet

[Vasyl_Onopriienko]

Try to delete some options from your responses API request. Use just model, instructions and input

[_j]

I trained up some AI. Let’s let it have a look.

Error: Error code: 400 - {'error': {'message': 'Invalid content type. image_url is only supported by certain models.', 'type': 'invalid_request_error', 'param': 'messages.[0].content.[1].type', 'code': None}}

Conclusion

There is nothing in documentation saying thatnanomodel is unsupported for vision after fine-tuning.

OpenAI can either process a refund for AI trained with the intention of vision, or fix the API.

(For me, inconsequential cost, but that could be multiple epochs on 1500 token images at training cost for you)

[Colm_Roche]

Thanks for taking the time to share this level of detail. So that we can progress this with the best placed team, can you please email this and any other information you believe to be relevant to support@openai.com?

[_j]

This is nothing to be personally emailed, and it is nonsensical to have an OpenAI account who’s entire engagement on the forum is to say “email us”, even previously posting their AI template here (as if that isn’t information already known, and a help button on the platform site is there also, if we want to submit ourselves to further bot-like responses that want videos of the problem they’ll never watch to delay action.)

This is an issue I was able to easily replicate on behalf of the original poster, not coming here to report the issue, but already being here to verify the issue. Thus, platform-wide.

I performed a fine tune training and also was unable to submit a call with an image part of a user message.

THE ISSUE

Fine-tuning of gpt-4.1-nano model then blocked from including user images to chat completions endpoint for language generation.

Simple as that. The rights of a ft:model type are not provisioned correctly in the API specification.

[OnceAndTwice]

When trying to reproduce, I noticed it is no longer possible to fine-tune gpt-4.1-nano with any examples that contain images.

The job failed due to an invalid training file. Invalid file format for this model. Images are not supported for gpt-4.1-nano-2025-04-14. Please remove all images from your examples and try again, or use a model that supports images such as gpt-4o-2024-08-06.

The base model accepts images just fine.

For OpenAI, this was ftjob-FPQSbC2eK82R8nQ40rSP3N0h

[_j]

Uploading file...
File uploaded successfully. File ID: file-JpJy...
Starting fine-tuning job...
Fine-tuning job started. Job ID: ftjob-ruSS...
Fine-tuning job status: validating_files
Fine-tuning job status: validating_files
Fine-tuning job status: failed
Fine-tuning job has completed.

Error details: {'code': 'invalid_training_file', 'param': 'training_file', 'message': 'The job failed due to an invalid training file. Invalid file format for this model. Images are not supported for gpt-4.1-nano-2025-04-14. Please remove all images from your examples and try again, or use a model that supports images such as gpt-4o-2024-08-06.'}

-mini also

Error details: {'code': 'invalid_training_file', 'param': 'training_file', 'message': 'The job failed due to an invalid training file. Invalid file format for this model. Images are not supported for gpt-4.1-mini-2025-04-14. Please remove all images from your examples and try again, or use a model that supports images such as gpt-4o-2024-08-06.'}

full gpt-4.1 continues…

Fine-tuning job started. Job ID: ftjob-g2vV...
Fine-tuning job status: validating_files
Fine-tuning job status: validating_files
Fine-tuning job status: validating_files
Fine-tuning job status: queued
Fine-tuning job status: queued

[OnceAndTwice]

Seeing as mini is also affected, this makes me think vision fine-tuning is disabled because these models use “patches.”

I really hope this is something OpenAI (please) implements for us. Thanks in advance btw.

[vunhan.contact]

I have replace my fine-tune model by original model gpt-4.1-nano, every setting still work correctly, so I don’t think the problem come from my settings

[vunhan.contact]

So, conclusion that gpt-4.1-nano fine-tune model have problem with image analyze and OPENAI do not mention it in docs. Is it true ?

[_j]

The conclusion is: they were blocking inference from using images.

In reaction to my flagging the issue, they also block training.

So at least you can’t spend money on uselessness.

The ability should be made available though. Disallowing user images when running any fine tune model based on 4.1 mini or nano is quite odd. Perhaps these models after fine-tuning are not intelligent enough to follow OpenAI’s hundreds of tokens of “vision safety rules” that are injected. Or OpenAI would have to eat training on more hidden tokens than your whole example.

[vunhan.contact]

Do you have any other suggested models that can analyze an image, but it still save money like gpt-4.1-nano. ( Is gpt-4o-mini suitable?)

[OnceAndTwice]

I don’t entirely know what you mean by this, but you can still fine-tune gpt-4.1 and gpt-4o with vision. It’s just strange that it’s unsupported for the smaller models and that this limitation isn’t documented anywhere by OpenAI.

[alex11]

So the same thing just happened to me as well. I fine-tuned gpt-4.1-2025-04-14 and just learned i can’t use any tools including files with it. I searched for ever and couldn’t find any documentation other than Chat telling me this was the case and then locating this same situation here.

[elianb]

Hi everyone, same issue here, I trained:

gpt-4o-2024-08-06

I fine-tunned it successfully with images but when I try to give an image as input for those models I get an error:

[OnceAndTwice]

You can’t use image inputs to fine-tuned models on Responses. And this limitation isn’t documented at all, by the way.