I tried to vision finetune gpt-4o using DPO, however came across the following error: " The job failed due to an invalid training file. Invalid file format. Example 1 Invalid content type. image_url is only supported by certain models."
We currently do not support fine-tuning vision inputs with DPO
1 Like