I have a question: Yesterday I finetuned a 4o mini model for creating teasers and at first glance I noticed significant improvements compared to prompting.
Does anyone of you have experience with whether I can expand the finetuned model for other tasks such as creating roof lines and titles or whether I have to train one model for each task because I always give it a specific role via jsonl? Or can I change the role accordingly in other lines of code?
I would say it depends. If you trying to ask the model to complete the tasks in a single API call, then you can consolidate them under one fine-tuned model. To achieve that, you can create an additional data set that includes examples covering both the existing task plus the new task.
It all really comes down how you would like to structure your tasks.
I hope that makes sense. Let me know if you’d like me to clarify further.
You basically follow the same process as when you normally fine-tune the model. However, when you create the fine-tuning job, you reference your fine-tuned model instead of the base model. But as for the training data set, it also needs to be in a JSONL file and the same format.
@jr.2509 If allowed, I had another question: Can I finetune the model via Chat in Playground further by RLHF and the thumbs down button or has the button no lasting effect on the finetuned model??
No. The only way to further optimize a model fine-tuned via the OpenAI fine-tuning endpoint is by going through the fine-tuning process again with additional data in line with our exchange above.
Hey @jr.2509, I just fine tuned my already fine tuned model with new training data, but after the job was finished, a new model was created… That I don’t understand. Has the first model now the new information or do I have to use the second model? Thanks in advance!
You should use the new model which incorporates both the original and the additional training set.
The good thing about this is that if the second model does not work as intended for some reason, you can always go back to the first fine-tuned model and, if needed, use it as basis for yet another fine-tuning job.
So we have to change the API each time we improve the model, good to know!