I finetuned a model. Since doing so, I’ve changed some things very slightly. Namely, how I prompt the model, and I’ve slightly modified the functions it calls.
The responses I’ve been getting from this current model, have been a bit of a mixed bag, so to drill in some more consistency, I feel like it’s a good data to gather even more finetuning data. So, I’ve gathered more exemplar responses made by this model with the new adaptations made to it. These responses are generally better than the previous examples I used to train the model with originally.
Should I re-finetune the current finetuned model with this new data, or finetune a new model on this new (and improved) data?
Hi - Personally, I’d go with creating a new fine-tuned model. It sounds like you’ve made quite a few critical adjustments and thus it could lead to disruptions if you fine-tune the existing one.
Did you have any particular concern that has so far kept you from creating a new one?
Yeah well the only thing keeping me from creating a new one is I feel I haven’t made too many changes.
I’ve changed one or two of the properties, and have slightly adjusted the system prompts. Maybe they’re bigger, and more impactful changes than I had originally been led to believe…
That being said, the current finetuned model is based on the gpt-3.5-0613 model, and I am considering switching to the newer 0125 model that’s just been released.
If I were to do so, I would have the choice of finetuning the model with just the new data, or (at least some of) the old data too. I’m presuming you’d favour the prior of those two options…?
Well, it sounds like the existing fine-tuned model isn’t providing you with satisfactory results. If this is correct, then rather trying to fix sth that may be broken, you are better off creating a new one with the updated data provided you are confident that it will perform better.
In terms of data volume, which is the key cost driver, it is in my view often less about quantity of training data but quality. So yes, I would just use the new data.