I’m trying to train a model to produce interactive lessons for users on any topic. This is a fairly complex, nuanced task, which involves relatively large function calls.
I’m finding it really hard to gather enough data to train the model on, as I need to go through these nuanced examples myself and try and make them as good as possible, which takes a long time.
To overcome this, I’m going to finetune a model on a small amount of data, and then get that model to produce more examples (which I’ll still have to go through myself, but I will hopefully need to do less editing as the responses should be of a higher quality).
Nevertheless, for the first round of training with the small dataset, do you recommend modifying any of the finetuning hyperparameters?