Continuously fine-tuning a model with more data over time

I’m doing a project where I plan to host a sql database where I store all the completions I’m using so that they’re viewable to the people I’m sharing this project with. I plan on accumulating many more completions over time and want to re-train the model with batches of new completions. I just plan on exporting data from my sql database to a jsonl, but would reusing some of those same completions impact the training any? should I make separate jsonl files for each new batch of completions or train a new model with the full set each time? or can I just retrain with all the completions each time?

I also have the same idea and hope to get professional advice!

You would take a base model. In the future this might be GPT-4, when OpenAI releases the API for that model. So you would essentially start with your own model which is a “copy” of GPT-4. Let’s call it damp_01. You fine tune it with all 700 entries in your json file. After some time you will have added another 300 lines and now have a combined set of 1000 entries. You can continue training model damp_01 by just providing it your new 300 entries.

Later, some time in the future, OpenAI will release GPT-4.2. Now you create a new model damp_02 which will essentially be a copy of GPT-4.2. You begin to fine-tune it — you should be providing all 1000 entries and start from scratch.

According to this OpenAI documentation: OpenAI API

Fine-tuning is currently only available for the following base models: davinci , curie , babbage , and ada . These are the original models that do not have any instruction following training (like text-davinci-003 does for example).

Am I missing something?

Hi @SomebodySysop, I guess you are right if you want to express that fine-tuning is not available for GPT-4. This is why I wrote that it might be GPT-4 in the future.

Yet, we could predict that fine-tuning will be one day available and until then we can already begin to assemble our json files for this purpose.

1 Like