As we know OpenAI will be releasing the API for fine-tuning GPT 3.5/4 later this year.
I’m wondering what the API is likely to be. Do people anticipate the same kind of simple prompt -> answer training data that the current fine-tuning API uses? Or is it likely there will also be ways to train the model on plain text in an unsupervised way, train it on full conversation trees, etc…
I’m guessing you will define a JSONL file, similar to fine tuning right now, but with User/Assistant pairs, with System being optional. I doubt full conversation trees, or unsupervised, but who knows.
What is impossible to predict but is ominous is the amount of tuning they could leave in. A double edged-sword, as conversational competency is almost expected now (and such assumptions of instruction following the usual failure in understanding what a fine-tune base GPT-3 model can do), but it also could put potential applications in jail.
Yeah, all these GPT-3.5 and GPT-4 models have a large amount of fine-tuning on top of them. I can see why it’s not an option right now. (fine-tune some other fine-tune?)
Maybe if they have base GPT-3.5 and GPT-4 models available, then they will essentially create a similar layer, but only off your fine-tuned inputs. That would be cool!
That’s exactly what I’m expecting. For me ideally, the API would be essentially just adding more knowledge to the instruct models.