Fine-tuning problem, multiple completion

[willycha0119]

I am preparing jsonl

I want to train the model to use the prefix of the text to predict the sentence I want to type

For example

“The weather is nice today”

So when I type “t”, “w”, “i”, “n”, “t”

I hope he can answer me “The weather is nice today”

But it could also be “That’s why I need teamwork”

So I prepared the dataset as below

{“prompt”:“t w i n t”,“completion”:“The weather is nice today”}
{“prompt”:“t w i n t”,“completion”:“That’s why i need teamwork”}

Is this preparation in the right direction?

Or is there something I need to modify?

[ruby_coder]

@willycha0119

{“prompt”:“t w i n t”,“completion”:“The weather is nice today”}
{“prompt”:“t w i n t”,“completion”:“That’s why i need teamwork”}

No. Your data is JSONL compliant but it does not meet the OpenAP data formatting requirements for fine-tuning.

Reference:

Preparing Your Dataset

See Also:

Fine-Tuning In a Nutshell with a Single Line JSONL File and n_epochs

Those who frequent this community easily notice that developers struggle to get good fine-tunings results. In this multi-part tutorial, I’m going to use the davinci base model and fine-tune it multiple times with the same single-line prompt using different values for n_epochs. I will take a completion with each of these models, and demonstrate that if you set your n_epochs value high enough, you can get excellent results fine-tuning with a single-line prompt! Before we get started, I would …