So when I use OpenAI CLI to process the dataset, the "prompt"part is just left with one column, and other two columns are deleted automatically when transferred into Jsonl document.
I don’t how this happen, should I integrate the 3 parts into one before finetuning?
What are your 3 columns?
Fine tuning with JSONL has just two parts, prompt and completion
The "prompt"part includes a “narrative”, “dialogue” and the “interlocutor”.
We developed a tool which validates, gives suggestions and reformats your data:
openai tools fine_tunes.prepare_data -f <LOCAL_FILE>
This tool accepts different formats, with the only requirement that they contain a prompt and a completion column/key.
Yeah, I got some tools to make these sub-columns into one. Thanks!