Thank you. I had initially read to convert my .txt files into .jsonl files, but this also failed for me.
Representing the data as .json has allowed me to upload my data successfully.
Here is the format I am using:
[
{
"prompt": "a string representing a prompt",
"completion": ["response data 1", "response data 2"]
},
{
"prompt": "etc",
"completion": "you can also pass a single completion string as opposed to an array"
},
...
]
I have no idea if this format is good, but the models do appear to be able to pull specific data, appearing that it has learned.
From experimenting for only a few minutes so far, it appears that the GPT Assistants are not even learning from the data, but essentially just searching the data and then supplementing prompts with this extra bit of text. Not to undermine how cool this technology is, but if you’re trying to actually fine-tune a model and account for custom data to yield a new model with unique weights, I don’t believe this is what you’re looking for; however, the results may still be better than what you can get with other available models out there, so it’s all very interesting regardless! I have no idea if what I am saying here is true, just my first thoughts from playing with it just now.