My first experiment was training a model on Moby Dick, line by line. One line was the prompt, the next was the response, and so on.
Next I did a few on using each line as the response, with the prompt blank.
Both techniques absolutely bias the output with the language and content of the training data.
Next I’m thinking about training word pairs with each other. IE one word prompt, next word response.
Any thoughts on this topic would be welcomed.
Depends on what you’re trying to achieve. Finetuning in this manner loses all context. One or two sentences floating on their own cannot convey the rest of the story. You might be interested in checking out my series on finetuning for novel writing.
What are you trying to do?
I’m trying to preserve the “voice” of the training data while erasing any context.
Also wondering more broadly, what is the effect of putting an entire paragraph or page in the response, versus single words?
One way to do this with fine tunning would be to set an empty prompt and the content would be the answer I think.
Reference: OpenAI API