I’m currently working on a building a fine-tuned model on GPT-3 that takes a sentence as input and and generates a new sentence based on a pre-defined ‘heuristic’.
The data was initially in an .xlxs format, and after preprocessing it I’ve created a “prompt” and “completion” column for the fine-tuning later. The “prompt” column essentially consists of the original sentences and the “completion” column consists of the heuristicized sentences. The original sentences have all been selected according to a particular heuristic.
I had a couple of questions regarding this problem:
- Is the current approach feasible for solving this problem? If not, then are there any other routes that could potentially work?
- I’ve converted this dataframe into a .jsonl file but I’m a bit confused about the next steps. How exactly do you incorporate this into a fine-tuned GPT-3 model?
- What GPT-3 model would be the best choice for this problem?
- Is there any way to include the heuristic used as a parameter while setting up the fine-tuned model?
Thanks in advance! Let me know if there’s anything I need to clear up.