I tested a set of conversation (prompts, user inputs, and assistant messages) in the Playground, and the model responded as expected. Then, after creating a JSONL file based on the documentation and fine-tuning a model, the new model’s responses seem uninformed, as if it lacks knowledge of the prompts and all the examples provided.
Any ideas on what might be causing this?
Your problem is “creating a JSONL file based on the documentation”, and expecting that to instill understanding of documentation.
An AI language model produces entailment, predicting an output based on an input, along with its pretraining (and OpenAI post training) shaping how patterns are expected to be followed.
Send the exact same system message and user message as constructed in lines of your fine-tune file. To then receive an assistant response like the fine tune file had the assistant producing. If you do not use the inference input the same way as you trained, or you did not train on exactly how the AI is expected to be used, even in growing conversations as examples, then you cannot expect to receive the desired behavior.
You will also find on the forum that the correct way to answer from documentation is to use an external retrieval solution, that new knowledge is not the kind of thing fine-tune will be good for (unless you can produce beyond thousands of example completions on that knowledge.)