In the documentation it says clearly that if you fine-tune a model, there’s no reason to do any prompt engineering… This is rather surprising to me, because with prompt design your query (as it were) is made in the context of the examples you’re giving the model. On the other hand, it seems that when you fine-tune the model with those same examples, the model might not be able to realize what kind of query it is you’re making—i.e. it might not be able to connect your query to the examples that it was trained on. Or does it, because the fine-tuning data were the last inputs it saw?
That is, what’s the difference between
- fine-tuning with:
{"prompt": "<example_prompt>", "completion": "<ideal_generated_text>"}
{"prompt": "<example_prompt>", "completion": "<ideal_generated_text>"}
{"prompt": "<example_prompt>", "completion": "<ideal_generated_text>"}
....
then prompting with
<actual prompt>
and 2) just prompting with:
<example_prompt>
<ideal_generated_text>
<example_prompt>
<ideal_generated_text>
<example_prompt>
<ideal_generated_text>
....
<actual_prompt>
besides the cost/speed. Is the quality of the output from the former case really strictly better than that of that of the latter? Is the optimal solution not some combination of the two strategies?