Thank you so much for the prompt reply. But just to clarify this further for myself, here is how I construct the messages when I am making a call to the API (just the gpt-turbo model, no fine-tuning):
messages = [
{“role”: “system”, “content”: “You are a helpful assistant.”},
{“role”: “user”, “content”: prompt},
{“role”: “assistant”, “content”: instruction},
]
The prompt is the same prompt I am using to construct my examples for fine-tuning, the instruction though is a long paragraph explaining to the chatgpt what to do.
In creating examples for fine-tuning, however, instead of instructions, I am providing examples of how to do the task.
Is there any benefit in adding the instruction to the prompt (if that’s the right place to add it)? Is there any harm to the fine-tuned model if I remove the instructions completely? Are these examples of “how to do the task” actually acting as if the instructions were provided? Any discussion would be helpful. Since fine-tuning is not cheap, I am trying to avoid trial and error as much as I can.