When fine-tuning a model, the prompts you use during training can have a significant impact on the model’s performance. The model learns to respond based on the prompts and responses it sees during training.
If you change the prompt significantly when using the fine-tuned model, it may not perform as expected. In your case, you’ve trained the model with the prompt “Identify whether the following is an objection or not”.
If you change the prompt to “Summarize the text given below. Also identify whether the following is an objection or not”, the model might not perform as well because it hasn’t been trained on this specific prompt.
However, there are a few strategies you can use to improve the model’s performance:
-
Include the original prompt in every training example : OpenAI recommends including the set of instructions and prompts that worked best for the model prior to fine-tuning in every training example. This can help you achieve the best and most general results, especially if you have relatively few training examples [source (https://platform.openai.com/docs/guides/fine-tuning)].
-
Use separator sequences : If you have enough training examples, you can fine-tune a custom model without instructions. However, it can be helpful to include separator sequences (e.g.,
->or###or any string that doesn’t commonly appear in your inputs) to tell the model when the prompt has ended and the output should begin [source (openai-cookbook/articles/how_to_work_with_large_language_models.md at main · openai/openai-cookbook · GitHub)]. -
Use the same prompt as part of the system message : If you want to replicate a specific behavior in the fine-tuned model, you might need to use the same prompt again as part of the system message [source (System dialog box remains empty)].
Remember, fine-tuning a model doesn’t give it new knowledge, but rather learns the writing style you are giving it. If you want it to learn new information, you might need to use other techniques, such as embeddings [source (Does fine tuning improve gpt3.5/4 retrieval speed?)]. Finally, if you find that the fine-tuned model is not performing as expected, you might need to retrain it with the correct data and prompts [source (Fine tuned with wrong data initially)].