Hi all, I am working on an application, and we have encountered an issue. We initially fine-tuned the OpenAI GPT-3.5-turbo model with a specific dataset for one scenario, and it produced good results.
Later, we used the fine-tuned model as the base model and added some additional data for the same scenario with the same system prompt. The resulting fine-tuned model also worked well with the training dataset.
However, when we fine-tuned the model a third time using the most recent fine-tuned snapshot for a different scenario with a different system prompt, the output model performed well for the new scenario. But when I provided input related to the old scenario with old system message, the model did not respond appropriately.
What happened to my model? It seems that the recently fine-tuned model should have knowledge of both scenarios, but the new scenario might have overridden the old one.
Ok. So in your case you are trying to expand your existing fine-tuned model with an additional scenario (scenario 2). At the same time you’d like the fine-tuned model to still properly respond to scenario 1 cases.
In order for the model to recognize that it is supposed to differentiate between two different scenarios, you need to make this clear in your training data. That means, in your new training data set you need to not only include examples for scenario 2 but also again examples for scenario 1. This way the model should be able to handle both cases.
I believe that if you only train the existing fine-tuned model with new cases, then it essentially overrides the existing training and the model will only recognize the new cases.