Can anyone explain how changes made to system prompt content effect the fine-tuned model? In another word, does the revision made in system prompt content after fine tuning misleads the fine-tuned model? Regards
The answer depends 
I’d say that smaller changes to the system prompt that do not change the “gist” of what you are trying the model to do should only have a negligible impact. However, more material revisions that involve changing the nature of the task are likely to lead to adverse outcomes.
Thanks 
I have just radically shortened the system prompt by keeping logic unchanged.
After shortening, I am able to get similar responses by using short content costing 350 token instead of long one costing 1300 token.
So, how about the effect of such amount of shortening where the gist kept unchanged?
1 Like
Great to hear it works. Not sure I understand the final question. Can you rephrase perhaps.
My original first question was about the effect of revisions in system prompt on behavior of fine tuned model. You said “it depends on how you revise it”
Now I am asking, what if my revision is only radically shortening the prompt by keeping the gist unchanged. Does that type of revision misleads the finetuned model? I am trying to decide whether to fine tune it from scratch or not.
I thought you had accomplished that successfully? Where is my misunderstanding coming from?
The long prompt version is in production. I am testing the short version only in playground. Although the short version is also successful in playground, I am hesitating to go live with the short version based on limited numbers of trials in playground. My use case is very sophisticated.
Anyway, I will increase the number of trials in playground and then decide to go live or not. Thank you very much for your help.
1 Like