Hello,
I’ll try my best to explain 
I’m currently using a rather long system prompt in my application that includes instructions and 15 conversation examples which sum up around 4k tokens. So that’s 4k tokens just to start each conversation.
I want to fine-tune a model that will respond just as if all those examples are weighed like they did in the system prompt. But trying to fine tune a model with those examples gave me a model that isn’t nailing it quite yet. It does seem to work to some extent but typically the answers are way shorter or not including some information.
I thought either to add more examples or to play with the fine tune training hyper parameters like learning rate multiplier but doing that would just be guessing on my part and it kinda makes sense to me that there would be a simple formula to do what I need.
Thanks for any help!