I’m watching the AI Engineering Summit Day 2 talks and just watched a talk about fine tuning. There are a lot of great opinions on here about when you should use fine tuning to training things like a binary classifier but just wanted to add some color about when you’d fine tune an LLM for reasoning… It’s when you don’t like the answers you’re getting from the model.
If you can use RAG to show a model the information you want to ask questions about and you like the answer you get back you don’t need fine tuning. If however, there’s nothing you can show the model that will result in the answer you want. It needs to be tuned.
As you might immediately guess, there are very few cases where you would need fine tuning but they do exist. They’re typically going to be around complex reasoning. Given this argument between two people who is right? That’s very much open to opinion and you may not agree with the models answer. That’s a case where you can tune it to think more like you.
The more important thing is to be able to recognize when a task you’re asking the model to perform could benefit from fine tuning… For me that’s one rule… When you don’t like the answer you’re getting and nothing you do seems to fix it…
To be completely transparent I have yet to fine tune a model as it’s expensive and so far I’ve always been able to prompt engineer myself out of corners. I’m mainly stating what it would take to get me to resort to fine tuning a model.
I can definitely see cases where nothing you’re going to do in the prompt is going to result in the answer you want so there are plenty of cases for fine tuning LLM’s.