This might be a stupid question but when I’m fine-tuning a model am I not essentially adding some more data to the entire dataset of the model?
If so then why does the response quality drop with fine-tuning?
I tried to fine-tune davinci on kubectl documentation. The idea was to ask questions in natural language and it’d return the appropriate commands and some complex variants of the command.
Few shots’ results are better than the fine-tuned ones.
My dataset has ~340 examples.
The feedback loop is slow and expensive.
Could someone please tell me what happens during fine-tuning, any links or references to improve the quality would be really appreciated.