I’m running experiments on LLMs which involves fine-tuning on research papers (for knowledge acquisition). I did this autoregressively with davinci-002, where the data were formatted as prompt-completion pairs with an empty prompt (technically a single whitespace because empty strings aren’t allowed anymore) and the completions was the content of the paper. For example:
{"prompt": " ", "completion": "<text from research paper>"}
I would like to know if there’s any way to do autoregressive fine-tuning like the above for chat models instead of typical supervised fine-tuning
I imagine I would have to format the fine-tuning data as:
{messages: [{"role": "user", "content": " "}, {"role": "assistant", "content": "<text from research paper>"}]}
Is this sensible?
Additionally, is there anything unprincipled about doing autoregressive fine-tuning on chat models?. For instance, do the chat capabilities decrease performance on a task like this (since the model “expects” question-answer formatting}, or does this kind of tuning cause the chat capabilities to be lost?