Fine Tuning Model on Conversation Chains

API
1

Note: I am not referring to Langchain even though I am using chain as a term in the title.

I have been using 3.5 and 4 across a bot for some time and have amassed a number of conversations (both single q/an and multiple back and forth responses). I have cleaned and modified the dataset to exhibit the output I want to see stylistically as well as content wise.

When fine tuning a model what are the best practices to handle multi turn conversations? Do I need to pair them off so that each part of a multi turn conversation is a single q/a or system/user? Additionally if I am using 3 roles (system, assistant, user), and I want to fine-tune on conversations that include notes from the assistant is that possible?

1 Like
2

Hi Kunal - it’s possible to chain conversations, i.e. include multiple rounds of Q&A, in a single example provided you stay within the token limitations. You would want to group them logically so the model can detect a pattern in interactions And yes, you can include all three roles in a single example.

Here’s an example structure:

{“messages”: [{“role”: “system”, “content”: “System message”}, {“role”: “user”, “content”: “User message 1”}, {“role”: “assistant”, “content”: “Assistant response 1”},{“role”: “user”, “content”: “User message 2.”}, {“role”: “assistant”, “content”: “Assistant message 2”}]}

1 Like
3

Ah great, thanks.

Would the same go for function calling as well? So if I have an extended history of function calls and their intended response I want to tune a model on to improve its accuracy on the function call I can pass the function role into the conversation chain?

For example if I wanted to fine tune a model on classification, I could just make user/system chains out of the function results or I could also include the function role into the chain as well.

4

I personally have not fine-tuned with function calls so far and don’t have any insights to share I’m afraid. I’m sure someone else can share some wisdom in that regard :slight_smile:

1 Like