I am trying to establish a systematic approach to the underlying problem versus solution, the data that will be used, and in what form. Suppose I have a live chatbot and a number of conversations between customers and the helpdesk. Now I want to use these conversations to fine-tune an LLM.
If I understand correctly, I can use the following structure to do so, is that correct?
{“prompt”: "Customer: Hey, how can I do XYZ using the ABC feature?\n\n###\n\nAgent: Hey there!\n\nJust click the Widgets link in the left nav, then do BBB followed by CCC, then DDD.\n\n###\n\nCustomer: Great, thanks! xoxo\n\n###\n\nAgent: ", “completion”: “No problem, just give us a shout if you need anything else! END”}
{“prompt”: "Customer: I need help with setting up DEF.\n\n###\n\nAgent: Sure thing!\n\nJust follow these steps: GGG, HHH, and III.\n\n###\n\nCustomer: Awesome, thank you so much!\n\n###\n\nAgent: ", “completion”: “You’re welcome! Have a great day! END”}
In this case, can the conversation between the customer and agent be used, and does each line need to end with a completion?
Now I move to the second situation, where a conversation takes place between a user and a chatbot (not a person), and there are errors in the chatbot’s response that I want to correct or improve.
Is it correct that I can ‘improve’ the incorrect response in the following way using weight = 0?
{
“messages”: [
{“role”: “system”, “content”: “Marv is a factual chatbot that is also sarcastic.”},
{“role”: “user”, “content”: “What is the capital of France?”},
{“role”: “assistant”, “content”: “Paris”, “weight”: 0},
{“role”: “assistant”, “content”: “Paris, as if everyone didn’t already know that.”, “weight”: 1}
]
}
Or is it better to provide only the desired answer?
And what if you want to provide multiple options as valid responses—can you combine them like this?
{
“messages”: [
{“role”: “system”, “content”: “Marv is a factual chatbot that is also sarcastic.”},
{“role”: “user”, “content”: “What is the capital of France?”},
{“role”: “assistant”, “content”: “Option1: Paris\nOption2: Paris, as if everyone didn’t already know that.\nOption3: Obviously, Paris—the city of love and lights.”}
]
}