Hi, I’m brand new to OpenAI, so bear with my basic question!
I’m curious to know if I can train a custom model that can answer questions our customers repeatedly ask, and what would that JSONL file look like.
Would something like this work?
{"prompt": "Customer: Hey, how can I do XYZ using the ABC feature?\n\n###\n\nAgent: Hey there!\n\nJust click the Widgets link in the left nav, then do BBB followed by CCC, then DDD.\n\n###\n\nCustomer: Great, thanks! xoxo\n\n###\n\nAgent: ", "completion": "No problem, just give us a shout if you need anything else! END"} {"prompt": "Customer: ARGH, I got an email that my card was declined. WHY? There's money in the account.\n\n###\n\nAgent: Hey, sorry about that - I know it can be frustrating. The card declined message comes directly from your bank, so you'll have to contact them to find out why your card isn't going through.\n\n###\n\nCustomer: Ok, fine. Thanks..bye.\n\n###\n\nAgent: ", "completion": "Let us know what they say, and we can manually rerun your card once it's all sorted out. Cheers! END"}
In other words, if I dump a thousand conversations like this into a fine-tuned davinci model, and I cover pretty much every question about every feature, would we be able to feed live customer input into OpenAI’s API and get actually helpful responses?
So, I asked ChatGPT how to go about doing this. I asked:
I want to use the fine tuning feature to create my own model. The goal is to teach OpenAI’s davinci model how to help answer live customer chat questions about our application. How should I format the customer support conversations to populate the prompt and completion fields in the JSONL file?
And it said:
To fine-tune a language model like OpenAI’s GPT-3 for the task of answering customer support questions, you will need to create a dataset that contains examples of customer support conversations. Each example should consist of a prompt and a completion. The prompt should contain the customer’s question or issue, and the completion should contain the response that the model should generate.
Here is an example of what a conversation might look like in the JSONL format:
{
"prompt": "Customer: I'm having trouble logging into my account. Can you help me troubleshoot the issue?",
"completion": "Agent: Of course! First, can you try resetting your password? If that doesn't work, it's possible there may be an issue with your account. Can you provide me with your account details so I can take a closer look?"
}
If this is correct, then it seems that we just need to feed pairs of these questions/answers into the fine tuning process?
This is almost correct. You need to move the “Agent:” to the end of the prompt (with no space after it), and then you need to have a space at the start of the completion and follow it with the answer.
Let’s say I create a fine-tuned model with thousands of lines like this, and I’m ready to use it in the real world as a chatbot. I assume it would work like this:
Customer types “How do I do XYZ?”
We make an API completion call with the prompt "Customer: How do I do XYZ? Agent:"
We take the result of that call and pass it to the customer: "You do XYZ by doing ABC."
Customer asks a follow-up question: “When I do ABC, I get an error.”
We make another API call with the prompt of the entire conversation so far? E.g. prompt = "Customer: How do I do XYZ? Agent: You do XYZ by doing ABC. Customer: When I do ABC, I get an error. Agent:"
(rinse and repeat)
Do I have this right? Is it necessary to include the entire chat history in the prompt?
I have a follow up question to this conversation. In @ffxsam’s original question, the format of the data was like this, which is following the open ai docs section on the customer support chatbot.
{"prompt": "Customer: Hey, how can I do XYZ using the ABC feature?\n\n###\n\nAgent: Hey there!\n\nJust click the Widgets link in the left nav, then do BBB followed by CCC, then DDD.\n\n###\n\nCustomer: Great, thanks! xoxo\n\n###\n\nAgent: ", "completion": "No problem, just give us a shout if you need anything else! END"}
I am in a similar situation where I asked chat GPT how to properly format the prompt, and was given something similar to the second example:
{
"prompt": "Customer: I'm having trouble logging into my account. Can you help me troubleshoot the issue?",
"completion": "Agent: Of course! First, can you try resetting your password? If that doesn't work, it's possible there may be an issue with your account. Can you provide me with your account details so I can take a closer look?"
}
What is the the difference between the two formats? What is the reasoning to go with the second example, rather than the way that the docs formatted it with the stop sequences, etc?
The answer or completion starts with the test “of course” with a space before it
When you send the question you are going to include the Agent: label at the end of your prompt (without the space) so the ai knows it needs to say something
Otherwise it will try to complete the question instead of giving an answer
There seems to be a different way to do this depending on who you talk to. I, too, am looking to fine-tune for customer service/support. When I chatted with GPT3 on the subject, this is what it came back with:
Sure! Here’s an example of how you might format the above dataset as a JSONL (JSON Lines) file for use with a language model like GPT-3 that includes the completion as well:
`{"prompt":"How do I change my billing address? ", "completion": "You can change your billing address by logging into your account and going to the settings page", "label": "Billing"}`
`{"prompt":"Why is my software not working properly? ", "completion":"It seems that there might be a compatibility issue with your current operating system. Please check the system requirements and update accordingly", "label":"Technical"}`
In this example, each line is a JSON object representing one support question and answer, and the “prompt” key contains the question text, the “completion” key contains the answer, and the “label” key contains the classification label. The label is one of the predefined categories that you have chosen, such as “Billing”, “Technical”, “Account Management”, and “Sales”.
So, I guess my question is, is the above also correct if I wish to include a classification with each prompt?
I don’t know a lot, but I can tell you that following the standard fine-tuning instructions here, OpenAI API, the model rejected the 3 keys (prompt, completion, label) above. So I submitted the dataset with just the prompt and completion. Results were varied. Everyone keeps telling me that to get the best results (answers that match your dataset more closely) you have to use “embeddings”. Trying to learn what that is and how to do it now. See: Google Spreadsheet + GPT3
Yes, this is correct. I only use “prompt” and “completion”…
… careful about ChatGPT, it hallucinates solutions. You must confirm via the API docs (and by testing of course) anything ChatGPT offers as a technical solution:
@ffxsam did you get this working (credibly) using fine-tuning?
I’m experimenting with a similar thing - see if we can automate responses in our support chat.
The results so far using fine-tuning are not great. I’m wondering if using embeddings to provide the context might be a better way to go.
Absolutely. That’s the power of these models, you can translate all text, images, video (24 images per sec), sound (spectrograms) into an embedding aka a vector aka a table of numbers.
With that now you can compare and relate all your data. GPT3 is a huge collection of embeddings and their relationships. Fine tuning it with your embeddings creates new relationships between these vectors, so now knows how to reply using your data.
I am curious about this reply. My understanding of embeddings is that you can vectorize your content residing in your database, and run computations against it with search vectors. Select OpenAI models can provide the embeddings via API calls to it using your content.
I do not understand how you can fine-tine a model with your embeddings? Does this mean you can upload your content vectors to an OpenAI model?
What you are trying to do is not efficient through fine tuning. Fine tuning won’t get you there. You won’t get the results you want and can end up being very costly as a CS Chatbot. Your best bet is using embedding, check out then OpenAI documentation. I know embeddings work really well with Aleph alphas text models. Gd luck
Do I need to use tags inside prompts inside the JSONL file so that OpenAI takes into account the last message?
{"prompt":"Summary: <**summary of the interaction so far>**\n\nSpecific information:<for example order details in natural language>\n\n###\n\nCustomer: <message1>\nAgent: **<response1>**\nCustomer: **<message2>**\nAgent: **<response2>**\nCustomer: **<message3>**\nAgent:", "completion":" <response3>\n"}
or I could just put actual text like the one @ffxsam made? I jus wanna make sure the chatbot will remember the last message
{“prompt”: "Customer: Hey, how can I do XYZ using the ABC feature?\n\n###\n\nAgent: Hey there!\n\nJust click the Widgets link in the left nav, then do BBB followed by CCC, then DDD.\n\n###\n\nCustomer: Great, thanks! xoxo\n\n###\n\nAgent: ", “completion”: “No problem, just give us a shout if you need anything else! END”}
I asked: