I’m having issues with fine-tuning ChatGPT to act as a specific agent, which is slow responding and giving bad/irreverent responses. Is there a better process like combining embedding with fine-tuning so ChatGPT will respond better with respect the context of having it act as a specific agent?
You can’t fine-tune ChatGPT. What fine-tune are you referring to?
I mean OpenAI’s API fine-tuning the davinci model.
Go with embedding to retrieve your specific information to provide the AI context on your expected response. Fine-tunes don’t add facts, they act as filters that shape the input, or classify the input.
Thanks Curt. So if the user asks a question for help, and instead of the current response where GPT responds with, “Yes, I can help you with that”, and I want it to respond with something along the lines of “I can not help with you that…”, isn’t this the purpose of fine-tuners?
The fine-tune won’t add much useful information. But embeddings will. So the user says: “I am looking at buying a new TV”. You embed this question, then look at all your previous Q/A that you have to offer, and then retrieve the relevant Q/A and have the AI (DaVinci in your case) answer the question “I am looking at buying a new TV” with what TV’s you have, or whatever your typical intro would be. (This is one basic case, but read below for how to use a fine-tune and embeddings together)
The fine-tune would just take “I am looking at buying a new TV” and it could say the right thing, but because it wasn’t likely trained on that exact same question, it tries to overgeneralize and make things up.
So embeddings use correlation to stuff the prompt with your data. and then you tell the AI to answer based on this data. Whereas fine-tunes are very limited in performing well at retrieving facts that you provide it in the training file. It’s not impossible for fine-tunes, especially if you train it with a high amount of epochs, it just seems like embeddings work too well for this.
Also, you can add new items to your inventory quickly through embeddings, whereas a fine-tune will involve a re-train or additional training, which is time and money. Also, you can’t easily get a fine-tune to “forget” about an old item, so that might involve a total re-train, which can get pricey.
The situation where fine-tunes really shine for me is categorization. So “I would like to buy a TV” would be categorized as, say, “new TV” [Note: or " 0" if I am shooting for a 1-token categorizer which performs better and “0” represents new TV’s.] in the output of the fine-tune (after training it, of course), and then I would kick off a process to offer the latest new TV’s from my inventory. So context recognition, semantics, intent, etc. are all good things you can train a model to be proficient at with a fine-tune.
You would then use a fine-tune up-front to understand intent of the user, then use embeddings to retrieve the latest information on this topic. This is a good way to get the two to work together, and utilize their strengths as a team.
1 Like
Thank you, Curt! That was very helpful!
1 Like