We are building a chatbot and want to get personalized answers for each user – User A would get a shorter, concise and funny answer, while User B would get a longer, more intelectual answer.
We don’t know how to differentiate a personalized answer form a general one since both make sense.
We don’t know how to define a bad answer for one user vs a correct answer for the same user.
So how do we evaluate the correct responses and what’s the best approach to request and get personalized answers from GPT-3.
Any comment is greatly appreciated.
I’m writing a book about this. The short answer is there is no short answer. This problem has many technical challenges. The simplest way is you can keep preferences for each user and create custom prompts for each user. But this lacks flexibility.
thanks for you reply, we are creating an app based on the premise that we can give personalized answers using GPT-3 any other insights you might have would be really helpful. Could you please share them?
Personalized answers to what? You might risk running afoul of the rules.
personalized answers to each user with the chatbot, what rules are in risk?
Yes exactly, we want personalized answers for every user.
As I said, this is a nontrivial problem. You have to start by recording every interaction, and then you must extract relevant information from those records. Then you must inject that information back into the logic of generating responses in realtime.
Ideally both, but our main focus is personalized answers to their character, preferences and behavior.
We would have our own model that classifies our user, gpt-3 would be used for the answers
Yeah that’s our strategy, you’re not missing anything, the problem arises later when we don’t know what’s the best output for each user, how do we know the better answer for each user? how do we know that it is correctly personalized?
Exactly this kind of problem we faced while creating our essay service with personal touch thing
what did you do? or what was your approach?
yeah that would most likely be the best option, measure the accuracy of the model with KPIs and prefer those answers with better performance.
thanks a lot for your insightful comments I really appreciate them.