Context: consumer facing FAQ chatbot.
I currently have a setup where a user question is handled by a semantic search in my redis DB. The query sometimes gets data that isn’t that relevant, for example:
User: Thanks chatbot!
Response: The way to handle a receipt is etc…
This is mostly happening with questions that do not have a specific enough response. (Any other question is handled appropriately, even the ones i dont want to answer.)
Not handling the response in a proper way makes my chatbot seem unintelligent.
I have tried handling this through engineering a system prompt, but none have so far worked well enough.
I am thinking it might be an issue with how “clean” my starting data is. The semantic search might be picking up on the similar uses of languages, and returning those. (Which i was hoping the chat completion api could pick up on and filter out)
Is there a more standard way of handling this issue? A Google search or two has so far not resulted in anything for me.
Hi there!
Based on the example you provided it looks like user messages such as “Thanks chatbot” are also by semantic search? Is this correct? One option would be to put in place some initial filtering through which those messages/questions are identified that should be handled by semantic search versus those that should be handled through a bots common capabilities.
Hey! Thanks for replying.
Yes the “thanks chatbot” messages also use semantic search.
Your solution should probably work, hadn’t thought of that one. Thanks!
Is there a more intelligent way of doing it rather than just creating a long list of responses that i filter on?
I don’t think you need to be rules based here. You can just classify these under some umbrella term such as “courtesy phrases” or some such (didn’t come up with a better term right now).
What kind of a system do you have in mind for this?
For now the setup is:
Chat > SemanticSearch > Question+Context >OpenAi Api
I am assuming some kind of a system would filter between Chat & SemanticSearch?
One option would be to build in a function call for your semantic search. This would give you the flexibility to define as part of the function description the conditions/circumstances under which a user message should be passed on for semantic search.
I’m about to board a flight and don’t have the link handy but if you have not dealt with function calling before, then have a look at the developer platform which has some dedicated guidance around this.
1 Like