Need help! How to deal with overlapping of data?

I have built a chatbot which can do a search on my data by creating embeddings solution. But if I have overlapping of data then answers are not accurate. Is there any way that I can send a recommend/follow-up question back to the user thru a chatbot to be more specific with the document. For example- User query is “share scope of work”. I have 10 documents where I have scope of work then what should be in a prompt to send a recommendation to the user to be more specific with the vendor’s name to get the right answer.

I have not tried it myself but there is this technique called Hypothetical Document Embeddings (HyDE) that seem to be promising and might interest you.

1 Like