Vector database QnA answering based on info from multiple replies

Prompting
1

Hello everybody,

What I am working on

I am working on a question-answering system. Our data is in the format of Question-Answer pairs.
I am using a vector database to search for matching questions.

Initial and current state

Initially, I was embedding only the questions but soon realized that sometimes, because of how the user asks the questions, the vector similarity search would not match the “question” field in the vector database, but the answer that the user is looking for is present in the “answer” field in the vector database. Then I combined the Question and Answer when computing the embedding. This improved the result, but I still found a case where it was not working well:

Current problem

I am using only the last user query to search the vector database, but people usually expect the bot to use detailes specified above. Of course, using multiple “user”, “assistant” messages in ChatGpt is possible, but when searching the vector database, I don’t use the whole conversation.
One example is if the user asked a question, got an answer from the bot, but wants a clarification. The user says: “I actually have X type of appointment”. When querying the database, I would like to query with a phrase that contains both the question that the user previously asked + the details that he provided in the last reply to get more relevant responses.

Question

Is there a method of improving the flow of the conversations, like using chatgpt functions, a scratchpad…? With these types of systems, normal users expect them to behave like humans, and they ask the question as if they’re talking to a human.
Thank you

1 Like
2

Hi and welcome to the Developer Forum!

You could pass the last few thousand tokens worth of interactions (system, user and assistant) to the model and prompt it with this:

(prior system/user/assistant role messages)+
“Given the above context, what is the best vector retrieval search prompt to use to get the most relevant embedding chunks for this query ###{new_user_query}###”

This will allow the model to produce the best search term for the users latest question and then you can use the retrievals from that search term as context for the question asked

1 Like
3

(prior system/user/assistant role messages)+
“Given the above context, what is the best vector retrieval search prompt to use to get the most relevant embedding chunks for this query ###{new_user_query}###”

What would be the role of the last message? User?

4

Yes user most likely, since the previous array element would be assistant, this is also called HyDE. Very powerful way of getting the AI to create phrases that correlate with the conversation, to aide your correlation.

1 Like
5

If you’re doing this, my best advice is to play around with the prompt, to find a way of generating something that has the largest correlation with your data, ie, if you have a dataset full of scientific abstracts you’re searching through, then you need to generate something similar to that.

1 Like