I think it’s a cool idea (and was talking about just HyDE). I read up about your idea and it sounds cool. I’m skeptical about it’s practicality with chat-bots, and also generally about it drifting.
But I’m excited to see you prove me wrong.
I still want to backup my initial comment about the standalone queries. Regardless of the situation work will need to be done on the initial query to at the least decontextualize it, so might as well perform some additional work to filter it into a pure form and then try to match your documents to the queries. Work needs to be done on the query!
To be fair, my experience is with domain-specific datasets that I ran into issues when trying out these methods.
I think an important thing to consider is the complete chaotic web that a conversation is. Realistically there are going to need to be many separate functions ran on specific types of queries.
Going back to domain-specific I built(used) a recommendation engine so I needed to gather and maintain a log of user preferences and selections. The type of question would modify how I would approach it so I needed it to be as clean as possible.
So for Platcorp not only does it need to have all of this knowledge, but it also needs to be able to recommend and understand what the user is looking for. Which means carrying all of this knowledge and being able to perform work on it. Maybe not Platcorp, but for my example I also needed to be able to aggregate numerical data from datasheets.
So for me, having a distilled query is absolutely necessary to know what I need to perform on it.
I think given that you started with langchain, you want to use indeed pinecone for the context but add a motorhead node to remember the chat, then chain both to your vhat model.
Try flowise, it’s a langchain UI, in the “marketplace” there is an example that does just what you ask for, and the visual workflow makes it very clear how it works in langchain.
I have done precisely this, inspired by Haystack’s agent component, although I have not used that.
I use an additional prompt, passing in the conversation history and the question, asking it to generate a question that can be understood independently of the history, incorporating the context. Happy to share more detail if you’d like.
As you are using Langchain, I would suggest you install flowise, it’s an UI for langchain, it show very easily in the workflow where you error is.
Go to marketplace and look for a “conversation retrieval workflow”. You will notice that the context itself is embedded in a pinecone vectorial DB, but the conversation itself uses another DB, Motorhead in the example.
So basically you need a DB for the context and a db for the conversation itself, try that and it should work.
All the other suggestions I read make is so difficult when in fact you have the solution within the langchain framework itself, even if you can code the UI makes it very easy for you to visualise the various possible architectures.
Hope this helps
Hello Mark
Would you be so kind to tell me if you manage to do this summarizing in gpt-3?
I only had luck with gpt-4. Tried a lot of variations in gpt-3 but it just dont summarize important things in the chat history.
This is what my last attempt looks like (works well in gpt-4)
Given the following chat history :
CHAT HISTORY :
“”"
USER : (1st question…)
ASSISTANT : (1st answer…)
…about 5 of these interactions
USER CURRENT QUESTION : (current user question)
“”"
generate a question that can be understood independently of the history, incorporating the context of the conversation.