I’m new to the field of AI and I’m currently working on creating a Chatbot tailored to engage with customers using personalized information.
Moreover, I’m interested in continuously augmenting my chatbot’s knowledge base to ensure it remains up-to-date. I’ve come across the term “semantic search” which may do it or maybe not, but I’m uncertain about its implementation and the approach.
I’m seeking advice and guidance from the community regarding the best approach to address this challenge. Are there alternative vector databases apart from Pinecone that you would recommend? Pinecone appears to be beyond my budget at the moment.
I appreciate your assistance and insights. Thank you.
Personally I use ChromaDB, there are many others if you search for open source vector database.
If you append all of the text from the user and from the LLM into the vector database you should have a vector searchable addition to the context, how you distribute that data and how you incorporate the searches back into your future prompts will be what defines your application from other “MemoryGPT” systems.
I don’t fully understand, can you shed a bit more light on this?
Let me give you an example scenario. I am building an application where I have 4 documents to provide info about and I train my system for those 4 documents, in future, I add another document, and I don’t want to re-train my model.