How to use multiple vector stores

API
1

Hi,

I have multiple vector stores created using ChromaDB and different prompt templates

  1. VectorStore1 (Embedded 100 documents related to topic A)
  2. VectorStore2 (Embedded 50 documents related to topic B)
  3. VectorStore3 (Embedded 200 documents related to topic C)

Prompt 1: Prompt related to queries asked for topic A
Prompt 2: Prompt related to queries asked for topic B
Prompt 3: Prompt related to queries asked for topic C

Now how to select the particular prompt template and VectorStore on the basis of user question and run a chain to get the correct answer.

I was looking into this article:

Please provide a solution, if any or let me know if the usecase mentioned above is valid?

3 Likes
2

Can you please justify why you need 3 vector stores? What is wrong with ONE?

1 Like
3

I want to keep data separate from each other, so different vector stores.

1 Like
4

I can’t see the point of doing that. Can you justify? (“want” doesn’t really provide insight into the reason)

5

There are multiple stakeholders, so I thought I should keep their documents in separate vector database.

The physical documents will be kept in separate locations and a particular team will have access to those folders. That way they can keep on adding / updating the documents which can be indexed to vector DB.

Please let me know if my approach is not correct.

1 Like
6

Ah, ok that makes more sense.

But if that’s the case why not deploy different instances instead of mixing them in one instance?

Or simply add an attribute to the vector store to allow you to filter by stakeholder? (Although that might kill some types of index)

7

Actually I refferred to the following article:

So I was thinking to keep all vector Db on in one instance. And on the basis of user query, select the prompt and run the particular chain.

However was not getting a breakthrough.

8

Identify the user, establish organisation, filter the single table by their org id in your search query. Done.

You don’t need to limit yourself to langchain. Do this from first principles so you can create your own efficient bespoke solution.

Frameworks are great but sometimes suboptimal and limiting (Assistant API is a good example of that!).

2 Likes
9

Sure, will try this. Thanks for the suggestion.

1 Like
10

Did you crack the solution? I am also having the similar problem to solve you started discussion with i.e. routing the query to correct index

1 Like
11

Also in the same place as you.
Interested to know what @axysharma did here.

I’m looking to implement this but with many vectorstores (different stakeholders as previously mentioned)

image
image2893Ă—693 139 KB

Source

1 Like
12

Hello everyone, has anyone discovered a solution for this yet? Here’s what I’m aiming for: I have a considerable number of documents that I’ve categorized by either topic or level. For instance, I’ve created vectors for common information, regional (state) specifics, and even further for municipal details. Now, what I’m striving for is to assemble assistants using combinations of these vectors. For instance, I’d like one assistant to comprise common and regional (state) information, another to have solely regional (state) information, and yet another to blend common information with municipal details. The key here is reusability; I want these vectors to be adaptable across multiple assistants, granting me finer control over their content and specificity.

1 Like
13

If you’re using Pinecone, you can query with metadata filters: Filter with metadata - Pinecone Docs

2 Likes
14

yes i have same issue here. need to serate vector stores and need to identify user questions to direct correct vector store