I am trying to set a specific ‘namespace’ from a Pinecone index so that I can have different chat bots answer from different datasets without having to create separate indexes. I am setting the name space as follows:
docsearch = Pinecone.from_existing_index(
index_name,
embeddings,
namespace=namespace
)
This works for the returned ‘sources’, which are from the correct ‘namespace’, however it will still provide answers based on information from all namespaces. So it seems to ignore the namespace setting when answering the question, but uses the namespace when retrieving the sources. Is there any way to get it to be strict for both?
Here is the QA chain I am using:
qa_chain = ConversationalRetrievalChain.from_llm(
llm=llm_retrieval,
retriever=docsearch.as_retriever(search_kwargs=search_kwargs),
return_source_documents=True,
combine_docs_chain_kwargs={“prompt”: prompt},
max_tokens_limit=3500,
rephrase_question=False
)