How to use chroma db as retriever

API
1

Hello all,
I am developing chat app using ChromaDB as verctor db as retriever with “create_retrieval_chain”.
Question:

  1. How can we check vector store data?
  2. how can we check whether the question got any supporting document from vector db retriever?
      # Fetch the vector database (CHROMA DB)
        vector_db = get_vector_db()
        
        # Initialize the language model with the OpenAI API key and model name from environment variables
        llm = ChatOpenAI(
                api_key=os.environ["OPENAI_API_KEY"]
                , model_name=os.environ["OPENAI_API_GPT_MODEL"]
                , temperature=0.2
            )
        # Define the prompt template for the document chain
        document_chain_prompt = ChatPromptTemplate.from_messages([
            ("system", "Answer the user's questions based on the below context:\n\n{context}"),
            MessagesPlaceholder(variable_name="chat_history"),
            ("user", "{input}"),
        ])        
        # Create the document chain using the language model and the prompt template
        document_chain = create_stuff_documents_chain(llm, document_chain_prompt)    
        
        # Define the prompt template for generating a search query based on the chat history
        history_aware_retriever_chain_prompt = ChatPromptTemplate.from_messages([
            MessagesPlaceholder(variable_name="chat_history"),
            ("user", "{input}"),
            ("user", "Given the above conversation, generate a search query to look up to get information relevant to the conversation")
        ])   
        # Configure the retriever to search for similar documents with a similarity score threshold
        vector_db_retriever = vector_db.as_retriever(
            search_type="similarity_score_threshold"
            , search_kwargs={
                    "score_threshold": 0.5
                    , "k": int(os.environ["related_doc_count"])
                }
        )
        # Create a history-aware retriever chain using the language model, retriever, and the prompt template
        history_aware_retriever_chain = create_history_aware_retriever(llm, vector_db_retriever, history_aware_retriever_chain_prompt)
            
        # Create a retrieval chain combining the history-aware retriever chain and the document chain
        retrieval_chain = create_retrieval_chain(history_aware_retriever_chain, document_chain)
        
        # Invoke the retrieval chain with the chat history and user input
        response = retrieval_chain.invoke({
            "chat_history": chat_history,
            "input": prompt
        })
        
        # Update the chat history with the new human messages
        chat_history.append(HumanMessage(content=prompt))
        
        # Update the chat history with the new AI messages
        chat_history.append(AIMessage(content=response["answer"]))
2

This works on my mac and rhel9:

So, youll need an embedding. This can be done locally using the “all-MiniLM-L6-v2” model, or you can farm it out to openai.

The way i use it is the vector db only holds IDs and vectors.
The query text is submitted to the embedding model to generate an embedding.
That query-embedding is used as the vector to check for closeness in ChromaDB.
ChromaDB returns a list of ids, and some other gobbeldy gook about the ranking of the result.
Then use the Id to fetch the relevant text in the example below its just a list. But you could write an datastore to hold your text. Then you could put other things in the database like dates, what role (user/assistant/offline knowledge import) produced the text.

I treat chromadb as a 3rd party fuzzy-logic-index to my persistence store.

I dont put “everything” in the vector store, but thats me.

# export LD_LIBRARY_PATH=/usr/local/lib:/usr/lib
#
# pip install chromadb
# pip install sentence-transformers
#

import chromadb
from sentence_transformers import SentenceTransformer

model_path = "all-MiniLM-L6-v2"
model = SentenceTransformer(model_path)

chroma_client = chromadb.PersistentClient(path="vector.db")

try: 
    chroma_client.delete_collection("vectors")
except:
    pass

collection = chroma_client.get_or_create_collection("vectors", metadata={"hnsq:space": "cosine"})

documents=[{'id': "1", 'content': "This is the haiku, it was written by a fool, it will not be good."},
    {'id': "2", 'content': "There are two paths that you can go by, but in the long run, theres still time to change the road youre on."},
    {'id': "3", 'content': "The apple iPhone was released in June of 2007."}]

for d in documents:
    vector = model.encode(d['content']).tolist()
    collection.add(embeddings=vector, ids=d['id'])

## test the index with a query

#query = "poetry"
#query = "product launch"
query = "choose a way"

qvector = model.encode(query).tolist()

results = collection.query(query_embeddings=qvector, n_results=1)
ids = results['ids'][0]

print ("Query="+query)
for d in documents:
    if d['id'] in ids:
        print ("Match="+d['content'])
3

Thank you for the reply. I found solution of chroma db open after persist in drive. My install version does not support that. I have to update chroma db to 0.5.0 to work in my environment.