I am using RAG but i find alot vector databases so what is the best vector database to use with (RAG) and what is the best solution to fit chatgpt4 tokens limitaion
1 Like
Weeeell, have you worked with DBs before? If you have, you may be able to find a vector DB that may integrate with or resemble closely what you like 
Are you looking for Free? Open Source? Enterprise Grade / organization / business? Ease of use? How big is the data?
Down in section “Benchmarks, comparisons, standardization” are a few references that have comparisons (for sure not complete): Vector Databases (are All The Rage) | by Christoph Bussler | Google Cloud - Community | Medium Maybe you could use this, at least as a start.
1 Like
Yes, I used (qdrant, chroma, and faiss) I think Pinecone is good but I didn’t use it
-I have big data to deal with and right now I use qdrant
-I need it Open Source
@Macha
1 Like
I really appreciate your help. Thank you.
@chbussler
Soo, this?
2 Likes
Yes, I am using it for my data.
Then just stick with that? I don’t see any reason to use anything different 
. Qdrant is a very popular and well respected choice for this.
I feel like it didn’t work well and I there might be something more efficient.
@Macha
In what ways? What was inefficient about it?
have you tried SuperDuperDB? It can do that and much more
1 Like
When I insert many tokens in one place, it may split into two or not take all the text, corrupting the meaning.
@Macha
No, I have never tried it. Thanks, I will try it right now.
@fern2gue
I think Chroma or Faiss would be great!
I tried using ChromaDB but it didn’t work. I think Faiss is a better option.
@fern2gue
Hi,
I tried SuperDuperDB
It worked really well,
You can try to use their in-memory or lancedb configuration for vector search support.
Thanks
1 Like
Thank you, I will give it a try.
@frankocean
Funny nobody mentioned Milvus… google “milvus vector database” and you’ll be on your way
1 Like