Embeddings not working as well as I hoped

[PaulBellow]

Welcome to the forum.

I’d take a look at this new thread…

Document Sections: Better rendering of chunks for long documents

I wanted to share with the community some work I’ve been doing around Retrieval Augmented Generation (RAG) using long form documents. I’ve developed a new technique called “Document Sections” which I feel is superior to traditional techniques being employed today. Here’s an example of the chunks returned by a vector DB like Pinecone. This is the chunk output from Vectra, my local vector DB project, for a query over a small corpus of documents from the Teams AI Library I designed. [image] Th…

Good stuff that might help you…