RAG and ranking/weight documents

Hi, can I easily rank/weight answers from documents differently in the demo-solution available in Azure (RAG)?

Let say, if I have different documents that cover similar things, but maybe could answer the same question in different way. Can I specify that the content from one document is more valuable than the other?

In another word, force it to take answer from the document that have precedence if the answer is found in both?

Guessing that it will be building a custom indexer + and making some logic about scoring profiles, but im not sure.

Is there some other simpler approach out there?

Welcome to the forum!

Why not do the obvious and just see if the first document has an answer, if not check the second. If you want to do the queries in parallel that can work just do the selection in sequence.

You could put a weighting factor on each document.

The default weight for each document would be 1.0. Any others would be bigger than 1.0 if they are more important.

But let’s say you have a really important document that you’d like to boost your ranking on. Then, put that weighting at, say, 1.1 or 1.2.

When you do the cosine similarity, just multiply the value by this weighting factor if the similarity is positive. Divide by the factor if the similarity is negative.

Be sure not to exceed the bounds beyond -1 or 1 after multiplying/dividing if you want to take the inverse cosine to get the angle.

Intreresting,
I have only seems weights for fields in scoring profiles, but is there also a setting that could be set per document somewhere?
Or do you mean the content field (which actually is the document)

I guess “per document” is not “out of the box”-functionality?
Is it to set the weight per “document” when indexing that the way to do it?

At the moment i have one container with different pdfs.
Some documents are more importent then others.

I could split it up to different containers depending on importance of the documents if it make anything easier, but it could be many containers.

Independent on splitting on different containers or not, it sounds to me i have to build a custom indexer anyway?

I’m assuming a custom implementation. So your in-memory structure has three fields

Hash/Index, EmbeddingVector, DocWeight.

So correlate on EmbeddingVector in memory, get your top hits, multiply/divide by DocWeight, and use Hash/Index to retrieve the document in another database. Or if you want to avoid a DB and store the text in memory, then you need another field DocContents containing the text.

You could containerize/shard by importance too. Then multiply/divide using a global constant per container.

The same idea come over me at the same second i posted the question.

One way could be to do it in sequence, checking if i got any answer or not.
If no answer, I do a new search against the less important documents and so on…

I have to think about how it would be to administrate such logic
Also the grainularity, if it would be set on documents or containers with documents.