How can RAG systems be improved for more complex queries

RAG generally works well answering questions with concrete answers that can be extracted from one or a few chunks of text from the document. On the other hand, document summarization tasks are better performed using map-reduce or refine techniques.

There are however certain complex questions that are a bit in between those two categories. For example: “what are the requisites to apply for this grant?”, where the actual requisites might be spread over several pages of text. The usual top-k similar chunks approach would be insufficient to answer these kinds of questions, as it would require the parsing and synthesis of extensive sections of text to give a valid and complete answer.

Determining the type of question in advance, while developing RAG systems capable of handling both straightforward and complex questions, doesn’t seem to be an easy task either.

I wanted to ask the community for your thoughts about this issue and any ideas or best practices to approach it. If you know of any references, papers or related active areas of research would be awesome too.


Wild guess: HyDE

“Precise Zero-Shot Dense Retrieval without Relevance Labels” by Luyu Gao, Xueguang Ma, Jimmy Lin and Jamie Callan

1 Like

Thank you @EricGT! Will read it in depth


HyDE is used by Discourse for parts of the AI in this forum.

That is where I first learned of HyDE.

1 Like