You can embed at different granularities, like word/sentence/paragraph/page, etc. Search across all the granularities, then fuse the results to form an overall result (RRF/RSF). Also, you can weight each granularity differently … so paragraphs more than sentences more than words, or whatever you decide.
If each granular chunk had an index indicating the location within the corpus, you could also grab adjacent chunks to provide more surrounding context.
So search at all your granularities, fuse the result, grab the highest fused results, expand each result by some radius, then these are your chunks for RAG.