Best practice for a big RAG

Honestly, I’d probably break it down into smaller chunks, like 1000-1500 words each. That way, I can focus on one chunk at a time and use ChatGPT3.5 to help me extract the key points. It’s like eating an elephant, one bite at a time!

Another approach I’d take is to use some keyword extraction techniques to identify the most important phrases and keywords. That way, I can quickly see what the document is about and what’s most relevant.

If I had to get really manual, I’d just sit down and read the thing, taking notes as I go. It’s old school, but sometimes that’s the best way to really understand what’s going on.

Lastly, I might use some other tools, like spaCy or NLTK, to help me preprocess the document and extract key points. It’s like having a team of experts helping me out!

So yeah, that’s how I’d tackle that beast of a document!

1 Like