Biggest problem I’m seeing with RAG is keeping the LLM focused but maintaining context. I know I know, preaching to the choir. An example, you’re creating a document that is complex, maybe an RFP (request for proposal). You have the document that you are creating and you have a document that describes how to respond to the RFP. You want the LLM to interact with you to complete and grade each section of the RFP. You can’t drift RFP sections are very specific. Right now I create a parent child doc definition. I use the parent doc to provide context, like AI “hey you’re missing section 1A and 3C. Which one would you like to work on?” Human “3C”. Similarity search for 3C description and guidance. Use this section specific guidance for this chat. Currently I’ve set up a “plan and execute” agent using the main doc description to drive the plan then iterate over the sections using an “evaluate” tool with the agent to drill into each section.
If someone has a better strategy I’m all ears, I’m, I mean eyes :