I am currently working with a use case where we are reducing a pdf to a feature vector through an LLM pipeline, much similar to a RAG architecture. We use LangChain, Vector Stores, Chunking, etc. The goal is to stuff a prompt with relevant data, and then have an LLM reduce data to a specific feature vector. I guess you could call it data extraction, because we are looking for certain data points within the pdfs.
We are thinking of evaluating the prompt design and optimal chunk size with RAGAS. I do have some questions:
- Would you consider this as a RAG architecture, or is it something else?
- How to evaluate the output – are there any good options to RAGAS?
- Do you know of any similar use cases I could have a look at?
Med vänlig hälsning,