Problem with doing RAG with 300k pages of PDFs

[SomebodySysop]

@apt.matrix.gs

Hi everyone, I am developing a project where an AI will have to answer questions based on 300k pages of legal PDFs. The questions the user ask will not vary that much. But what I wanted to ask is how do I build this?

GPT4 Tutorial: How to chat with multiple pdf files - The Chat Completion Process (R.A.G. / Embeddings)

• https://youtu.be/Ix9WIZpArm0?si=tKIb0RzffnU-3UPe
• My post:
  • https://community.openai.com/t/converting-pdf-files-text-into-embedding… Intelligent Search to a Rails Application with Weaviate