CHATGPT API with 200 massive PDF files

[Diet]

Welcome to the community!

Your options with retrieval are pretty limited, especially if you don’t want to edit and clean your documents.

In terms of using RAG in general, there’s vigorous debate on the matter. You can join the fray here:

Using gpt-4 API to Semantically Chunk Documents

I am creating this topic based upon the thread of conversations which developed from this post: RAG is not really a solution - #43 by SomebodySysop While a number of us have different approaches to the first tier of “Semantic Chunking”, like @sergeliatko 's API: RAG is not really a solution - #50 by sergeliatko we were all still looking for a way to have the LLM do that last bit of “sub-chunking”, where instead of simply chopping the chunk up by size we have the model look at it and determine t…