I understand that we can attach up to 10,000 files to each vector store. But I am wondering how granular should the content of each file be. For example, should I keep 10 FAQ points in a single file or split them into 10 files, with one FAQ point in each file?
Would there be a difference in accuracy, cost, and/or speed?
With the chunking of documents for Assistants’ vector store, you don’t have control of the split points or the size of chunks.
So the size of the files itself is what can affect the chunks. Can’t split up documents to 800 token chunks if all of the documents are 400 tokens! This can also make a useful cost limit on a system that will load up the AI with 20 chunks of uselessness when it searches; you can make the chunking yourself with small documents.
Then we get to the point of your matter: you want an AI to answer questions. Does it need to answer “show me your FAQ” - or does it need to answer based on individual facts. Will the relevance be higher or lower on an article that has ten points. Will you displace uploaded documents when combining with tons of factoids?
The ultimate problem is that assistants has a search function that must be queried, the AI not knowing what it will find, a retrieval that doesn’t have a threshold of relevance cutoff, a chunking that doesn’t attempt to split semantically but instead wants you to pay for text overlaps, a document extraction that over-promises, etc - all are general-purpose and ideal for no application.