Overcoming many small files using Assistants Retrieval

API
1

Hey Open AI folks :wave:

I’m interested in building an app that pulls in a bunch of news articles, summarizes them, and cites its sources.

One problem I’m facing with the way the Assistants API is structured is that it seems to lend itself to individual larger files, per the below limit:

…a maximum of 20 files per Assistant, and they can be at most 512 MB each

Because of the file number limitation I can’t treat each article as an individual file.

Some potential workarounds I’ve thought of:

1. Grouping articles together in the same large file
This feels the most natural, but based on the API responses in the playground, it doesn’t seem like I will be able to determine the which articles were actually used for the summary, as the response only includes the file IDs use for the summary, not the line numbers

2. Creating many small assistants
This will require finding some scheme for grouping the articles in groups of at most 20. Adds a lot of technical complexity and doesn’t seem like the way assistants was intended to be used.

3. Building my own RAG
This is simply leaving OpenAI and developing my own system for this.

Let me know your thoughts! Is there a better approach I can take?

2

You’ll need to use some sort of retrieval system to determine which assistants to use, which defeats the purpose.

This in my opinion is the best option. The current retrieval system is incredibly lacking and not usable in production. One powerful feature in RAG that retrieval lacks is hybrid search which combines semantic search alongside keywords.

You can use Weaviate and find much more control, much better results, and for much cheaper.

2 Likes
3

Yeah I’ve been playing around with Neum / weaviate as well but saw this offering and thought it might save me time. All good thanks for the response!