RAG questions with assistants V2

API
1

Hi there!

I am starting to work on developing a production RAG chatbot with a company’s knowledge base and I have a few questions that I’d love some help on.

  1. Does the file search tool use the metadata attached to vector store objects in any way currently? I noticed that in the known limitations section in the file_search docs it said “pre-search filtering” with metadata is an upcoming addition, but is the metadata used in the vector embedding of the document, or used in any way by file_search?
  2. Are there any workarounds to perform a ‘structured’ query prior to the hybrid search for similar chunks? All I can think of right now is to have an approach that first selects which assistant to send the query to, with separate assistants having separate vector stores. I am guessing that with the upcoming addition of metadata based presearch filtering this will be solved but any other ideas in the meantime?
  3. When annotations are returned, a start_index and end_index are included, how do I interpret these? I thought maybe they are counts of tokens in a document but they don’t seem to line up as far as I can tell?

Thanks for any input!

2 Likes
2

Hey there,

I am also wondering about this… Regarding the metadata, I suppose not. As you said they mentioned they are working on adding support for this. To my knowledge no metadata is included at all at the moment.

Regarding the structured approach: What we did at first is create separate vector stores for certain “file categories”. In our case the user selects which category they would like to inquire about in the front end. This then triggers the client.beta.assistants.update which updates the assistant to use the corresponding vector store, they same could be done at thread level. In your case you could do it automatically with another assistant or with one assistant with a forced function call before performing the file search. However, I found that this does not necessarily improve the accuracy of the file search. I also tested it with putting all files into one VS (roughly 3000 files) and it surprisingly works really well. This of course depends on your data…

Unfortunately we don’t have a lot of information about how File Search works at the moment… It would be great if the OpenAI Team would expand the documentation a bit. I posted a thread here: Assistants API File Search and Vector Stores