I’m currently working with the OpenAI API to allow users to ask questions about specific documents. My goal is to restrict the search to only one document within a vector store. However, I’ve run into a few challenges that I hope the community can help me with.
– What I’ve Tried So Far:
Using File Search Tool: I attempted to use the “File Search” tool to limit the search scope. However, the “File Search Resources” parameter only accepts the entire vector database, not a specific file or document within that database.
Instructions and Context:*I’ve tried providing detailed instructions in the API call, instructing the assistant to focus on a specific document, but it continues to search across all files within the vector store, not just the one I want to target.
–What I’m Looking For:
I would appreciate any suggestions or alternative methods to limit the file search to a single document within a vector store.
If anyone has faced a similar issue or has any creative solutions, I’d love to hear your thoughts!
Thank you in advance for your help!
Best.
PS : I’m working on make.com if it can help. But i can make direct API call if necessary.
You can’t. Best strategy I’ve found is to grab some semantically relevant information from the files themselves and add those into the message request. This has the effect of triggering the model to search out similar content … which will then reach out to those files.
I’ve found that once it “hits” the file, then the rest of the conversation it stays locked in.
If you only want to search one document in a Vector Store, then why not only form a Vector Store from that single document?
Vector Stores can be created more-or-less spontaneously. So, if you know the file you want to target, you can create a flow that attaches a File to a newly created VS that persists for the duration of your search, searches the file, provides context, then deletes itself.
You’d only want a VS with all of your files if you are trying to relate the information between different files. As you’ve noted, a model has to waste a lot of Input Tokens because it searches across all files by default.
In this case, I have had success reducing Input Tokens by using descriptive file names, explicitly telling the model which file to look at, and providing a guess on the location of the information I’m looking for.
Furthermore, there are very new ways of using File Search Ranking Options, which were quietly released this past week. I don’t know much about them yet, but they look key.
In addition to this, you can also save a mapping of Vector Store IDs saved somewhere and using function calling you can select a specific Vector Store during the conversation.
I upload the file via the API with the “purpose”: Assistants
(Vision, Batch, and FineTune don’t seem relevant to me)
Then I attach this file to the chosen vector base so that I can then use it with a wizard.
I ask my assistant for the vector base and the question with the file search tool activated.
But maybe I’m not using the right method.
@jim
Thank you very much for sharing this method, but it seems too complex.
@thinktank
Yes, it’s an option, but it poses several problems:
In terms of pricing, if I have 8,000 vector stores with a single document, I doubt they’ll charge me anything, right?
I don’t know if it’s 1GB free per vector store or 1GB for the whole account (i.e. all Vector Stores)? 1Gb free per vector store seems me too good.
And in terms of management, it’s a mess too. To think about deleting and doing a bit of sorting from time to time.
But if the price issue is resolved and it doesn’t cost more, I’ll probably go for this option.
On the subject of price, do you by any chance have an idea on how to recover the number of tokens (input and output) used in a request to a wizard (to calculate the cost), because in the API return with the wizard I don’t have this info. I need to make a request to a Run? Then on a Thread?
The first screenshot you shared, you selected File Search as tool but you attached Vector Store Ids to Code Interpreter resources and not file File Search resources. Is that a mistake or is it intended.
Vector Stores are charged per gigabyte per hour that they persist. So 8,000 small vector stores will only be expensive if you make them persist forever. I think they give us 100 free GB of persistent File Storage. (But the cost should be the same even if you have one large VS with 8000 files.)
But, TBH, mate, I think your needs are already exceeding capacity of the platform you’re trying to use. Input Token / Output Token counts are a standard statistic that the API response includes—so you’re just waiting on the producer of your platform to include the statistic.
I suggest doing your experimentation on the platform.openai.com. I am far more comfortable with Graphical User Interfaces myself. Even now that I’m learning to code Python, I still use the GUI. (I usually have vs code on one screen and the platform on another.) They’re doing a good job keeping it on par with coding access, too.
