I’m adding a PDF file to a vector store, then using that file as part of my search and would like to delete it again after the fact. However, even though I’m getting a deleted = true response, the file remains in the vecotr store. Is this a bug people have seen previously? (or am I doing something wrong? I would think the fact it gives me a positive response to my request to delete means it alos thinks it is being deleted) Thanks
1 Like
Hey there and welcome to the forum!
How soon are you checking for its deletion? It may just be taking some time updating.
Conversely, I’m pretty sure there was a point where OpenAI expressed it may hold data for a certain period of time before it’s completely wiped. I am not sure if what you’re seeing is an instance of that or not.
May I ask why this approach though? This seems rather odd, because a vector store is meant to do exactly that; store stuff. If you’re constantly dumping and deleting the same data, that would mean it’s important enough to stay somewhere, and the whole point of vectorizing data is to be able to pull up information at-will that’s already stored in an efficient manner. Meaning somewhere in your pipeline you’re deciding to add data to let a vector store decide whether or not to add that data.
1 Like
Hey. Thanks for the response and apologies for the delay.
I could definitely do this better but the file I’m uploading contains a list. That list grows with each run of the application so if I continued to upload the file each time without deleting I’d just end up with a huge replication of the data.
Appreciate there are likely a thousand other (better) ways to do this but I guess I’m just playing around to familiarise myself with how it all works rather than trying to achieve any tangible outcome just for now.
I do appreciate you coming back on my original post. And I agree with your commentary. I’ll have another think about my approach. The files are still there undeleted even though I get the positive response from the delete request.
There are two methods:
- disconnect an assistant file from a vector store
- delete a file from storage
If you want the old file gone from your OpenAI file storage, you have to take extra action. A file uploaded with purpose:assistants can be connected to multiple vector stores, and may also be left there waiting for future use, thus it is not auto-deleted at any point.