Vector Store List Files Bug - Deleted files show as In Progress

I see kind of similar posts to issue with vector stores and file issue, but either OpenAI considers Vector Store an afterthought and has no desire to make it work for professional use or it is just buggy?

Issue #1:
Using API

https://api.openai.com/v1/vector_stores/{vector_store_id}/files

It returns a bunch of files that are all “in_progress:” which were deleted via playground a while ago.

Issue #2:

https://api.openai.com/v1/vector_stores/{vector_store_id}/files/{file_id}

Using that endpoint and deleting the listed files… does nothing! It reports as deleted, but then whe reading the list of files again, I still see all the files…

Now when using https://api.openai.com/v1/vector_stores to get list of stores, the files count does show 0 which should be correct.

So there is a 100% disconnect between endpoints. Is it a bug or a “feature” that OpenAI doesn’t plan on addressing? Hoping to use the entire OpenAI ecosystem of agents/chatkit/vector store or should I look for more reliable alternatives?

Update:
The behavior seems to happen where deleted files get “stuck” into in_progress and keep showing in endpoint to list files for a vector store. They can’t be found if using endpoint to retrieve a file and they do not count in file_counts when getting details about the vector store (Expected behavior).

So clearly a flagrant bug that makes that part of API 100% unusable :frowning:

I have been facing a similar issue for quite a while now. I have created workflows that update vector store daily. By update I mean, detaching older files from the vector store (https://platform.openai.com/docs/api-reference/vector-stores-files/deleteFile?lang=python) and then permanently deleting the file object itself (https://platform.openai.com/docs/api-reference/files/delete?lang=python).

The workflows upon completion report no error but when I print the files inside a vector store, I get a 404 ‘file not found’ error. Upon more debugging, I figured out that the status of these files is still stuck at “in_progress”.

I have given up all hopes on the Vector Store endpoints (@OpenAI) but since I have built some of the workflows on this, I would appreciate any help. (*beggars cannot be choosers*)

Given the fact that the bug exists in the first place, it has not been fixed in months most likely, It’s probably fair to assume that the whole vector store is probably a vaporware.

I would have thought the vector store would have actually been an important part of the effort to get more developers to stay in an Open AI ecosystem. It ties nicely with other SDK, it makes it effortless in many ways. Instead it is pushing my team (Tier 5 customer) to look for alternatives ASAP and reduce our independence on OpenAI to the bare minimum.

But we still can use 1-800-chat-gpt to communicate with chat gpt! :wink: