Vector Stores are not retrievable via list all

Currently the list vector stores endpoint in the OpenAI API is not returning any vector stores. Getting the vector store directly via id is working fine, but using the list all will yield no results.

I used the following endpoint as described in the API reference:

curl --location 'https://api.openai.com/v1/vector_stores' 
--header 'Authorization: Bearer OPENAI_APIKEY' 
--header 'Content-Type: application/json'
--header 'OpenAI-Beta: assistants=v2'

I can even reproduce this behavior directly on the platform.openai.com site.

Pressing ā€˜Create’ on the Vector Store tab in the Storage page will successfully create a vector store. Refreshing the page and going back to the vector stores tab will not list any vector stores still. Searching for the vector store by providing an id in the URL will show it: https://platform.openai.com/storage/vector_stores/vs_someid

This behavior is happening across all our projects since today morning.

Thanks!

2 Likes

I called a few times and reloaded a few times. Seemed healthy on my few old examples, which is all I have.

Here’s some Python written to not fail on you.

  • You can put api_key="sk-proj-your_api_key_string" directly in code if local for assurance.
  • You can put in a loop and keep asking for your vector stores with a bit of time.sleep(5) to report on intermittent issues, or increase the default count.
  • Default results are max 20, and the API maximum return is 100. The playground pages 10 at a time.
import os
import json
import gzip
import zlib
import urllib.request
import urllib.parse

def decode_content(data, encoding):
    """
    Decompress response body according to Content-Encoding header.
    Supports: gzip, deflate. Other encoding values are returned as-is.
    """
    if not encoding:
        return data
    enc = encoding.lower().strip()
    if enc == "gzip":
        return gzip.decompress(data)
    if enc == "deflate":
        return zlib.decompress(data)
    return data

def list_vector_store_ids(
        limit=20,
        after=None,
        before=None,
        order="desc"
    ):
    """
    Fetch vector store entries from OpenAI and return a list of IDs.

    Parameters:
      limit   - max number of items to return (1–100, default 20)
      after   - cursor ID to fetch items after this ID
      before  - cursor ID to fetch items before this ID
      order   - "asc" or "desc" sort on created_at (default "desc")

    Returns:
      A list of string IDs for each vector_store in the response.
    """
    api_key = os.getenv("OPENAI_API_KEY")
    if not api_key:
        raise EnvironmentError(
            "OPENAI_API_KEY environment variable is not set"
        )

    # build query parameters
    params = {}
    if limit is not None:
        params["limit"] = str(limit)
    if after:
        params["after"] = after
    if before:
        params["before"] = before
    if order:
        params["order"] = order
    qs = urllib.parse.urlencode(params)

    # assemble the full URL
    url = (
        "https://api.openai.com/v1/vector_stores"
        "?" + qs
    )

    headers = {
        "Accept": "*/*",
        "Accept-Encoding": "gzip, deflate",
        "Accept-Language": "en-US,en;q=0.5",
        "Authorization": "Bearer " + api_key,
        "Cache-Control": "no-cache",
        "Connection": "keep-alive",
        "Host": "api.openai.com",
        "OpenAI-Beta": "assistants=v2",
        "Pragma": "no-cache",
        "Priority": "u=0",
        "User-Agent": (
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:143.0) "
            "Gecko/20100101 Firefox/143.0"
        ),
    }

    req = urllib.request.Request(url, headers=headers, method="GET")
    with urllib.request.urlopen(req) as resp:
        raw = resp.read()
        encoding = resp.getheader("Content-Encoding")
        body = decode_content(raw, encoding)
        data = json.loads(body.decode("utf-8"))

    ids = []
    for item in data.get("data", []):
        vid = item.get("id")
        if vid:
            ids.append(vid)
    return ids

if __name__ == "__main__":
    # Example usage: fetch up to 10 most recent stores
    vector_ids = list_vector_store_ids(limit=10)
    for vid in vector_ids:
        print(vid)

'''
The vector store list call returns a list of items. If there was only one:

{
  "object": "list",
  "data": [
    {
      "id": "vs_abcdef1234567890",
      "object": "vector_store",
      "created_at": 1700000000,
      "name": "example vector store",
      "description": null,
      "usage_bytes": 2048,
      "file_counts": {
        "in_progress": 0,
        "completed": 1,
        "failed": 0,
        "cancelled": 0,
        "total": 1
      },
      "status": "completed",
      "expires_after": null,
      "expires_at": null,
      "last_active_at": 1700000000,
      "metadata": {}
    }
  ],
  "first_id": "vs_abcdef1234567890",
  "last_id": "vs_abcdef1234567890",
  "has_more": false
}
'''

The API endpoint itself has rate limits for calls that are non-consuming of AI resources. They are pretty high but reachable.

Hi and welcome to the community!

I wasn’t able to immediately reproduce the issue.
What I did notice is that the curl example in your post is missing a header.

curl https://api.openai.com/v1/vector_stores
-H ā€œAuthorization: Bearer $OPENAI_API_KEYā€
-H ā€œContent-Type: application/jsonā€
-H ā€œOpenAI-Beta: assistants=v2ā€

This is from the API reference to get a list of vector stores.

1 Like

Hi, yes this header is present in my API requests. I copied it when I tried it without it. Will update my example.

Furthermore did you try to reproduce it on the platform UI itself? I think this is the easiest way to try it.

Hi thanks for the example.

The rate limits are no issue. Furthermore it is also happening on the platform UI.

I don’t have a python environment set up, so it will take a bit to try it.

Also lets try to be clear here, I’m getting a response from the API but it is empty:

{
    "object": "list",
    "data": [],
    "first_id": null,
    "last_id": null,
    "has_more": false
}

To note: when you use a project API key, and upload files with purpose ā€œuser_dataā€ as shall be done to discount the value of ā€œAssistantsā€ as a purpose, the information is scoped to a project in a highly-undocumented manner, to where you can only rely on it to break calls.

Ensure that the file upload to storage, the creation of a vector store, the attachment of the files, and the subsequent listing are all being done by the same API key, and in the platform site, the selection of its project at upper-left.

I’m using the ā€œassistantā€ purpose when uploading the files to the vector store. But this issue is already happening even though the vector store doesn’t even have files.

Furthermore it is happening even though I use the same API key for all the purposes you listed.

And it is happening on the platform UI as well, so it doesn’t seem to be an issue of incorrect usage.

Everything was working fine until this morning. Vector Store creation and listing was working still yesterday evening. No settings were changed since then.

1 Like

I think given the transition from working to non-working, and not just a failure in getting it to work, you need to blast this similar message to ā€œhelpā€ via the platform site: that your existing application with the same calls being utilized is now failing intermittently and returning no values, and it appears to be either an organization data retrieval issue or a wider problem with a few anecdotes of the same on the OpenAI forum.

Fellow developers can’t run database corruption reconciliation or whatever it would take to restore access to the data.

The workaround is the work: re-creation of the resources and see if they get listed. Or keep your own list and poll for the successes of individual uploads against individual vector store IDs.

Several days-long outages recently should be informing whether this product is serviceable to build on at all.

2 Likes

One of my first steps was to list all existing vector stores on the platform, which worked as expected. From our conversation, I gather you’re not seeing any vector stores at all, though they can still be retrieved by ID.

I’ll keep monitoring this, but in the meantime, @_j is right - submitting an error report at help.openai.com is the best next step. Be sure to include a link to this conversation to help speed up resolution.

1 Like

Thanks! I will try to get some help there but I’m not sure how as this page is not showing anything to get a report out (except the AI support which was not helpful)

I’ll also tag the OpenAI support here @OpenAI_Support

Hi,

I am also facing the same issue. I can only see vector stores created yesterday and earlier in the OpenAi Platform. When creating a new vector store using either the APIs or the UI in the Platform is says that it is created, but it is not a part of the list when I use the API to list all vector stores. If I use the vector_stor_id from the created vector store (not shown in the UI) in an API call, I get all information about that vector store. To me it seems like the List Vector stores API are having issues :face_with_spiral_eyes:

2 Likes

I am now able to confirm this issue with newly creating vector stores which are not retrieved on the platform.

I am also facing this issue, is there an estimate for when things are back to normal?

Unfortunately it is not yet acknowledged by OpenAI, so no news on that part.

Linking the second topic covering a similar/related issue here:

1 Like

I’m having the same issue. Created a vector store via API, and added documents to it via API. I can fetch the Vector Store by ID, but it is not visible in the web ui or listing stores with the API.

1 Like

Same issue here, commenting for visibility and to get notified if anything changes.

Same issue and it is blocking as we delete previous vector store after creating a new one with same name.

1 Like

I’m having the same problem.

Thanks for the reports! We’re able to reproduce it as well, root-causing it actively.

5 Likes