Error in creating vector store

Hi,

We are unable to create vector store

vector_store = client.beta.vector_stores.create(
        name='Case Documents', file_ids=uploaded_file_ids)

Getting Error openai.InternalServerError: Error code: 500 - {‘error’: {‘message’: ‘The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID req_aa8caa30d3e9f13e00e0df6993d2604e in your email.)’, ‘type’: ‘server_error’, ‘param’: None, ‘code’: None}}

I tested last time couple of days back when it was working perfectly fine but now its not working at all. Let me know whats the problem here!

Thanks!

Could you try again now please?

Still same error

openai.InternalServerError: Error code: 500 - {'error': {'message': 'The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID req_a2b6994191f4a2407dca88c15fdf5a83 in your email.)', 'type': 'server_error', 'param': None, 'code': None}}

I’m able to, over and over…

Attempting to upload files...
  Uploading file: example.txt ...
    Successfully uploaded 'example.txt' with ID: file-VqKt5WgcYZwo7ChJwrZ248
  Uploading file: text1.txt ...
    Successfully uploaded 'text1.txt' with ID: file-Xj3ZfXfzRvvvgo4TEtbXX1
All files uploaded successfully.

Attempting to create vector store named 'Talk to Dox dB'...
Successfully created vector store with ID: vs_67a48159b2088191b70f30540301d68d

Polling the vector store for completion status...
  Vector Store vs_67a48159b2088191b70f30540301d68d status: in_progress (1/2 completed, 1 in progress, 0 failed, 0 cancelled)
  Vector Store vs_67a48159b2088191b70f30540301d68d status: in_progress (1/2 completed, 1 in progress, 0 failed, 0 cancelled)
  Vector Store vs_67a48159b2088191b70f30540301d68d status: in_progress (1/2 completed, 1 in progress, 0 failed, 0 cancelled)
  Vector Store vs_67a48159b2088191b70f30540301d68d status: in_progress (1/2 completed, 1 in progress, 0 failed, 0 cancelled)
  Vector Store vs_67a48159b2088191b70f30540301d68d status: completed (2/2 completed, 0 in progress, 0 failed, 0 cancelled)
All files in the vector store are completed.

Attempting to create assistant named 'Talk to Dox assistant'...
Successfully created assistant with ID: asst_leWIAXUiQKMVOIsYBIa8MXfv

Attempting to create thread...
Successfully created thread with ID: thread_Aup2Wgto8yRXkM6qAplA93Re

Attempting to create and poll a run for the thread...
Successfully created and completed run with ID: run_DZ0Ncxo3cw8r4oueGPxzh1qK

Attempting to retrieve the newest message from the completed run...
Successfully retrieved message. Below is the response:

{
  "id": "msg_CMz0z04b83l099ZLyZ5MYtiU",
  "assistant_id": "asst_leWIAXUiQKMVOIsYBIa8MXfv",
  "attachments": [
  ],
  "completed_at": null,
  "content": [
{
      "text": {
        "annotations": [
{
            "end_index": 464,
            "file_citation": {
              "file_id": "file-Xj3ZfXfzRvvvgo4TEtbXX1"
            },
            "start_index": 452,
            "text": "\u30104:2\u2020source\u3011",
            "type": "file_citation"
          },
{
            "end_index": 1145,
            "file_citation": {
              "file_id": "file-Xj3ZfXfzRvvvgo4TEtbXX1"
            },
            "start_index": 1133,
            "text": "\u30104:0\u2020source\u3011",
            "type": "file_citation"
          },
{
            "end_index": 1157,
            "file_citation": {
              "file_id": "file-Xj3ZfXfzRvvvgo4TEtbXX1"
            },
            "start_index": 1145,
            "text": "\u30104:1\u2020source\u3011",
            "type": "file_citation"
          }
        ],
        "value": """
          To attach a vector store to OpenAI assistants, you need to follow these steps:
          
          1. **Create a Vector Store**: You can create a vector store using the OpenAI
          API. Here's an example of how to do this in Python:
          
             ```python
             from openai import OpenAI
             client = OpenAI()
          
             vector_store = client.beta.vector_stores.create(
                 name="Support FAQ"
             )
             print(vector_store)
             ```
          
             This will return a vector store object that you can use later【4:2†source】.
          
          2. **Attach the Vector Store to the Assistant**: When configuring your
          assistant, you need to specify the vector store ID in the assistant's
          configuration. The configuration should include a `file_search` tool that lists
          the vector store IDs. You can attach only one vector store to an assistant at a
          time.
          
             Here’s a snippet of how you might configure the assistant:
          
             ```json
             {
                 "tools": [
                     {
                         "type": "file_search",
                         "vector_store_ids": ["vs_abc123"]  // Replace with your vector
          store ID
                     }
                 ]
             }
             ```
          
             Make sure to replace `"vs_abc123"` with the actual ID of the vector store you
          created【4:0†source】【4:1†source】.
          
          3. **Using the Vector Store**: Once attached, the assistant can utilize the
          vector store for file searches and other operations that require vector
          embeddings.
          
          For more detailed information, you can refer to the OpenAI API documentation
          regarding vector stores and assistants.
        """
      },
      "type": "text"
    }
  ],
  "created_at": 1738834280,
  "incomplete_at": null,
  "incomplete_details": null,
  "metadata": {
  },
  "object": "thread.message",
  "role": "assistant",
  "run_id": "run_DZ0Ncxo3cw8r4oueGPxzh1qK",
  "status": null,
  "thread_id": "thread_Aup2Wgto8yRXkM6qAplA93Re"
}
 
Beginning cleanup of all created resources...
  Attempting to delete file with ID: file-VqKt5WgcYZwo7ChJwrZ248...
    Successfully deleted file: file-VqKt5WgcYZwo7ChJwrZ248
  Attempting to delete file with ID: file-Xj3ZfXfzRvvvgo4TEtbXX1...
    Successfully deleted file: file-Xj3ZfXfzRvvvgo4TEtbXX1
  Attempting to delete vector store with ID: vs_67a48159b2088191b70f30540301d68d...
    Successfully deleted vector store: vs_67a48159b2088191b70f30540301d68d
  Attempting to delete assistant with ID: asst_leWIAXUiQKMVOIsYBIa8MXfv...
    Successfully deleted assistant: asst_leWIAXUiQKMVOIsYBIa8MXfv
  Attempting to delete thread with ID: thread_Aup2Wgto8yRXkM6qAplA93Re...
    Successfully deleted thread: thread_Aup2Wgto8yRXkM6qAplA93Re
Cleanup process complete.

The AI answers how to use vector stores in that console log, because it uses file_search to learn :laughing:

You don’t show how you are getting the files there. This should do the trick.

file_paths = [
    "example.txt",
    "example.doc",
]
uploaded_file_ids = []
vector_store_id = None
try:
    """
    Upload Files
    """
    print("Attempting to upload files...")
    for file_path in file_paths:
        print(f"  Uploading file: {file_path} ...")
        with open(file_path, "rb") as file:
            response = client.files.create(
                file=file,
                purpose="assistants",
            )
            uploaded_file_ids.append(response.id)
        print(f"    Successfully uploaded '{file_path}' with ID: {response.id}")
    print("All files uploaded successfully.\n")
except Exception as e:
    print(f"handle your exception: {e}")

Make sure the files are supported. The API can do some funky inspection, calling a text file a CSV and rejecting it, for example.

You could recheck the response object of each file upload, made with purpose:assistants, and ensure it is successful. Example:

{
  "id": "file-abc123",
  "object": "file",
  "bytes": 120000,
  "created_at": 1677610602,
  "filename": "salesOverview.pdf",
  "purpose": "assistants",
}

A vector store creation like yours - with all the parameters ready for you to tune:

try:
    """
    Create a new Vector Store
    """
    print(f"Attempting to create vector store named '{name} dB'...")
    vector_store = client.beta.vector_stores.create(
        name=f"Case Documents",
        file_ids=uploaded_file_ids,
        expires_after={
            "anchor": "last_active_at",
            "days": 60,
        },
        chunking_strategy={
            "type": "static",
            "static": {
                "chunk_overlap_tokens": 200,
                "max_chunk_size_tokens": 600,
            },
        },
    )
    vector_store_id = vector_store.id
    print(f"Successfully created vector store with ID: {vector_store_id}\n")
except Exception as e:
    print(f"handle your exception: {e}")

Files are pretty much immediately available to add. You could put a time.sleep(5) in there to ensure the blob storage settles down anyway.

1 Like

I don’t why but I keep on getting same error again and again. I tried deleting whole assistant and recreated back again but still no luck.

You can also try: creating a vector store without files.

Then add them one-by-one in individual API calls.

You can also try to create a vector store via the platform.
A maybe related issue has been reported here:

Also, please note the the comment in the first post:

I just confirmed that I can attach files to the thread vector store manually through the platform. It seems to mainly be failing on the API

they broke the api similar issues here w code that hasnt changed and jsut stopped working today

Update, I just tested again the assistant api and everything went through successfully!

Going to run the test few more times and will report back again if there’s any issue.

1 Like

Several bugfixes have been deployed yesterday. AFAIK the problem did not affect all users and it will be helpful if you can confirm that the problem has been resolved or follow-up with any ongoing issues.

@olajideogun123

1 Like

Initials test seem to be looking good. It would be nice to have communication next time or maybe i do not know the proper channels to get updats, but hard to build a product like this

1 Like