Assistant API - Error with files

[sam.b]

Hi- I created an assistant that uses retrieval and I uploaded the files using the web interface. It works using the test playground but when I when I call it from the python API, I get this error:

Since you have uploaded a file that is not accessible with the myfiles_browser tool, I won’t be able to directly make use of the contents of the file for assisting you.

[joshg]

This is killing me. I have a 42% failure rate with reading files now, all with this kind of error. Sometimes re-running with the same file works, sometimes it fails. Seems like some data cluster consistency issue maybe?

[sam.b]

Painful
Now I am getting this error:
It seems there has been a misunderstanding. The file you have uploaded is not accessible using the myfiles_browser tool as it may be.

[joshg]

It seems to cycle through a few different messages, all on the same theme of myfiles_browser not working.

[joshg]

FWIW - I added a 10 second delay between uploading a file and using it and it improved my success rate. Either that or something was fixed on the API side

[james40]

I am also getting a lot of wonky behaviour with the Assistant API and files. Lots of “I cannot access the file” even though it was uploaded minutes ago, and it’s just a simple text file.

[mattrosine]

I just posted this in a different thread:

When I try to upload a .csv to the playground assistant I get this error:

Failed to create assistant: UserError: Failed to index file: Unsupported file file-tYMNCZxn29GoTc9xtg4kAu6D type: application/csv

Even though it is a correct .csv and the data is prepared properly.

Its frustrating not knowing if I’m doing something wrong or if this is to do with the outages or rollout.

[joshg]

Late last night I started getting a new error

openai.BadRequestError: Error code: 400 - {‘error’: {‘message’: “Failed to index file: () The index ‘document_chunks_hnsw_432’ for service ‘retrieval-search-index-prod-11’ was not found.\nCode: \nMessage: The index ‘document_chunks_hnsw_432’ for service ‘retrieval-search-index-prod-11’ was not found.”, ‘type’: ‘invalid_request_error’, ‘param’: None, ‘code’: None}

[camil.popa.nica]

@joshg

• {‘error’: {‘message’: “Failed to index file: () The index ‘document_chunks_hnsw_432’ for service ‘retrieval-search-index-prod-11’ wa

Hi, did you find a fix for this ?

[apariyallc]

I create the assistants and everything was working perfectly fine until I had to go on a trip and when I returned and tried my application, then everything started to fail with this error! So, is this issue still open and no expertise could address it?!

[IAmJackHarper]

Getting the same error with the new gpt-4-0125-preview

[h.gurgenci]

The same problem for me. I have posted a full description of my experience here. It is similar to what people reort above.

[imam.muhammad]

I also have the same problem for me. is there any solutions for this ?

[ashot1]

I also have the same issue, 1 month later. Wonder why they don’t address this, or respond here.

[mrjonesbot]

Having the same issues here with gpt-4-turbo-preview

[mrjonesbot]

Hi all,

Given the latest changes to assistants (now OpenAI-beta => assistants=v2), I was able to follow this guide to get file search up and running with my assistants: https://platform.openai.com/docs/assistants/tools/file-search/quickstart?context=streaming&lang=python

The guide describes two ways to access files:

1. Create a vector store, upload a file and attach to vector store; then modify you assistant to reference the vector store. Then do the thread and run creation.

or

2. Upload file, create assistant, create a thread with a nested message which includes the file_id as a file_search attachment (will show an example), then create a run.

I’ve had success with #2, the assistant is able to access the file and I don’t need to handle vector stores or modifying assistants directly (2 fewer api calls).

Here’s some partial examples:

Create assistant:

  34   def create_assistant
  35     url = "#{@base_url}/assistants"
  36
  37     body = {
  38       model: 'gpt-4-turbo-preview',
  39       name: "My assistant",
  40       instructions: "You will analyze documents",
  41       tools: [
  42         { type: 'file_search' }
  43       ]
  44     }.to_json
  45
  46     res = HTTParty.post(url, body: body, headers: headers)
  47
  48     res
  49   end

Upload a file (body):

 140   def upload_file_body
 141     {
 142       file: @file,
 143       purpose: 'assistants'
 144     }
 145   end

Create a thread:

def create_thread(file_id:)
 119     url = "#{@base_url}/threads"
 120
 121     body = {
 122       messages: [
 123         {
 124           role: 'user',
 125           content: content(file_id),
 126           attachments: [
 127             { file_id: file_id, tools: [{ type: 'file_search' }] }
 128           ]
 129         }
 130       ]
 131     }
 132
 133     res = HTTParty.post(url, body: body.to_json, headers: headers)
 134
 135     res
 136   end

Headers for all api calls:

 147   def headers
 148     {
 149       'Authorization' => "Bearer #{@api_key}",
 150       'Content-Type' => "application/json",
 151       "OpenAI-Beta" => "assistants=v2"
 152     }
 153   end

Creating the assistant, uploading the file, and creating a thread in this way allow my runs to succeed as the assistant is able to locate the file and run analysis.

Be sure to OpenAI-Beta header to v2 as well.

Hope this helps!

[tristanjw]

This is interesting, thanks.

From my reading of the documentation, the file_search tool uses vectorStore files. This is because they are broken into vector components that facilitate easy and rapid search.

Therefore, you need to add the file to a vectorStore and then add that store as a tools_resource in your thread.

It looks like your thread payload doesn’t put the file in a vectorStore nor does it give the ‘file_search’ tool access to a vectorStore.

I’m not too sure what to make of this if your assistant is operating effectively!

[mrjonesbot]

You can bypass creating a vector store directly, since threads created with a Message attachment, will do so automatically; this is described in step 4 of the documentation previously linked.

You can also attach files as Message attachments on your thread. Doing so will create another vector_store associated with the thread, or, if there is already a vector store attached to this thread, attach the new files to the existing thread vector store. When you create a Run on this thread, the file search tool will query both the vector_store from your assistant and the vector_store on the thread.

https://platform.openai.com/docs/assistants/tools/file-search/step-4-create-a-thread

All that should be needed, is the file_id from step 2 (upload files), referenced in your prompt.

[StudioProjectie]

The OpenAI backend needs some time to transform the file into a searchable vector index and add it to the vector database. The time that takes changes (depending on server load and filesize). I find that in most cases it will have the file available in 30 seconds.

However, documentation says that it will normally try for 60 seconds. You will need to add polling functionality to check when the vector database is updated.

[james.novotny]

I’ve found that even if you have a Vector Store attached to an Assistant, it will not automatically be available via the API. When you retrieve the assistant via API, you can see the tool resources. However, the Assistant will not have access to them. What I’ve done is retrieve the Assistant, pull the tool_resources from it and then attach those to the Thread. I’d say there is a bug here. If a Vector Store is attached to an Assistant, why have to jump through hoops to use it via the API?