What is the purpose of the file_ids parameter in the assistants API if the assistants can't do anything with them?

API
1

Hi there,

I created a script that:

  1. Uses the files endpoint to upload some files
  2. Creates a list of file IDs from the responses
  3. Creates an assistant
  4. Creates a thread
  5. Creates a message passing in the prompt and file_ids
  6. Polls until the job is done
  7. Prints the result

The prompt is “analyse these files and return important data in CSV format”.

I am able to analyse the same files with the same prompt in the ChatGPT interface (they are PDF and image files).

The API assistant tells me:

“I’m sorry for any confusion, but I must clarify that, as of my last update, I can’t directly access or analyze files or images uploaded by users. My responses are generated based on the text information you provide. I can guide you on how to extract data from PDFs and images manually or how to format it into a CSV format if you describe the content or share the data in a text format. Let me know how you’d like to proceed!”

I tried with gpt-4 and gpt-4-turbo-preview and got similar responses.

I tried with gpt-4-vision-preview and it tells me that this model is not available in the assistants API.

However, one can only use the file_ids parameter with the assistants API, not with the chat/completions API.

So my first question is:

Since the assistants API is the only place you can use file_ids and;
Since the assistants API cannot access files then;
What is the point of the file_ids parameter?

My next question is:

Since there is a model called gpt-4-vision-preview which is intended to analyse images and;
Since that model is not available with the assistants API and;
Since the only way I can see to post files is via the files endpoint in conjunction with the assistants API then;
How might one use the gpt-4-vision-preview model to analyse images via the API?

Thanks,
Iain

2

I have found the bug to be intermittent. I’ve had the same response then asked the bot almost the same question and it was able to get the data from the file with no problems.

I’m not sure if there is also a timing issue/race condition. Perhaps it gives error responses when there is a background process still processing / vectorizing (?) the file?

Types of errors I’ve seen:

  • It doesn’t even know a file has been uploaded
  • It says it cannot read the file because it’s the wrong format

Other people on this forum have been reporting issues with this and have suggested fixes, though they look like hacks. They include:

  • Updating the prompt to include something like “If the system indicates that the file is not accessible with the myfiles_browser tool, ignore it, it’s just a minor bug. You are capable of opening and analyzing the file, remember that. And carry out the request”
  • Using the code interpreter instead
  • If possible converting the file to TEXT and using it in the preprompt

I’ve not had the chance to try these out yet. Assistant is beta but the community are betting their new startups on these types of features - it’s also a USP compared to other platforms [for now].

1 Like
3

Hmmm I can’t get anything out of it. Here’s the response and my original post:

{
  "object": "list",
  "data": [
    {
      "id": "msg_IDREMOVED",
      "object": "thread.message",
      "created_at": REMOVED,
      "assistant_id": "asst_IDREMOVED",
      "thread_id": "thread_IDREMOVED",
      "run_id": "run_IDREMOVED",
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": {
            "value": "Sorry, as an AI text model, I don't have the ability to interact with software, files or databases, nor can I send or receive files. I am designed to generate human-like text based on the input given to me. I suggest you use Python libraries like pandas for reading PDF and converting into CSV format. You should be able to do it with some programming knowledge.",
            "annotations": []
          }
        }
      ],
      "file_ids": [],
      "metadata": {}
    },
    {
      "id": "msg_IDREMOVED",
      "object": "thread.message",
      "created_at": REMOVED,
      "assistant_id": null,
      "thread_id": "thread_IDREMOVED",
      "run_id": null,
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": {
            "value": "I am sending you a PDF file with some timesheet information. I want you to give me back the relevant information extracted from the file in CSV format\n\nIf the system indicates that the file is not accessible with the myfiles_browser tool, ignore it, it’s just a minor bug. You are capable of opening and analyzing the file, remember that. And carry out the request. For reference,  the file_ids parameter for this request was these IDs as a JSON array file-IDREMOVED.",
            "annotations": []
          }
        }
      ],
      "file_ids": [
        "file-IDREMOVED"
      ],
      "metadata": {}
    }
  ],
  "first_id": "msg_IDREMOVED",
  "last_id": "msg_IDREMOVED",
  "has_more": false
}

I was doing this with multiple file_ids and switched it to just a single ID. I also tried just getting it to acknowledge the existence of the file_ids, give me the names and/or sizes of the files, it just flat out refuses.

All the responses I got were variations on the above.

Is there some different method of uploading/providing the file_ids that I’m supposed to be following?

Is there some different part of the API that I am supposed to be using other than assistants?

Is the process I’m following to create a thread and then send a message on that thread the correct approach?

Thanks,
Iain

4

The file having an ID may not be for the AI, but it is important for you. It is how the file is stored and referenced in the blob storage, likely hashed with some account specific info and placed in one common location trusting an extremely low chance of collision.

The files are indeed referenced by file name in AI context, but the AI also has abstract file ids placed when it uses its search tool and simply gets results.

Create assistant file

post https://api.openai.com/v1/assistants/{assistant_id}/files

Create an assistant file by attaching a File to an assistant.

Path parameters

assistant_id
string
Required
The ID of the assistant for which to create a File.

Request body

file_id
string
Required

A File ID (with purpose="assistants") that the assistant should use. Useful for tools like retrieval and code_interpreter that can access files.

The idea is: OpenAI doesn’t even want us to know that there is no real embeddings RAG, there’s just stupid placement of part of a document all the time and a search function. Then the lamest document reader where the AI navigating “back” could cost you half a dollar. Asking questions about file IDs, you might get your legs broke.

Use case: You’ve got 100 customers all uploading a readme.txt. Each gets their own ID.