I just tested this on my end and it seems to be working. Are you trying this out in the playground? Is the file_search tool enabled?
You may want to include something like “Answer questions based on documents provided” in the assistant instructions and then, ask a question about something inside your file, instead of asking if the assistant can see the file or not.
Today there is new problem when “file_search” tool is enabled, the run stays in incomplete state.
2024-05-01 12:05:04,004 - INFO - _process_messages_non_streaming - Processing run: run_sdXgpSFJDmNhPEUtOlac1dUY with status: incomplete
When I cancel the run, I get error back
2024-05-01 11:57:47,537 - ERROR - process_input - An error occurred while processing the input: Error occurred during processing messages: Error occurred during non-streaming processing run: Error code: 400 - {‘error’: {‘message’: “Cannot cancel run with status ‘incomplete’.”, ‘type’:
So I cannot really test the original issue at the moment due to whole file_search functionality seems looping in incomplete state.
I am having the same issue. My files are not being read properly through the API I set up. As I am new to this I assumed it was my prompt and my code. But when I ask the assistant to list the contents of the reference document it just makes it up.
This is the thread ID: thread_x7UDFy0N7OwrMSvrK22JSvpE
Update - I have tested all of the models with the API calling the assistant. The only one that searches the file is the grpt 4 turbo. ( I only did one test per model). However - with further testing I have found that to be inconsistent also. I’ve tried txt files, Pdf’s and word documents - all have the same issue. Looks like file search broke!
I used Assistant Playground and found the same issue too. The assistant is just not reading the attached file in the thread as is. It often outputs data that doesn’t exist in the attached file.
same for us. assistant don’t able to read files content and answer question based on it, even though file is attached to the thread. thread id: thread_YQSyJ7MyyK2S4nfH0M6HCr75
also it’s quite often has a problems sending request in general after file was uploaded. seems like file searh and file upload have some problems in general. we’re very often experience this issues
It looks like the model is invoking code_interpreter in the thread you listed (instead of file_search). Could you try prompting the model to use file_search instead? Alternatively, you can turn off the code_interpreter tool for that particular run using https://platform.openai.com/docs/api-reference/runs/createRun#runs-createrun-tools
Hello, I tried using your proposed solutions but I don’t get consistent results.
Here is a thread ID that you can look into: thread_VZDnS8w95AyjgpUYxp5oKQWS
Here is my code:
from openai.types.beta.threads.message_create_params import Attachment, AttachmentToolFileSearch
from OpenAI import client
# Upload your pdf(s) to the OpenAI API
file = client.files.create(
file=open('nda.pdf', 'rb'),
purpose='assistants'
)
# Create thread
thread = client.beta.threads.create()
print(thread.id)
# Create an Assistant. It has to have
# "file_search" tool enabled to attach files when prompting it.
def get_assistant():
return client.beta.assistants.create(
model='gpt-4o',
description='You are a data retrieval assistant.',
instructions="You are a helpful assistant designed to return all the data from the attached files.",
tools=[{"type": "file_search"}],
name='My Assistant Name',
)
# Add your prompt here
prompt = "What is included in the attached file? Return the full text please. Please use the file_search tool."
client.beta.threads.messages.create(
thread_id = thread.id,
role='user',
content=prompt,
attachments=[Attachment(file_id=file.id, tools=[AttachmentToolFileSearch(type='file_search')])]
)
# Run the created thread with the assistant. It will wait until the message is processed.
run = client.beta.threads.runs.create_and_poll(
thread_id=thread.id,
assistant_id=get_assistant().id,
timeout=300, # 5 minutes
tools=[{"type": "file_search"}],
)
# Eg. issue with openai server
if run.status != "completed":
raise Exception('Run failed:', run.status)
# Fetch outputs of the thread
messages_cursor = client.beta.threads.messages.list(thread_id=thread.id)
messages = [message for message in messages_cursor]
message = messages[0] # This is the output from the Assistant
assert message.content[0].type == "text"
# Output text of the Assistant
res_txt = message.content[0].text.value
print(res_txt)
# Delete the file(s) afterward to preserve space (max 100gb/company)
client.files.delete(file.id)
file = client.files.create(
file=open('invoice.pdf', 'rb'),
purpose='assistants'
)
client.beta.threads.messages.create(
thread_id = thread.id,
role='user',
content=prompt,
attachments=[Attachment(file_id=file.id, tools=[AttachmentToolFileSearch(type='file_search')])]
)
# Run the created thread with the assistant. It will wait until the message is processed.
run = client.beta.threads.runs.create_and_poll(
thread_id=thread.id,
assistant_id=get_assistant().id,
timeout=300, # 5 minutes
tools=[{"type": "file_search"}],
)
# Eg. issue with openai server
if run.status != "completed":
raise Exception('Run failed:', run.status)
# Fetch outputs of the thread
messages_cursor = client.beta.threads.messages.list(thread_id=thread.id)
messages = [message for message in messages_cursor]
message = messages[0] # This is the output from the Assistant
assert message.content[0].type == "text"
# Output text of the Assistant
res_txt = message.content[0].text.value
print(res_txt)
# Delete the file(s) afterward to preserve space (max 100gb/company)
client.files.delete(file.id)
The issue seems to be, that the files attached to a message are added to a thread vector store and that, depending on the load on the openai side, the vector store did not process the files when the creation of the answer starts. If you create a vector store for the thread yourself and upload and attach the files to to it and wait for the vector store to process the files, the only other thing you need to do it to create an additional message stating, that all user queries refer to the files (you need to list the file names).
When you do that, afterwards the files are properly used and your query results are based on the file content.
Most use cases require to upload new files upon each request/message.
I understand that the file might not be processed right away which is a bummer but your proposed solution is not ideal at all. I can think of 2 possible solutions:
The API should only return back a response once all the files are processed and ready to use.
They should expose an endpoint or whatever to check the processed status of the attached file
Until then, this issue is still open.
Edit:
This behavior is totally unexpected and unacceptable since we follow the guides created by OpenAI for a reason and we expect the things to work right away. They don’t even mention this limitation anywhere as far as I checked. For each little thing, we have to create an inefficient workaround to make it work for a general use case, most of the time.
I agree, that the amount of workarounds needed is far from ideal. My impression is also, that OpenAI invests much more effort into ChatGPT than into the Assistant API. But it is like it is at the moment.
Btw. what you can also always do instead is not to use the OpenAI Vectorstore infrastructure and to host your own Vectorstores and augment the prompts with the data yourself.
That is what I did, also because of the fact that there is a 100GB file storage limit for the whole organization which is not enough in the long run. I had to create my own functions to generate the embeddings for the files and supply the Assistant API with the relative chunks of data based on the user’s query.
I have the same issue. On the playground, it is working without any problem but when I try to perform the SAME request via API, the assistant can not find the file. I have uploaded the file id for OpenAI but couldn’t find a proper API to retrieve the file to see if it is uploaded correctly.