Description: I’ve encountered an issue with the Assistant API related to file handling. When using the Assistant API’s playground, it appears to have access to files uploaded to the assistant. However, when the API is called programmatically, its behavior is inconsistent.
Steps to Reproduce:
Upload files to the Assistant.
Test file access in the API playground - files are accessible.
Make an API call to query the files (e.g., “What files do you have?”).
Attempt to perform operations on these files through the API (e.g., summarizing content or answering questions based on the file contents).
Expected Behavior: The API should be able to access and interact with the files consistently, both in the playground and when called programmatically. This includes performing tasks like summarizing content or answering questions based on these files.
Actual Behavior: The API successfully acknowledges and lists the files when queried. However, when asked to perform tasks involving the content of these files, it fails to access them, despite the fact that file IDs are attached and visible in the API response.
Has anyone else encountered a similar problem? I’m experiencing a specific issue with the Assistant API related to file handling and I’m curious to know if this is a common challenge. Any insights or shared experiences would be greatly appreciated
i have problems to upload some files. Some js files are uploadable and others are not. They are just 2kb. And sometimes the file names are missing, and then suddenly the are back again.
I get this Error: There was an error uploading the file Exit.js: Invalid file format. Supported formats: [‘c’, ‘cpp’, ‘csv’, ‘docx’, ‘html’, ‘java’, ‘json’, ‘md’, ‘pdf’, ‘php’, ‘pptx’, ‘py’, ‘rb’, ‘tex’, ‘txt’, ‘css’, ‘jpeg’, ‘jpg’, ‘js’, ‘gif’, ‘png’, ‘tar’, ‘ts’, ‘xlsx’, ‘xml’, ‘zip’]
Yes, I also have this problem and have had it for a couple of weeks now. It is inconsistent, most of the time it can access the CSV file with no problem but 10% of the time it will tell the user that it can’t access the files. There is no reason at all for this as it works again soon after. This is impacting my service greatly. Any known fixes?
I am currently experiencing a challenge with the Assistant API concerning file handling. In the Assistant API’s playground, files uploaded to the assistant seem to be accessible without any issues. However, I’ve noticed inconsistencies when the API is accessed programmatically. Could anyone provide insights or feedback on this matter? Your assistance would be greatly appreciated.
I am having a similar experience. Using ‘retrieval’ a 5 pages pdf file is provided to the assistant. I use both the API and playground to interact with the resulting assistant. Behaviour is inconsistent:
sometimes I get the right answer immediately (and then it is like magic )
sometimes I get a wrong answer and I tell the assistant to check again and the 2nd or 3rd time I get the right answer. (“Sorry I was not providing the right answer. I will check again…”)
sometimes it just keeps saying that the requested answer is not available in the provided document(s), while it clearly is.
The inconsistency that I see is making the assistant unreliable, hence useless. I am talking about 50% errors, not just the last 5%.
I have a similar problem using the API- answers that I’m looking for are all located within the uploaded files; when I test assistant in playground, I get great, well-synthesized answers; when I access via API, I get answers of poorer quality that are often totally incorrect. What’s the difference in how files are handled in playground test vs. via API?
I have a related issue where I am using function calling to have an assistant find relevant scripts in my Unity project and add them as retrieval files, which it does just fine. But it will then hallucinate content until I explicitly tell it that it does have access to that particular script. Then it can answer questions about it.
I’m noticing another issue with Retrieval. Not related to my use of the API but with the use of Playground and prompts (one-shot long prompts).
If you have a prompt that is supposed to trigger retrieval (e.g., use our previous conversation to remind me XYZ), it does work, it looks at, I don’t know, older data I have put in there; so I wonder if there is a cache issue; but it doesn’t automatically and correctly retrieve the information from “my last session” while it’s asked in my prompts.
However when conversing, as a user in interaction, if I explicitly say it’s not correct and ask to retrieve information, Retrieval is activated and the right information is retrieved.
So it doesn’t work automatically through “instructions”, but it work if I prompt it in during the interaction.
Did you observe similar things?
4 months down the line. The issue persists. thanks @b1willaert , summarised the issue well. Hope openAI fixes this issue. Is there any way we can report a bug or something ? as you said assistants are not very useful with such a high rate of inconsistent behavior.
I have run into this as well. I send a prompt (via the API) that should be answered in the uploaded files, but it does not work. It works better in playground.
With the API, I tried asking “what files do you have access to?” and it correctly answered with the list of files. Then following that prompt, retrieval was working correctly. So that seemed to prime the conversation.
Explicitly prefixing your request with “using retrieval, …” also works (as previously noted), but that’s not a great solution.
UPDATE:
I have had some success adding “Instructions”. For example, “You have access to several files containing documentation. When the user asks a question, you should use ‘retrieval’ to answer concisely.” When I added this, I more consistently got answers using retrieval.
Note with a long instruction set, it seems to work better putting this earlier. At first I added it at the end and it didn’t seem t o help, then I moved it closer to the beginning.
Another observation I have is there is some correlation to the question itself that you are hoping will trigger retrieval. If you are referencing proprietary terms in your documentation, then there is a good chance it will pull using retrieval. If you, however, are using generic terms that it understands in it’s general training, there is a chance you will get a non-retrieval response. However, once you “break the retrieval barrier”, it will start using retrieval for the latter type.
)