Assistant file search exhibits strange behaviors

API
,
1

Not sure if anyone else has experienced these issues with Assistants file search? If so, could you please tell me what you have done or the reason why it occurs?

  1. I am trying to get assistant to prefer file search tool over function tool. When given a function that searches the internet as well as file search tool, I give the instruction to use file search to find the answer. For the first few searches the assistant obeys. However, after creating a few threads the assistant only uses the function tool instead of file search. Nonetheless, it may still put a few citations in the answer. However, according to my logs, it never actually calls the file search tool. Not sure if the citations are real.
  2. I never see the assistant call both file search and the function tool, even when I instruct it to do so.
  3. The assistant will include parts of answers from the previous thread in the current thread
  4. When assistant provides citations, it will not provide a quote along with them, only provides the file id.
  5. When vectorizing large files above 10MB it will fail (vector store displays a red dot). However the advertised limit is 500MB. Uploading works fine.
  6. Randomly get errors about not being able to search files. May be because attached vector store has red dot (one of the files was not vectorized). However, I can’t delete the file that has the error.
  7. These strange behaviors for the most part do not happen in playground.
1 Like
2

I have had significant struggles when the “file search” is active. Through the playground the assistant will follow it’s system instruction many (not all) times, though through the API it will ignore all it’s system instructions. If you force it to use it’s system instructions by sending a “follow your system instructions” with each prompt. Then it ignores the file search.

1 Like