Hi! I am using the Assistants API and have created a vector store which contains reference material my Assistant can draw from to answer user queries. If the files in the vector store contain information relevant to the query, it should use them, but otherwise, it should answer from general knowledge.
My problem: when the Assistant does not find an answer in the files, it tells the user that the files didn’t contain the information, which I don’t want it to do.
Examples:
The files provided do not contain X
It seems that the files you uploaded contain detailed instructions on X, but not Y
The documents do not specifically mention X
Additional context: The end user does not know or care that there are files in the vector store, they just want the answer to their query.
Any suggestions? I have tried adding various instructions to the prompt, e.g. 'You have access to the attached files which you can use to answer user queries. If there is no relevant information in the files, just answer from your general knowledge.; but this doesn’t help/
So, what exactly is the information being retrieved? As in, what kind of assistant are you trying to generate?
Could you also share your system / developer prompt? I think I know what’s going on, but I want to confirm. There’s something in the way that instruction prompt is being set up to where the model is confusing the role its meant to be playing.
Hey, thanks for your response! Unfortunately I’m unable to share specific details on the assistant or system prompt, but it is a single conversational agent designed to engage in conversation and answer user queries on a particular topic, much like a customer service agent. The system prompt contains an overview of the assistant’s role, guidelines, and constraints, and does not mention anything about the files uploaded to the vector store.
I suspect that the ultimate solution is a multi-agent approach with a dedicated retrieval agent, but I’m wondering if there is a workaround stop these types of responses in the short term. Any hypothesis you can share would be much appreciated!
The instructions they give to the AI model about the file search tool says directly within that “the user uploaded files”.
You can spend a lot of effort at trying to counter the file search tool text which you cannot fix by giving a bunch of instructions.
Or you can spend the effort to make a RAG solution portable to chat completions instead of Assistants, or portable to any AI provider’s platform who is more responsive to such concerns.