When my AI Assistant does not have enough information about something it always responds with something like this
“the document does not specifically discuss this topic”
How can make the Assistant stop referencing the document to the end-user?
We have seen this issue reported quite a bit, and our own experience is that the assistant tends to try and use any and all documents it has access to.
The only ways I know to limit it are:
- Specify this same issue in the instructions: “do not reference documents in your responses unless the user explicitly asks for informacion from your knowledgebase or asks about specific documents”
- Have a layer above your assistant that evaluates the user query and decides if documents should be used or not. If not, remove them from the thread / start a new thread with the old messages but no file ids. If yes, append to the user question which documents the assitant should rederence. “use only the following documents when answering the query…”
Happy building!
Thank you! I did try to specify in the instructions but it seems it did not follow the instructions the way I wrote it at first. Now I’m trying this version:
If the document does not provide detailed information about a question please politely decline to answer without mentioning the document.
This seems to be working OK for now. Seems like you have to try to write the instructions in different ways to see what works. P.S. I actually want my assistant to always reference the documents, I just don’t want it to mention anything about it’s sources to the end-user.
Uf that can be more complicated. Because the system prompt of an assistant using the Browsing tool tells the assistant to use a quote function to log everything that it uses from the original doc.
if not sharing information about the doc or the fact that it read is a hard requirement, you may want ot add a simple gpt-3.5 layer after the assistant response, to reformat it to avoid any mentions. To this layer you can basically ask (in more elaborate prompting) “take this response from our system and eliminate any mentions of the document that has been used, maintaining all the information, and avoiding direct references”…
Otherwise, more heavy prompt engineering in the system prompt where you use the same language as their core system prompt. This can be more effective at cancelling that default behavior.
Good luck!
1 Like
ah gotcha makes sense and good idea… I’ll consider implementing something like that if needed. So far my assistant seems to be omitting any reference of “the document” now after those above instructions. So far so good, let’s see.
1 Like