"Use this document info to reply to the users: {file metadata (name and description and stuff here…)} "
Cases:
When score_threshold is 0.7, I assume that no related document was found, it completely ignores the instruction and constantly reply with “I cannot find any related document to…”, I tried to add a lot of emphasized words like “ALWAYS”, “MUST”, it’s still ignored.
When score_threshold is 0.5, it can retrieve some related document, and the instruction works flawlessly
So my question is, does the file search do more than just search for related content and immerge it into the response, I feel like it’s forced to say no when nothing is found even if the instruction tells it not to do so.
You have to talk about what is behind the tool. Not “use this document” that the AI never sees.
The AI has to write effective queries that would match up with documents.
A tool return is the result, a list of file chunks – or no return as a result of calling the function.
The threshold is a cutoff where the match of query vs document below your setting is not returned. The AI reports correctly that there’s no file_search results if you set to such a high threshold, and it can’t do anything about your choice.
Sorry for not clarifying, my instruction is dynamic, which means that the “Use this document…” will contain some metadata of the documents, for example:
“Use this document: title: book_1.pdf, lang: english, description: book about…”
And I only want the bot to use that to respond that it has that document with the name book_1.pdf, written about something, not that I want it to retrieve the content of the document.
The “additional instructions” is way back in the system message.
You also have file_search on, though?
The AI is in a pattern of calling tools, and getting tool results. It’s kind of answering what the tool return gives or produces.
Now the question: Is OpenAI jammin’ up your application with messages counter to your application being returned like “you must report that you have no documents found”?
The symptom says so. The return message for me not going thru the full work of setting a high threshold, but just no files:
NoDocumentsFound
No documents found in conversation. User must upload files before tool can be invoked.
That’s going to be followed before the AI thinks hard about other stuff seen to write about. Showing here:
I’ve never gone to Assistants because of other things internal that also are contrary to good practice, and discover similar now when using the internal tools in Responses also.
