Assistant API ignoring custom files

API
1

ey everyone,

I’ve built a custom integration for the Assistant API (chatbot) and added supporting files that it should use during conversations. One of those files is urls.txt, which contains specific URLs the assistant should reference when a user asks for a link.

In the system prompt, I clearly instruct the assistant to only provide URLs from the urls.txt file. However, it keeps ignoring this and instead returns random URLs from my domain, not the ones listed.

Has anyone run into this issue? How can I make the assistant reliably use the contents of the urls.txt file only?

Thanks in advance!

2

You will have to place your own instructions as text.

You have the additional_instructions field if the instructions need to be dynamic, and not persisted in the chat every turn by adding messages.

Otherwise, if using URLs is what the assistant does and must always know, they can occupy the assistant instructions.

Why?

File search (“added supporting files”) in Assistants is not suitable for instructing the AI. Nor is it even really suitable as a RAG solution for improving AI knowledge.

  1. File search tells the AI the files came from a user

The image shows a detailed description of the "myfiles_browser" tool, explaining its functions, syntax, usage instructions, and guidelines for browsing and searching uploaded user files. (Captioned by AI)
The image shows a detailed description of the "myfiles_browser" tool, explaining its functions, syntax, usage instructions, and guidelines for browsing and searching uploaded user files. (Captioned by AI)621×409 15.4 KB

  1. File search gets you damaging messages added to your chat:

A system message indicates that the user has uploaded files, which are available for searching with the "msearch" tool. (Captioned by AI)
A system message indicates that the user has uploaded files, which are available for searching with the "msearch" tool. (Captioned by AI)776×142 3.88 KB

  1. The AI then produces complete falsehoods because of the developer’s use of file search for their own knowledge database

The image shows a conversation where a user asks for an explanation of their action, and an AI responds by stating that the user has uploaded files, which can now be searched using an internal tool called "file_search." (Captioned by AI)
The image shows a conversation where a user asks for an explanation of their action, and an AI responds by stating that the user has uploaded files, which can now be searched using an internal tool called "file_search." (Captioned by AI)508×250 6.4 KB

  1. The AI has no idea what is in the vector store database

  2. The information can only be accessed and seen when the AI feels the need to call a tool, a tool that is offered purposelessly to it

  3. The return is based on a query the AI must write, and a semantic similarity search returns chunked results. Is a query based on user input such as “sprockets that can interface with a widget” even going to return a list of URLs, something completely different than than the query that could be displaced by other files that are actually language?

Responses: no better.

(The past threads containing these diagnostic outputs are easy to find, as I have no use for Assistants other than to show others its issues.)

3

Thank you for your reply, wont the prompt be to long for the assistant to understand if i added all the data stored in the files? And what is the purpose of the vector store database if the assistant api can’t read them?

4

“File search (“added supporting files”) in Assistants is not suitable for instructing the AI. Nor is it even really suitable as a RAG solution for improving AI”

What IS it suitable for?

5

It’s suitable for treating you the developer as a consumer and a user and only delivering ChatGPT as a product with carbon-copies of the internal tool messages, where users can upload their own files - that they then can’t understand why the AI can’t answer everything about.