Assistants inconsistency with attached vector store

API
,
1

Hi everyone,

I’m having a lot of trouble getting Assistant to be at least 90% consistent with using the Vector Store and general knowledge at the same time. I’ve got a rather complex setup including functions and VS. All is using 4o model.

I am trying to deal with two issues:

  1. I want to prevent assistant from mentioning the uploaded files. This is not solved by explicitly stating in the context that this is not allowed, using additional_instructions doesn’t help either. Guiding a response with “You must start your responses with…” helps just a little but not enough.
  2. Assistants answers are not good with vector store as it often says that the information I am looking for is not in the uploaded files (also causing problem #1) instead of using general knowledge.

So far I’ve tried passing VS to the thread or attaching it directly to the assistant, several combinations of contexts, answer guiding, system instructions, additional instructions, passing instructions to the thread or run. Whats also worthy mentioning is that my context is quite long (but still within the limits of course).

What are your best experiences when dealing with such case? What would be the best practices when dealing with Assistant that uses general knowledge AND vector store AND function calls? Would using two separate assistants and then combining the result be an viable option? Is there a difference in behaviour depending on if VS will be attached to the assistant or passed during thread creation? How does the assistant pick if it should use VS or general knowledge?

Thanks for advance for any help.

2

This is an issue often encountered, and for which I’ve answered with workaround suggestions too often.

The vector store search tool, in two different variations currently, directly talks about the files as being uploaded by the user. This hinders the tool, with the symptoms articulated well by you.

A full topic I wrote, besides many other similar issues I could link:

We will see if AMA tomorrow has OpenAI willing to address this, with my question and recommendation (lower in the snippet pictured).