Anyone know (in the API) how to attach a vector store to a thread (and actually get it to work)

API
1

So, I am trying to dynamically via the API allow the user to select what vector store to include in a Thread. It’s not working, the Thread cannot ‘see’ any of the files in the specified vector store.

What I’m doing via API

  1. Creating a Thread and specifying 1 vector store
const threadParams = { messages: [] }
  if (allVectorStoreIds.length > 0) {
    threadParams.toolResources = {
      file_search: { vector_store_ids: [vectorStoreIdGoesHere] },
    }
  }

  1. I then retrieve the newly created Thread via API to check it got set up correctly and can see in the console log messages that it has tool_resources.file_search.vector_store_ids[‘the correct vector_store’]

  2. I then create a Run with assistant id = ‘the assistant id with a vector store’ and tools[{ type: ‘file_search’}] and a message “What files can you see?” and run it.

  3. It comes back with “It seems there are no visible files or content available…” → and this is the problem obviously, I expect it to list the file names that are in the vector store that I’d previously attached in step 1.

Yes I have double-checked the correct vector store id is used, and via Dashboard checked that the vector store contains a file.

What am I doing wrong? or is this a bug?

Note that until a few days ago, the Assistant Playground seemed to have a problem where a vector store attached to an Assistant was not visible to a playground prompt.

I believe I can have 1 vector store attached to each of the Assistant AND Thread, i.e. 2 vector stores active in total.

When I create the Run to ask what files it can see, it cannot see any files attached to the Assistant or dynamically attached to the Thread.

2

I’ve found it to be very inconsistent. Sometimes it will see it right away other times it takes a couple tries. It denies there is any file attached, but then if I mention something within the file all of a sudden it can read that material (and when it can it works great).

Make sure you check out the new “detail” view where you can see all the scores for retrieved data. I’m doing the save thing you are and I was really surprised that the vectorstore attached to the thread doesn’t show up at all some times.

Still trying to figure out why or how.

1 Like
3

I’ve found while a query of “What files can you see?” most of the time fails (GPT says it can’t see any files???), I am seeing reliable usage of file content in API queries.

If I create a Thread with a vector store id, then after I’ve done a Run and retrieved the Messages, the message entries will have an attachment that lists a citation to one of the Files in the Vector Store, i.e. this is what I expected.

My approach at the moment is a 2 step approach

  1. Specifically requesting a file_search to get it to try to retrieve relevant files, ie “What do you know about <topic/question>”… which then typically does a file retrieval, then
  2. Another query this time with a function, since I want structured data back, but the messages used in this step start out with “Using what you just retrieved…”

I can then force a tool_choice in step 1 to be file_search and step 2 to be function.

this is ok, I was hoping for a 1-shot approach where it forces a file_search and then puts the results into a function/structured data response, but if I have to do it in 2 steps that’s ok

4

That’s a good approach.

My problem is: users will upload a file, and then ask “what do you think of this?”. Semantically, “what do you think of this?” has no relevance with the file they just uploaded so it ends up looking elsewhere to respond (my current implantation has two vector stores: one on the assistant and one on the thread).

I’ve had good success lately embedding some semantically relevant material into the message itself before sending it off to be run. This “triggers” a response that incorporates additional semantically similar content from the file they uploaded.

5

re: when you said “My problem is”…

In my app, www.architectureinmotion.com.au, I have users entering a specific question they want to generate content for. Its Enterprise Architecture content, and so they could be asking things like “Create a capability model for ”, “Perform an initial risk assessment for ”, etc,…totally free form.

So what I do in step 1 is query for “What do you know about <the user’s question>”, which in my case I think addresses the good point you raised.

I think that’s what you’re saying in your 2nd paragraph.

I’m also now extracting the citation …

data.response.data[0].content[0].text.annotations[0].file_citation.file_id

… and then retrieving the name of the file so when I present the results to the user I can explicitly show them which of the file(s) they provided was used to help shape the results.

1 Like