Hi,
i have a strange problem with the vector-stores. I have a chatbot that uses the responses API and a lot of documents (PDFs) in a vector-store. It is for a Podcast and every episode has its own PDF, including http links to the episode on spotify and YT.
Now when a users asks “give me the link to Episode 22” or “give me all episodes that cover issue xyz” the system will return and often give the correct info. But often also not. Then it will give wrong links, either to other episodes (it says “here is the link to episode 22” but the link leads to episode 28) or simply dead-links that look correct, but lead to a 404 on the target platform.
I tried to make it very clear in the instruction that only real links should be used, reduced the temperature, changed models (even with Mistral and Gemini) - but it will not go away.
In the case above when he gave me the wrong link for episode 22 when i asked back and said “hey, that is the link to episode 28” he will respond and apologize and give me the correct link…
So the correct info seems to be available, he just wont use it.
Any idea what is going wrong or what i should change?
Thanks in advance!
Try to include in assistant instructions this text.
“If the user’s question is likely to be covered by my PDF (vector store ID: ), search the vector store first.
If a relevant answer is found, reply using only the exact text passages or bullet points from the knowledge base, without any changes, summaries, or modifications.”
The vector store may be retrieving the correct document or vector but failing to properly match it with the specific query. This might result in links pointing to the wrong episode (e.g., episode 28 instead of 22).