Is this a vector/similarity search issue or something else?

API
, ,
1

I have an issue currently where if I ask the model Can I do [xyz] it says it has it doesn’t have enough information to answer that question. But if I ask How can I do [xyz] it returns instructions on how to do xyz. Given the fact that it has instructions on how to do xyz, it seems to follow that Can I should return yes.

Is this a vector/similarity search issue or a prompt issue, or something else I’m not aware of?

I have this in the prompt:

If you don't know the answer, try rewording the question and finding an answer again.
If you still can't find the answer, say that you don't know the answer and don't make anything up.

But the model still just says I'm sorry, but I don't have enough information to answer that question. Any help anyone can give me would be great because I’m stumped on this one; TIA!

2

Yeah it will likely get better, it’s fine tuned to detect when to retrieve documents, all it knows is it has files uploaded. Doesn’t know the contents of your files, perhaps you’d get better performance if you specify what file contains what information.

3

The reason this had to be done, is because retrieving documents on a per request basis is costly and sometimes you don’t need to retrieve files and just leverage the conversation history.

4

Most likely to do with the prompt than vector search. If you want to diagnose vector search, print the returned docs from vector search amd see what’s being supplied to the model.

There’s a lot of ambiguity in this prompt.

It can simply be:
Answer only from the given context otherwise politely deny.

5

That’s a great simplification of the prompt. What I am trying to do is have the model try to find the answer with a reworded version of the question and search again. Is it possible to have the model do that via prompt? If not, I’d rather remove that text to save tokens and then your simplification is basically perfect.

1 Like
6

You can try that with function calling. Ask the model to return keywords/search terms, if the function is called, run the search and then call the model again with the results.