I have a File Assistant which seems to be working fairly well. But sometimes, for the exact same question, it “randomly” decides not to use any of the vectorized files and makes up its own answer with no annotations.
It’s nice that it uses far fewer tokens for such answers, but unfortunately its made-up answer often has mistakes.
If it matters, I have temperature set to 0.16 and Top_P set to 1.0.
It seems that once it starts not using annotations, it won’t for a while, regardless of question.
What causes this? What is the best way of handling it?