I am having a similar experience. Using ‘retrieval’ a 5 pages pdf file is provided to the assistant. I use both the API and playground to interact with the resulting assistant. Behaviour is inconsistent:
- sometimes I get the right answer immediately (and then it is like magic
) - sometimes I get a wrong answer and I tell the assistant to check again and the 2nd or 3rd time I get the right answer. (“Sorry I was not providing the right answer. I will check again…”)
- sometimes it just keeps saying that the requested answer is not available in the provided document(s), while it clearly is.
The inconsistency that I see is making the assistant unreliable, hence useless. I am talking about 50% errors, not just the last 5%.