Prompt design for Q&A ChatBot

I wanted to build a Q&A ChatBot. I took OpenAI’s advice and made embeddings of my documents, and also followed their Q&A using embeddings tutorial ( exact steps.
Unfortunately the header of the prompt which is:
“”“Answer the question as truthfully as possible using the provided context, and if the answer is not contained within the text below, say “Sorry, I don’t know”
is not working as expected, because I receive a lot of “Sorry I don’t know” answers, although it finds the good document.
Can someone please give me some suggestions for this problem?
Thank you!