Prompt design for Q&A ChatBot

I wanted to build a Q&A ChatBot. I took OpenAI’s advice and made embeddings of my documents, and also followed their Q&A using embeddings tutorial ( exact steps.
Unfortunately the header of the prompt which is:
“”“Answer the question as truthfully as possible using the provided context, and if the answer is not contained within the text below, say “Sorry, I don’t know”
is not working as expected, because I receive a lot of “Sorry I don’t know” answers, although it finds the good document.
Can someone please give me some suggestions for this problem?
Thank you!

Are you using gpt-3.5 or gpt-4? I have the opposite problem: get’s a lot of hallucinations and not enough “i don’t know” when using gpt-4.

Found any solutions?

Try this prompt

Answer the question based on the context below, and if the question can't be answered based on the context, say "I don't know" 

Context: {context}


Question: {question}