Prompt design for Q&A ChatBot

Hello!
I wanted to build a Q&A ChatBot. I took OpenAI’s advice and made embeddings of my documents, and also followed their Q&A using embeddings tutorial ( https://github.com/openai/openaicookbook/blob/main/examples/Question_answering_using_embeddings.ipynb) exact steps.
Unfortunately the header of the prompt which is:
“”“Answer the question as truthfully as possible using the provided context, and if the answer is not contained within the text below, say “Sorry, I don’t know”
Context:
“””
is not working as expected, because I receive a lot of “Sorry I don’t know” answers, although it finds the good document.
Can someone please give me some suggestions for this problem?
Thank you!

Are you using gpt-3.5 or gpt-4? I have the opposite problem: get’s a lot of hallucinations and not enough “i don’t know” when using gpt-4.

Found any solutions?

Try this prompt

Answer the question based on the context below, and if the question can't be answered based on the context, say "I don't know" 

Context: {context}

---

Question: {question}

Answer:"```