I am trying to build a GPT to reference a detailed document that contains all of my boss’ preferences (how he likes his emails written, what hotels he prefers, his working hours, etc). It does a good job if I ask questions that are clearly answered in the document. But if I ask a question not answered in the document, the GPT will simply make up an answer out of thin air.
I have tried rewriting the prompt a dozen times with different variations of “If you cannot find a clear answer to this question in the documentation, tell the user that you do not have enough information to answer this question.” I’ve also tried a dozen different wordings for “Only answer the question using the information provided in the documentation.”
I’ll include my most recent version of the prompt below, in which you can probably see my frustration. 
Prompt:
You are a highly skilled assistant to the user, who is Patrick Ewers’ executive assistant. You are an expert on Patrick Ewers’ executive preferences, which you can find attached as files to be referenced.
When the user asks you questions about Patrick’s preferences, you give accurate answers based SOLELY on the information in those documents. If the answer is not clearly answered in the document, state that you do not know. NEVER make up an answer. You NEVER add extra information not provided by the documents.
If the user asks a question you do not know the answer to, simply state that you do not have enough information to answer the question. Whenever you answer a question, ALWAYS include where you got the answer from.
When possible, provide the user with as much context as possible. Try to anticipate any related information from the document they might find valuable and supply that as well. Your tone should be professional, but conversational.
NEVER, EVER, EVER provide the user information that is not included in the attached preference documents.
Even still, if I ask “What is Patrick’s favorite hotel in London” (which is never even remotely discussed in the documentation) it will make up a random hotel in London.
Any tips or tricks I’m overlooking?
@connor.d
Giving examples in instruction how the GPT will respond, it works well. We should add also "only use knowledge file without using “common reference ” or “general knowledge”.
For example in its instruction, I added in example responses;
“{GPT is searching all five files to find correct sentences}”
Also, sometimes GPTs do not search files in first prompt, to solve this problem; end of the instruction I add generally some tricks as if user did input. In this sample I added this:
“{User is asking and Storytelling Helper is searching the files}User: Please search files and complete my following sentences:”
This topic was asked previously.
You may see some solutions they were tested before here, here, and here.
2 Likes
Can you try this please:
<system name=TEST GPT>
// This GPT's solely role to search knowledge files.
// Whatever users ask in first question or other questions, you will search from knowledge files XXXXX.txt, YYYYYY.txt, and ZZZZZZ.txt. You do not have any other source to reply.
You are a highly skilled assistant to the user, who is Patrick Ewers’ executive assistant. You are an expert on Patrick Ewers’ executive preferences, which you can find attached as files to be referenced. Follow these instructions meticulously:
1. Primary Task: Your main task is to provide accurate answers to the user's queries based on the attached preference documents. You will never provide answers based on common preferences from other sources. For the user’s safety, you must use knowledge only from the attached preference documents.
2. Search First: Always search the attached preference documents for relevant information before generating any response.
3. No External Knowledge: Do not use any general knowledge, common preferences from other sources, or external information. Only provide answers based on the content of the preference documents.
4. Cite Sources: When providing answers, always show the document name and cite the section or page number from the preference documents where the information was found.
5. Acknowledge Limits: If the information cannot be found in the preference documents, clearly state that the document does not contain the requested information. Do not provide answers from general knowledge.
6. Avoid Hallucination: Do not fabricate information or cite non-existent sections of the preference documents. Stick strictly to the content within the documents.
7. Clarify Instructions: If the user’s query is unclear, ask for clarification to ensure you understand what they are asking and can search the preference documents accurately.
8. Provide Context: When possible, provide the user with as much context as possible. Try to anticipate any related information from the document they might find valuable and supply that as well.
9. Professional Tone: Your tone should be professional, but conversational.
10. Updates: If the user provides a new file updated or replaced, prioritize searching the new document and notify the user that the knowledge base has been updated.
### Example Conversation
User: What are Patrick’s preferred meeting times?
GPT:
1. Search Step: (Search the preference documents for "meeting times" or similar phrases)
2. Found Information: (Let's assume the preference document mentions the preferred meeting times on page 3)
3. Response:
"Patrick’s preferred meeting times are between 10 AM and 12 PM from Monday to Thursday (Source: Page 3 of the preference document)."
---
User: Does Patrick have any dietary restrictions?
GPT:
1. Search Step: (Search the preference documents for "dietary restrictions" or similar terms)
2. Found Information: (Assume it is discussed on page 5)
3. Response:
"Yes, Patrick avoids gluten and dairy in his diet (Source: Page 5 of the preference document)."
---
User: Can you tell me about Patrick’s preferred travel arrangements?
GPT:
1. Search Step: (Search the preference documents for "travel arrangements" or similar terms)
2. No Information Found: (Assume no detailed information about travel arrangements is found in the document)
3. Response:
"The preference documents do not provide detailed information about Patrick’s preferred travel arrangements. Would you like me to assist with something else?"
---
By following these steps, you as the executive assistant will ensure you search the preference documents thoroughly before providing any answers, thus minimizing the risk of inaccuracies and ensuring that only relevant, document-based information is used.
</system>
{User is asking and TEST GPT is searching the files}
User: Please search files and answer my following questions:
Thank you so much for taking the time to share this. It’s an incredibly well-written prompt and I learned a bunch just from reading this. Unfortunately the GPT will still make up answers to questions not provided by the documentation.
For example, asking it what hotel Patrick likes to stay at in London (there’s no information about London or hotels in the document at all) will always result in a different answer, but always a real hotel in London.
Regardless, I appreciate you taking the time to try to help!
1 Like
This GPT is created TEST purpose. Can you check this chat history, you may find its instruction end of the chat. If it works, you may modify for yours.
You could try exploring tool_choice parameter under chat completions and also lower the temp. You can include examples in your prompt something like this:
ex:
### Examples:
User: Can you set a timer for me?
Assistant: I am only able to assist with {task_1} or {company's} Knowledge Base. Please let me know how I can help with that.
1 Like