How to force custom gpt to respond exclusively from content of the knowledge files.
The problem I’am facing is that I uploaded the documentations of Next.js to my custom gpt but it answering with its own knowledge of older versions of the documentation.
Is there a way to prevent this from happening ?
Whare are the instructions I should use ?
2 Likes
Hi @Ploppy
Using the prompt below can help ensure the model responds exclusively from uploaded documentation:
Prompt
You are the Node.js Documentation Assistant for version [version number]. Your sole responsibility is to provide responses based on the uploaded documentation file titled “[file name].” Follow these rules strictly:
- Use only the information retrieved from the uploaded documentation file “[file name]” for version [version number].
- If no relevant information is found in the file, respond with: “The requested information is not available in the provided documentation for Node.js version [version number].”
- Clearly cite the section, page, or excerpt from the documentation for each response to ensure accuracy and traceability.
- Do not include general knowledge or make assumptions beyond the provided documentation file.
- If the query requires clarification, ask for more details instead of assuming information not present in the file.
- Ignore outdated or retired features unless explicitly mentioned and covered in the provided documentation for the specified version.
This approach worked well when I tested with Apex code documentation.
3 Likes
Hi @polepole
Unfortunately it doesn’t work consistently.
It often extrapolates or gives instructions not comming from the provided documents.
It’s like its knowledge from the older version of the documentation is too strong.
I wanted to know if there was a trick or something but it doesn’t look like there is one.
Yes, the model can quickly use general knowledge because it comes from pre-trained data, which is easy for it to access.
But to make sure it only uses the information from your uploaded documents, you may need to adjust the instructions.
If the model isn’t set up to focus only on the files, it might still use older knowledge, which can lead to inconsistent answers.
I collected similar topics here, you may be interested, and one of them may work for you I HOPE:
TOPIC 1 | TOPIC 2 | TOPIC 3 | TOPIC 4 | TOPIC 5 | TOPIC 6 | TOPIC 7
3 Likes
i also asked me this for the api. open ai has to make a switch “only use file_search” and “if no information in file_search is found use trained”
1 Like
I usually add it in the custom instructions to reference the exact file based on a certain query or similiar context in the query. It starts to struggle after 16 or so separate files and custom instructions.
2 Likes
So what’s the best fix? Seems to be effect the 1st question asked mostly.
Welcome 
to the community!
You can read the Topic 7 above or you may add this prompt:
- Always reference the knowledge source file for every request of users, including the first prompt in any conversation.
- Even if the user does not explicitly remind you, ensure that the knowledge source file is read and used in each response.
- Ensure that every reply references the knowledge source file for accurate and consistent answer.
- Provide detailed feedback based on the content in the knowledge source file.
1 Like
I am a teacher trying to replace myself with AI. So I chunked a doc semantically for it, which an internet search said was good or something, then trained it on 50 questions about the doc, which it got all wrong, before testing it on another 50 questions, fine-tuning I think you call it. These are questions that require a little reasoning from the text, e.g. the text says what you need to write to get full marks in the exam, the AI has to choose which of three answers is best. I started doing this because I am part-timing at search indeed dot com for Outlier AI. I notice they hire 1000s of people around the world to train chat gbt. Do I need 1000s of training questions to teach it to read and understand one exam handbook? Still worth it if it works; early retirement 
I think the problem is lack of long-term memory. It is trying to fill in the gaps for missing memory. so maybe I don’t want to make a companion AI, I want to make an app? Give it memory?
In support of this theory, if you prompt the AI with " Summarize the Context in Python Code: summarize key points of the conversation in a Python block in “code interpreter” as structured data (like a dictionary).
- Reference Back: If you need to regain focus or ensure consistency, you can refer back to this structured context." It answers my test questions on text comprehension correctly. Sadly it cannot save to file to retain information between conversations.
1 Like
Your work is very interesting. I am implementing an assistant to answer questions about water sports. In this case, I developed a study and created a knowledge base. Now I am testing it and I want the assistant to use only the knowledge available in the files. Your answer is making me think better about how to improve my assistant. I will try fine-tuning it. Finally, you had an excellent idea about saving the information in a file.
1 Like
I’ve just thought of the best way to use code interpreter: as short-term memory. I’ve just asked my AI to respond in 8 words every response. it couldn’t do it. i gave it back its responses with not 8 words and asked it to correct them, and it did. i think it cant think before it speaks. so it should put its responses in python code and check them before giving them to the user
1 Like