I am using document retrieval to pull sources from a knowledge base and adding them into the prompt for GPT-3.5-turbo to answer questions, but it seems like sometimes it is pulling information elsewhere. I even tried to specify in the prompt that it should not use information outside of the documents retrieved but it still seems to pull information from whatever information it was trained on occasionally.
1 Like
Is it hallucinating or is the external bot providing factually correct information that you are not seeking to include?
If it’s a hallucination, I’d try a knowledge graph approach to constrain output (this solution is far more efficient where the sources are large)
If it’s adding factually correct information, you could try in order:
A. Adding a section in the prompt where you specify the output format, so it’s forced to follow a specify step-by-step information extraction process
B. Train it with sample Q:A pairs either via finetuning (overkill most of the time) or via a vector store (most popular
C. Using a KG as in above
1 Like
Please provide a sample prompt and question. Let us see if we can’t help you improve your prompt
1 Like
May sound simple but just in case, verify temperature is low to zero. I’ve had very good luck with 1) system prompt being explicit that it should not use external info 2) temperature at zero.