I am exploring integrating GPT into our own chat sessions based on our own custom knowledge base.
I’m using LlamaIndex to generate an index of the custom knowledge base in my company, then asking questions about the material. This works well, but it still answers unrelated questions. For example if I use the index to query I can say “What is the diameter of the moon?” and it will still answer it.
Obviously I don’t want these type of questions to be answered in our business chat sessions. How exactly do I limit the scope of the conversation? It should instead say something like “Based on my research i’m not able to find an answer to your question” or something like that which I can program.
Several proposals. In my experience, you get the best behavior when you actually combine all of them:
Clearly specify the questions that should not be answered via prompt-engineering. Stuff such as “You should always refuse to answer questions that are not related to this specific domain” should help a lot.
Include binary classifiers that determine whether a question is “on-topic” or “off-topic” for your particular use case. You can use cheap fine-tuned OpenAI models for this or open source stuff (Huggingface).
Include a minimum threshold of similarity when retrieving documents to answer questions. If no documents surpasses this threshold, decline to answer politely (with a pre-specified formula).
Use content moderation (OpenAI’s free endpoint) to filter out inappropriate requests.
Include reg-exp filtering to add a extra security layer to stuff such as prompt-injection (especially if you’re exposing your app to external customers).
