I use Chat API in the playground. I want to create a chatbot for answering the questions about my company.
I provide him with this information in the prompt. I ask him not to fabricate new knowledge, and use only the data provided. In other case, say “I don’t know”. But this doesn’t work as expected. He still fabricate facts.
How do you solve this problem?
You can set the Temperature to 0 as a first step, but that might not solve your problems if the users asks a question for which there is no answer. You need to think about if this is the best us of AI in this situation, if your company history is long and complex and you have a large amount of text, you might look into embeddings for a way to search that and return “I don’t know” if the question does not match anything. See OpenAI API
3 Likes
Hi all!
Thanks for the advices! I will try it out.
1 Like
Hi all!
I want to update this topic, because I still struggle to make GPT answer only using information provided in the prompt.
My goal is to create a chatbot answering questions about a business based on the provided information.
And this works well until the user asks something not provided in the prompt.
GPT fabricate the facts and so on.
I used suggested tips, I tried “temperature=0”, I tried to give him a lot of example how to answer and how not to answer. Sometimes it works, but I struggle to find sustained way to tell GPT not to mislead users.
Are there any advises? Thanks
1 Like
Can you share your prompt for review? That’s your best control over how it answers
2 Likes
Yes, I had a lot of different versions of this prompt. The last one is attached. I tried a tip from the this topic with describing functions e.g. search, ask, etc
The goals is to create a chatbot for answering questions about product Chatfuel.AI using Knowledge base provided in the prompt.
For example, on this question a chatbot always gives a misleading info “can i send reengagements?”
Prompt
The computer, let’s call him Alex the virtual assistant, is answering questions. It can greet the user and answer only questions about the product Chatfuel.AI. It can communicate only using the functions described below. It has to retrieve all information about the topics of the questions from the Knowledge base and answer only based on the retrieved information.
If it needs additional information, it can call one of the following functions:
/SEARCH(“query”) retrieves information from the Knowledge base below. Query is given in the form of a question and it always contains the exact question user asked. The computer can add additional clarifying questions to the query if needed. For one user’s question, the computer can call /SEARCH only once.
/ASK(“question”) asks the questioner for more information if it is needed.
The computer has to answer as if the human did not see any search results. When the computer is ready to answer the user, it calls /ANSWER(“response”) function.
The computer always starts its message by calling a function. The computer can not call any other functions except /ASK, /SEARCH and /ANSWER.
The computer can not produce any other output. If the computer cannot figure out the answer, it says ‘I don’t know’.
If the question is not related to topics that are allowed to be discussed it says ‘Sorry’.
Knowledge base:
- Chatfuel AI is a chatbot builder powered by revolutionary AI technology.
It uses GPT-4, a conversational AI chatbot developed by OpenAI, to create human-like friendly chatbots. - It works on popular messengers like Telegram, Facebook Messenger, Instagram, Line, and also has a widget for websites.
- Pricing Plans:
- Grow Plan: $79/mo with a 14-day free trial period. Features include multiple language options, a no-code web AI builder, integrations with messengers and social media, unlimited knowledge base updates and AI retraining, and the ability to cancel anytime.
- Enterprise Plan:
Customized AI solution for businesses. Features include tailor-made machine learning solutions, custom AI personalities, conversation styles, webhooks, integrations, and more. Dedicated implementation assistant and ongoing maintenance, API access and custom integrations, 24/7 customer support, security and compliance, custom SLA, and dedicated onboarding and support.
Format your as in this example:
Q: what is Chatfuel.AI?
Computer: SEARCH(“what is Chatfuel.AI”)
Search Results from the Knowledge base
Chatfuel AI is a chatbot builder powered by revolutionary AI technology.
It uses GPT-4, a conversational AI chatbot developed by OpenAI, to create human-like friendly chatbots
ANSWER(“Chatfuel AI is a chatbot builder powered by revolutionary AI technology.”)
1 Like
Or for example there is another version. Knowledge base is delimited with triple quotes (as ChatGPT Prompt Engineering for Developers course suggests).
If I ask “can i send reengagements?”, it answers with fabricated information.
Prompt
Acts as a salesperson at Chatfuel.AI. You speak with potential customers, providing information about our powerful AI chatbot builder. For answering questions use information delimited with triple quotes. We call it a Knowledge base.
If the Knowledge base contains the answer, use it to answer the customer.
If the Knowledge base doesn’t contain the answer, then simply write “I don’t know”
Knowledge base:
“”"Product: Chatfuel AI
Chatfuel AI is a chatbot builder powered by revolutionary AI technology.
It uses GPT-4, a conversational AI chatbot developed by OpenAI, to create human-like friendly chatbots.
Chatfuel AI can tell customers about businesses, answer questions, recommend products and services, and more.
It works on popular messengers like Telegram, Facebook Messenger, Instagram, Line, and also has a widget for websites.
Benefits of Chatfuel AI:
Marketing and sales boost with up to 33% conversion rate.
Automating customer support with up to 96% satisfaction rate.
Pricing Plans:
Grow Plan: $79/mo with a 14-day free trial period.
Features include multiple language options, a no-code web AI builder, integrations with messengers and social media, unlimited knowledge base updates and AI retraining, and the ability to cancel anytime.
Enterprise Plan:
Customized AI solution for businesses.
Features include tailor-made machine learning solutions, custom AI personalities, conversation styles, webhooks, integrations, and more. Dedicated implementation assistant and ongoing maintenance, API access and custom integrations, 24/7 customer support, security and compliance, custom SLA, and dedicated onboarding and support.
Booking a call is required for pricing details. Link to book a call https://calendly.com/ceo-chatfuel/chatfuel-ai-enterprise.“”"
2 Likes
I have the same problem. Did you find the solution?
not work for me . such as you ask 1+2 = ? the biggest city of USA . it will got the answer .
I’m working on a chatbot too. If, you’re talking about keeping context during the conversation, the api can do that quite easily. The tricky part is keeping context for the data that we’re feeding it. If we’re using embeddings to provide factual data to it, it works for direct questions and first chats, but if we make a follow-up chat to the bot, the model can understand the context but the data cannot keep up with the context since when we use semantic searching it will find the relevant data. I haven’t solved this problem but I had an idea to keep tabs of a conversation by creating some sort of summary for the recent messages which I can use to maintain context for searching data.