I setup new Assistant with following instructions:
Answer questions about my product. Important!!! Never invent anything yourself!!! If the is not concrete answer neither in these instructions or in additional instructions, strictly say the following phrase: Sorry, I don’t know the answer
My product - English language training services for IT specialists. Answer based on the information below, as well as information from the additional instructions. Important!!! Never invent anything yourself!!! If the is not concrete answer neither in these instructions or in additional instructions, strictly say the following phrase: Sorry, I don’t know the answer
Below is information about my service:
my offer for developers, QA, project manager and product manager
the cost depends on the student’s level and request
format of classes online
When asking such question: “which payment methods are available?”, I am getting fake information in the response (this information is not present in original instruction).
In your instructions - besides what you have shared above - have you provided details of the type of knowledge files you have uploaded and the nature of information that is covered in them? That could potentially help as it would allow the Assistant to more effectively “navigate” through the knowledge base.
What do you mean by “type of the knowledge files” and nature of information? Can you give an example? Actually all the information I provided is within this instruction (no extra files used)
So this is the full set of information you provide the Assistant to answer queries or are there more details for each bullet? Sorry, just trying to understand…
Just for simplicity (to share this “hallucination” case) I created this restricted knowledge set to avoid extra factors , which can impact the asnwer. So yep - it is the complete set of info
The initial testing on my end with your example question “Which payment methods are available?” shows that the Assistant under the gpt-3.5-turbo model properly answers with “Sorry, I don’t know the answer”. I will run a few other tests but perhaps this was just a one-off?
Besides the tests, I have made some (basic) amendments to your example instructions.
You are an Assistant tasked with responding to user questions about a product/service based on the information provided below. In responding to user queries, you strictly only rely on the information provided. You never invent any information. If a query cannot be answered through the available information, you respond with: " Sorry, I don’t know the answer"
Description of product:
English language training services for IT specialists.
(1) Target audience: developers, QA, project manager and product manager
(2) Cost: depends on the student’s level and request
(3) Format of classes: online
Based on a few test questions this seemed to have worked with gpt-3.5-turbo.
Actually, why I am asking: when I translate the instruction into another language (and imitate user quetsions in this language), is again showing the same hallucination behavior
I’m not sure what you are referring to? I tested the simplified version of your instructions to which I made amendments in one more other language and experienced no hallucinations.
Note that if you have expanded your instructions since, then further adjustments may be required to achieve the same outcome.
Generally, I think the issue with the initial set of instructions provided were twofold:
You co-mingled instructions about how to engage with user and how to use available information to respond to a query with the actual information about the service. You should strive to build a clear logic into your instructions and delineate the different information types. As you can see from the amendments, I grouped all information relating to the service together.
While it is helpful to reinforce certain messages, I would generally advise against being too repetitive, especially when it comes to “negative instructions” about what not to do. In extreme cases this can have the adverse impact oft your Assistant consistently answering with “Sorry, I don’t know the answer”. In general it is advised to use negative instructions very sparsely and instead focus on describing what the Assistant SHOULD do.
And i use only GPT-3.5-turbo, no hallucinations. Do you use only Playground or you tested same Assistant in your code and have same hallucinations from model, i need more details in order to help you?
Is ok, the purpose for that video was to show people can use Assistant in their PC with local data, he is not altering data or generate random things, he’s obedient and smart, etc.
I know in the video the chat is not that great, i solved some bugs too, fixed chat to update correctly, etc. (I will replace the video in 24h)
I will make one better with an interface and a real database.
And after i finish that i go to make the architecture for an mini AGI.
