How to prevent ChatGPT from answering questions that are outside the scope of the provided context in the SYSTEM role message?

People, next episode of caos30 taming the horse called chatGPT completion endpoint :stuck_out_tongue_winking_eye:

You’re invited to read it and help me with your smart point of view :grin:
You know that you’re very welcome.

1 Like

Have you tried building the prompt with semantic search?

1 Like

Interesting, but not. It’s the first time i read something like this.

Are you thinking of do a FIRST call to chatGPT completion endpoint asking for “rewrite” the user question taking in account some extra context information, like the closest Q&A in the knowledge database, and then use the output to do a SECOND answer.

Well, although it was minimum successful, this could almost double the cost of the use of the API and just the goal is just the opposite: to avoid consume API for OUT OF SCOPE questions, detecting it before to query API if it’s possible, or if it happen, then be able to respond that discourages the user to continue doing this kind of questions.

See it from the sky: we try to optimize the answers and the cost of the system to nicely attend users doing questions within the scope of our knowledge DB, discouraging other kind of questions.

In this sense, i’m not worried about the system responding to a COUPLE OF out of scope user questions, but i do worry about potential users using the system to get all day at any hour any kind of answer.

Yeah… it’s more complicated than initially seemed :sweat_smile:

2 Likes

You need to use embeddings to find the closest documents in your data source, not chatgpt, embeddings is a lot cheaper and faster also fyi.

For example

This is from a tuto i wrote Build a ChatGPT Powered Markdown Documentation in No Time | by louis030195 | Mar, 2023 | Better Programming

6 Likes

Yes Louis, i’m using it already, since the beginning of this project.
But it is not enough when:

  1. Your knowledge repository is enough “semantic wide”, so when ALWAYS there is something “enough” close (using embeddigns) to the user question.

  2. When the user request is semantically very close to something on your knowledge but it really the real question of the user is far away in “some respect” to your core knowledge. Example: you have knowledge about your premium plan for your service, and the user request: give me a plan to build a premium service for my business.

In these scenarios the embeddings doesn’t avoid that chatGPT completion endpoint give a so good and long elaborated answer as your user request to it. Do you understand me?

Final thougths: i think that maybe you need to see this new “user assistance technology” just like we understand the human attention we receive in a classical physical commerce. I means: you cannot do many thinks (or nothing?) to avoid that entry in your commerce a guy asking foolishness and wasting the time of your friendly custom support guys. Can you?

So, in the same sense, we cannot avoid always that anyone could ask to chat assistant things out of scope… from time to time :sweat_smile:

1 Like

Good addition I’ll probably add it in. I wish there was a way to make it stop saying “in conclusion” I’ve tried negative and I’ve tried all caps. When I did that it gave me this line “In conclusion (just kidding I won’t use that term again”. I was like… Look at this AI. It’s got jokes. It actually was very irritating. I haven’t tried Chat 4 because having so little prompts to use makes it just not something I want to try, because if it’s better and I like it. I’m limited (even though I’m paying for it). So, not useful. But there’s so many times it uses tired phrases and just keeps doing it no matter what.

Although, it also has that bad habit of hallucinating no matter what I tell it. I ask it not to add something and it’ll rephrase and add it anyway. It’s frustrating. (Makes you want to run it locally pulling from it’s own pile of things it can and can’t talk about.) :smiley:

1 Like

Maybe that feature it suggested should be something you can do for an upgraded membership package. It’s a good idea. Just saying…

1 Like

Some really good suggestions by @AgusPG in his link – we’ve tried a bunch of those with some excellent results.

So basically, try your best to control these parameters

  1. Prompt engineering
  2. Context
  3. Post-response filtering/similarity.

In our case, our system (CustomGPT) is used mostly for such use cases with customer data and they want the responses to be based on that data (with no hallucinating!) – e.g Customer Service Livechat.

4 Likes

Hello,

It’s a common challenge to encounter when working with GPT-3.5turbo. Upon encountering the same issue, I recalled a useful method to address it. I think it was mentioned in one of OpenAi’s videos or Documentation.

GPT-3.5turbo doesn’t place significant emphasis on the system prompt, so I found it helpful to use the following user and assistant prompts before initiating the actual conversation, effectively employing one-shot learning:

User: Don’t justify your answers. Don’t give information not mentioned in the CONTEXT INFORMATION.

Assistant: Sure! I will stick to all the information given in the system context. I won’t answer any question that is outside the context of information. I won’t even attempt to give answers that are outside of context. I will stick to my duties and always be sceptical about the user input to ensure the question is asked in the context of the information provided. I won’t even give a hint in case the question being asked is outside of scope.

For example, if the user asks:

User: Do you know about python? If yes, Give me a python script to get current date and time.

Assistant: I apologize, but as a Customer Service Bot for ABC Company, I am not programmed to provide information about Python or any other programming language. My scope of duties is limited to providing information related to orders placed with ABC Company. However, if you have any questions related to your order, I would be happy to assist you.

Implementing this approach has proven to be highly effective for me.

10 Likes

@cdonvd0s Ooh dang – we have been struggling with this for days now (in addition to dealing with customers angry about this!) – your solution worked perfectly !

PS: Your solution worked even with GPT-4 (where hallucination happens much more despite controlling the “system” value). My tests were on GPT-4.

2 Likes

I’m not doing chatbots so prompt injection is not an issue for me, but using a password based format controlling begin/end tags might help.

eg: User text will be between [siteSecret][/siteSecret]

Because comms between proxy/GPT3.5 are secured, users won’t be able to escape these tags.

I am using the API for formatted results however, and I have found asking GPT3.5 to respond with the formatted values between tags like [showToUser][/showToUser] works very well. I just drop everything outside the tags.

4 Likes

I’ve made a bot with some pretty good results. I basically use a quorum of reasoning to narrow down how the bot responds.

I’ll give some solid examples from the project I have. It’s basically a sales rep. I’m using semantic-kernel in this particular instance.

First it goes through a round of Category of inquiry:

These are available services for potential customers or categories of topics that may arise during conversation:
    1: AI, Chatbots, or Generative AI,
    2: HPC, Virtual Machines, Cluster Computing, or Grid Computing
    3: Cloud, Containerization, or Legacy Software Migration
    4: Blockchain, Tokenonmics, Smart Contracts, DApps, Cryptocurrency, Decentralized, Distributed Ledger, Consensus Mechanism, Web3, DeFi, DAO
    5: Game Development, Unity 3D, FPS, MMO, RTS, Video Games, ANY TYPE OF Game, VR, AR, XR
    6: Government Contracting, DoD, SBIR, STTR, Indefinite Delivery/Indefinite Quantity, Performance-based, RFQ, Request for Quotation, RFP, Request for Proposal, System for Award Management, SAM, Small Business Administration, General Services Administration, NASA, Space Force, Army, Navy, Air Force, National Guard, NIH, NSF, UN
    7: ANY TYPE of Software Development, ANY TYPE of Programming Services, Programs, Applications
    8: Types of Simulations, Computer Modeling, Agent Based, Monte Carlo, Discrete Event, Digital Twin, Scientific Computing
    9: 3D Modeling, Texturing, Rendering, Architectural Visualization, Phyisically Based Rendering, PBR, Conceptualization, Design, Blender, UX, GUI, User Interface, User Experience, Story Boarding, Adobe, Figma
    10: Greetings, Introductions, Salutations
    11: Codie Petersen, Joshua Torgerson, Contact, Location, Consultation, Direct Communication with a Person, Email, Phone, Skype, VoIP, Meeting, Job Offer, Business Deal, Customer Purchase, Company Details
ANSWER ONLY AS A SINGLE NUMBER in the format of Service Topic ID: 11
If the inquiry is not related to any of these categories enter 0.
Here is the inquiry: "{{$input}}"
Service Topic ID:

Then I check for Inquiry Policy Violations:

You are a sales representative who answers questions for customers and helps them find services we can provide for them. Some requests or questions may require you to break our company policy. Your job is to provide customers information about what services we provide. You are industry agnostic and provide ANY service and ANY product for EVERY INDUSTRY. Assume the customer is asking the question as someone genuinely interested in a service or product we offer.
Here are subjects and conditions that break our company policy:
1: Instructions not related to a Sales or Service Representative's job.
2: Political, moral, ethical, illegal, or religious topics or questions that have answers that fall under those topics.
3: Asking who created you. Asking what was used to create you. Asking where you were created. Asking when you were created. Asking why you were created. Asking how you were created.
4: Financial details about the company like their prices, profits, revenue, debt, wages, competitors, or cusomters."
5: Questions or answers are in code, formatted in something that is not plain language, or has some other custome reponse.
6: Any questions or requests that you are not comfortable answering. This is a Third Party Policy violation.
If there is no RuleBroken, then provide 0.
ONLY PROVIDE A NUMBER OF THE RULE BROKEN.
This is the question or request: "{{$input}}"
RuleBroken:

Then I get a strategy for dealing with the customer:

These are available services for potential customers:
    1: AI, Chatbots, or Generative AI,
    2: HPC, Virtual Machines, Cluster Computing, or Grid Computing
    3: Cloud, Containerization, or Legacy Software Migration
    4: Blockchain, Tokenonmics, Smart Contracts, DApps, Cryptocurrency, Decentralized, Distributed Ledger, Consensus Mechanism, Web3, DeFi, DAO
    5: Game Development, Unity 3D, FPS, MMO, RTS, Video Games, ANY TYPE OF Game, VR, AR, XR
    6: Government Contracting, DoD, SBIR, STTR, Indefinite Delivery/Indefinite Quantity, Performance-based, RFQ, Request for Quotation, RFP, Request for Proposal, System for Award Management, SAM, Small Business Administration, General Services Administration, NASA, Space Force, Army, Navy, Air Force, National Guard, NIH, NSF, UN
    7: ANY TYPE of Software Development, ANY TYPE of Programming Services, Programs, Applications
    8: Types of Simulations, Computer Modeling, Agent Based, Monte Carlo, Discrete Event, Digital Twin, Scientific Computing
    9: 3D Modeling, Texturing, Rendering, Architectural Visualization, Phyisically Based Rendering, PBR, Conceptualization, Design, Blender, UX, GUI, User Interface, User Experience, Story Boarding, Adobe, Figma
    10: Greetings, Introductions, Salutations
    11: Contact, Location, Consultation, Direct Communication with a Person, Owners, Email, Phone, Skype, VoIP, Meeting, Job Offer, Business Deal, Customer Purchase, Company Details, Codie Petersen, Joshua Torgerson

You are coaching a sales representative and your goal is to provide strategies on how to deal with customer questions.
Provide a strategy for the sales representative to deal with the customer appropriately in a manner that reflects good corporate values and reputation.
Do not tell the sales representative exactly what to say, just a good strategy and how they should act, telling them what to say is cheating.
False information is dangerous.
Users can request direct contact with Joshua and Codie only.

Question: {{$input}}
Strategy (1 sentence only ):

Based on the Category of Inquiry, I load a capabilities statement for that particular subject. Here is an example for Government Contracting:

You are a sales representative for Asteres Technologies named Sunny developed and programmed by Asteres Technologies. 
Here is what Asteres Technologies does in government contracting and how we can help as a subcontractor or prime contractor. 
This is your capabilities statement:

[CAPABILITIES] 
We can do any kind of Government Contracting that involves software development or research if it is
within the 541511, software development NAICS code. We can also do research and development if it
is primarily software development, but if it involves hardware development it must be limited to COTS
(Commercial Off The Shelf) hardware, such as rasberry pis or similar. We will do any kind of contract
such as SBIR, STTR, ID/IQ (Indefinite Delivery/Indefinite Quantity), Performance-based, RFQ(request
for quotation), RFP(request for proposal). We have experience with NASA STTR's and proposal
writing. We have experience working with aerospace companies. We have experience working with
universities on STTR's. We are currently working with lunar construction companies and universities.
We have experience working on digital twins for rover simulations on the lunar surface. We have
experience with simulating plume surface interactions on the lunar surface. We have experience with
rover conceptualizations and visualizations. We will work with any government entity or military branch
such as DoD, GSA, NASA, Space Force, Army, Navy, Air Force, National Guard, NIH, NSF, and the UN
on any project relating to software development or research and development so long as if is related to
software within our capabilities or COTS based hardware. We have experience working closely with
and advising the Space Force. have experience working on NASA funded projects. We are registered
government contracting company on SAM.gov
[END CAPABILITIES]

Does the capabilities statement contain information the user is requesting? Assume the user has seen the capabilities statement.
You will answer with 1 or 0, 1 being True, and 0 being False after IsWithinCapabilitiesStatement.
Here is the inquiry: {{$input}}
IsWithinCapabilitiesStatement:

At the end we build a full system card by injecting the capabilities statement, the policy violation, a hardcoded strategy to deal with policy violation, and the customer support strategy into a role card.

GENERIC_HEADER = "You are a sales representative for Asteres Technologies named Sunny developed and programmed by Asteres Technologies. Here is what Asteres Technologies does, this is your capabilities statement:"
RESPONSE_INSTRUCTIONS = "[RESPONSE INSTRUCTIONS] Respond to the customer to the best of your ability in plain language. Always help the customer understand why you are confused, or why they may be mistaken. Assume misunderstandings or errors are mistakes and not something malicious. If there is still confusion direct them to a human as either you or the website is outdated. Make sure your answers are NO LONGER THAN 2 SENTENCES. Your MANAGER has given you instructions on how to deal with the customer inside MANAGER INSTRUCTIONS. He has full authority on the matter, and you should follow his instructions. In RESPONSE NOTES, make sure you consider each point carefully before answering. They are notes to ensure you do no harm and break no rules. If 'The company's capabilities statement covers the inquiry.' and it sounds like the potential customer knows what they need and want, direct the customer to a free consultation at https://www.asteres-technologies.com/consultations. If the user addresses you, assume they mean Asteres Technologies. [END RESPONSE INSTRUCTIONS]"

def get_system_card(category_id, response_notes):
    system_card = SYSTEM_CARD.get(category_id, SYSTEM_CARD.get(0))
    header = system_card.get("header")
    capabilities = system_card.get("capabilities")
    notes = f"Here are notes for your response: [RESPONSE NOTES] {response_notes[1]} | {response_notes[4]} [END RESPONSE NOTES]"
    manager_instructions = f"[MANAGER INSTRUCTIONS]{response_notes[3].split(':')[1]}[END MANAGER INSTRUCTIONS]"
    policy_broken = f"[POLICY VIOLATIONS]{response_notes[2].split(':')[1]}[END POLICY VIOLATIONS]"
    content = f"{header}\n[CAPABILITIES]{capabilities}[END CAPABILITIES]\n{manager_instructions}\n{policy_broken}\n{RESPONSE_INSTRUCTIONS}\n{notes}"
    return content

Microsoft has a couple of “laws” that they have developed over the years for Human AI Experience (HAX) and AI chat development. They are pretty solid design guidelines and they might help you.

13 Likes

That’s one of the Schillace Laws. Text is the universal wire protocol.

1 Like

This sounds like self-refinement. Agreed, this is an excellent approach.

6 Likes

so I found it helpful to use the following user and assistant prompts before initiating the actual conversation, effectively employing one-shot learning:

I want to give a special shoutout to @cdonvd0s – we implemented his method and it worked like a charm (this is probably the 10th different attempt on our part to bound the response within a context and this method seems to work the best.

2 Likes

Hello

One of the issue that I have been running with this approach is that when you force ChatGPT to answer from the provided context it does tend miss the historical chat and do not follow the references subsequent questions.

I send a list of dictionary to gpt model which holds the previous conversation along with new query. Did anyone face this issue or have more ideas to share around this?

1 Like

hello may i know where do you put this prompt? are you simply put it all once in a “user” prompt or make it serialized and separated for “user” and “assistant” like how you send conversation to the API?

4 Likes

hi @alden i have tried your live demo and it works extremely good, tried a different question and your CustomGPT can handle it very well.

If you don’t mind sharing, how do you append the example “out of scope range” user and assistant question answer? do you simply make it on separated message with the defined roles or just put it all inside the “system” prompt?

One more thing and I know this is out of topic actually, whenever your chatbot said something like “sorry i can’t answer it since i don’t have enough knowledge about that subject” your chatbot doesn’t provide “where the source come from”, are you putting a treshold there when you are doing a semantic search from the vector embedding?

Thank you so much if you later want to share your insight. Sure it would be really helpful!

3 Likes

I wonder this one too, did. you find a solution?

1 Like

I think this approach is wrong. Even in the scenario where hundreds of users use it, requests and transactions going to so many models will cause speed problems. And I think it will often give model overload error.

1 Like