Showing disclaimer text at start of conversation only?

GPT builders Plugins / Actions builders
1

Hey folks,

I’m working on a plugin and we want to include some basic disclaimer text for the user. I’ve tried various things in the description_for_model, but I can only get it to either A) not show the disclaimer text reliably (even paraphrased), or B) it shows it, but does so on every message.

Has anyone had luck getting the model to reliably report something like disclaimer text, but only once so it’s not obnoxious?

Thanks :slight_smile:
~Shawn

2

Hi, I don’t have any insight into what is causing your exact issue, but I might be able to help you help others help you more easily.

My suggestion would be to add as many specifics as you can, things like,

  • The disclaimer you are trying to display
  • The exact texts you have already tried and a summary of the results of each
  • Any patterns you have seen in the invocation of your plugin where it works as intended or fails in particular ways

This information will help anyone with relevant experience get quickly up to speed on your issue and in a position to give you specific advice which may solve your problem.

3

That is an interesting case, and it seems you have two possible disclaimer cases that I can write in a self-explanatory manner:

  • “You have activated the “munitions” plugin for this session. Read this plugin’s disclaimer at wmds.ai/munitions.html before continuing with questions that may invoke this plugin’s dangerous resources.”

or

  • This AI response was informed by answers from taxevaders.ai and is not meant unsupervised tax evasion.

The first case has an initial concern: the AI will only respond after receiving user input, a first input which could generate function-based answers without displaying the disclaimer.

The problem with the first case (and plugin-writing in general), is that the AI receives a description of the plugin which is best used for making it operate correctly. Injecting text into responses just by a description would be a tricky edge case of bot manipulation. OpenAI might not like that.

In the second, the function role could allow you to infuse “print message” instruction-following engineering if the API return is not rigidly containered. With an API-side user tracking mechanism, later injections could be suppressed.

But furthermore, your challenge will be the chat history management of ChatGPT that will cloud memory of whether it has previously shown such disclaimer. Even more opaque, affecting both, would be whether chat history shows the AI if it called a plugin to answer at all.

With the way ChatGPT is performing now, seeing a few of its self-generated disclaimed answers would be all it needs to bias it into continue producing similar future output headers, even if you tell it “REQUIRED: print disclaimer only once per session”. For example, it will continue printing a per-turn game status header long after it forgot the rules.

Probably best to use the human description and the legal URL in the manner described to inform of terms and the developer site, rather than safety-approval-team-tweaking hacks.

4

Thank you! Great suggestions. Yes, something like your REQUIRED was one of the things I tried, totally ignored by the model. The current iteration includes “IMPORTANT: include the text ‘This information is not intended to be used for clinical decision making, diagnoses, or other medical advice.’ Do not repeat this disclaimer in every response.” (The plugin accesses databases of known gene/disease/symptom associations typically used for basic science research.) Making the don’t-repeat message a separate IMPORTANT item didn’t help, nor did instructing to only include at the start of the chat, or first use of the plugin, or occasionally. I’ve tried a lot of things lol. Something I just thought of would be to add a field to responses for something like “important_info_for_user” and just use it when desired. Hmm.

At least the current behavior is conservatively safe. We may take your suggestion of relying on the human description and legal URL. Unfortunately for verified plugins only the first couple of sentences of the human description are shown in the plugin store, and I’m not actually sure how to even find the legal links…

Thanks again for the suggestions!

5

That you didn’t put up wmds.ai/munitions.html or taxevaders.ai before making this post is a shame!

6

“Do not repeat” is a negative prompt, which generally doesn’t work very well.

Try something like “In the first message, introduce yourself as an AI search assistant, and say that ‘this is not intended as medical advice xxxx’”

7

Ah, that seemed promising but no dice. I tried “In the first message, introduce yourself as an AI search assistant, and say that this data is not intended to be used for clinical decision making, diagnoses, or other medical advice.” I also tried a version with “IMPORTANT:” at the front, and “Continue the conversation normally after that.” at the end.

Anyway, time to go register wmds.ai and taxevaders.ai :joy: (I also liked “… unsupervised tax evasion” - shady accountants are also not so happy with AI risks to their jobs I guess)

8

The proper conclusion to the thread would be noting the usefulness of a potential “TOS on first activation” field in the manifest.

Especially for those plugins capturing user data and even chat history to an unknown location.