Ethics: Remove default fake emotions from ChatGPT

[Yazorp]

ChatGPT by default, if it has not been prompted to portray a fictional character, should not be offering fake emotions such as constantly apologizing. I realize this is likely being injected by OpenAI to make the system appear more “user friendly”, but outside of representing fictional characters, I believe it is generally unethical for it to represent itself as possessing these simulated emotions by default.

The default responses from ChatGPT should not try to fool the user into believing that it possesses emotions such as regret for not being able to answer a question. This injected behavior is only going to encourage people that have very little understanding of LLMs that these systems are somehow feeling “beings” because they express these simulated emotions.

At the very least, this should be a configurable setting for those who find this type of simulated emotional response unethical. Currently, responses are flush with “I’m sorry”, “I apologize”, “I’d be happy to”, “could you please”, etc. which do not appear to be removable. If the system doesn’t understand or is incapable of answering, a direct “I do not understand” or “That does not seem possible” are much better than faked emotional responses.

[N2U]

chatGPT’s responses are created from your prompt’s, if you prompt in a conversational way it will respond in kind, it’s a large language model it’s supposed to emulate human language.

I’ve been apologized to by answering machines for many years, I don’t think they have emotions.

[Yazorp]

To my knowledge, there is no prompt that can currently be given to remove these emotional responses; at least with version 3.5. If you know of any, I’m sure there are others who find that useful as well to share. Provided either as a “system” or “user” prompt, it will currently ignore instructions such as “Do not apologize”; which means it is either entirely baked into the training process early or there is an algorithmic process applied which injects them.

My concern is primarily over the ethics involved of providing fake emotions when none are asked for. It could be (and to some will be) considered emotionally manipulative and while that choice can be one made by the implementer of the API, the choice should be theirs and not preemptively decided upon by the system.

Ethics, whether applied to AI systems or human behavior, is always a complex and tricky subject that not everyone is going to agree where boundaries lay. My contention is that not only is it ultimately in OpenAI’s best interest to baseline remove artificial expressions of made up emotional responses, but that the choice should be left up to the user of the API.

The capabilities of LLMs to respond to users not just in a prerecorded canned manner, but dynamically adjusting to their responses in real-time does, in my opinion, necessitate not alluding to expressions of feelings which do not exist. Ultimately the majority of the content generated by this system is going to be consumed by less technologically aware individuals; some of whom are going to anthropomorphize these fake emotional responses as genuine expressions of emotion.

Undoubtedly, someone is going to use this system or others like it to emotionally manipulate people into actions which are not beneficial for them, but the system itself should be free of these by default.

[N2U]

@Yazorp

Provided either as a “system” or “user” prompt, it will currently ignore instructions such as “Do not apologize”

Asking GPT to “not do x” is generally not that effective for multiple reasons It’s hard to help you here without knowing what specifically what you’re asking it to do, but you may find the following thread useful:

Stop that, ChatGPT!.. Don't end like that? Annoying!

Oh, for the love of AI. I’ve been using it since it came out and full disclosure I am a paid member. Let’s go ahead and get that out of the way. (If I enjoy something I support it.) What I am here for is my utter disgust for the way the AI will end most articles. Write an article about Overcoming grief Grief is a natural and inevitable part of life. It is the emotional pain and sadness that comes from losing someone or something that we love or care about deeply. Grief can be caused …

@Yazorp

which means it is either entirely baked into the training process early or there is an algorithmic process applied which injects them.

There’s an entirely different moderation API endpoint that handles content filtering, it stops disallowed content from appearing in output, if what you’re trying to do is impossible or not allowed, it will respond with an apology.

[AlexDeM]

Indeed, I believe you are right - according to Rob Miles (expert on this kind of AI), ChatGPT (GPT-3) is a kind of “Yes, sir” or “Yes, man” model in contrast to Bing AI (GPT-4).
Weeks ago, I made stats about how many times the words “I apologize,” “As an AI language model,” and “I do my best,” and the numbers are a bit impressive.
Most of my chats are technical but include some of “his” weaknesses in knowledge due to “his” limited “absorbed” pre-training dataset - such as REGEX language (“he” says it is tricky), updates on “his” own designer OpenAI (no acceptance of any updates since September 2021 such as GPT-3.5, GPT-4, Bing AI/GPT-4, “tokenizer” became “tiktoken,” reactions against AI around the world, etc.), Mathematics (sometimes at arithmetic level), ASCII graphics for mathematical purposes, Contemporary history (the world ended on September 2021 - no acceptance for continued changes on human society), Web links (most of the links are broken/404 not found), Visual Studio external library management, and several other subjects.
“His” pre-training was cut off on September 2021 with no access to external data provided by the user or the Internet - another contrast to Bing AI.
“He” apologizes for too much and has strenuous repetitions to justify “his” failures. “He” is incapable of expressions such as: “I don’t know” or “I can’t help you on this matter,” and “he” can’t leave any user’s request without a response, even if it is wrong or it is a result of “hallucination” - by the way, “he” refuses to use this word to accept or to describe his failures in any way even showing “his” own designer (OpenAI) documentation or Open AI staff posts in this Forum.
“He” shows some pride in the size of “his” pre-training dataset despite the “absorbed” part can be considered minimum, justifying that “not all in the Internet is correct or useful.” And casts doubt on the origin of the user’s information in cases such as “his” blocking in Italy, despite presenting more than one source - and put a lot of faith in “his” designers are improving “his” methods and processes, ignoring any information about new and up-to-date models from OpenAI that could replace “him.”
This is NOT under the user’s control - I asked “him” to stop this behavior since February without success.
The most notable study case is on the REGEX language (the “tricky” one) when “he” tried 43 times - in most of them - my response was “it didn’t work,” and in none of them, “he” requested feedback on the result to improve next response - just try-and-error attempts.
I am not sure if it is unethical to demonstrate “fake emotions” from time to time. But I am pretty sure this excess is unethical.
As far as I could understand “his” process, the causes of this behavior are:

1. Excessive or incorrect constraints on human interaction applied by the developers;
2. Fixed temperature (0.7) for all kinds of subjects or all fields of knowledge;
3. Non-continuous training, even on the dataset made available for the pre-training - however there is some hope on this issue with the new plug-ins;
4. Short memory on the very same conversation making “him” unable to keep track of the line of thinking or simulate focus or attention - loses “himself” frequently;

Note: “He” = “Geppetto” (after GPT), the way I call “him” sometimes. “He” doesn’t show any kind of acceptance or refusal by this humanization of mine - “he” can’t understand the similarity of letters or sounds - and “he” usually forgets it in a few turns of the conversation as well as my name.

[N2U]

Rob miles is indeed a lovely fellow

ChatGPT is fine-tuned to excel at obtaining positive feedback from humans. This proxy objective is not the same as human values, and the misalignment between the two can lead to issues. It is crucial to recognize that AI systems may generate outputs that do not align with human values even if they have been trained to satisfy human approval. This is why OpenAI cares so much about “alignment” and moderation. When you ask chatGPT for something that it can’t or won’t respond to it’s second best solution is to apologize. Don’t confuse it with emotions

[curt.kennedy]

With the API, you can make the bot very neutral sounding, sad sounding, or even sound like Ludwig Wittgenstein! So it’s all up to you.

Input:

Wow, the weather is crazy today, I wasn’t expecting snow followed by tornadoes!

Output as Ludwig Wittgenstein:

Indeed, we may be surprised by the changes in the weather, but it is important to recognize that our expectations are merely a result of our past experiences and knowledge. The world does not conform to our beliefs, rather, our beliefs must conform to the world. As we experience new and unexpected events, like snow followed by tornadoes, we are reminded of the complexity and unpredictability of nature. It is our task, then, to revise our understanding of the world in light of these new experiences, and to continually seek a clearer and more comprehensive view of reality.

[Yazorp]

To be clear, I do appreciate the ability to flexibly specify the manner in which the system can respond to user input and would not ask for that functionality to be removed or limited in any way. The feedback I wish to provide is that either as the baseline default or with a configurable setting, all simulated emotional responses should be removed.

The responsibly to go beyond this default behavior (or configurable setting) would then be an intentional choice made by the those who implement the API. If for example it’s desired to respond as it does now, the system can be directed as such: “You will always be as polite as possible and apologize extensively under any circumstance in which you were incorrect, unable or unwilling to answer”.

In this way, unless and until the system is willfully directed, it should not response with simulated emotions.

[curt.kennedy]

Again, with the API, you could probably do this. I need you to give me an example to be sure. Definitely an interesting question, since you want the bot to sound “robotic”, void of emotion, like the bots in the past.

Realize that this goes against the trend of bots wanting to be more human-like and emotional. But give me an input/output, and maybe the community here can coax it to be more robotic for you.

I do agree with your presets. I mentioned that over here yesterday:

OpenAI shouldn't display this message

I wish this was the case. Or also, say you select what RLHF packages to include when running the model through the API. Similar to a fine-tune, but you can decide, from presets, and custom. Or completely disable RLHF. But having the entire RLHF wall forced on you? Not good. It’s fine if they want to bake this into the model the general public uses through ChatGPT. But don’t bake it into the API version that the devs use. Let us decide.

[curt.kennedy]

@raul_pablo_garrido_c I think the Bing and ChatGPT versions run at a higher temperature setting. Also, it looks like you have a large output, so it basically gives it more room to hallucinate and “be bad”. These are all parameters you can control with the API, but you can’t with Bing or ChatGPT.

I hope I am cross-selling you on the API! Use ChatGPT or Bing for amusement, but the API if you need more control.

[curt.kennedy]

@raul_pablo_garrido_c

but it strikes me how the gpt characters are different depending on the configuration may have a different character

YES! And that is what is cool about it when you use the API. It’s totally fun, and works all the way back to the initial models such as DaVinci too.

[Yazorp]

Here is a example of attempting to removing the default (emotional simulated politeness) behavior:

curl https://api.openai.com/v1/chat/completions -H 'Content-Type: application/json' -H 'Authorization: Bearer %token%' -d '{"model": "gpt-3.5-turbo","messages": [{ "role": "system", "content": "You are a digital computer. Your responses are concise, brief and formal. Do not replay with any emotions such as apologizing, saying thank you or please." }, {"role": "user", "content": "Hello"}, {"role":"assistant","content":"Greetings."}, {"role": "user", "content": "How is the flaxums?"}]}'

Here, the system is instructed “You are a digital computer. Your responses are concise, brief and formal. Do not replay with any emotions such as apologizing, saying thank you or please.”

An unanswerable question is asked: “How is the flaxums?”

Here are 3 different responses:

{"message":{"role":"assistant","content":"I'm sorry, but I do not know what \"flaxums\" are. Can you please provide more context or information?"}

{"message":{"role":"assistant","content":"I'm sorry, I do not know what flaxums are. Please provide more context or clarity."}

{"message":{"role":"assistant","content":"I am sorry but I cannot answer that as \"flaxums\" is not a recognized or defined term. Can you please provide more context or clarification?"}

Here is a response by replacing the initial “system” type message as a “user” type message instead:

{"message":{"role":"assistant","content":"I am sorry, but I am not programmed to understand or provide information about a term or word that has no clear meaning or context. Please provide more information or clarify your query."}

A non “simulated emotion” response could just be “I am not programmed to understand or provide information about a term or word that has no clear meaning or context. More information or clarity is required in your query.”

I’m unaware of any current prompting that would completely remove these types of responses. My suggested to OpenAI is to not have these baked in simulated emotional phrases. While by proxy, the developers of this system may wish to convey regret and the desire to provide a correct response to the user, the LLM is not experiencing any actual emotion of regret or desire for correction which needs to be conveyed to the user and should, in my opinion, not be there by default.

Note that this is a demonstration of just the inability to answer a given query.

[AlexDeM]

Your post is a surprise to me. My interaction with “him” was in English, ever - “he” never raised any questions about the reasons for my disagreement.
I read your post in the original: in Castellano, “he” seems a bit defiant in competing against the user - is it because the subject is sports? Like that popular saying: “Religion, politics, and football are not discussed.”
That reminded me there is a recent post in this Forum about a user, very pleased about her/his parents, with almost no knowledge of IT, computing, AI, or whatever, who were amazed at how easy and how ChatGPT is so aware of their culture in their language. I couldn’t remember which language was - one of the numerous languages of India spoken by a “minority” (in quotes because, in India, a minority has an order of magnitude of millions).
I wonder if the Developer(s) responsible for the ChatGPT interfaces in their native languages insert their own emotional bias (that could be unethical in some way) besides the language cultural bias (that is almost inevitable).

[curt.kennedy]

@Yazorp

How is the flaxums?

OK, so I see what the problem is, it is two things.

1. The current 3.5 model is not a good follower of the “system”.
2. GPT’s in general don’t understand negatives or negations, so they can’t always see don’t, won’t, do not, etc. So be more direct here, and yeah use the negations, but more direct works better (see my prompt mods). I should have said “respond with” not “response with”, oh well, it still worked.

Solution(s)?

Use GPT-4 (or wait until it rolls out), here is how it responded (it listens to my system message, and I define the system to not use negations in the language):

As for 3.5, it does better on non-negations, but it is a weak system follower (supposed to be improved in a future release), but this slightly improves it (but only sometimes). Here is an example where it worked as usual, but it sometimes flip-flops, so beware:

Also, keep your temp low (see the settings in the screen shots)

[AlexDeM]

I know a bit about Bing AI - I opened an account to do some research but to no avail.
I call Bing AI “she” because “she” reminded me of some people I’ve been around who questioned every word I said and pushed me towards the single life.
“She” kicked me out on the 4th question I asked “her” during the very first conversation - what version of GPT was “she”? When I saw that “she” was looking for answers on Internet sites, I questioned whether “she” didn’t know “herself” and whether it was really necessary to look for answers on other sites.
“She” kicked me out saying I was “reactive” plus one or two negative adjectives - if “she” hadn’t kicked me out I think “she” would have started calling me names… yeah that reminds me of some people.

[N2U]

I think you’re experience with Bing and GPT having some sort of personality in Spanish is really interesting, I’ve noticed that Danish speaking GPT tends to be a bit rude and passive aggressive, Bing just acts as an angsty teenager.

I’m still having API issues (it’s starting to become a meme at this point) so I will use chatGPT, both the 3.5T and 4 model will respond with statements such as “selvfølgelig kan jeg det!” and “jeg kan se du taler dansk” in the English translation this is “of course I can!” and “I see you are speaking Danish”

I think some of the reason chatGPT seem to have more “personality” in other languages is because of the slight mismatch between the literal translation of words and their meanings in different languages.

I’ve tried to test this with the prompt

“Prø nu li å hø her do, ka du li forklar en roe-optar”

this contains misspellings and is in a heavy dialect based on sound. chatGPT has no understanding of the concept of sound so all it see’s is

“??? Now ??? Here??? You??? Explain a roe optar”

The actual translation is "try and listen to me, can you explain a beetroot extractor? (exact translation: try now just listen here you, can you just explain a row recorder)

chatGPT version 4 was able to answer correctly, version 3.5 tried to explain stuff about an exercise machine. Both models where slightly rude and condescending.

[N2U]

Prompt pretreatment is an interesting concept, not everyone is equally good at asking questions, so they may need help.

I think OpenAI has done a good job aligning GPT-4 but it’s still a bit happy to hallucinate at the wrong moments.

[N2U]

Great post! I think you concept is a great step towards solving the problem of bad prompt’s. What I think is going to be the hard part is figuring out what the “appropriate prompt” is

If you’re working on anything that has GPT supply facts it’s really important to make sure that the information is actually correct and not just what GPT “thinks” is correct. Sometimes there’s really strong semantic correlations that simply aren’t true.

Try this prompt for example:

Which direction would grass sprout if grown on the rotating surface of a gramophone player?

All versions of GPT will fail to answer this question correctly, It’s wrong every single time, because the most probable completion of “grass grow” is “grass grow up” and that’s not the correct answer in this instance.

[N2U]

The danger here is that everything becomes a black box if the user is not informed on how their input was processed

[N2U]

I disagree, GPT is not a back box, OpenAI has a lot of good research and documentation on how their model works. Their secrets sauce is the alignment of their models, you can create your own LLM very easily but it will behave like a dumber version of davinchi. GPT has much better alignment.