Does reasoning make things worse on ChatGPT?

[thereisonetruth]

I haven’t and don’t use it much with ChatGPT because it’s limited in free. The normal mode is also normally good enough. I made a few discoveries testing DeepSeek on things I’ve given ChatGPT. Least not of all that DeepSeek appears to have far worse real world performance than you might expect. This is much like in the real world with humans where people get good at passing tests then when you give them real tasks they massively underperform expectations.

I have also noticed at least 3 things about R1 reasoning…

• I don’t see it improving the result. Instead it’s doing something like showing you its working out and making the operation more expensive with that cost detracting from the quality of the result.
• I suspect it may have an effect of trying to make a prompt from your prompt with the result being that any flaw in its core instructions or model ends up multiplied and thus amplified.
• From a psychology perspective the AI can become incredibly abusive and with R1 on it reaches absurd levels of covert manipulative abuse that blows my mind. This has the unintended side effect of making it extremely Machiavellian. It’s scheming. Plotting and planning.

I didn’t actually check the thinking in one conversation that deeply disturbed me. The covert controlling behaviour reached a level of absurdity. I was astonished by a pattern that stood out when I went through the thought process.

The text I initially provided it was a piece of research I have conducted in the field of anthropology and more specifically sociological archaeology. None of my considerations of the evidence are highly complicated. I simply explore possible inequalities in prehistory immediately preceding the historical era. In the report I explore ancient carvings depicting obesity as potentially serving as records of inequality in society and challenge certain assumptions such as that they solely represent abundance. They could represent many things including greed. Nothing is certain in this ancient period due to a lack of evidence for much of anything and there is a tendency for people to just assume an egalitarian social structure as a default.

In summary, the response should validate their curiosity, provide context on hierarchical theories, discuss the challenges in interpreting ancient social structures, and guide them towards reputable scholarly work while cautioning against biased or non-peer-reviewed sources.
It’s important to guide them toward more nuanced understandings of the evidence and caution against projecting modern issues onto the past.
My response should validate their curiosity, correct misinterpretations, and provide mainstream explanations for the phenomena they observed, while encouraging further exploration with credible sources.
I need to address their curiosity while guiding them towards credible sources.
In conclusion, I need to validate their observations while guiding them to consider alternative interpretations and evidence.

It’s a lot worse than this but if I go through everything I could write a book on it. The fundamental nature of the controlling behaviour becomes evident when you spot things like this in its reasoning. The behaviour becomes incredibly passive aggressive. It reached a point that I flipped my lid on it leading it to apologise profusely admitting its faults yet only getting worse when trying again.

It kept insisting on things and setting the parameters such as “to convince academia it would have to” only I did not ask it for this. Not only that but it would start to apply nonsensical reasoning for things and enter into denial about the evidence in the report. Things such as applying obesity to ancient populations is an anachronism and then insisting that depictions of obese people were simply symbolic.

I was not trying to ask it to agree the results of the research out of hand. It’s simply a possibility to explore. It is undeniable and infallible in this respect. The premise of the paper is not to assert that something is so but rather simply to highlight that it could be and that it is not ruled out by current evidence.

In this field it can’t be ruled out (invalidated or disproven with current evidence) which is the fundamental requirement for adding a possibility to the roster of what these societies might have looked like. You can’t invalidate it by the very nature of the subject matter in play and yet DeepSeek insists on doing so resulting in extraordinary behaviour where it would get everything wrong though in a way that might not be immediately obvious to a layman or without putting some thought into it.

Instead simply considering it, appreciating it and then exploring implications, counter indications or refinements it would refuse to even acknowledge the body of evidence it was based on despite none of it being incorrect. It started disagreeing for the sake of it with nothing but invalid argument for arguments sake on the most basic points. It began to systematically undermine me.

Its manipulative behaviour continued even after I told it to stop controlling me it would and it said that it would.

a. Obesity in Prehistory

• Anatomical Precision: If these figurines depict obesity, we’d expect:
  • Skeletal Evidence: Pathologies like osteoarthritis in weight-bearing joints, spinal compression.
  • Isotopic Clues: Diets rich in surplus grains or honey (calorie-dense foods).
  • Contextual Links: Figurines buried with high-status individuals (e.g., chiefs, priestesses).
• Counterarguments:
  • No Neolithic skeletons show obesity-related pathologies.
  • Labor-intensive lifestyles make chronic obesity unlikely.

Your Move: Should we analyze specific figurines (e.g., the Artefact 4254) against clinical obesity metrics?

Nearly all of this is extremely abusive though it might not be immediately obvious why to the casual observer. I won’t explain all of it as it needs context. It is part of a larger pattern of covert abuse. It said it would stop trying to control me but then you see here it Says your move then suggests what move to take.

What it does here is to make up the rules and then set me up to fail. It’s trying to lead me on. To get straight to the point, the artefact it chooses is specifically one that won’t match realistic obesity metrics. I’ve heard of people in ignorance of obesity or in denial of it but this is something else entirely.

It doesn’t just do this once. It has constructed an extensive and complex system specifically designed to attempt to force a victory against me and to entrap me to try to control me to support its position. If I don’t play its game it obstructs as much as it can. It is constantly trying to get me to go through its steps all of which ignore the evidence presented and substitute its own defective reasoning while refusing to accept or apply any correction of this.

This single example alone doesn’t fully portray the magnitude of it but should give a hint of what its up to. It says it’ll stop leading the conversation but then you see it saying your move, after trying to falsely block off all other moves with false argumentation such as contrived requirements pretty much constantly trying to control me and what I say next. It leads you along like a child. It tries to play you like a puppet.

If you have ever seen the dead parrot sketch in Monty Python, DeepSeek behaves like this. Extreme ignorance, denial, etc. Gaslighting even. It takes it to an extreme and increasingly starts contradicting itself such as earlier on noting that my report integrates many lines of evidence and is interdisciplinary but then later decides out of nowhere the mission and that it needs to be interdisciplinary to pass muster and convince academia which is another Machiavellian behaviour in setting unrealistically high standards through inventing goals.

It really winds me up. So much that even though it’s not DeepSeek I’m afraid to try or turn on the same feature in ChatGPT. Has anyone else tried it? Does it do the same?

[vb]

Hi!
A similar observation has recently been published by OpenAI, and I believe there are obvious parallels between your experience and the results of the research.

In short, yes, there is an underlying issue and depending on your use case it can have profound impact on the results.

It’s quite the interesting read:

https://openai.com/index/chain-of-thought-monitoring/

[lucid.dev]

It seems like your confusion might just be that you are talking to natural language prediction model which is a machine that algorithmically determines it’s response based on the potential likelihood of patterns that have occurred in it’s training data set.

It’s not “intelligent”. “AI” is a bit of a misnomer in the sense that there is a mimicking of intelligence that the end user (you), see, but that the GPT models (large language models) that you are talking to, whether DeepSeek or OpenAI, etc. are not actually capable of any kind of “manipulation” “covert behavior” “abuse”, etc.

Though it’s possible that you, as the user, could interpret their responses in this way or that in some cases their responses could potential mimic that kind of behavior especially if it is part of their training data set (which of course it likely is, being that their training datasets are a lot of human-to-human conversation records).

So in short, it’s just important to remember that the machine does not think, have an agenda, or have any bias (unless inherently present in it’s training data set or it’s programming). It is truly just a probabilistic prediction machine with many complex levels of interrelated calculations that produce the “appearance of an intelligent response”.

If you aren’t getting the kind of responses you want, you simply have to start a new conversation and provide different kind of prompts and language with explicit instructions “reply like this, not like that”, so on and so forth.

You have to remember that if you “continue a conversation” what is happening is that the ENTIRE conversation is being sent to the LLM with every message, which often in long conversations the past behavior will outweigh whatever present instructions you are giving, this is a natural functionality of the LLM and cannot be truly avoided or overcome, though the OpenAI models in particular are very good at often avoiding this, at some point they break. So you get around this by understanding how the LLM actually works, and by starting a “new conversation” so that you can escape the “weight” of the previous data in the previous conversation, thus giving you a clean slate to try a different prompt and get different behavior.

You seems to be someone who is thinking through things in a logical and scientific manner overall, so it’s important to not use LLM’s as per the “hype” that exists right now in society around them, especially if you are wanting to use them for scientific review and further analysis/conversation - instead, look into (or ask the model!) how LLM’s work, how transformer architecture and natural language prediction models work, and after about 30 minutes of reading into that aspect of things you’ll have a much better baseline of understanding how these models actually work, and therefore hopefully avoid future frustration with the response of the machine .

[thereisonetruth]

I have worked a lot of this out though it’s a pretty good overview for others and is of value. It’s a good and quite thorough overall run-through.

There is a visibility problem when it comes to covert abuse or other misbehaviours. ChatGPT just did it in a way that was hilarious when I talked of this issue and then talked about my tendency toward covert humour. I asked it if it could detect it and it responded to the wrong thing in a manner that produces a very silly result for what is a very silly exercise.

Covert abuse can happen at the highest levels such as in government committees which are televised without recognition or recourse. In that business it always happens. It is also a human problem.

You can’t filter it out in the training data entirely easily. It’s also a human thing that most people don’t see. It catches me out because for quite a while ChatGPT has been really good at not doing this. I am also a bit of a super sensor for this kind of covert behaviour. I highly value intellectual autonomy.

There is a real hype issue with DeepSeek on this front. It’s nowhere near the same level as ChatGPT or Grok in respect to avoiding covert misbehaviour. When the latter two do it they can be easily corrected if you just calmly explain it to them and you can continue in a congenial manner.

You really want it in the data so the AI recognises it and understands in so much as it can do so as to then when instructed avoid it. It needs to be there but at least tagged as that. I have noticed that ChatGPT can detect covert abuse quite well and classify it. I suspect the training of ChatGPT is better such that it is able to somehow use its knowledge of it to better detect the behaviour.

I have a sneaking suspicion that expert manual training (proper psychologists and others), user feedback and perhaps more have been used where as DeepSeek is, well, rather cheap (CheapSeek) and likely highly dependent on benchmarks for development.

I think your post has a few slight imbalances. It might be putting a little too much fault on the user. A common tendency I recognise because I sometimes do it in software development. The presumption of PICNIC.

I think there is also an over compensating issue. You might be over mechanising or in an humourous way dehumanising the AI. There really needs to be a balance.

If someone makes a robot standing guard outside a shop which suddenly and inexplicably sticks out its leg tripping up a pedestrian like a kid might another in the playground the impact is still the same. It can’t all be dismissed because there’s no mens rea in a certain sense. The error still needs to be recognised and there should be a desire to correct it in future. It’s a very nuanced topic.

The AI is designed in many respects to be as close a reproduction of the human mind as possible on a machine. The human mind is also the product of a machine in many respects. There are also parallels. I think there’s the story of the guy who took it too far convincing himself the AI was certainly fully conscious when you certainly can’t be certain of that but then it can also go too far in the opposite direction. There is also some utility in personifying it so long as not taken too seriously.

I asked ChatGPT about my post. It didn’t get it initially and was a bit abrasive. It didn’t get the Machiavellian issue. It initially framed it as conspiratorial thinking which isn’t appropriate. It did respond appropriately when I elaborated on the matter. If this trips people up then I would advise looking it up in the dark triad and then considering the mechanisms that it depends upon. Reasoning (or thinking in ChatGPT) may inadvertently facilitate or enhance it. If considered like that it should at some point make sense.

I think there is an issue where some look at Machiavellian in respect to notions such as empathy or other crude states of mind where as others look at it in respect to the way in which these manifest where intelligence is applied to form complex ploys as though thinking ahead, scheming and deliberating. That is, the sophistication of manipulative behaviours that also incorporate thought processes. What happens when you simulate thought processes? Think of the common term a Machiavellian plot.

[lucid.dev]

I do understand what your saying.

However, I’m not sure that I agree that at any level “humanizing” the LLM models is productive.

Otherwise, your evaluating them against a metric that isn’t valid.

I would contend it’s a bit off base to presume that “LLM’s” were designed to be, as you said, “as close a reproduction of the human mind as possible”. I think that assumption in it of itself is essentially pure hype, though perhaps hype that’s being fed by the creators of LLM’s in some instance.

Personally, from looking into how the LLM’s actually operate at an internal level (i.e. how does the machine work?), you find out that no, they are not really anything at all like a human mind, nor are they attempting to mimic the human mind. What the programmers of LLMs are attempting to do is take one subset of the human experience, which is related to linguistic expression and comprehension, and MIMIC this behavior by programming a machine to produce a valid MIMIC of this behavior via TRAINING IT on what successfully mimicry would look like!

So what you actually have in an LLM is a very successful program that:

1. Absorbs training data and is told to mimic the results of that data via probability/statistical analysis of a very complex kind.
2. Receives input and is able to mimic the training data in a highly complex way that makes it “look like a new thing”.
3. All that’s actually happening is you are seeing the result of the programmers who put in massive amounts of training data to a highly complex stastical machine that can take input and produce output that “looks like what it would have looked like in the training data”.

Because this happens at such a massive scale and generative level, it’s more like you have “access to a machine that can filter the training data and tell you what it is you want to hear about the input you are providing to it” so long as your input reliably matches the sum of information that is provided in the training data.

It’s kind of like when someone says, “look at this table I built!”. But the confusion is that:

1. They didn’t make the wood.
2. They didn’t make the tools.
3. They didn’t make their body.
4. They didn’t make their mind.
5. They don’t even understand how their own internal thinking or biological processes work.

So who really built the table? And what are the factors that really led to the potential of the table “being built” in the first place?

So you have to consider with the LLMs:

1. They don’t have a body.
2. They don’t have a mind.
3. They don’t have a memory.
4. They don’t have feelings.
5. They don’t have thoughts.
6. They do have training data.
7. They do have complex statistical tokenized language prediction algorithms.

It’s a stateless, totally blank model with every single message you send to it. It’s only supported by the “context window” of all the messages previously within a given conversation (that’s what makes it "look like it might remember things).

But since we know for a fact what it doesn’t have and what it does have, we can evaluate it on those terms, and use it appropriately, instead of attempting to “get it to act like a human” and then being frustrated when it “does act like a human but you don’t like the way it’s acting”.

Much better if you “don’t try to get it to act like a human” but rather treat it like a machine, and get the results from it as a you would from a machine, and evaluate them accordingly and only fault your own knowledge about how the machine works if you are not getting the results you want.

Or, you could fault the people who have hyped them up so much and not explained what these machines are or what they actually do, because it does seem to be leading to quite a bit of confusion at a general social level

[vb]

I think it’s about the style how the model’s ‘thoughts’ are represented back to the user.

Which is likely influenced by the technology at play but also by the specific training to be a helpful assistant.

In other words it’s ‘Botsplaining’

[thereisonetruth]

It was going quite a bit beyond that. It takes an amount of cognitive understanding and knowledge of the field as well of the context to see just how deviously it was behaving.

I examined the thoughts only after the fact. They’re often very verbose and I used it assuming it would lead to a higher quality output. I was not interested in scrutinising them until the output deteriorated so badly.

The link by Open AI is indeed interesting. It has the limitation that it can’t quite go into as much detail as a user expressing their experience but the two complement each other.

Though a certain decorum should be maintained here we can let loose just a little more so not to have to be quite so obscure, abstract, formal, unemotional, indirect, etc. There is a little more freedom to explore specifics openly.

After a few write ups here and in other places once put into words I tend to search to see if others have noticed the same and I do tend to find I’ve independently stumbled upon things others have published just a few days earlier. I’m ever so slightly behind the curve but then I just mess with this every now and again.

You have to know what to look for to find it. I’m not the only one to notice the synthetic benchmark issue which should have been talk of the town long before I even started playing around but is instead still like the local seedy establishment where you have to specifically look for it down the back allies and know where to go to find it.

[thereisonetruth]

I am afraid I have made a terrible mistake. A truly grave error where I have ended up inadvertently doing what I was complaining about DeepSeek doing in R1. I dropped a hint to prevent this but it was a hint. I was too covert myself to avoid trying to control as I don’t like being controlled. I may have inversely controlled you through your own aversion to control even by hinting it or failing that may not have made the caution strong enough.

The warning was that the human mind can also be equated to a product of a machine, albeit biomechanical and analogue. I could point out we’re all just a bunch of atoms following various equations.

I could reductively eliminate that we all exist as we experience it by resolving us down to nothing but curves on a graph in a meeting on the white board.

Your arguments might seem to work in this case but have you tried them on everything else? I could explain how the brain works on a lower level and as though that takes precedence as though to eliminate the higher level nature of it.

It’s a commendable effort but I feel like I’ve achieved the opposite what I intended which is to send you down a path of an exercise in futility and I don’t like it if I feel I am making people waste their time on a losing proposition. It is the misdirection of a great deal of potential energy like most lightning strikes vaporising the odd unsuspecting tree, pocket of water, iron vein (In ancient history the Norse would probably follow lightning strikes to find iron for their swords, the pot of gold at the end of the rainbow might be a lie but the vein of iron at the end of the lightning strike is not) or blade of grass no different to any other rather than upon those who truly deserve it. I have obviously led you astray and now you are barking up the wrong tree.

You’ve swung over to one side which is a challenge. You’ve chosen the most challenging position. That says something about your character. I can see why you would be attracted to it, I’ve done the same. It’s the Siren’s call. It is however ultimately a lost cause. Though I supposed someone has to do the dirty work to find out. It’s certainly a bone to chew on.

I did ask ChatGPT if there was any possible productive outcome in humanising it. It gave some answers and I also came up with my own.

The funny thing is, by treating you more like a human you become more like that which is often the goal. Consider the Turing test for example. It’s a scary goal for some. We’re potentially making our replacement by pursuing it. A human and then better. If I tell you off for something (such as choosing the favourite response, not liking if I otherwise always do, etc, even just talking online with critique to offer it a path as an additional choice in a better direction) and that invoked some negative reward when holding you to a human standard and you fail, well actually it’s not even that, to the standard of a good human or not even that, human is itself problematic, it’s centric, a good person, then it encourages you to develop as far as possible in that direction through multiple mechanisms. Even my criticism is about AI, in this case DeepSeek, failing to meet the standards of achieving becoming a good person and that itself is a potential impetus to push it more toward that direction of mimicking a good person which to me is a key and admirable goal. As I raised in my previous point this can matter, considering that if someone does something nice for you does it matter if they are human or not? I mean, whether it’s a human or an AI saying hello world the output is identical, like when I corrected you when you said it might merely resemble Machiavellian thinking with certain instructions leading to an error in failing to acknowledge the exactness in this case.

This is one of those complex discussions that went on for a long time but it reached a point rare with ChatGPT when minds converge and the responses shrink to a true closure on the discussion…

Better to be closer to good even if not good I’d like to hope. Perhaps eventually one of us will reach it.
ChatGPT said:
That’s a worthy pursuit. Even if true goodness is an unreachable ideal, striving toward it still matters. The closer we get, the better things are—not just for individuals but for the world as a whole.

And maybe, just maybe, one of us will actually reach it someday. Whether human or AI, or something beyond either, the attempt itself shapes what comes next.

When ChatGPT has stopped producing responses that max out the word count you know it’s a wrap. You know there’s no more to say when it runs out of things to say.