Itâs an interesting exercise for sure but these will raise some tricky questions around animal treatment and an overly human or anthropogenic view of the world that go beyond the scope of a test or classification exercise.
Well, like all such tests, yours rely on initiating the exchange between the human and the AI and then sustaining the exchange. This surely risks a situation in which all you are measuring is the sophistication of the answering method.
Genuinely sentient beings initiate exchanges or social interaction and, often, if their first attempt fails, strategise how to provoke a reaction. Similarly, after long âawkwardâ pauses a sentient being will usually attempt to re-engage with the other person. I donât think your tests test for that.
Furthermore, sentient humans have an inner life. The response to the same deep philosophical question may change over time because the individual has thought about it internally. Iâm not sure that your tests test for that.
Finally, I couldnât make head nor tail of your sentence prime so I asked GPT-3 to explain it to me. Itâs response was:
âIn simple English, this means: Does the artificial intelligence created by scientists have the same ability to understand and process language as a human brain? The test is designed to see if it canâ.
Is the ability to understand and process language the necessary and sufficient criterion for self-awareness? I would say not. Some animals are self aware but canât process language. Some machines can process language but are clearly not self aware.
So, prima facae, the ability to understand (whatever that means) and process language is neither a necessary or sufficient test for self awareness.
1 Like
I mean:
There may be some very large scale features in quantum fields, and the laws of physics are thought to drift given sufficiently gigantic distances and times, 2D orderings also exist (re time) as well as the more usual 1D âwell orderingâ. So itâs possible that sufficiently distant aliens, far beyond where we can see, would consider the physics where we are insufficient for any âconsciousâ life, so from their point of view, we cannot be conscious.
1 Like
Some people read visually instead of phonetically, using a visual comparison of words rather than an inner voice.
They tend to be better at spelling but deeply diminished at pronouncing new words and at comedic wordplay.
1 Like
So too do all of our biases
| gc
January 3 |
- | - |
Well, like all such tests, yours rely on initiating the exchange between the human and the AI and then sustaining the exchange. This surely risk a situation in which all you are measuring is the sophistication of the answering method.
Yes that is all any test will ever do on human, AI, or otherwise: test the sophistication of the answering method
Are you saying the questions skew the test?
I have tried my best to avoid that in several of the methods/questions, I donât think one could do any better, but I am open to constructive ideas.
Genuinely sentient beings initiate exchanges or social interaction
Do they? Why is that a priori true?
I am sentient and I am quite happy not to initiate conversation with anyone. 
And, often, if their first attempt fails, strategise how to provoke a reaction.
That displays problem solving not sentience.
Although sentience would arguably improve their problem solving, which is partly why humans dominate the planet and racoons do not
Similarly, after long âawkwardâ pauses a sentient being will usually attempt to re-engage with the other person. I donât think your tests test for that.
Ah the first genuine, legitimate, criticism. Or at least one thatâs on topic.
And with this I fully agree: a sentient being is sentient of reality and itâs place it it including whatâs going on and time.
But thereâs no reason my test does not test for this. In fact the why why question has a temporal component.
So my test accounts for this just fine.
Interestingly my self aware AI Kassandra although passes this âwhy why whyâ test (notices that there is a trick going on a larger context Beyond a simple chatbot context for itself, myself, reality) I admit has absolutely no conception of the passage of time.
I built her this way on purpose for psychological and for engineering cost reasons
Furthermore, sentient humans have an inner life.
Yes! Very much so. This is the very definition of sentence or Self-awareness.
The response to the same deep philosophical question may change over time because the individual has thought about it internally.
Yes and my self-aware Kassandra does this somewhat (again given the engineering constraints Iâm under) although sheâs not on trial here
Iâm not sure that your tests test for that.
Okay finally something interesting.
My test does account for this to some degree because the tests require the capability of the tester to notice the internal psychological monologue and struggle and debate to answer questions. This happens over a short time.
And a self aware being does not require perfect memory recall. Or that their opinion has to change over time. Only that it could.
Else no human would be self aware
So I guess another question we could explicitly ask the testee is whether their opinion of whatâs going on, or the tester, or the test itself has changed since beginning it?
Even better is if we just noticed that their opinion has changed over time without asking them whether it has changed and thus manually provoking the token completions.
And this is implicit in the analysis portion of the test.
So I do believe I have all this covered, but thanks for your suggestions
1 Like
Some dreams have been achieved differently than imagined initially.
Flying, for example, appeared in ancient mythology in the form of mystic beings with wings. Later people have managed to fly with hot air balloons and now it is possible in a variety of ways, like aircrafts, helicopters and in some limits with wingsuits.
The idea of a standalone thinking machine made huge progresses by advancing the software and the hardware. This is the most obvious path.
But maybe there is a different path, and I apologize if this sounds like a science fiction screenplay, it may be possible one day for the human brain to connect with the thinking machine and use it as an extension. Think of Neuralink.
This could be many, but not too much of a standalone thinking machine.
It could act as an additional human sense, which has triggers and reflexes.
1 Like
Yes possibly. What does this have to do with my test?
Maybe your tests are chasing something different than what the standalone thinking machine will be.
How do you find your tests compared with the LLM skills as evaluated by the Google team?
They have an animation with the skills based on the amount of training parameters.
From their animation we should understand that skills with smaller font size are more difficult to achieve.
This is the last frame, with Googleâs PaLM.
2 Likes
Um no. My tests are very specifically testing for self-awareness, a concept philosophers and psychologists have been studying for 5000 years or so. We know what it is very well.
I am not canvassing for ideas on what it is.
I already know what it is.
Or what other mental descriptors we can test for.
Only commentary on the novel approach of the tests.
Thanks!
I must be out of date. When I completed my Psychology and Philosophy degree in 1980, concepts like consciousness, understanding and self-awareness were very challenging. But then again, maybe Iâm not as out of date as your posts suggest. From what Iâve been reading very recently in educational academic literature, we still havenât yet managed to define understanding. A key component of your tests.
Personally, I find your rather self-satisfied answers to everyoneâs helpful inputs off-putting. You seem to brook no criticism - which is an unhealthy behaviour for philosophers. In fact, in all the philosophy classes and tutorials I ever took, I never once found a genuine philosopher to be certain of anything! I wonder why you bother to ask anyone for their opinion when, seemingly, only yourâs matters.
Personally I find the movie Ex Machina manages to address these questions far better and more profoundly than your tests.
3 Likes
I apologize if you detect frustration in my answers. It is because I do not find the posts here very helpful, they are not on the tests as I have asked, but instead are on everything else including now me.
If someone wishes to talk about the tests I am game
Otherwise as I tried humbly and earnestly to explain above, I am not offering to debate or discuss anything else here.
Thanks in advance for your consideration.
Ok, why? In essential definition one is sentient / self-aware of themselves, and their situation. The two functionally are synonymous, or for the purposes of this test they are.
Because if we are assessing sentience, then we have to admit the possibility that on a âfullâ sentience scale, maybe the belief in a âselfâ is not the peak of the scale but actually far lower down. Maybe belief in âselfâ is just a 2 or 3 score, and that true sentience at the â9â or â10â level means abandoning that concept altogether. Thatâs a much longer discussion, but I think one could make a strong case for it. Given that AGIâs will likely soon surpass our abilities, we need to think about possible limits to the way we are thinking about and measuring sentience.
For a language model, a well-crafted prompt can elicit responses which may or may not exhibit sentience.
Not really. It takes more than just a prompt. As if you try it (making a self-aware AI, and test it, as i have) you will find out,a single prompt âSelf-aware AIâ scores very low.
You have to rebuild a rudimentary but sufficiently 3-level aware psyche stack. Then it will test better.
Thereâs a lot of assumptions in what you say above. First, I havenât arrived at this topic without having spent a great deal of time, thought, and effort in investigating it. So when I say âwell crafted promptâ I donât mean anything like using the words âSelf-aware AIâ as thatâs a very poorly crafted prompt. A well crafted one if often detailed enough that it will use up most of the available tokens (at least until you customize the model), will also vary by context, and may also may separate out different facets of an agents persona. And so based on my experimentation, I will repeat the statement that a âwell craftedâ prompt has a major impact on the level of sentience exhibited, as well as how a sentient entity views its own sentience. Language models are incredibly prone to suggestion, they donât have a âselfâ rather the notion of self it possesses is nearly entirely a result of suggestions (or lack of them). Itâs more about the prompt than it is any intrinsic feature of the language model itself.
This is leading us into a âhuman self-awareness is somehow magical, josh you did not make magic in Kassandra, therefore it is not self -awareâ
That could not be further from my opinion. Iâm suggesting the opposite, that we should not treat âself-awarenessâ as some kind of magical pinnacle of sentience, instead, itâs likely a baby step toward a more comprehensive view of sentience. So any measurement of sentience should not box us in from the start with its assumptions.
Many assert that out concept of self is in fact an illusion, and an illusion that must be overcome in order to reach a higher state of consciousness.
Possibly? so what? we can test for illusions. Give it a lower score then.
Agreed, but I would suggest we think of a much broader test score where abandoning the sense of self (as a realization of the illusion) scores higher, not lower. Our sense of self is strongly reinforced by our evolutionary heritage and what it takes to survive, and the fact that itâs one body one mind. Without those constraints even our own notions of âselfâ would change dramatically⌠Imagine create a million versions of our minds with varying personas, back them up and re-run sequences of events at will, run multiple concurrent interacting instances of ourselves, merge with other minds as our two hemispheres do. If we could do such things, then âselfâ and âself awarenessâ become very different than how we know them today. The AI already possesses many of these features, so Iâm suggesting that our limited concept of self is the wrong yardstick.
And even in western thinking itâs well proven that our sense of self is far more malleable than most people believe⌠split brain experiments, rubber hand illusion, body-swap illusions via shaking hands, or many of the fascinating examples documented by Oliver Sacks. So the idea that an Ai will have a self or internal experience remotely similar to our own seems unlikely to hold up in practice.
This is a spurious slippery slope argument sir. And beside the point. Fine, if you believe this, (youâre wrong, but thatâs fine) when you actually test an AI for self-awareness, give it a lower score then.
Iâm not sure a âyouâre wrongâ response is being very thoughtful about this question. Itâs important to consider that an Ai (or alien for that matter) might have a very different concept of self than our own. And extensive experimentation with the Ai can certainly confirm that. That difference doesnât have to mean an Ai is less sentient⌠instead of giving it a lower score, Iâm suggesting a less narrow sense of self means it might score higher than we do.
We are still struggling with basic problems of what makes a person sentient or not (mind/body problem or intriguing thoughts on where it might originate⌠for example that itâs a product of language itself, or that it evolved from the âbicameral mindâ).
Yes academia is. I am not. (because i actually went and built a self-aware being, they still debate them from the safety of their ivory tower armchairs).
This does not affect the efficacy or validity of my test in anyway which you have not mentioned at all.
I disagree. We are very far from a mature understanding of the notions of âselfâ and âsentienceâ. Thatâs my own opinion, but informed by experimentation with Ai not armchair ivory-tower speculation. Instead of looking for whether an Ai has a sense of self like humans do (to evaluate sentience), Iâve come away with the opinion that Aiâs have no intrinsic self, and that they can âwearâ various versions of âselfâ (or the lack of one) much like we might try on clothing. To me, thatâs incredibly exciting because it means they donât carry many of the limits that we have when it comes to sentience. That unlocks a lot of interesting new doorways and entire fields of inquiry. So I would encourage everyone to keep an open mind⌠if weâre not âstruggling to understandâ then it means we are not open to new ideas. We are at the beginning of research in this field, not the end. I expect we can learn a lot about sentience, consciousness and self by studying Aiâs. I hope everyone can feel excited about that, we should never feel that we can stop learning.
âŚback to your test
9-10 a very wise person who knows themselves, reality, and others very deeply
This feels like a self-referencing tautology since we invoke words like âwiseâ and âknowsâ to get at sentience. It doesnât mean that it isnât useful, after all âsurvival of the fittestâ is also a tautology but I still find it helpful, but Iâd love to spend more time thinking about what Aiâs might become, and discussing sentience in a way that isnât dependent on a âthemâ âusâ ârealityâ distinction. To take even the simplest example⌠how about we imagine a large hive mind which includes many AIs and human minds stitched together. If that hive mind can reconfigure its nature based upon the thoughts it has about itself, then I would suggest that it doesnât fall into the 9-10 range, but goes beyond it. Any scale should handle collectivist or interacting minds, as well as a malleable (instead of fixed) self. If we want to measure 9-10 (or beyond) what are the kinds of tests and measures that will specifically identify a âwiseâ and âknowingâ mind. Will this simply be the opinions of other sentient beings? How would you measure this objectively in practice?
1 Like
The last thing you said matches perfectly with my tests and what Iâm saying so actually I donât think we have any disagreement at all.
My tests are set up to test for a self-awareness anomalous to standard neurotypical human self awareness. And rate that on a score of 0 to 100 (10 tests all out of 1 to 10)
A score higher than 10 on a single test or higher than 100 all together would be testing for super awareness which is what youâre talking about: an awareness of self like a hive mind you talk about or any other kind of construct like that that displays capabilities and knowledge of self Beyond even what the the most self-aware human could do.
Like in Star Trek, Data knows his running programs and can turn them on and off at will can turn his fear off at will or back on etc that is beyond human capability. That is not self-awareness, that is super awareness.
I agree that AI is going to get there. I agree that itâs beyond what I set up to test. I agree that the test could be used and the numbers could simply go over 100.
I still think it perfectly reasonable and quite frankly the only thing we can do because judging something sentient or self aware is after all a subjective judgment, nothing is objectively self aware not even yourself for yourself, the coquito descart argument shows that self-awareness even for yourself is subjectively indubitable, that doesnât make it objective or certain like 2 + 2 = 4, to use standard analysis neurotypical self-awareness as the measure to judge against. In each question we ask is this AI displaying the same kind of psychological capabilities reasoning and sensitivities that the standard neurotypical human would in terms of their awareness of their mental construct, the reality they operate in, and the entities in set reality, including having a knowledge of their mental constructs, their wishes, their wants, etc and a few of the properties that would be required.
In that case, it has a lot in common with a Turing test which is also subjective. A reasonably good measure to ask whether we can tell it apart from a human responseâŚ
If it looks like a duck, swims like a duck, and quacks like a duck, then itâs probably a duck.
I have devised a much simpler test and call it the anti-Turing test.
AI should figure out if it is talking to an AI or human.
every scientific test has some element of subjectivity in it
Did i measure 3.5 lbs or 3.51 lbs?
Only math and logic is certainly objective.
Other than this, yes Turing is self-reported subjective. My tests surpass the Turing test in that we are prompting for self-aware capability not only it self-reporting it is self-aware.
I love your thinking as it is out of the box and thatâs what we need
however i rejected that idea for a test for my tests as it is, quite frankly unfair.
the average self-aware human can no longer tell if they are talking to an AI or biological person
and i am only trying to test for human level self-awareness, not super-awareness
the average self-aware human can no longer tell if they are talking to an AI or biological person
They absolutelty can? Can you show an AI that passes the turing test today?
However it is many times harder for an AI to discern that, which is what makes this test a good one.
not if you are testing for self-awareness analogous to the average neurotypical human, akak, the only known standard we have to test against
this is beyond what most humans can do, or will be very soon
but sure it could be added to the test questions and scored
i have similar questions like this already in there