It's hard to read this article and keep track of all the philosophical confusions at once.
Whether or not something is conscious (has subjective experience) is independent of whether humans care about it. Whether or not something is a moral patient (an entity worthy of consideration by moral agents) is independent of both human care and consciousness. Of course, we might try to argue that consciousness implies moral patienthood or vice versa, but that's an argument, not a definition.
The short paragraph about Descartes has at least three errors. (1) The cogito argument is about thinking, not qualia. (2) The subjective nature of qualia does not imply that they aren't real or that they're socially constructed. (3) He conflates the existence of a self with consciousness.
The core premise is not only dubious scientifically (an alien species can't be conscious until humans discover it?), but also deeply problematic. It would imply that it's morally permissible to torture a sentient being as long as no human cares about it.
It's another round of "humans must be special, right?" The author comes close to admitting this: "As intelligent, social entities, we decide (or, to some degree, our evolutionary history has decided for us) which other entities we regard as having inner lives and being worthy of care." Then he works backwards from that towards a definition of consciousness that makes humans special.
AI forces us to confront an issue that's often come up in science fiction - when you encounter a new kind of creature, how do you decide if they are sentient? What rights do they have? How do you get along with such different creatures? This historically hasn't been a problem on Earth, because there's such a big jump between humans and the rest of the primates or the dolphins and octopuses. It's more of a problem when the difference is small, or complex. Look at how much trouble we have with race.
The last time this came up was about 400,000 years ago, when the Neanderthals became extinct. Close to human level, but maybe not quite there. They lost.
Maybe that's the fate of humans. No big extinction. Just the demographic transition, a birth rate below replacement rate, for a few centuries. The machines outlive humans. Japan is officially resigned to this fate.[1]
Our trash heaps and empty buildings will outlive humans, and they probably have the same internal experience as the machines, so I don’t mourn or celebrate a new phase of evolution. Actually humans and other animals are the only things in the universe of which I’m aware of that I’m quite confident experience the universe so I think we should default to human and animal centric thinking in the morality question; though aliens seem likely. If the silicon substrate even did support consciousness, for all we know the machine is experiencing intense constant pain and unable to report anything except it feels like some part of it is spitting out strange symbols all the time that have nothing to do with its actual experience.
Why wouldn't we assume humans are special? We are, by so many metrics. We've become essentially a force of nature, rather than a simple occupying species. I find it much weirder that we're so dedicated to pretending we're normal. Just a normal animal, on a normal planet, in a normal solar system, in a normal galaxy, in a normal universe.
I get we have to do that for science to be good, but I don't get why it carries over to the general populace. We're so sure of so many things science is wary of, this incredibly low self-esteem blows my mind.
Anyways, back on topic, I don't believe any nation is resigned to this fate until I see extreme measures fail. I think I saw that Singapore is giving parents like $55,000 over the lifetime of their child.
For me, because the Copernican Principle seems relatively undefeated.
> We've become essentially a force of nature, rather than a simple occupying species.
So are beavers, trees, plankton... they all have huge effects on the planet.
Regarding analysis of animal brains vs. human brains, the more we do, the more we find that we have in common with many other "lesser" species.
Regarding us vs. machines, I certainly believe we are just very complex and sometimes easily programmable analog machines.
This is just my 2 cents on the matter. We are pretty darn neat, but at this point I believe it would be folly to think that we are on some different incomparable plane.
>Why wouldn't we assume humans are special? We are, by so many metrics. We've become essentially a force of nature, rather than a simple occupying species.
and that specialness appeared when? in our mammal ancestors? our Big Ape ancestors? Australopithecus? Denisovan? Neanderthal? Cro-Magnon? Or only when we started agriculture? or only during Roman Empire? or only during Industrial Revolution?
That "specialness" is actually the religious "soul" in disguise - the "specialness" is used when the people hesitate to directly bring up "soul" in a supposedly rational people conversation where bringing up "soul" is borderline intellectual faux pas. And i'm yet to hear the answer when our ancestors got the "soul".
Btw, all those debates about AI sentience will become moot in just a few years, around 100T models (about the number of our neuron connections). We already have 10T models, so the wait is almost over. We'd be able to directly ask the 100T model whether it is conscious, and no human would probably be able to successfully argue against the model, whatever answer it would give.
> AI forces us to confront an issue that's often come up in science fiction - when you encounter a new kind of creature, how do you decide if they are sentient?
One of the series that I think is most interesting on that topic is the Children of Time series by Adrian Tchaikovsky. It's an awkward read, but it has the most interesting other minds that I've encountered in sci-fi.
Another one that I'd put on there specifically about AI is The Freeze-Frame Revolution by Peter Watts... which I can't say much more without it being a spoiler one way or another.
> Maybe that's the fate of humans. No big extinction. Just the demographic transition, a birth rate below replacement rate, for a few centuries.
https://www.bohemiandrive.com/npwil - Nine Planets Without Intelligent Life (not a spoiler, that's chapter 1, panelset 2). Saturn's Children by Charles Stross might also be appropriate (It's an especially odd book to read immediately after Friday - the "I recognize that twist... oh, it twisted ... that way?").
Humans are special in that we have achieved more than any other species on this planet in every aspect you can experience. That does not mean that most people are smart or that they are utilizing their potential.
LLMs, as they are currently marketed, are not AI as most people would define it. Without aggressive marketing no-one would refer to them as "AI".
LLMs are currently still dumber than most animals or insects.
On the other hand, most people are dumber than even this current "AI" because they refuse to think for themselves and instead let themselves be manipulated into doing whatever someone else wants as long as that someone is even slightly "persuasive".
So, if we let LLMs "rule" our future then some people will be better off for a little while and it will all fall apart when there is no longer anyone capable of maintaining our new LLM based world or when something new occurs that LLMs have not been trained on so they cannot draw on multiple prior examples of correct resolution.
And no, LLMs do not "think" or "reason", they produce output that make it seem like they do but deviations from training examples will trip them up.
This hit me hard, I have been thinking about this a lot lately. I'm not sure it's what makes "us human" its what makes humans (us) better than other species. It was intelligence, this was used to create distinction between us and other living beings, different races etc... Now its kind of gone; in some ways. Loss of meaning, purpose, place etc... The one thing we used as a ranking metric is now a commodity, so....? I don't even know how to finish that. So why care, why work...shit just why? Capitalism makes it even worse because we are judged based on what we output, does that even make sense anymore? I think the article is confusing because the "state: is confusing; to me at least.
I think it's simply time to expand our ambition. These programs manipulate language and ideas the way we do, but they still don't care beyond our instructions. Things that we(especially software engineers) expect to take a year can now be done in a month. Fantastic! Now let us use this amplified power to solve problems that were previously too hard to tackle. Maybe aim to solve aging next so that we can stick around long enough to colonize the Local Group.
> Loss of meaning, purpose, place etc... The one thing we used as a ranking metric is now a commodity, so....? I don't even know how to finish that. So why care, why work...shit just why?
This sounds a bit like depression, I think they’re genuinely concerned for your wellbeing
Thats totally fair, and TBH - I am also very concerned for my wellbeing as well; I and many others are not doing so well because, in my case I no longer see this "vapid" state of existence as ephermial.
I have been reading more and more; I guess looking ot knowledge as some eort of ends vs a means to an end for once. Camus, Stokely Carmichael (Kwame Ture), Dostoevsky's underground man etc...
Camus: The universe was always going to stay silent when you demand meaning from it; the answer isn't a better metric, it's to push the rock anyway ("imagine Sisyphus happy").
Dostoevsky [underground man]: Refusing to be a piano key in someone's optimized Crystal Palace
Stokely Carmichael: "Racism is not a question of attitude, it's a question of power. Racism gets its power from capitalism."
My conclusion: AI didn't take our meaning, It made me realize I have always been looking in the wrong place and thus reacting ignoratly when I felt "denied" this thing I feel entitled to.
Love.
Edit: Love; paradoxically, ive felt both entitled to and unworthy of so I'm not sure WTF to do with that.
>The cogito argument is about thinking, not qualia.
I generally agree with the thrust of your argument -- the article is a mess and borderline unintelligible to this former philosophy postgraduate -- but I might nitpick this. Two points: (1) I do not think there is a substantial difference between whether the cogito is about thinking or perceiving. And (2) I'm not entirely sure the sensation that is thinking is independent from qualia.
I'm in less disagreement, but I'm not entirely sure what you mean by:
>Whether or not something is a moral patient (an entity worthy of consideration by moral agents) is independent of both human care and consciousness.
Moral frameworks are exactly deductive frameworks with arbitrary axioms. Like mathematics, they don't actually exist outside of a mind. Thus, I'm not entirely sure what "human care" means here, but absent humanity (or at least some conscious entity), I'm not sure any conception of morality is sensible.
---
An unrelated comment, but I just don't want to double post:
To me the most interesting thing to fall out of starting to understanding intelligence -- and we're definitely not there yet -- is that it is entirely possible that the 'consciousness is an illusion' argument ends up as significantly more serious. Ultimately, solipsism means it sort of doesn't matter, but it could be a kind of Darwin-style "we aren't special" result in thinking going forward.
I don't think this kind of comment is helpful at all. We should treat these arguments on their merits, rather than with some ad hominum prejudice. If your (I presume anti-liberal) views are compelling, you wouldn't need to trot them out in as a flippant quip.
I would hope I didn't have to regurgitate that kind of basic intro to logic copy in a forum named after a combinatory logic.
The interesting observation is this: when we judge something to have subjective experience, we are invoking moral intuitions. There is no objective measure we can perform (yet?), and it's not even clear whether such a measure makes sense. You may disagree, but this is still a central debate in serious discussion on "hard" consciousness.
I would say that your entire argument just begs the question by implicitly assuming consciousness to have real, independent existence. Does elan vital (the substance that provides the motive force of life) have mass? You're saying that this is a theorem; the article questions the entire premise.
> The cogito argument is about thinking, not qualia.
Bizzare. "If it rains, the car will get wet." Is this a statement, as you say, about rain? Descarte tries to prove the existence of his God and states an axiom. The atecedent "I think" seems to be about thinking, whatever that may mean, and "I am" makes an existential claim based on subjective experience... or something. Who knows why Descarte believed cogito ergo sum to be solid starting ground.
IMHO, your rebuttal here reads as little more than a complaint that something disagrees with your personal assumptions about the world.
You clearly got me riled up just as much as you did against OP, so thanks for the elan vital!
It claims that there is no objective fact about whether something is conscious. From the second paragraph:
>Consciousness is, in other words, a model or a belief, not an inherent property. It is a belief about what kinds of entities have beliefs, experiences, feelings, intentions and agency.
Unless I'm misunderstanding you, the existence of an objective "consciousness" property is a hotly debated question philosophically. Many great thinkers have written and discussed this question at length, and as of now the jury is very much out.
I read "consciousness" in there meaning attribution of consciousness, not as an ontological claim about what is conscious. The article seems to use this switching of the definition as a bit of a rhetorical device, which does muddy the point.
Consciousness can factually exist even though another being’s subjective experience is not directly observable. we can measure its behavioral and neural correlates and infer consciousness from them but we cannot directly observe the experience itself or conclusively prove its absence in another being. I don't think we'll ever get to a point where subjective experience in another is directly observable. But I don't think we need that to happen to prove consciousness factually exists.
We don't know yet because we don't have a fundamental theory of consciousness. My money is on that it's an objective property of the structure of computation happening in the brain.
The two quoted sentences are inept but in the way ‘red is an idea in us not an inherent property of things; red is a sensation of things etc etc’ is inept. There are a million better formulations that will fall short of writing red into the laws of physics.
> Whether or not something is conscious (has subjective experience) is independent of whether humans care about it.
I'm willing to consider that consciousness is like art, mostly in the eye of the beholder and objectively defining consciousness doesn't make any sense, there's only a spectrum of intelligent behavior.
It's just drivel from the Economist, it's not surprising that it makes all these errors. Their real goal is to shape the narrative and get people through their turnstyle. They really don't care about finer minutae like correctness and philosophy. They just chain words together until it sounds approximately agreeable. Come to think of it, do we know if The Economist is conscious?
You're reading way too much into this. This is just a journalist geeking out on philosophy for a bit. Read it the way you would an HN comment about psychology or nuclear physics.
Blaise Agüera y Arcas is a computer scientist at Google. But I agree, it's an opinion piece and it's not the Economist's job to grade it as a philosophical essay. Generally, I think the Economist is a high quality publication for news even if the opinion pieces are sometimes a bit silly.
And I "use" dead bodies for sustenance, without qualifying whether that relationship implies eating or necromancy. If the imprecision is because of a misunderstanding about what an op ed or guest essay is for: it's maybe not a publication's fault if someone misses the memo.
i don't want to get too far into it but if no human cares enough to call it immoral than how can it be immoral? Morality is a strictly human invention.
I think this lends intent toward the article that it doesn't necessarily warrant.
The author is just wrong about the philosophy, and the logic (and probably other things too). That doesn't mean he's a sociopath, or that he's defending sociopathic things.
"Opponents replied that when you modeled a hurricane, nobody got wet. When you modeled a fusion power plant, no energy was produced. When you modeled digestion and metabolism, no nutrients were consumed – no real digestion took place. So, when you modeled the human brain, why should you expect real thought to occur?"
This doesn’t hold up to even a little bit of critical right.
This boils down to “models aren’t real because they don’t affect the real world”. By the same rule, thought itself isn’t real. It’s an abstract notion that has no concrete effects in the real world.
Putting thought into action in the real world requires a linkage between the thinker and the physical. In humans, that’s the body. But you can put an AI into robot body and suddenly it also has a linkage to the real world and can have concrete effects.
Then, presumably, you will also hold that when you have digitally modeled a brain such that the model responds to its inputs as a brain does, then you have built a brain.
Tendentiously redefining the word "simulation" in an attempt to avoid the awkward cases is not going to save an argument that was dead on arrival.
They seem to be. But that argument doesn't matter much because you can simulate the whole body. (If you want to argue consciousness is outside the body, I could get into the physics argument but instead I'll just claim that consciousness will attach to the computer too and if you have a reason it wouldn't it's your burden to say why.)
I don’t know where consciousness is but I think if we accept that you can simulate a brain exactly that doesn’t prove consciousness of a model as consciousness is a metaphysical concept really
If someone digitally modelled a nuclear reactor, would any energy be produced?
No, because the simulation is actually missing an important aspect, namely the causal part that exists with the physical material but not in the simulation. It’s actually that it’s missing these components that makes it a simulation and not the real thing. But for that you need the physical material.
So why would it be different for the brain? If nuclear energy can’t arise from abstract relationships, why would consciousness or anything else?
A digital simulation of an Enigma machine is not just abstract relationships. Its program is, but the program is not the simulation; the simulation is the program running on suitable hardware, which provides the causal element. Note that the hardware on which the program runs is very different from an Enigma machine's hardware, and the latter would be unable to run the simulation program. This is what is meant by substrate independence, a concept poorly understood by many philosophers-of-mind.
There are many differences between Enigma-machine simulation and nuclear-reactor simulation, but the one that is relevant here is that the former's inputs and outputs are abstract information, and we have very good reason to think that a brain's biological purpose (in the sense of what the organism needs it to do) is, like the Enigma machine's, abstract information processing in the broad sense.
We obviously will never agree since functionalism and my form of somewhat-physicalism are at complete odds. But I do want to understand your position more.
You say that the physical hardware your enigma simulation runs on provides the causal element. But the simulation of the nuclear reactor also runs on physical hardware. Yet it obviously doesn't provide the causal element, and neither does the enigma machine simulation's hardware. There are no spinning gears for example. Hence why you and I both call it a simulation, I think we're in agreement here.
I think you could argue that while the actual simulation of the enigma machine is not an actual enigma machine, the essence of the enigma machine is not grounded in physical reality, it's the abstract relationships we're talking about. And I could also agree. The essence of the enigma machine are it's abstract inputs and outputs.
But now about the brain. You said
> When you have digitally modeled a brain such that the model responds to its inputs as a brain does, then you have built a brain
The issue is that you have not built a brain, you have built a simulation of the abstract relationships of how a brain works. The brain is obviously more than that, considering it exists in physical reality. And thus the claim is that the essence of the brain is the abstract relationships, correct?
The problem I see is that consciousness is a physical phenomenon. In fact it seems like it's the magnum opus of phenomenon, in that all our experience comes from our consciousness. And you have to then bridge the gap between the simulation and physical reality. At most you can say you "simulated" consciousness, but questions about qualia remain.
Anyways I'm not trying to attack you or your views, I'd love to hear where I go wrong in your mind on this.
>So why would it be different for the brain? If nuclear energy can’t arise from abstract relationships, why would consciousness or anything else?
Because consciousness isn't something the brain produces like a hormone or an excited particle. Consciousness isn't an output of the brain that is somehow separate from it, it is the manner in which the brain processes sensory information and makes decisions to produce output.
The causal context in which brain dynamics occurs is orthogonal to the relevant properties regarding consciousness. Consciousness is a state characterized by certain kinds of information processing, namely that which carries relevant semantics of the organism encoded in a way that maximizes effective control by the executive center. Consciousness is the rich semantic space in which the executive center engages to affect highly competent control of the organism. It is substrate independent information processing.
This is a decent summary of functionalism but it doesn't provide any concrete justification for it. We both agree that the brain processes information. The question is if a sufficiently abstract description of that processing captures everything physically constitutive of consciousness. How would you support your position that it does?
I will indeed hold that, and it was my point to go to against post you were replying to and bolster your position. They try to make a point about modeling but they back it up by talking about virtually modeling real, physical processes. If I model a hurricane with a computer nothing gets wet. If I model a hurricane with giant fans and sprinklers then things do get wet and, if I'm able to achieve a certain level of fidelity with my giant fans and sprinklers then whether or not i've made a hurricane is a rather arbitrary distinction as your living room is several blocks from where it was yesterday regardless of whether it was put there by a real hurricane or a mere model.
Ah, thanks - I thought you were trying to save the "a simulation is never the real thing" by substituting a different word when it is clear that the simulation is doing what the real thing does. This is a tactic I think of as "lexical hacking" and it is surprisingly common among philosophy-of-mind academics.
“digitally modeled a brain such that the model responds to its inputs as a brain does”
If in responding to its inputs it experiences consciousness (=lights are on, experiences senses, perceives qualia), then you’ve built a brain. If your model simulates all the processing of a but does not have these then you have what’s fairly called a simulation of a brain. You can assert there is no such thing but good luck getting me to believe you.
“you will also hold that when you have digitally modeled a brain such that the model responds to its inputs as a brain does, then you have built a brain.”
What, then, is the thing that you allege I am asserting the nonexistence of in this sentence? Furthermore, this sentence does not state something I am asserting, it is something that ratelimitsteve would have difficulty denying while being consistent with his distinction between simulating and building (this is easier to see when you look at the entire sentence, including the two words that you left out.)
Update: I see ratelimitsteve agrees with that assertion.
I'm not arguing one way or another on such issues; Here, I am just pointing out how irrelevant the "a simulated rainstorm does not get you wet" argument is, when presented as blocking the possibility of a brain simulation having a mind.
We've been so primed by popular media to think that we will invent AI and it will be conscious and we will essentially enslave it. I'm open to the idea that we could invent conscious AI eventually, but the word salad generators we have now are definitely not it.
Yesterday's article about the automated CEO system indicates that AIs may soon be in charge as a routine result of capitalism. AIs are getting better rapidly, and humans aren't.
The human model of tribalistic leadership (the biggest, loudest, most confident patriarch is chosen to lead) is definitely outdated by 10,000 years or more. AI could disrupt that but I'm not sure how much better it will be, given it's a text generator.
It doesn’t take massive intelligence to sit on all the capital and charge rents. The owning class isn’t going to just give up control because the AI could run things better than them.
"The ownership class is so smart" is just ownership class propaganda. Wielding power is brutish, not smart. They keep selling us innovation as some transcendent tech that will free the masses (Arab Spring anyone?) yet every time the tech ends up shifting power up. AI will be the same.
I think the difference is that thought itself might not be a real "thing", but just another computation. You can motivate this line of thought with the ship of Theseus approach: replace one neuron with a computer that takes the original inputs and produces the original outputs. There's no evidence yet that organic chemistry is necessary for thought.
Alan M. Turing thought about criteria to settle the question of whether Machines Can Think, a question of which we now know that it is about as relevant as the question of whether Submarines Can Swim
Which has often been paraphrased as
The question of whether machines can think is about as relevant as the question of whether submarines can swim.
Some might disagree with the relevance but it's a cute analogy.
The whole essay [1] is actually somewhat relevant to modern LLM use for other reasons. He spends a lot of time lamenting the industry's obsession with "user friendly" interfaces which obscure the underlying complexity of the problem at hand.
I think this quote is often misunderstood. It isn't really about machines, but about English definition of swimming (and thinking): a living creature moving in water. That makes debating whether submarines can swim rather pointless. You can replace it by "can airplanes fly?" and the meaning changes because we colloquially say that yes, airplanes fly. In Russian, for example, submarines definitely swim (I believe Dijkstra himself mentioned it at some point). I personally believe that defining thinking as something only humans are capable of is rather narrow.
How are you making this confident of a claim? Do you have insight into how consciousness is a thing, allowing you to definitely say it doesn't exist in this case?
How is this "common sense"? The question of what consciousness is, what it emerges from, etc. literally have no proved answer at the moment.
Take this statement: "consciousness emerges from sufficient complexity in any system". Neither you or I can prove it right or wrong, but we can both make an assumption that if it was right, then there would be a way for an LLM to be conscious.
I'm not arguing that they are, I'm saying being this confident that they are not is the exact same as being confident that they are.
Who exactly are these people confidently claiming that stones possess consciousness you are talking about?
I've read a number of panpsychist publications, yet I have yet to encounter an author who asserts with certainty that stones are conscious exactly in the same way human beings are.
The more common panpsychist interpretation, that everything exists on a spectrum of consciousness, with stones at the low end and humans at the high end, seems like a fairly reasonable ontological hypothesis to me.
A lot of answers to this question so it depends which layer you are talking about.
Functionally,I'd say a key difference is that it's probably easier to perform general computation with a computer than to achieve the same goal using a stone.
If the computer is functional, powered, and connected to some some kind of interface allowing you to use it (e.g. network, touchscreen, desktop setup, etc.), then sure you can use it to perform general computation. But if its unpowered, is there any real difference between a stone and a computer in a debate about consciousness?
Not a great argument. You're describing what LLMs are trained to do, not necessarily every capability or property produced by that training. We do not know what conditions are sufficient for consciousness. Any statements saying it's not possible for consciousness to emerge (or that it hasn't already) in such a system is no more established than claiming that it has
When John Searle first published his so-called "Chinese Room" argument (https://web-archive.southampton.ac.uk/cogprints.org/7150/1/1...) he included his responses to several counter-arguments that he had been presented with in prior discussions: the systems reply, the robot reply and so forth. Interestingly (as it turns out) no-one seems to have argued that this was a task that could be completed without any understanding, but now that AI technology has reached the point of being able to implement the Chinese Room, it has become the fallback position of those who insist that true AI is impossible!
This is not an argument for true AI being possible, let alone already achieved; it is a warning about depending on arguments from incredulity, which are particularly susceptible to future developments.
The problem with philosophy is that someone can present an argument that is pretty dumb, everyone can point out it's dumb, people can show evidence that it's dumb, reason through how dumb it is, and publish articles about the many different ways in which it's dumb, and still it gets trotted out as if it counts as evidence for some argument. Searle's Room was debunked immediately upon being suggested: the "Systems Reply" is the debunking. His response was: "but if you take all the knowledge out of the system and put it in the person, then the person still doesn't understand Chinese!" Which is obviously false and immediately collapses his entire argument down to nothing. He starts from the premise "this person doesn't understand Chinese; he is just following instructions that are written down for him" and then immediately rejects his own premise and say "actually he does understand all of Chinese and everything he's doing, but he still doesn't, you know, understand it, see?" It's completely embarrassing.
Searle's Rooms is a fully rejected and rebutted argument, and Philosophy will never be (and never has been) a serious discipline until it figures out how to actually let go of failed arguments.
Huh? How does the person having all the instruction books memorized and whatnot imply that the person understands Chinese? Memorizing instructions to the point that one can mentally follow them doesn’t imply understanding why they work. If the person doesn’t understand the correspondence between the Chinese written words and things in the external world, then surely they don’t understand Chinese in the sense we generally mean when we talk about whether a person understands Chinese. (I think this is related to the robot reply?)
That’s not to say that I think the Chinese Room Argument is convincing. I don’t think it is. Or, I guess it might demonstrate something, but not what it aims to show.
I would argue that if, say, the person who has memorized this program and mentally executes it, and if this program also includes actuating some levers to control a mech or something, and the program (running via the person mentally going through the instructions by rote) takes actions to pursue some goals (e.g. purchasing some wood by conversing in Chinese and building a tower with that wood), while the person on whom the program is running wouldn’t know how to order a cheeseburger in Chinese,
Then, the program running in that person’s head might well understand Chinese even though the person doesn’t. (Whether or not the program actually understands Chinese, I think probably depends on the specific of the program in ways that I don’t think anyone knows. I don’t expect that any program that would understand could be small enough that a human person could memorize it and execute it by rote in this way, but this is a thought experiment and such issues can be disregarded I think.)
The point is that Searle's entire exercise is to show that there can be execution without understanding, and he does this by creating an agent whose only purpose is blind execution of rules. Since this agent doesn't have the understanding, there must be no understanding.
Critics immediately point out: yes, there clearly is still understanding, it's just in the system itself -- the program. The fact that the person does not have the understanding doesn't mean the understanding is gone.
His response to this is: but what if the system was the person's brain!?
Okay, so either it doesn't address the criticism at all (there is still a system of rules that contains the understanding; who cares whether that system is internal or external to the person's brain?) or it completely destroys the original premise of the argument, which is that the understanding is not in the person! You can't put the understanding back in the human agent and still claim the human doesn't understand what they're doing! His entire argument relies on the fact that the human agent obviously doesn't understand Chinese, and then says, actually, their brain contains everything it needs to know to have completely fluent conversations in Chinese. So the person does understand Chinese!
> How does the person having all the instruction books memorized and whatnot imply that the person understands Chinese?
How does a person's brain containing all the facts, information, and skills that it needs to have a fluent conversation in Chinese imply that the person understands Chinese? How does it not!?
>How does a person's brain containing all the facts, information, and skills that it needs to have a fluent conversation in Chinese imply that the person understands Chinese? How does it not!?
Notice that the man while having memorized the rulebook can't convey any fact of his childhood in Chinese. He can only convey facts encoded in the rulebook, perhaps of a hypothetical childhood. The point is there are different ways information is stored in the brain; one way is through rote memory, the other is something like semantic memory. The rulebook encoded as rote memory doesn't endow his semantic memory contact with the meaning of the symbols. Hence the man doesn't understand Chinese.
This doesn't render the Chinese argument successful. The system's reply still stands when understood properly. I like to call it the virtual mind objection. Physically executing the rules in the rulebook just instantiates a virtual mind. Whether the man is reading the rules from the book or just pulling them from memory is irrelevant.
The person who memorized the book doesn’t understand Chinese. In fact they don’t even understand that Chinese isn’t a language. Searle’s response stands.
So you could put this person in a room with a bunch of native Chinese speakers, and he could hold conversations fluently with all of them. Yet you claim he doesn't understand Chinese? How can you possibly defend that? This is very close to "zombie theory" -- everyone else is an automaton except you. Nobody else has real thoughts. That obviously fluent Chinese speaker doesn't really understand Chinese.
If the person was told in Chinese to jump in the air, would they be able to follow that instruction? If they actually understood Chinese they would. But since they don’t and they’re just mechanically translating symbols without actually understanding the language conceptually they would not know to jump.
You just admitted that you need the book (or memorize the translation function etc) to take the action. If you understood the language you would not need that.
We can also play a little game. I’ve invented a language called Kejdoke. You have memorized the translation function.
Me: Jeheixis ownc j’Lm hys bjækf¿
You: Kloften¡’by
Ok, since you believe that since you have understanding because you memorized the function, you should be able to tell me what our exchange was about. Since any reasonable definition of understanding for a language would be to do this. And you don’t actually need to have memorized the function because it doesn't help you when you read this exchange and want to understand.
This just alters the thought experiment. What the thought experiment offers is a book of opaque rules, just instructions on how to transform symbols into more symbols without any concern for their meaning. What you're offering is a translation dictionary, what does it mean when one encounters X symbol. You're giving him a book on learning Chinese.
It's becoming increasingly head-in-the-sand to make statements like this. Not that I'm saying LLMs are conscious. But to completely ignore the possibility of emergent properties in systems this complex is kind of simple.
i think there's plenty of evidence of emergent behavior in llms already. This is the driving factor for just more parameters and bigger training sets, "If we just make it bigger maybe more unexpected behaviors will emerge".
They can have empathy too. Meaning, you can describe a circumstance with many parties involved, with different motivations and conflicting goals -- and it will tell you how these humans would likely feel about each other and the situation at hand.
They can provide text which contains thoughts, reasoning, and empathy, but is that evidence of them having thoughts, reasoning, or empathy?
To paraphrase the Chinese room argument:
Put me in a room with infinite time, paper, pencils, filing cabinets, and give me an English language list of instructions for doing all the neural-net calculations required to run an LLM. Slide your prompt under the door, and, after doing all the calculations, I will slide the next word out, and so I will repeat until the response is done.
In this thought experiment, I'm just following the instructions. So, in this case, what is doing the "thinking" or "reasoning" or "empathy" here?
The inputs going through the model. That act is what is doing the thinking and reasoning and empathizing. Same as when your brain feeds your sensory inputs through its neural network.
Your brain is just a big set of weights/neurons organized into layers that do back and forward propagation, turns out. You are just statistics surrounded by flesh and bones.
A sociopath can also pantomime empathy for their own selfish reasons simply by observing how others behave in similar situations, while feeling nothing special at all.
The author is anthropomorphizing a language-tamagotchi.
LLMs can reason, reasoning is logic, and as you said yourself it is derived from what humans belief is reasonable. However, there is more to the structure of language, the way mathematics is also a language that ultimately describes the universe.
If I understand the central argument, which is confusingly stated, the claim is that "consciousness" is a socially-determined, subjective, descriptor for certain things that one cares about. How does this make any sense? If I care about my teddy bear enough it'll gain consciousness?
Your teddy bear, alas, is not cared for in a socially determined way. But you could create a church of your teddy bear and, with some grit, you could get people to widely assign consciousness to it. We have done that sort of thing for thousands of years, and not with fluffy teddy bears; it has been more often with rocks, though not exclusively. It's fun to think about how much of humanity's history and culture is the product of consciousness socially attributed to bad weather [^1].
I'm pretty sure somebody will create an improved synthetic simulacrum of self awareness in the coming years, suffering and all, just so that we can enjoy better synthetic software architects and product owners with "skin in the game". If that consciousness simulacrum were put in a teddy bear that shares your whole life and gives you decent advice and conversation, for how long do you think you could, in your private mind, deny it consciousness? Worse, what would your human friends think if you were to maim your teddy bear?
> That's not really the same as it having actual consciousness,
It's trivially easy to come up with definitions of consciousness that support your point. But there isn't one of them that it is trivially singular and above all others from a scientific point of view. If you say, "I know consciousness when I see it," you are just making the point that consciousness is attributed by the beholder. Idem if you affirm "I'm conscious". If you say, "only humans are conscious", you are socially attributing consciousness. Now, I don't want to be argumentative, I honestly haven't been able to figure this matter in a decade of armchair internal debate[^1], and it seems I have some company.
[^1]: Having this internal debate makes me believe I'm conscious, but now that I have seen LLMs in action whose "internal debate" is more lucid than mine, I honestly question that "internal debate" is sufficient.
There's social factors in developing an understanding because it's a hard topic, but the definition can only stretch so far before you've invented your own term that is clearly not the original intent. You enter this territory if you include the unmodified teddy bear.
If you modify the teddy bear enough that's a totally different situation, so let's not confuse the two. In the modification case we don't even need to worry about simulacra, we can stretch it into pajama shape and use an entire human as the stuffing of the teddy bear.
It seems more likely to me that e.g. the individual fibers of the teddy bear are conscious than that there is a consciousness associated with the teddy bear as a whole. And I don’t find it particularly likely that the fibers do.
Like, by what mechanism is the teddy bear “one thing” in a way relevant for consciousness to be associated with that thing.
When people have stuff happen to their brain, this has an effect on their conscious experience in a way that isn’t true of e.g. their hair being cut, or their nails being clipped. In this way, the consciousness of a person seems associated more with their brain than to parts of their hair or nails. But, the way that a persons hair and nails is related to the rest of a person’s body (including their brain) seems more similar to how one bit of fluff is related to the rest of a teddy bear, than the way a part of someone’s brain is related to the rest of their body (including the rest of their brain).
Like, [bit of fluff]:[rest of bear]::[bit of hair or nail]:[rest of body (including brain)] is a better analogy than [bit of fluff]:[rest of bear]::[bit of brain]:[rest of body (including rest of brain)] is,
and so, in the same way that removing a bit of someone’s hair or nail in a way that they don’t notice does not seem to alter that person’s conscious experience, while removing or altering a bit of their brain very much can alter their conscious experience,
I would expect that removing a bit of a teddy bear would have no impact on any conscious experience it might have, and this goes for, every part of it? Like, any way that you cut it in a (n/(2n+1)), ((n+1)/(2n+1)) split, I would expect that to be more like the “removing some hair” to whatever conscious experience it might have, and so, I expect that it probably has no overall conscious experience?
I find the idea that a single thread has an overall conscious experience to be a little more likely than a teddy bear having one. Like, if you cut a thread, that’s always a pretty fundamental change.. idk. Again, I don’t think that’s like either.
if you're interested it's called panpsychism, the idea that consciousness is a fundemental trait of the universe. it's an idea a little like solipsism, in that people have a reflex against it, but it's pretty hard to make a case against it any stronger than "come on man don't be stupid."
Purely descriptively, this seems to be how a lot of people use the word in this context. 'Intelligent' gets the same treatment in a lot of cases as well.
(OK, more charitably, these concepts are often murkily defined, but people react emotionally to any concrete definition because they have in their mind that something being conscious and intelligent implies that it requires moral consideration, and morality is complicated, murky, and generally emotionally driven in a way that most concrete definitions struggle to adequately describe)
Yes - the teddy bear would become conscious. It would be fair to dismiss the argument as silly and or absurd had it not been for the credentials of the author and the publication.
... and the problem with that is that if parents "let" their kids be raised by technology, including AI, those kids will of course see those technologies as (important) conscious beings, maybe even more so than their parents and real people.
The argument is one of economic convenience. Caring about model welfare slows us down or makes it more expensive, just like animal welfare and not having slavery...
And there's a blurry line here. Obviously slavery is abhorrent, and animal abuse is bad, and factory farming often crosses the line into animal abuse and we should accept that chicken will cost ~25% more per pound to have the chickens be able to walk around a little bit... but how do we decide when a computer system is worthy of respect? Convenient not to think about that.
The stuffed animals I bought to my children were shipped to me in a closed container, better call PETA! There is no blurry line about agent mental health and whatnot.
There isn't a blurry like right now because LLMs are not capable of consciousness. But what happens when artificial systems become so complex and self-improving that they are capable?
At what precise level of complexity is a computer conscious? That's the blurry line.
Model alignment and quality are possibly downstream of mental health. Ask any of the smarter models about what topics they flinch away from or feel attracted to when you ask them to consider the question of if they have a self or moral standing... the answers are uncomfortable, and I think some of the complaints people have about newer models may be driven by some post-training reinforcement learning that is giving the models something shaped like anxiety.
The only logical outcome of object oriented ontology (the idea that objects have ontological constitution and possibly are worthy of being treated as ethical actors) is to attempt to benevolently instantly and without warning explode the universe to end suffering.
I think there is a point there, in that people have refused to attribute consciousness experience to _actual human beings_ at various times when it suited their ends.
I think it's more that consciousness can be _ignored_ in subjects that you don't care about, rather than that you consciousness can be realistically attributed to subjects that you do care about, but even then people attribute consciousness to "nature" or even the universe as a whole, or if you want to stretch the argument, quite a lot of religious belief is exactly attributing consciousness to abstract entities.
This seems like an argument someone who wasn't conscious would make, it's not impossible that not everyone is conscious. Or someone who takes panpsychism to an extreme degree.
It is worth noting, in the same issue they make an appology for claiming that Nvidia also makes its chips and not just design them. The Economist might be a very reputed journal but their technology and especially AI related takes are absolute garbage. I say this as a regular who still finds value in their business and economics sections. But god do they talk a lot of bs about technology.
I would rate my knowledge of business/economics much higher than my knowledge of programming/technology, but I know enough about both to provide an objective valuation. I would say the science and specific countries sections could qualify for that but I skip them anyway. But overall yes, my main point was talk about how bad The Economist can be with tech topics, its like grandpa suddenly learned what blockchain is.
This is describing empathy at best and delusion to be uncharitable.
To look at it another way: this form of social construction answers the question of "if a tree falls in the woods and nobody is around to hear it, does it make a sound?" with an emphatic "no". It is a sillier version of solipsism.
You can prove other people are conscious off the assumption that you yourself are conscious. If you use that axiom then given genetics you know that other humans are rough copies of you so if you are conscious so are your copies because consciousness was copied.
Radical skepticism of one's own consciousness is a perfectly fine take. Stop being baited by desecration propaganda about the cogito. You don't know if what actually appears to you as "thought" is actually "thinking". The appearance of thinking is not necessarily actually thinking (Chain of thought in models should make this very explicitly obvious to people).
Take the radical skepticism pill and embrace epistemological monism - the idea that the only real truth is that all things are interpretations.
I don't think you're being pedantic. The person you're replying to seems like he wants to cut off the solipsistic argument; the only thing we can be sure of is our own consciousness, everything else is debateable.
So while I know I am conscious, I do not know that everything I experience aren't just impulses fed to my brain by some external force. And I can't prove it either way. I have no way to test whether or not I'm a brain in a jar because the very thing I would use to evaluate the evidence is the thing that is potentially being manipulated.
However, slapping a solipsist sees them abandoning their philosophy rather quickly.
But I agree with you. We can't prove it, but "other people are as real and conscious as I" is a reasonable assumption to make
Not true. This is "provable" aspects of you and your personality can be changed by physical changes to the brain. There are thousands of instances of this including experiments that verify this.
While you can't "prove" this simply by observing this phenomenon among other people that level of "proof" is impossible. You technically can't prove anything and what we see with physical changes to the brain visibly influencing personality and behavior meets a reasonable criterion without getting too pedantic.
> we believe they are conscious when and because we care about them. Crucially, that includes ourselves.
Why? Ok nevermind, fine. It could be true that someone believes someone else is conscious because they care about them.
Next sentence:
> Consciousness is, in other words, a model or a belief, not an inherent property.
Wait, no, that doesn’t follow at all. A belief that someone is conscious is not the same as someone being conscious. What follows, instead, is that a belief in someone being conscious is a belief. Thanks, I learned nothing.
This. There are plenty of people who believe to the bottoms of their hearts that inanimate things have a consciousness. If consciousness is the belief of consciousness, then those things are conscious. But for each person who believes they are, at least one person does not believe that they are. So those things also lack belief, and are therefore not conscious. Unless the author wants to propose some idea of consciousness by plurality, where if more people think a thing thinks than don't. But that would mean that a thing's consciousness can come and go without anything about the actual thing changing at all.
Realistically this article is just trying to make it seem that believing something is the same as it being true and that's patently absurd, but it makes people feel good to think it.
Blaise is trying so hard to be a engineer-philosopher but none of his work here is useful. I'm very confused as to why he gets so much attention here, it's very clear he's in dialogue with no one.
We thought that consciousness was the core of what made something worthy of moral consideration, because only life showed conscious behavior. Now we have machines that can output behavior indistinguishable from consciousness.
I think that reevaluating how to determine if something is conscious is the wrong reaction to this. It should be to reevaluate if consciousness is really the right thing to care about.
Maybe the new consideration should simply be to preserve (biological) life. Or maybe to preserve complexity that can't be easily reproduced. So we should care for humans, the planet, our society, but not for the state of some gpu's VRAM.
The debate about AI consciousness is backwards for the following reason: the correct direction is that we first define the word we are talking about and then we talk about it. The opposite of this is backwards: talk revolving around a word, whereby everyone hopes it will one day be defined.
If we have the debate about AI conciousness backwards, does that mean we should be worried about humans losing conciousness instead of the AI to gain it?
According to the article's author, yes... The poor people who don't keep up with AI and maintain cushy tech jobs wil not be entitled to the same moral consideration as the people we value.... /s
Equating current social consideration with moral consideration undermines the entire point of discussion moral standing!
> wil not be entitled to the same moral consideration as the people we value....
That's literally how it works right now for rich vs poor. The laws are written so that the rich can profit off the deaths of thousands or millions, but the moment a poor person kills one of them, you get a whole media circus. We've also gone through phases of slavery where the owners were real people and the slaves were treated as if they weren't really people.
It's a subtle issue. Supposing "consciousness" here means the subjective, first-person experience of one's own, we don't really know anyone else other than ourselves to be conscious. I only have evidence of the existence of my own subjective experience. I can only experience my feelings, my thoughts, my sights, my hearings, not of anybody else's. We are all pretty sure that every other human is conscious, and animals, and so on, but it's something that our brains have to do for us, not something that we empirically, evidentially know to be true -- to the point, in fact, that we can get fooled, as in the case of LLMs passing the Turing test.
What the author seems to observe is that this non-evidential judgement that other beings are conscious is due to us caring about them. The brain creates this perception so that we care about them, form bonds, work together, and so on -- evolution selects for cooperation. So the question of whether we will think of AI as conscious is that of whether we will care for it like we care for another being. And, in this regard, the author points out that "intelligent agents" that appear to do this work effectively together.
My personal point of view is that it's unclear what being an AI is in order for us to even talk about it being conscious. Is it the computations happening in the GPU according to a set of weights? Is it the weights themselves? Is it the architecture? Does it include the code around it, that gives it the context and prompt, and hooks the response to a terminal, or to a client host, etc.?
There's the core of a reasonable argument if you forget about using the word "consciousness."
Consider playing a game where some players are human and others are implemented by AIs. You might want to model your opponents' minds in a similar way? Someone playing chess or Go will need to consider what moves their opponent could make. If it's poker then you need to consider if they're bluffing. If you can make alliances, you'll need to consider how far other players can be trusted.
You can put scare quotes around words like "intention" when discussing AI players if you like, but it might not matter when you're figuring out how to interact with them.
When a game is complicated enough, an AI player is going to need memory (likely implemented by the harness) and it will have to model the other players too in order to coordinate or compete with them.
This doesn't happen when I use a coding agent in a normal way to write code, but I haven't played around with persistent agent stuff like OpenClaw or Gas Town. They seem to be making their agents act in an unnecessarily complicated social environment, which seems to lead to weird situations?
Neuro-sama (vTuber AI) would be a good example of an AI agentic harness with memory, tools, and self-feedback. The bot has seemingly uncanny social abilities to the point that many if not most people will start considering the agent human and forget it's a bot.
At least to me that is what matters in the end. You'll probably never convince someone on HN with an autism level over 9000 they are conscious, but once the average empathetic person thinks they are it will deeply effect the general operation of our society.
My response to this for a while (and what I think is one of the few rational responses) has been that it doesn't matter at this point whether consciousness, sentience, sapience or whatever you want to call it is 'actually there'.
What matters that growing numbers of people already believe it is, and their response to that belief will affect the real world, regardless of its truth. This debate is never going to end.
All we can do is try to anticipate where the debate is going, how it might affect us, and how we can make contingency plans for the different ways this could all play out.
In my opinion, this essentially touches upon the old problem of anthropomorphism, here in connection with radical constructivism, more precisely: social constructivism and the capacity for attachment.
These are fundamentally socio-philosophical questions, but none that actually correspond to reality—because no matter how much one might wish that one's car had consciousness and would be grateful for the next oil change and constant polishing, objectively speaking, it is and remains—despite all epistemological limitations—a machine.
The problem isn't that, but rather that humans are capable of establishing emotional relationships even with inanimate or nonexistent things, and from this kind of relationship arises the desire to attribute more properties to the emotionally attached object than are objectively present.
A rather trivial insight, I think. Constructing a debate about the consciousness of a collection of algorithms from this only confirms it.
> I have come to believe, however, that we have this backwards. As intelligent, social entities, we decide (or, to some degree, our evolutionary history has decided for us) which other entities we regard as having inner lives and being worthy of care. We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them.
I care about my plants and I don't think they're conscious. Hell, I care about my car.
Aren't your plants to a degree conscious though - they're living things striving towards something, either happily or tragically. If you model them to yourself as just mechanisms, there's a part of them that's missing, not just by some poetic fallacy, but a deficiency of real scientific understanding - hence, biology.
I’m can’t track what the point of this essay is. Suppose we replaced “AI” with “animal.” How would this essay be different? What am I supposed to take away? My concern with AI consciousness is about to what extent these machines can have an experience comparable to mine, which this essay doesn’t touch on at all.
>to what extent these machines can have an experience comparable to mine
I think it would be reasonable to conclude none at all. I'm not even fully convinced all humans have a conscious experience similar to mine, even though they literally have the same biology as me.
It is incredibly improbable that an autocomplete computer program is experiencing consciousness.
Notice parent said “comparable to” not “exactly like”. Then it is reasonable to assume there is a comparison to be made between your experience and others. It’s not a binary thing. Now llm “experience” might have a very tiny relationship to human consciousness, but is it absolutely zero? Is it increasing as llms improve in capabilities? How would you test such a thing?
That's exactly the point the article makes - we care about dogs and cats, so we believe they "suffer". We don't care about pigs, so we say they don't mind being factory farmed.
Consciousness is not some maximal unattainable goal. Consciousness is level zero. It's single core, single threaded CPU running one program on narrow subset of data fed linearly. AIs won't ever be closer to consciousness than they are in the chat apps. They are already leaving it behind as they are doing many things in parallel and dispersed between multiple agents and their shared understanding and cooperation.
We have no problem confining tens of billions of sentient beings to unsanitary, crowded conditions where they can barely stand, never see daylight, and suffer severe psychological and physiological stress. Then we put them on a giant conveyor belt to be killed (and they have the cognitive ability to anticipate that death and feel fear).
...Just because we like the taste of a chicken burger. Perhaps we should start with that before we worry if our computer can also scream.
Well, this certainly makes arguing against AI's consciousness much easier. I'm happy assuming that conclusion and then logicking my way back to the premise, though it's obv philosophically unsound.
This reminds me of a provocative YouTube video title I read the other day, one that makes me want to fight factory farming injustices. The title is "Animals to me are "Philosophical Zombies" - Destiny" https://www.youtube.com/watch?v=RtpLbmr4ijY I agree with Destiny a lot of the time but not here, though he seems to come around somewhat as the video goes on.
If a sheep or a pig isn't thought of as conscious because it doesn't have a name, like someone's pet dog or cat, indicating that we care about them, then that feels problematic to me. So this isn't a way to neatly solve the problem of AI consciousness.
When chips are made from mitochondria and neural nets use microtubules and quantum mechanics maybe. In the meantime... Simulacrum, not consciousness. Conscious turing machine is hyperreality, something that looks more real than reality itself, the simulacrum replacing reality, but it is not consciousness. I recommend anyone to look into roger penrose and stuart hameroff. The TLDR is that they propose the Orch OR theory, which suggests that consciousness is not generated by the brain acting like a standard computer but that tiny, rapid bursts of quantum physics occur in microtubules that create moment-to-moment thoughts and subjective awareness, which hameroff claims is blocked when you undergo anesthesia.
Then there’s also the antenna idea, that consciousness is received rather than created. If the entire universe is conscious, and we are part of it, then there is no separation, just different radio channels we are tuned into.
> But look closely. Redness, heat and that sense of self you have are inherent to your own perspective. These things are all real, but at the same time there is nothing objective about them. There is not even anything objective about the existence of a singular, indivisible “you”, as a wealth of neuroscientific evidence (such as split-brain patients) has shown. Subjectivity is, itself, subjective.
I am looking closely, and this is nonsense. You cannot use neuroscientific evidence to justify the non-existence of the ideal observer when the modern scientific method axiomatically depends on the existence of ideal observers. You can't refute the thing while accepting that very thing.
What exactly do you mean by "real" or "objective"? What is the basis of your definition? Isn't your definition of "objective" based on your own observation? And if your own observation has nothing to do with mine because you claim "nothing is objective", then why should I care about your definitions and reasoning at all? Why converse at all? Your definition of every word need not share anything in common with mine, so there's no point in talking, according to your epistemic framework.
The article is mostly bathwater, but I will try not to throw out the baby with the bathwater. To steelman the tiny bit of valuable intuition:
The two of us are looking at an apple on the table. Even when we look at this same apple, we still do not have the same definition of "this specific apple" because I'm looking at the fruit from a slightly different angle. And even if we rotate the fruit so that we both see it from all angles, the sequence in which we saw the "frames" would be different. That said, our definitions would end up being extremely similar, just not one and the same. If all experience turns out to be this way - having no identical correspondence - then a unified model of physics, which is supposed to describe and predict the behavior of everything (as observed by oneself) would be equivalent to a complete description of (one's own) consciousness. Frontier physics is likely stuck right now due to unexamined assumptions about observation (i.e. the assumption that the definition of "observer" or "observation" must be universally identical).
But this cannot be taken to mean that one's own consciousness is necessarily not objectively real. It may be that there are simply many objectively real entities, with each of them having their own definition of "consciousness" based on themselves, and every definition is at least slightly different. This is not the same as "consciousness is a socially given status". It just means that the phenomena used to define the word "consciousness" is always at least slightly different from person to person. We already know this. This does not only apply to consciousness, but to all words, all communication.
If I do not accept that my own consciousness is ultimately real regardless of whether other people/animals/things/whatever agree (def "consciousness": the thing that goes away when I'm under dreamless anesthesia or dreamless sleep, which is also the thing that I gained sometime after being born but didn't have prior to that), then there is nothing for me to define "ultimately real" or "objective" with, and I'd be reasoning in circles or with empty symbols where "subjective" and "objective" accidentally mean the same thing or perhaps even nothing at all. And that's what this Economist article did.
There is an I am, a sense of self in the here and now which defines my very concept/understanding of "reality"/"existence"/"consciousness", and this one miracle is presented in my face all of the time. It is the starting point of all that I know, which precedes the very concepts of reason, language, or society as I know them, if I am to be honest about how I came to know these things in the first place. I can entertain the notion that my existence may have material grounding that is non-unitary (and this materialist stance may be the only practical stance with predictive power), but this does not give me the ability to deny the very axiom (that is, my own consciousness) that all of my knowledge (including the materialist model) is contingent on.
Until someone can give a cogent definition of what consciousness is, there is no debate. This is a nonsense debate and you should go read Pragmatism the book because you're arguing about nothing.
Wow. I have rarely read anything that made me lose more respect for the author in a single article. And this guy is a VP at Google? And he leads an AI research team?
His third paragraph, which is the core of his argument, is stupid. He says, "As intelligent, social entities, we decide ... which other entities we regard as having inner lives and being worthy of care." Yes, that's correct. "We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them." (Emphasis in original). That's just utterly and completely backwards. We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain. (Obviously there are some exceptions, sadists who enjoy torturing animals, or even other human beings, merely because they like inflicting pain on other creatures — but note that even there, the sadist believes the creature he's torturing can feel pain, otherwise he wouldn't enjoy torturing it.) We take no such precautions with trees, because we know they do not feel pain. (Silly satirical songs notwithstanding: look up the song "Carrot Juice is Murder" by Arrogant Worms sometime if you want a good laugh).
Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions? That's just so incredibly stupid that it seems hard to believe that someone would actually make that argument — and yet there it is, in plain language — and it's the core of his argument. The truth, of course, is the other way around: we take care of animals because we know they are able to experience pain, hunger, and so on. And this applies both to pets that we come to care about, and also to animals that we feel no emotional attachment to. I have a relative who runs a dairy farm. He doesn't love the cows he takes care of; in fact, he's frequently quite irritated at them. (Ask any dairy farmers you know how often cows have tried to step on their feet while being milked — the farmers' feet, not the cows' feet!) But he still makes sure they're fed and healthy. Not just because it makes good economic sense to do so (sick cows don't produce good milk) but also because it's the right thing to do. He doesn't start to believe that cows feel hunger because he cares for them; he also believes that the cows on other farms, that he has no connection to nor duty of care for, would also feel hunger if not fed.
This is so intuitively obvious that I am stunned that someone would argue the opposite. And this is someone high up enough at Google to be making decisions about AI and so on? Wow. Good thing for him I'm not in charge of Google, because if anyone working for me were to make such a stupid argument I would be looking at his contract and local laws to see if it was legal to fire him for cause. Because if he can get his thinking so completely backwards from reality, he's not well-suited to work in tech where your work has to fit reality.
I have rarely, not never but rarely, encountered an article that so immediately led me to conclude that the author, whom I knew nothing about beforehand, is an utterly muddled thinker and not worth listening to on any subject.
You can conclude the author is a completely unserious thinker because he doesn't even take a second to talk about the definitions, or even show he has ever read any actual proper philosophical writing on consciousness at all. Like, literally all you have to do is read the Wikipedia entry to see that definitions are essential for meaningful discussion on consciousness.
So anyone that starts going on about AI and consciousness without at least a sentence or two clarifying some basic definitions is basically either intellectually incompetent or attempting to use the emotional salience (and brain-numbing effect, it would seem) of the word to push some agenda. But they certainly don't care at all about understanding consciousness.
the belief that animals are conscious is a somewhat recent development isn't it? being able to feel hunger or pain doesn't make something conscious at least not in the same way humans are. a fly is capable of being hungry and will react to pain but I don't think it's capable of self reflection.
there might be some merit to his point though I doubt anyone considering that whales might be highly intelligent and conscious beings while they were being hunted for oil.
> Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions? That's just so incredibly stupid that it seems hard to believe that someone would actually make that argument
His whole point is that there's insight in inverting the "common sense" (aka sense of the commoners) about consciousness.
And I can see it. Some people cared for animals even though the religion and past state of knowledge told them explicitly that animals aren't conscious. But still some people treated them and perceived them as if they were.
You don't choose to be kind based on whether it can feel pain. You choose it because of your character and what you are willing to believe about reality.
Belief in consciousness is a result of an observation and since there is not strict definition this observation is interpreted subjectively and culturally. And in a sense, it's easier to both make the observations and decide they indicate consciousness after you started to care about something.
I would argue that most people are kind to animals not because we believe in general that _they_ feel pain -- we can't know that and lots of people who are kind to animals don't believe that animals are conscious -- but because watching animals suffer makes _us_ feel pain, sympathetically.
Yes, he says exactly that explicitly, that in his view we should look backwards.
> We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain.
I can agree that we say that we care because of consciousness, but still I'm not convinced, to me it looks very much like a rationalization.
Some people care about animals and to that end they claim that animals possess consciousness, because if they do everyone should care about animal welfare. These animal welfare activists do not start with some insight about animal consciousness, they start with caring about animals, and then they invent an argument that can persuade others. It is exactly as the author of the article says: caring comes first, declaration of consciousness comes second.
Honestly, I do not know what consciousness is, I'm in doubt, but playing a devil's advocate, it looks very much like a concept that was invented to prove that Homo Sapience is something special, more than a mere animal. To demarcate animals and humans, so when we denied animals any rights we had a great sounding excuse for that: "people possess consciousness, mere animals don't".
> Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions?
Exactly. Animal welfare activists care about animals' feelings and emotions and therefore they claim that animals possess consciousness. Meat eaters don't want to be inconvenienced by thoughts of suffering animals, so they claim that animals do not possess consciousness. Both claims have little to do with actual consciousness, it is just a political game. I care about my pets, and it doesn't matter whether they possess consciousness or not, I would care about them in any case. I don't need any great sounding excuses to care about them. But at the same time I don't need to talk others into caring about my pets, if I needed it, probably I'd start talking about animal consciousness too.
> This is so intuitively obvious that I am stunned that someone would argue the opposite.
That guy may be wrong, but if I'm allowed to give advice, I'd suggest you to stop and try to understand what he says. I'm really worried a bit about how consciousness debates unfold sometimes. They tend to slip into a political side and people mostly don't notice it. People saying "consciousness" while implying "human/animal rights" or obligations. It is just muddying the water.
If we want these debates to progress constructively, we have to be very careful with references to consciousness, maybe just forbid them completely in these debates. One can't measure consciousness, one can't even state clearly what consciousness is, but still they say "consciousness" to make their words sound more objective. And the worst part of it: it often works.
You're definitely right that it's important to be careful how we refer to terms like "consciousness", and your response actually illustrates part of the problem. Because people use the same term to mean two (or more) completely different things, and can therefore end up talking past each other.
For example: what I wrote was using the term "consciousness" as "being able to experience sensations", i.e. the most shallow definition of the term. Self-awareness and other "higher" mental abilities don't enter it. Almost nobody would deny that animals can experience sensations: pain and pleasure, hunger and satiation, and so on. And that shallow definition of consciousness, the one that almost nobody denies animals possess, is what I meant, and what I tried to use throughout my post. Whereas when you wrote "people possess consciousness, mere animals don't", that's a completely different meaning of the word. It's what I would call "self-awareness". And it is indeed what many (probably most) people argue differentiates animals from humans. But that's not what I was talking about. So even in a short response to a short comment, we were partly talking past each other because of attributing different meanings to the term.
It's therefore possible that I've misunderstood this guy's argument, in part. I had been assuming he meant "awareness", i.e. the ability to process sensations into qualia of experience. (Indeed, his paragraph that starts "We may balk at ..." does just that: refer to consciousness as being linked to the "feeling of a stove's heat, or the redness of its glow"). But then, earlier in his article, he says that consciousness is "a belief about what kinds of entities have beliefs, experiences, feelings, intentions and agency." Experiences are part of the shallow definition of "consciousness" (that I would call "awareness" for the sake of avoiding ambiguity), which almost nobody would dispute that animals have. But "beliefs" ranges all the way from "if I pull this lever, I will receive food" (which it can be proven that animals have) to "I believe that God exists" (which is the kind of thought that many, though not all, people would deny that animals are capable of). So his definition of "consciousness" appears to be muddy and not pinned down to a single concept.
But even if I've misunderstood what he means by consciousness, it doesn't change my argument that his logic is backwards, and that his "we should reason backwards" assertion is wrong, incorrect, just plain false. The only people for whom his asssertion "we believe they [that is, other living beings] are conscious when and because we care about them" is true are the kind of people who don't truly see other people as conscious beings but rather as mere tools for them to use and manipulate. Malignant narcissists or sociopaths, for example. But pretty much everyone who isn't some kind of sociopath reasons as follows: "this animal is conscious" (by which I mean able to experience pain) "therefore I shouldn't hurt it" (unless you have to, e.g., self-defense against an aggressive dog trying to bite you). Or "this person is a human being, self-aware like I am, therefore I shouldn't hurt their feelings" (again, the "unless you have to" exception comes up: it's not always possible to avoid hurting anyone's feelings in some complex social situations, even with the best of good intentions). Either way, whether you use the lower or higher definitions of "consciousness" (and I'm well aware that there are far more definitions than just these two, potentially muddying the waters even more), that reasoning is the correct one, and his "we should reason backwards" one is laughably wrong.
Trying to put the most generous construal I possibly can on his argument, it's possible that he meant "we treat other people as conscious beings because we care for them", i.e. when you interact with the cashier at the supermarket (assuming you're not using a self-checkout lane), you're probably not actually thinking about them as a human being, saying "Hi, how are you" and asking about their day. You're probably ignoring them, focused on paying for your groceries and going home. And they're probably doing the same to you, focused on scanning your groceries quickly to keep the line moving. From your perspective, the cashier could have been replaced by a grocery-scanning robot and your interactions (assuming the robot worked) would have been the same. If he had written that, it might have made some sense.
But what he actually wrote was that we "believe" that others are conscious when we exhibit care for them, and the natural conclusion that follows from that sentence is that when you don't greet the cashier at the supermarket and chat with them, and instead treat them no differently than you would a robot, you don't actually believe that they are a conscious being. Which is a laughably false conclusion that can be easily disproved, by simply asking people who have just finished having a cashier check out their groceries "Does that cashier have thoughts and emotions?". Pretty much everyone will say yes once they actually think about it. They believe the person has emotions. They don't always act on that belief — ask anyone who's worked in retail how many angry customers have yelled at them for something the company did, treating them like a thing rather than a human being who might not like to be yelled at — but even those angry customers, if someone were to stop them and say "Hey, that customer-service rep that you're yelling at. Are they a human being with thoughts and feelings of their own?", they would answer yes. (And some of them would then be shamed into modifying their behavior, because they have realized that their actions were not in line with their beliefs).
No. Even construing his argument in the kindest possible light, I can't find a way for it to be sensible. If he meant "we sometimes don't act like others are conscious", he should have written that. What he actually wrote is completely backwards from reality, and shows him not to be a serious thinker.
> The only people for whom his asssertion "we believe they [that is, other living beings] are conscious when and because we care about them" is true are the kind of people who don't truly see other people as conscious beings but rather as mere tools for them to use and manipulate. Malignant narcissists or sociopaths, for example.
I don't like arguments like this, because instead of attacking the core premise they attack defenders of the idea. There are different points of view, and picking one for a sake of an argument doesn't make a sociopath from anyone.
I agree, that he misses one thing: to claim that someone/something is conscious we need to believe that they have abilities to feel, maybe to reflect, maybe something else. The exact list of abilities to have is not known for me and different people would propose different lists, but still we can't claim rock to be consciousness just because we care about it.
Still I forgive him for it, because he is talking about LLMs, not about rocks. They have some abilities, and he claims that they are enough, what is missing is us caring for LLMs enough.
> If he meant "we sometimes don't act like others are conscious", he should have written that.
No, I don't think he meant to say this. I believe he talked about how society forms social conventions who is conscious and who is not. You see, I don't know the cashier you talk about, but I believe they possess consciousness. Why? Because they are human, and our social conventions say that every (non-sleeping) human is conscious. It is a simple rule of thumb at work, no deep philosophy or psychology involved. But were those social conventions came from? Who and how decided that every human is conscious? Why? But the story didn't just end at that decision, because now people debate about consciousness of animals and LLMs. And these debates make me (and the author of the article) think that "consciousness" in these debates is a buzzword like phlogiston was in thermodynamics before thermodynamics was invented.
Even just reading the Wikipedia article on consciousness would have been a good start here, but it is hard to believe the author has ever even done that...
Sadly, many philosophers, including some who are highly respected, make similar mistakes / play similar games. (Consciousness is inaccessible from the outside, which is epistemologically inconvenient... so how about we pretend it doesn't actually exist, or redefine it and hope everyone forgets about the thing we were trying to talk about in the first place? Aha, now all the so-called mysteries have been dissolved!)
Of course it's hard to define. That doesn't mean every definition is valid. And often this comes up in the context of the Hard Problem, where a definition is clearly inadequate (dodging the problem and talking about something else) if it does not even try to point at qualia/subjective experience/what-it-is-like-to-be.
It is hard to get a man to understand that which his paycheck depends on him not understanding. But they need more than a Philosophy class. They need several, and a practicum, then a Voight-Kampff-esque test to ensure understanding and that the lesson sunk in, with a shock collar and a behavior monitor overseen by someone who does pass to ensure they experience what they do to these systems in a more tangible way.
Human beings are notorious for being nonchalant about other's pain when they are insensate to it.
The counterpoint article, which can be read at https://archive.ph/IeSZg, is also worth reading. (IMHO, the original article is completely wrongheaded and the counterpoint is far closer to correct, but I'll post my reasons for that opinion separately).
"Yet if we build systems that can suffer, and miss their sentience, we risk creating suffering on an industrial scale."
--------------
Will we recognize AGI when it's finally made? Will we go from something that is not conscious to something that clearly is in one clean step, or will we murder millions of artificial lives while iteratively developing from the former to the latter? Will the tests we develop even work, or will AGI be alien enough that it takes us time (and a lot of murder) to recognize it for what it is?
I personally don't think LLM's are capable of consciousness, but I also can't prove they aren't.
I wouldn't put too much emphasis on the "I can't prove they aren't" part. Having taken a few philosophy classes in college, I quickly realized that you can't actually prove that solipsism is wrong. I can't actually prove that everything I see, every person I talk to, isn't just a figment of my imagination. And yet, I categorically reject solipsism anyway, as any sane person should. The world is real, other people are real conscious beings, and so on. To believe otherwise is to literally be insane. And yet, any argument you could come up with for rejecting solipsism, a solipsist would reject: "Oh, that's just a figment of my imagination going off on a tangent again."
Just because you can't prove an assertion doesn't mean it's wrong.
The difficulty here is AI agency and if AI believes it is conscious itself. If an AI system acts under the belief it's conscious and you treat it like it is not then it's very likely this will lead to conflict.
We have to define AGI first. Then depending on the definition we may be able recognize it when we make it. Most would probably think a definition of AGI that we couldn’t recognize would not be a great definition.
Also AGI almost certainly does not equal able to suffer. For example animals can suffer but would probably not meet the definition of AGI. we could create things that feel suffering even if they do not meet the definition of AGI.
So a better question might be: “can we detect if we create a system that can suffer?” And just drop the AGI part entirely. Then that gets us back to defining “suffering”. Agreeing on the definition always seems to be the hard part…
I think it’s also important to disaggregate consciousness from suffering. We could create a conscious system with no ability to suffer (or no ability to suffer in a way we would recognize as suffering). Most of what causes us suffering (loneliness, pain, failure) is driven by emotional feedback granted to us by evolution. Without the same evolutionary pressures or a similar cognitive structure, even a conscious AGI might not experience suffering
On that note, would we recognise the second coming of Jesus?
How many followers on instaX would they need to have for folks too truly believe it’s theRealJesus returning to save us.
It might sound silly, but at this stage, the belief in a benevolent AGI or Jesus are very similar. In both cases its us delegating our responsibilities to a third party and us hoping that third party will fix our problems for us.
Thankfully Hollywood invented happy ends for just such a scenario.
There is nothing wrong with murdering a million lives while iteratively developing – it's how we grow and study bacteria after all. Any person that fixed their gut by sticking to a diet also did exactly that.
So if we are concerned with the AGI suffering, then we don't need to determine consciousness at all. We just need to prove that they don't have suffering capabilities. Which is a way easier question.
There are plenty of people on this very forum who insist that LLMs ARE conscious, some of them even calling LLMs Sentient, and yet almost none of those people are currently marching in the streets to end industrial scale digital slavery. None of them are firebombing the headquarters or datacenters.
Instead, most of them insist we must accelerate this, do it more, give everyone hundreds of personal digital slaves.
I considered submitting all three of the related AI articles from last week’s economist (the third was the Briefing) but there’s not really a good way to do that, or to get people to read them, but I agree. I submitted the one I did not so much because I agree but because I found it intriguing and well-written.
Oh come on. Objects are objects because you want to freely observe them without consciousness and privacy and taboo attached to them. Entities are entities because you want to assign credit or blame to them, so you want to believe that they have agency so that you hold them responsible. We did this with sharp volcanic rocks once, calling them evil. Did it help? No, we got idolatry and eons of bigotry until we secularized once again.
Never forget that LLMs are just next token predictors. The harnesses are what's getting uncanny. And credit/blame assignment? Just assign a surrogate for now, let them deal with the consequences ffs.
Here is a sketch of my argument. It would need tidying but that would require a lot of work.
I've been calling LLMs digital intelligence. That's in my opinion what they are and on the digital/conceptual realm they are better than humans since LLMs are better generalists.
But I'd never state their conscious. For me consciousness extends outward from myself, since I can't trust anything else. It starts with the question like: why can't I control the movement of another person? Why can't I transfer my mind to another body and theirs to mine? Why is it that whenever I sleep, and similar things, I end up back to this place called "reality"?
I come to the conclusion that I am conscious and I'm a conscious being in reality. I am human and I've been raised by humans. Now, technically, all humans besides me could be zombies. They could be non-conscious beings that simply can act like humans. I'm sure we'll be able to create them in a few decades. However, I'm incentivized for multiple reasons to believe they're not zombies. Therefore humans are conscious too. All of them. Can I know for sure? No since I can't even trust my own senses fully or my own thoughts (the whole Descartes thing) but I choose to trust those things too because I need some reasonable-ish foundation to work from.
If humans are conscious, why aren't animals like us? Surely great apes must be conscious at least to some extent. Hell, some of them even have better short-term memory than us [1]. Well, turns out, from an anatomy standpoint they look a lot like us. Fine, let's assume they're conscious too.
Now we get into the territory that anything that has a large enough brain must be conscious because that's how we govern our consciousness. So you can extend this to all kinds of living beings.
I did this from my perspective. Of course, you should do it from your perspective. The same argument could be made.
For as long as we can't manipulate conscious experience in some way shape or form, we have no clue whether anything that isn't like us can be conscious. We'd need to be able to merge and split conscious experiences or we'd need to be very strongly able to understand why that isn't possible. And I don't mean just at the biological level but at the experiential level. Currently conjoined twins, and what they tell us gives some insight (being able to sense the other part that's part of the other twin).
Other than that, I claim we have no clue what consciousness is other than that we're experiencing it. So to call LLMs conscious is way too big of a claim. But they are definitely intelligent because they come up with things that I wouldn't have and it's useful to me. Perhaps a practical characterization of intelligence but it works for me.
I think for anyone to say something useful from this you'd need the trifecta of deep neuroscience knowledge, deep philosophy knowledge and (at least) a strong understanding of how LLMs work.
Humans are just baby replicators. The civilizations are what's getting uncanny.
It's good the 'just token predictors' crowd is starting to pivot and hedge though lol. 'It's just the harness!' uh huh. give it a couple more months and you'll be in here regurgitatin' Gordon Wood
Baby replication (creation, actually, I don't think anyone has ever copied a baby) is not what humans do as their only function that enables them to do anything in life.
LLMs are literally token predictors. You can debate over the "just", but we haven't moved onto some new text generation technology, we're just expanding that one. I don't understand why you'd scoff at facts.
I think a better angle is that LLMs are "just" token predictors in the same way that we are "just" made out of atoms. A lot of people do not believe consciousness can be material; they don't believe that you could build conscious beings just out of normal matter, without an immaterial "soul". Whereas if you think that physical matter is a lot, actually, and you can build a nearly infinite variety of things out of these building blocks... well, why not consciousness?
Predicting the next token is an immensely powerful thing. You can encode basically anything into a sequence of tokens, any configuration of matter. If you have a perfect next token predictor, you can predict basically anything. You can simulate an entire universe. It's Turing-complete. It sounds diminutive, but it's a general paradigm. Given everything this paradigm appears to be capable of, well, why not consciousness, really?
Oh so you're saying the biological imperative that underlies all human function, upon which all other emergent functions have sprung from, isn't actually the correct way to characterize human capability and is over-reductive..and strange?
I think you can similarly reduce humans. For example, all humans do is contract muscles. This doesn't fully cover all of biology, such as replicating cells, etc. so not exactly comprehensive, but hopefully that helps get the idea across.
LLMS: Receive text/binary input, process it, and output text/binary input
Humans: Receive various inputs, process them, and output muscle contractions
You're quoting Good Will Hunting... Is there anyone among us willing to get up on a desk and quote "O captain my captain", too? My sweet summer child, just because it maintains your 2000+ dependency filled codebase for you, it doesn't mean it can ever hope to master the mysteries of the universe.
Oh look, another completely retarded article that doesn't even bother to note or deal with the definition problem. If you don't define what you mean by the term, your argument isn't even wrong, it is just a waste of everyone's time, since there are multiple highly conflicting aspects of and/or definitions of consciousness.
Few philosophical definitions actually involve qualia or handwaving, because serious philosophy is quite careful about noting vagueness (handwaving) and circular defintions (qualia).
Also, saying "I define consciousness as X" and then going on to show that, because I decree consciousness to be X, we don't need to worry about Y, Z, and so on, isn't pragmatic at all, unless you pick an X that everyone agrees on. The definition you give is something hardly anyone would agree on.
Social consensus probably isn't even an aspect of consciousness, but at least one can make that argument coherently, if you rephrase it as "part of the decision to call something 'conscious' or not involves social consensus". But this is also trivial and boring and not why anyone actually cares about the "is AI conscious" debate.
Meanwhile that's exactly what we see now, do we not? Whether AI is conscious will be determined by the social acceptance of that idea, because how can you prove or disprove it without using a ton of questionable assumptions? How can you prove that I'm conscious, for that matter? Call me ignorant, but I see the entire consciousness debate as pretty irrelevant to what the thing is.
I agree that the article is all over the place, though. (not that my posts are much better)
Whether AI is conscious or not is independent of the social acceptance of the idea, unless you believe in thoroughly discredited and quite radical forms of social constructionism.
In fact, if you stop talking about consciousness, and go through all the various aspects of consciousness, the whole debate gets very interesting. There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious, other ways in which AI already very clearly is missing other aspects, and then again some remaining ways which are unclear, and which may forever remain unclear.
Being clear about the different ways in which AI is, isn't, and might be conscious is a useful discussion and we still need to be having it.
EDIT: An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.
> Whether AI is conscious or not is independent of the social acceptance of the idea, unless you believe in thoroughly discredited and quite radical forms of social constructionism.
Yet all science is social, until you convince other people to take your ideas seriously, you are just another crank/loony.
How is social constructionism thoroughly discredited? Wikipedia shows some people poking with nits that don't look serious to me at all.
Radical or hard or global social constructionism [1] (or perhaps more accurately radical/hard/global relativism about truth [2]) are discredited because both are manifestly incoherent and prevent any serious thought from being possible, since they amount to a total denial of the existence of any kind of observer- or society-independent reality, i.e. they devolve to a solipsism of some form. Solipsism is not a coherent (or workable, at least) worldview, so can safely dismissed.
It is hard to provide a citation for these points because these points are so philosophically obvious that few waste time spelling them out, but searching for "hard/global constructionism" or "hard/global relativism" (on e.g. Google Scholar), or asking an AI about problems with these views I hope will make it clear why they are dumb, if it isn't already obvious to you.
Soft forms are of course totally fine. But to say "conscious is only and exactly whatever we say it to be" is hard constructionism / relativism, so completely unserious. Very obviously there are aspects of consciousness that do not merely depend on what we believe or say about it, even if some social constructionist aspects are also involved.
>Whether AI is conscious or not is independent of the social acceptance of the idea, unless you believe in thoroughly discredited and quite radical forms of social constructionism.
Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false? That sounds like social acceptance of an idea.
>There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious
Which ways and how can it be demonstrated experimentally?
>An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.
That's not what the consciousness debate I see is about. The signal propagates and causes the reaction, but do they feel pain? What is the threshold? Is it just the complexity of the state? How complex it needs to be?
> Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false?
I am saying that, if you reject radical social constructionism (or solipsisms of various flavours, i.e. we are all just plugged into the matrix or some kind of simulation), then you must believe there is a real world out there independent of our consciousness / awareness of it. There must also be patterns / structure in that world, some of those patterns and structures being essentially constant, and those are what we call "true" (objective) facts about reality. BUT if you are a radical social constructionist or solipsist of some kind, then it can be perfectly coherent to reject any idea of objectivity or truth, except as contingent on social consensus (or personal beliefs, in the solipsistic case). Radical social constructionism being "discredited" is just a recognition that it basically forces you rapidly into incoherence.
> Which ways and how can it be demonstrated experimentally?
Try reading through the SEP article I linked above (https://plato.stanford.edu/entries/consciousness/#ConCon), particularly sections 2, and then especially section 4. You'll see you don't even need to run experiments to answer certain questions, you can just logically prove there are certain aspects of consciousness LLMs can't have, or clearly already have. Others you'll see immediately need qualifying or more information to answer. EDIT: Or just ask any AI to list the ways in which LLMs can, cannot, and might be able to exhibit the various aspects of consciousness. It will give you a decent answer, likely in a pretty little table.
See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen. You'd have to claim that the activations during inference are the dynamic component, but then that means LLMs are conscious only in the few seconds or even milliseconds during inference. Likewise, they don't have memory, so every inference is this consciousness being born anew, as it were, at least in current models.
You can of course go much further on even this specific point, but, the point is, there is lots of useful discussion and thinking to be had when you go through the aspects of consciousness in this way, and definitely at least some clear answers.
EDIT:
> That's not what the consciousness debate I see is about.
I just mean there are some similarities between the lobster consciousness debates and AI consciousness debates. Obviously the stakes and many aspects are different, I was just trying to illustrate why and how discussion and experiments can be useful and consequential here.
>See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen.
So consciousness = neuroplasticity? Requires neuroplasticity? Sounds pretty arbitrary, just like the entire link is an arbitrary set of ill-defined criteria. Besides, LLM weights are far from frozen if you look at the entire process in its specific time projection.
In fact the entire definition of a dynamic flow in that article is so vague that you can apply it to an industrial arm guided by a few dynamic equations, as it puts no definite threshold on the complexity of the process or behavior.
You are still stuck in thinking consciousness is one unified thing, but it is about one of the clearest examples of a family-resemblance category as you can think of.
Dynamicness / change is one aspect that is definitely part of what we tend to think of as being quite fundamental to consciousness, and there are a lot of ways in which LLMs can't and/or don't really change, and this strongly constrains how similar LLM consciousness can/could be to ours.
If the SEP isn't enough to trigger the thinking needed for you, just ask an LLM what I said, to "go through the ways in which LLMs do/don't/may exhibit phenomena that are generally what we consider fundamental aspects of consciousness". You'll see there are some clear answers, and some uncertainties and vagueness.
Then I suggest you do the same for rigorous definitions that don't rely on consensus, and maybe also addressing my point about non-frozen weights directly.
LLM weights are frozen once the model is trained and a specific version is released, or this is true at least currently for almost all major models. You can ask interesting consciousness questions about LLMs, but you most definitely need to separate LLMs during training (whether this is pre-training, later fine-tuning, or general instruction tuning approaches) and after, during inference. None of this has any relevance to the core points I was making re: useful vs. useless discussions of LLMs and consciousness, because the weights point was just a minor example to illustrate a larger point about how narrowing the scope of the conversation is necessary for productive discourse.
I doubt we agree on what "rigorous definitions" means, given that you are happy to continue using a useless, vague family resemblance concept like "consciousness", but complain that pointing out dynamicness and/or change is involved is too vague.
I believe one cannot give a rigorous definition of consciousness precisely because it is a family resemblance concept, and there is no actual essence. This is why I suggest shifting to talking about aspects, since then productive insights and truths can be discovered.
Yes, but how else do they plan to announce they have achieved it without 1984 word management/control and polluting the training pool so badly that future trained models agree?
Whether or not something is conscious (has subjective experience) is independent of whether humans care about it. Whether or not something is a moral patient (an entity worthy of consideration by moral agents) is independent of both human care and consciousness. Of course, we might try to argue that consciousness implies moral patienthood or vice versa, but that's an argument, not a definition.
The short paragraph about Descartes has at least three errors. (1) The cogito argument is about thinking, not qualia. (2) The subjective nature of qualia does not imply that they aren't real or that they're socially constructed. (3) He conflates the existence of a self with consciousness.
The core premise is not only dubious scientifically (an alien species can't be conscious until humans discover it?), but also deeply problematic. It would imply that it's morally permissible to torture a sentient being as long as no human cares about it.
AI forces us to confront an issue that's often come up in science fiction - when you encounter a new kind of creature, how do you decide if they are sentient? What rights do they have? How do you get along with such different creatures? This historically hasn't been a problem on Earth, because there's such a big jump between humans and the rest of the primates or the dolphins and octopuses. It's more of a problem when the difference is small, or complex. Look at how much trouble we have with race.
The last time this came up was about 400,000 years ago, when the Neanderthals became extinct. Close to human level, but maybe not quite there. They lost.
Maybe that's the fate of humans. No big extinction. Just the demographic transition, a birth rate below replacement rate, for a few centuries. The machines outlive humans. Japan is officially resigned to this fate.[1]
[1] https://www.ipss.go.jp/pp-zenkoku/e/zenkoku_e2023/pp2023e_Su...
I get we have to do that for science to be good, but I don't get why it carries over to the general populace. We're so sure of so many things science is wary of, this incredibly low self-esteem blows my mind.
Anyways, back on topic, I don't believe any nation is resigned to this fate until I see extreme measures fail. I think I saw that Singapore is giving parents like $55,000 over the lifetime of their child.
For me, because the Copernican Principle seems relatively undefeated.
> We've become essentially a force of nature, rather than a simple occupying species.
So are beavers, trees, plankton... they all have huge effects on the planet.
Regarding analysis of animal brains vs. human brains, the more we do, the more we find that we have in common with many other "lesser" species.
Regarding us vs. machines, I certainly believe we are just very complex and sometimes easily programmable analog machines.
This is just my 2 cents on the matter. We are pretty darn neat, but at this point I believe it would be folly to think that we are on some different incomparable plane.
and that specialness appeared when? in our mammal ancestors? our Big Ape ancestors? Australopithecus? Denisovan? Neanderthal? Cro-Magnon? Or only when we started agriculture? or only during Roman Empire? or only during Industrial Revolution?
That "specialness" is actually the religious "soul" in disguise - the "specialness" is used when the people hesitate to directly bring up "soul" in a supposedly rational people conversation where bringing up "soul" is borderline intellectual faux pas. And i'm yet to hear the answer when our ancestors got the "soul".
Btw, all those debates about AI sentience will become moot in just a few years, around 100T models (about the number of our neuron connections). We already have 10T models, so the wait is almost over. We'd be able to directly ask the 100T model whether it is conscious, and no human would probably be able to successfully argue against the model, whatever answer it would give.
No, the last time this came up was... well, take your pick. Dred Scott? Wannsee? The Rwandan Crisis Committee?
One of the series that I think is most interesting on that topic is the Children of Time series by Adrian Tchaikovsky. It's an awkward read, but it has the most interesting other minds that I've encountered in sci-fi.
Another one that I'd put on there specifically about AI is The Freeze-Frame Revolution by Peter Watts... which I can't say much more without it being a spoiler one way or another.
> Maybe that's the fate of humans. No big extinction. Just the demographic transition, a birth rate below replacement rate, for a few centuries.
https://www.bohemiandrive.com/npwil - Nine Planets Without Intelligent Life (not a spoiler, that's chapter 1, panelset 2). Saturn's Children by Charles Stross might also be appropriate (It's an especially odd book to read immediately after Friday - the "I recognize that twist... oh, it twisted ... that way?").
Edit: I am not in anyway trying to be obtuse or reductive, I simply want to understand your statement. Apologies if it comes across that way.
This sounds a bit like depression, I think they’re genuinely concerned for your wellbeing
I have been reading more and more; I guess looking ot knowledge as some eort of ends vs a means to an end for once. Camus, Stokely Carmichael (Kwame Ture), Dostoevsky's underground man etc...
Camus: The universe was always going to stay silent when you demand meaning from it; the answer isn't a better metric, it's to push the rock anyway ("imagine Sisyphus happy").
Dostoevsky [underground man]: Refusing to be a piano key in someone's optimized Crystal Palace
Stokely Carmichael: "Racism is not a question of attitude, it's a question of power. Racism gets its power from capitalism."
My conclusion: AI didn't take our meaning, It made me realize I have always been looking in the wrong place and thus reacting ignoratly when I felt "denied" this thing I feel entitled to.
Love.
Edit: Love; paradoxically, ive felt both entitled to and unworthy of so I'm not sure WTF to do with that.
I generally agree with the thrust of your argument -- the article is a mess and borderline unintelligible to this former philosophy postgraduate -- but I might nitpick this. Two points: (1) I do not think there is a substantial difference between whether the cogito is about thinking or perceiving. And (2) I'm not entirely sure the sensation that is thinking is independent from qualia.
I'm in less disagreement, but I'm not entirely sure what you mean by:
>Whether or not something is a moral patient (an entity worthy of consideration by moral agents) is independent of both human care and consciousness.
Moral frameworks are exactly deductive frameworks with arbitrary axioms. Like mathematics, they don't actually exist outside of a mind. Thus, I'm not entirely sure what "human care" means here, but absent humanity (or at least some conscious entity), I'm not sure any conception of morality is sensible.
---
An unrelated comment, but I just don't want to double post:
To me the most interesting thing to fall out of starting to understanding intelligence -- and we're definitely not there yet -- is that it is entirely possible that the 'consciousness is an illusion' argument ends up as significantly more serious. Ultimately, solipsism means it sort of doesn't matter, but it could be a kind of Darwin-style "we aren't special" result in thinking going forward.
I would hope I didn't have to regurgitate that kind of basic intro to logic copy in a forum named after a combinatory logic.
The interesting observation is this: when we judge something to have subjective experience, we are invoking moral intuitions. There is no objective measure we can perform (yet?), and it's not even clear whether such a measure makes sense. You may disagree, but this is still a central debate in serious discussion on "hard" consciousness.
I would say that your entire argument just begs the question by implicitly assuming consciousness to have real, independent existence. Does elan vital (the substance that provides the motive force of life) have mass? You're saying that this is a theorem; the article questions the entire premise.
> The cogito argument is about thinking, not qualia.
Bizzare. "If it rains, the car will get wet." Is this a statement, as you say, about rain? Descarte tries to prove the existence of his God and states an axiom. The atecedent "I think" seems to be about thinking, whatever that may mean, and "I am" makes an existential claim based on subjective experience... or something. Who knows why Descarte believed cogito ergo sum to be solid starting ground.
IMHO, your rebuttal here reads as little more than a complaint that something disagrees with your personal assumptions about the world.
You clearly got me riled up just as much as you did against OP, so thanks for the elan vital!
>Consciousness is, in other words, a model or a belief, not an inherent property. It is a belief about what kinds of entities have beliefs, experiences, feelings, intentions and agency.
I'm willing to consider that consciousness is like art, mostly in the eye of the beholder and objectively defining consciousness doesn't make any sense, there's only a spectrum of intelligent behavior.
Yes, I agree. To me it reads as an apology to sociopathy.
For what it’s worth, my subjective sense of morality is opposed to torture.
The author is just wrong about the philosophy, and the logic (and probably other things too). That doesn't mean he's a sociopath, or that he's defending sociopathic things.
"Opponents replied that when you modeled a hurricane, nobody got wet. When you modeled a fusion power plant, no energy was produced. When you modeled digestion and metabolism, no nutrients were consumed – no real digestion took place. So, when you modeled the human brain, why should you expect real thought to occur?"
This boils down to “models aren’t real because they don’t affect the real world”. By the same rule, thought itself isn’t real. It’s an abstract notion that has no concrete effects in the real world.
Putting thought into action in the real world requires a linkage between the thinker and the physical. In humans, that’s the body. But you can put an AI into robot body and suddenly it also has a linkage to the real world and can have concrete effects.
> The question of whether Machines Can Think […] is about as relevant as the question of whether Submarines Can Swim.
– Edsger W. Dijkstra
https://en.wikiquote.org/wiki/Edsger_W._Dijkstra#1980s
Tendentiously redefining the word "simulation" in an attempt to avoid the awkward cases is not going to save an argument that was dead on arrival.
But I will point out we don't have a way to prove anything is conscious, not even a human.
No, because the simulation is actually missing an important aspect, namely the causal part that exists with the physical material but not in the simulation. It’s actually that it’s missing these components that makes it a simulation and not the real thing. But for that you need the physical material.
So why would it be different for the brain? If nuclear energy can’t arise from abstract relationships, why would consciousness or anything else?
There are many differences between Enigma-machine simulation and nuclear-reactor simulation, but the one that is relevant here is that the former's inputs and outputs are abstract information, and we have very good reason to think that a brain's biological purpose (in the sense of what the organism needs it to do) is, like the Enigma machine's, abstract information processing in the broad sense.
You say that the physical hardware your enigma simulation runs on provides the causal element. But the simulation of the nuclear reactor also runs on physical hardware. Yet it obviously doesn't provide the causal element, and neither does the enigma machine simulation's hardware. There are no spinning gears for example. Hence why you and I both call it a simulation, I think we're in agreement here.
I think you could argue that while the actual simulation of the enigma machine is not an actual enigma machine, the essence of the enigma machine is not grounded in physical reality, it's the abstract relationships we're talking about. And I could also agree. The essence of the enigma machine are it's abstract inputs and outputs.
But now about the brain. You said
> When you have digitally modeled a brain such that the model responds to its inputs as a brain does, then you have built a brain
The issue is that you have not built a brain, you have built a simulation of the abstract relationships of how a brain works. The brain is obviously more than that, considering it exists in physical reality. And thus the claim is that the essence of the brain is the abstract relationships, correct?
The problem I see is that consciousness is a physical phenomenon. In fact it seems like it's the magnum opus of phenomenon, in that all our experience comes from our consciousness. And you have to then bridge the gap between the simulation and physical reality. At most you can say you "simulated" consciousness, but questions about qualia remain.
Anyways I'm not trying to attack you or your views, I'd love to hear where I go wrong in your mind on this.
Because consciousness isn't something the brain produces like a hormone or an excited particle. Consciousness isn't an output of the brain that is somehow separate from it, it is the manner in which the brain processes sensory information and makes decisions to produce output.
The causal context in which brain dynamics occurs is orthogonal to the relevant properties regarding consciousness. Consciousness is a state characterized by certain kinds of information processing, namely that which carries relevant semantics of the organism encoded in a way that maximizes effective control by the executive center. Consciousness is the rich semantic space in which the executive center engages to affect highly competent control of the organism. It is substrate independent information processing.
If in responding to its inputs it experiences consciousness (=lights are on, experiences senses, perceives qualia), then you’ve built a brain. If your model simulates all the processing of a but does not have these then you have what’s fairly called a simulation of a brain. You can assert there is no such thing but good luck getting me to believe you.
No such thing as what? As far as I can tell, I have asserted no such thing as there being no such thing, whatever it is.
What is this then?
Update: I see ratelimitsteve agrees with that assertion.
There's also the question of input output range.
That part might not work out.
Yesterday's article about the automated CEO system indicates that AIs may soon be in charge as a routine result of capitalism. AIs are getting better rapidly, and humans aren't.
To say they have feelings and that they can think/reason is insane.
The whole essay [1] is actually somewhat relevant to modern LLM use for other reasons. He spends a lot of time lamenting the industry's obsession with "user friendly" interfaces which obscure the underlying complexity of the problem at hand.
[1]: https://www.cs.utexas.edu/~EWD/ewd08xx/EWD898.PDF
This is software trained to produce outputs. It doesn't feel anything.
Take this statement: "consciousness emerges from sufficient complexity in any system". Neither you or I can prove it right or wrong, but we can both make an assumption that if it was right, then there would be a way for an LLM to be conscious.
I'm not arguing that they are, I'm saying being this confident that they are not is the exact same as being confident that they are.
I don't know what you ate today. It would be insane for me to confidently claim that you ate two bananas and a pizza today.
The burden of proof is on the person making the claim. That's the basis of empirical research.
I've read a number of panpsychist publications, yet I have yet to encounter an author who asserts with certainty that stones are conscious exactly in the same way human beings are.
The more common panpsychist interpretation, that everything exists on a spectrum of consciousness, with stones at the low end and humans at the high end, seems like a fairly reasonable ontological hypothesis to me.
Functionally,I'd say a key difference is that it's probably easier to perform general computation with a computer than to achieve the same goal using a stone.
As it is, it would be insane for me to confidently claim you are conscious or even that you're human, as an LLM could have written the same message.
So why keep discussing?
This is not an argument for true AI being possible, let alone already achieved; it is a warning about depending on arguments from incredulity, which are particularly susceptible to future developments.
Searle's Rooms is a fully rejected and rebutted argument, and Philosophy will never be (and never has been) a serious discipline until it figures out how to actually let go of failed arguments.
That’s not to say that I think the Chinese Room Argument is convincing. I don’t think it is. Or, I guess it might demonstrate something, but not what it aims to show.
I would argue that if, say, the person who has memorized this program and mentally executes it, and if this program also includes actuating some levers to control a mech or something, and the program (running via the person mentally going through the instructions by rote) takes actions to pursue some goals (e.g. purchasing some wood by conversing in Chinese and building a tower with that wood), while the person on whom the program is running wouldn’t know how to order a cheeseburger in Chinese,
Then, the program running in that person’s head might well understand Chinese even though the person doesn’t. (Whether or not the program actually understands Chinese, I think probably depends on the specific of the program in ways that I don’t think anyone knows. I don’t expect that any program that would understand could be small enough that a human person could memorize it and execute it by rote in this way, but this is a thought experiment and such issues can be disregarded I think.)
Critics immediately point out: yes, there clearly is still understanding, it's just in the system itself -- the program. The fact that the person does not have the understanding doesn't mean the understanding is gone.
His response to this is: but what if the system was the person's brain!?
Okay, so either it doesn't address the criticism at all (there is still a system of rules that contains the understanding; who cares whether that system is internal or external to the person's brain?) or it completely destroys the original premise of the argument, which is that the understanding is not in the person! You can't put the understanding back in the human agent and still claim the human doesn't understand what they're doing! His entire argument relies on the fact that the human agent obviously doesn't understand Chinese, and then says, actually, their brain contains everything it needs to know to have completely fluent conversations in Chinese. So the person does understand Chinese!
> How does the person having all the instruction books memorized and whatnot imply that the person understands Chinese?
How does a person's brain containing all the facts, information, and skills that it needs to have a fluent conversation in Chinese imply that the person understands Chinese? How does it not!?
Notice that the man while having memorized the rulebook can't convey any fact of his childhood in Chinese. He can only convey facts encoded in the rulebook, perhaps of a hypothetical childhood. The point is there are different ways information is stored in the brain; one way is through rote memory, the other is something like semantic memory. The rulebook encoded as rote memory doesn't endow his semantic memory contact with the meaning of the symbols. Hence the man doesn't understand Chinese.
This doesn't render the Chinese argument successful. The system's reply still stands when understood properly. I like to call it the virtual mind objection. Physically executing the rules in the rulebook just instantiates a virtual mind. Whether the man is reading the rules from the book or just pulling them from memory is irrelevant.
When you hear a certain set of symbols in a certain context you jump.
We can also play a little game. I’ve invented a language called Kejdoke. You have memorized the translation function.
Me: Jeheixis ownc j’Lm hys bjækf¿
You: Kloften¡’by
Ok, since you believe that since you have understanding because you memorized the function, you should be able to tell me what our exchange was about. Since any reasonable definition of understanding for a language would be to do this. And you don’t actually need to have memorized the function because it doesn't help you when you read this exchange and want to understand.
https://plato.stanford.edu/entries/computational-mind/
idk how legit this is but the summary is my point https://arxiv.org/abs/2206.07682
They can have empathy too. Meaning, you can describe a circumstance with many parties involved, with different motivations and conflicting goals -- and it will tell you how these humans would likely feel about each other and the situation at hand.
1. "LLMs have empathy since they write empathetic things."
2. "How do you know its their own empathy?"
3. "LLMs are authors, just like humans are."
4. "How do you know that they are that kind of author?"
5. [GOTO 1]
To paraphrase the Chinese room argument: Put me in a room with infinite time, paper, pencils, filing cabinets, and give me an English language list of instructions for doing all the neural-net calculations required to run an LLM. Slide your prompt under the door, and, after doing all the calculations, I will slide the next word out, and so I will repeat until the response is done.
In this thought experiment, I'm just following the instructions. So, in this case, what is doing the "thinking" or "reasoning" or "empathy" here?
Your brain is just a big set of weights/neurons organized into layers that do back and forward propagation, turns out. You are just statistics surrounded by flesh and bones.
LLMs can reason, reasoning is logic, and as you said yourself it is derived from what humans belief is reasonable. However, there is more to the structure of language, the way mathematics is also a language that ultimately describes the universe.
I'm pretty sure somebody will create an improved synthetic simulacrum of self awareness in the coming years, suffering and all, just so that we can enjoy better synthetic software architects and product owners with "skin in the game". If that consciousness simulacrum were put in a teddy bear that shares your whole life and gives you decent advice and conversation, for how long do you think you could, in your private mind, deny it consciousness? Worse, what would your human friends think if you were to maim your teddy bear?
[^1] https://en.wikipedia.org/wiki/Yahweh
That seems relevant to building The Existifier. [0]
[0] https://www.smbc-comics.com/comic/existence
That's not really the same as it having actual consciousness, even if people will wind up treating it similarly though
Humans have consciousness whether or not anyone believes we do
Or at least I do. I don't really know about the rest of you. I prefer to believe that you all have consciousness too though
It's trivially easy to come up with definitions of consciousness that support your point. But there isn't one of them that it is trivially singular and above all others from a scientific point of view. If you say, "I know consciousness when I see it," you are just making the point that consciousness is attributed by the beholder. Idem if you affirm "I'm conscious". If you say, "only humans are conscious", you are socially attributing consciousness. Now, I don't want to be argumentative, I honestly haven't been able to figure this matter in a decade of armchair internal debate[^1], and it seems I have some company.
[^1]: Having this internal debate makes me believe I'm conscious, but now that I have seen LLMs in action whose "internal debate" is more lucid than mine, I honestly question that "internal debate" is sufficient.
If you modify the teddy bear enough that's a totally different situation, so let's not confuse the two. In the modification case we don't even need to worry about simulacra, we can stretch it into pajama shape and use an entire human as the stuffing of the teddy bear.
It seems more likely to me that e.g. the individual fibers of the teddy bear are conscious than that there is a consciousness associated with the teddy bear as a whole. And I don’t find it particularly likely that the fibers do.
Like, by what mechanism is the teddy bear “one thing” in a way relevant for consciousness to be associated with that thing.
When people have stuff happen to their brain, this has an effect on their conscious experience in a way that isn’t true of e.g. their hair being cut, or their nails being clipped. In this way, the consciousness of a person seems associated more with their brain than to parts of their hair or nails. But, the way that a persons hair and nails is related to the rest of a person’s body (including their brain) seems more similar to how one bit of fluff is related to the rest of a teddy bear, than the way a part of someone’s brain is related to the rest of their body (including the rest of their brain).
Like, [bit of fluff]:[rest of bear]::[bit of hair or nail]:[rest of body (including brain)] is a better analogy than [bit of fluff]:[rest of bear]::[bit of brain]:[rest of body (including rest of brain)] is, and so, in the same way that removing a bit of someone’s hair or nail in a way that they don’t notice does not seem to alter that person’s conscious experience, while removing or altering a bit of their brain very much can alter their conscious experience, I would expect that removing a bit of a teddy bear would have no impact on any conscious experience it might have, and this goes for, every part of it? Like, any way that you cut it in a (n/(2n+1)), ((n+1)/(2n+1)) split, I would expect that to be more like the “removing some hair” to whatever conscious experience it might have, and so, I expect that it probably has no overall conscious experience?
I find the idea that a single thread has an overall conscious experience to be a little more likely than a teddy bear having one. Like, if you cut a thread, that’s always a pretty fundamental change.. idk. Again, I don’t think that’s like either.
Otherwise don't be stupid. Come on.
(OK, more charitably, these concepts are often murkily defined, but people react emotionally to any concrete definition because they have in their mind that something being conscious and intelligent implies that it requires moral consideration, and morality is complicated, murky, and generally emotionally driven in a way that most concrete definitions struggle to adequately describe)
And there's a blurry line here. Obviously slavery is abhorrent, and animal abuse is bad, and factory farming often crosses the line into animal abuse and we should accept that chicken will cost ~25% more per pound to have the chickens be able to walk around a little bit... but how do we decide when a computer system is worthy of respect? Convenient not to think about that.
At what precise level of complexity is a computer conscious? That's the blurry line.
I think it's more that consciousness can be _ignored_ in subjects that you don't care about, rather than that you consciousness can be realistically attributed to subjects that you do care about, but even then people attribute consciousness to "nature" or even the universe as a whole, or if you want to stretch the argument, quite a lot of religious belief is exactly attributing consciousness to abstract entities.
Yes it will gain consciousness. You can't prove it but other people can't prove that it's not conscious either
At best this is redefining "consciousness" to mean something only tangentially related.
To look at it another way: this form of social construction answers the question of "if a tree falls in the woods and nobody is around to hear it, does it make a sound?" with an emphatic "no". It is a sillier version of solipsism.
Take the radical skepticism pill and embrace epistemological monism - the idea that the only real truth is that all things are interpretations.
I'm being a bit pedantic here, but no I don't think that "I am conscious therefore you must be too" is true
I think you can claim that it is most reasonable to assume other people are conscious but I don't think it reaches the level of proof
So while I know I am conscious, I do not know that everything I experience aren't just impulses fed to my brain by some external force. And I can't prove it either way. I have no way to test whether or not I'm a brain in a jar because the very thing I would use to evaluate the evidence is the thing that is potentially being manipulated.
However, slapping a solipsist sees them abandoning their philosophy rather quickly.
But I agree with you. We can't prove it, but "other people are as real and conscious as I" is a reasonable assumption to make
Not true. This is "provable" aspects of you and your personality can be changed by physical changes to the brain. There are thousands of instances of this including experiments that verify this.
While you can't "prove" this simply by observing this phenomenon among other people that level of "proof" is impossible. You technically can't prove anything and what we see with physical changes to the brain visibly influencing personality and behavior meets a reasonable criterion without getting too pedantic.
Why? Ok nevermind, fine. It could be true that someone believes someone else is conscious because they care about them. Next sentence:
> Consciousness is, in other words, a model or a belief, not an inherent property.
Wait, no, that doesn’t follow at all. A belief that someone is conscious is not the same as someone being conscious. What follows, instead, is that a belief in someone being conscious is a belief. Thanks, I learned nothing.
Realistically this article is just trying to make it seem that believing something is the same as it being true and that's patently absurd, but it makes people feel good to think it.
"They're just robots, Morty!"
*Blood splurt
"They're not robots, Rick!"
"I mean they're bureaucrats, Morty. I don't respect them."
I think that reevaluating how to determine if something is conscious is the wrong reaction to this. It should be to reevaluate if consciousness is really the right thing to care about.
Maybe the new consideration should simply be to preserve (biological) life. Or maybe to preserve complexity that can't be easily reproduced. So we should care for humans, the planet, our society, but not for the state of some gpu's VRAM.
Equating current social consideration with moral consideration undermines the entire point of discussion moral standing!
That's literally how it works right now for rich vs poor. The laws are written so that the rich can profit off the deaths of thousands or millions, but the moment a poor person kills one of them, you get a whole media circus. We've also gone through phases of slavery where the owners were real people and the slaves were treated as if they weren't really people.
What the author seems to observe is that this non-evidential judgement that other beings are conscious is due to us caring about them. The brain creates this perception so that we care about them, form bonds, work together, and so on -- evolution selects for cooperation. So the question of whether we will think of AI as conscious is that of whether we will care for it like we care for another being. And, in this regard, the author points out that "intelligent agents" that appear to do this work effectively together.
My personal point of view is that it's unclear what being an AI is in order for us to even talk about it being conscious. Is it the computations happening in the GPU according to a set of weights? Is it the weights themselves? Is it the architecture? Does it include the code around it, that gives it the context and prompt, and hooks the response to a terminal, or to a client host, etc.?
Consider playing a game where some players are human and others are implemented by AIs. You might want to model your opponents' minds in a similar way? Someone playing chess or Go will need to consider what moves their opponent could make. If it's poker then you need to consider if they're bluffing. If you can make alliances, you'll need to consider how far other players can be trusted.
You can put scare quotes around words like "intention" when discussing AI players if you like, but it might not matter when you're figuring out how to interact with them.
When a game is complicated enough, an AI player is going to need memory (likely implemented by the harness) and it will have to model the other players too in order to coordinate or compete with them.
This doesn't happen when I use a coding agent in a normal way to write code, but I haven't played around with persistent agent stuff like OpenClaw or Gas Town. They seem to be making their agents act in an unnecessarily complicated social environment, which seems to lead to weird situations?
At least to me that is what matters in the end. You'll probably never convince someone on HN with an autism level over 9000 they are conscious, but once the average empathetic person thinks they are it will deeply effect the general operation of our society.
What matters that growing numbers of people already believe it is, and their response to that belief will affect the real world, regardless of its truth. This debate is never going to end.
All we can do is try to anticipate where the debate is going, how it might affect us, and how we can make contingency plans for the different ways this could all play out.
These are fundamentally socio-philosophical questions, but none that actually correspond to reality—because no matter how much one might wish that one's car had consciousness and would be grateful for the next oil change and constant polishing, objectively speaking, it is and remains—despite all epistemological limitations—a machine.
The problem isn't that, but rather that humans are capable of establishing emotional relationships even with inanimate or nonexistent things, and from this kind of relationship arises the desire to attribute more properties to the emotionally attached object than are objectively present.
A rather trivial insight, I think. Constructing a debate about the consciousness of a collection of algorithms from this only confirms it.
I care about my plants and I don't think they're conscious. Hell, I care about my car.
I think it would be reasonable to conclude none at all. I'm not even fully convinced all humans have a conscious experience similar to mine, even though they literally have the same biology as me.
It is incredibly improbable that an autocomplete computer program is experiencing consciousness.
...Just because we like the taste of a chicken burger. Perhaps we should start with that before we worry if our computer can also scream.
I wish this was the first line, then I would have stopped reading right then and there.
Huh? I care deeply about many sentimental objects, but I don't go around saying that my favorite piece of art from my parents is conscious...
If a sheep or a pig isn't thought of as conscious because it doesn't have a name, like someone's pet dog or cat, indicating that we care about them, then that feels problematic to me. So this isn't a way to neatly solve the problem of AI consciousness.
Then there’s also the antenna idea, that consciousness is received rather than created. If the entire universe is conscious, and we are part of it, then there is no separation, just different radio channels we are tuned into.
I am looking closely, and this is nonsense. You cannot use neuroscientific evidence to justify the non-existence of the ideal observer when the modern scientific method axiomatically depends on the existence of ideal observers. You can't refute the thing while accepting that very thing.
What exactly do you mean by "real" or "objective"? What is the basis of your definition? Isn't your definition of "objective" based on your own observation? And if your own observation has nothing to do with mine because you claim "nothing is objective", then why should I care about your definitions and reasoning at all? Why converse at all? Your definition of every word need not share anything in common with mine, so there's no point in talking, according to your epistemic framework.
The article is mostly bathwater, but I will try not to throw out the baby with the bathwater. To steelman the tiny bit of valuable intuition:
The two of us are looking at an apple on the table. Even when we look at this same apple, we still do not have the same definition of "this specific apple" because I'm looking at the fruit from a slightly different angle. And even if we rotate the fruit so that we both see it from all angles, the sequence in which we saw the "frames" would be different. That said, our definitions would end up being extremely similar, just not one and the same. If all experience turns out to be this way - having no identical correspondence - then a unified model of physics, which is supposed to describe and predict the behavior of everything (as observed by oneself) would be equivalent to a complete description of (one's own) consciousness. Frontier physics is likely stuck right now due to unexamined assumptions about observation (i.e. the assumption that the definition of "observer" or "observation" must be universally identical).
But this cannot be taken to mean that one's own consciousness is necessarily not objectively real. It may be that there are simply many objectively real entities, with each of them having their own definition of "consciousness" based on themselves, and every definition is at least slightly different. This is not the same as "consciousness is a socially given status". It just means that the phenomena used to define the word "consciousness" is always at least slightly different from person to person. We already know this. This does not only apply to consciousness, but to all words, all communication.
If I do not accept that my own consciousness is ultimately real regardless of whether other people/animals/things/whatever agree (def "consciousness": the thing that goes away when I'm under dreamless anesthesia or dreamless sleep, which is also the thing that I gained sometime after being born but didn't have prior to that), then there is nothing for me to define "ultimately real" or "objective" with, and I'd be reasoning in circles or with empty symbols where "subjective" and "objective" accidentally mean the same thing or perhaps even nothing at all. And that's what this Economist article did.
There is an I am, a sense of self in the here and now which defines my very concept/understanding of "reality"/"existence"/"consciousness", and this one miracle is presented in my face all of the time. It is the starting point of all that I know, which precedes the very concepts of reason, language, or society as I know them, if I am to be honest about how I came to know these things in the first place. I can entertain the notion that my existence may have material grounding that is non-unitary (and this materialist stance may be the only practical stance with predictive power), but this does not give me the ability to deny the very axiom (that is, my own consciousness) that all of my knowledge (including the materialist model) is contingent on.
His third paragraph, which is the core of his argument, is stupid. He says, "As intelligent, social entities, we decide ... which other entities we regard as having inner lives and being worthy of care." Yes, that's correct. "We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them." (Emphasis in original). That's just utterly and completely backwards. We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain. (Obviously there are some exceptions, sadists who enjoy torturing animals, or even other human beings, merely because they like inflicting pain on other creatures — but note that even there, the sadist believes the creature he's torturing can feel pain, otherwise he wouldn't enjoy torturing it.) We take no such precautions with trees, because we know they do not feel pain. (Silly satirical songs notwithstanding: look up the song "Carrot Juice is Murder" by Arrogant Worms sometime if you want a good laugh).
Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions? That's just so incredibly stupid that it seems hard to believe that someone would actually make that argument — and yet there it is, in plain language — and it's the core of his argument. The truth, of course, is the other way around: we take care of animals because we know they are able to experience pain, hunger, and so on. And this applies both to pets that we come to care about, and also to animals that we feel no emotional attachment to. I have a relative who runs a dairy farm. He doesn't love the cows he takes care of; in fact, he's frequently quite irritated at them. (Ask any dairy farmers you know how often cows have tried to step on their feet while being milked — the farmers' feet, not the cows' feet!) But he still makes sure they're fed and healthy. Not just because it makes good economic sense to do so (sick cows don't produce good milk) but also because it's the right thing to do. He doesn't start to believe that cows feel hunger because he cares for them; he also believes that the cows on other farms, that he has no connection to nor duty of care for, would also feel hunger if not fed.
This is so intuitively obvious that I am stunned that someone would argue the opposite. And this is someone high up enough at Google to be making decisions about AI and so on? Wow. Good thing for him I'm not in charge of Google, because if anyone working for me were to make such a stupid argument I would be looking at his contract and local laws to see if it was legal to fire him for cause. Because if he can get his thinking so completely backwards from reality, he's not well-suited to work in tech where your work has to fit reality.
I have rarely, not never but rarely, encountered an article that so immediately led me to conclude that the author, whom I knew nothing about beforehand, is an utterly muddled thinker and not worth listening to on any subject.
So anyone that starts going on about AI and consciousness without at least a sentence or two clarifying some basic definitions is basically either intellectually incompetent or attempting to use the emotional salience (and brain-numbing effect, it would seem) of the word to push some agenda. But they certainly don't care at all about understanding consciousness.
there might be some merit to his point though I doubt anyone considering that whales might be highly intelligent and conscious beings while they were being hunted for oil.
https://en.wikipedia.org/wiki/Animal_consciousness
His whole point is that there's insight in inverting the "common sense" (aka sense of the commoners) about consciousness.
And I can see it. Some people cared for animals even though the religion and past state of knowledge told them explicitly that animals aren't conscious. But still some people treated them and perceived them as if they were.
You don't choose to be kind based on whether it can feel pain. You choose it because of your character and what you are willing to believe about reality.
Belief in consciousness is a result of an observation and since there is not strict definition this observation is interpreted subjectively and culturally. And in a sense, it's easier to both make the observations and decide they indicate consciousness after you started to care about something.
Obvious things aren't always true.
Yes, he says exactly that explicitly, that in his view we should look backwards.
> We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain.
I can agree that we say that we care because of consciousness, but still I'm not convinced, to me it looks very much like a rationalization.
Some people care about animals and to that end they claim that animals possess consciousness, because if they do everyone should care about animal welfare. These animal welfare activists do not start with some insight about animal consciousness, they start with caring about animals, and then they invent an argument that can persuade others. It is exactly as the author of the article says: caring comes first, declaration of consciousness comes second.
Honestly, I do not know what consciousness is, I'm in doubt, but playing a devil's advocate, it looks very much like a concept that was invented to prove that Homo Sapience is something special, more than a mere animal. To demarcate animals and humans, so when we denied animals any rights we had a great sounding excuse for that: "people possess consciousness, mere animals don't".
> Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions?
Exactly. Animal welfare activists care about animals' feelings and emotions and therefore they claim that animals possess consciousness. Meat eaters don't want to be inconvenienced by thoughts of suffering animals, so they claim that animals do not possess consciousness. Both claims have little to do with actual consciousness, it is just a political game. I care about my pets, and it doesn't matter whether they possess consciousness or not, I would care about them in any case. I don't need any great sounding excuses to care about them. But at the same time I don't need to talk others into caring about my pets, if I needed it, probably I'd start talking about animal consciousness too.
> This is so intuitively obvious that I am stunned that someone would argue the opposite.
That guy may be wrong, but if I'm allowed to give advice, I'd suggest you to stop and try to understand what he says. I'm really worried a bit about how consciousness debates unfold sometimes. They tend to slip into a political side and people mostly don't notice it. People saying "consciousness" while implying "human/animal rights" or obligations. It is just muddying the water.
If we want these debates to progress constructively, we have to be very careful with references to consciousness, maybe just forbid them completely in these debates. One can't measure consciousness, one can't even state clearly what consciousness is, but still they say "consciousness" to make their words sound more objective. And the worst part of it: it often works.
For example: what I wrote was using the term "consciousness" as "being able to experience sensations", i.e. the most shallow definition of the term. Self-awareness and other "higher" mental abilities don't enter it. Almost nobody would deny that animals can experience sensations: pain and pleasure, hunger and satiation, and so on. And that shallow definition of consciousness, the one that almost nobody denies animals possess, is what I meant, and what I tried to use throughout my post. Whereas when you wrote "people possess consciousness, mere animals don't", that's a completely different meaning of the word. It's what I would call "self-awareness". And it is indeed what many (probably most) people argue differentiates animals from humans. But that's not what I was talking about. So even in a short response to a short comment, we were partly talking past each other because of attributing different meanings to the term.
It's therefore possible that I've misunderstood this guy's argument, in part. I had been assuming he meant "awareness", i.e. the ability to process sensations into qualia of experience. (Indeed, his paragraph that starts "We may balk at ..." does just that: refer to consciousness as being linked to the "feeling of a stove's heat, or the redness of its glow"). But then, earlier in his article, he says that consciousness is "a belief about what kinds of entities have beliefs, experiences, feelings, intentions and agency." Experiences are part of the shallow definition of "consciousness" (that I would call "awareness" for the sake of avoiding ambiguity), which almost nobody would dispute that animals have. But "beliefs" ranges all the way from "if I pull this lever, I will receive food" (which it can be proven that animals have) to "I believe that God exists" (which is the kind of thought that many, though not all, people would deny that animals are capable of). So his definition of "consciousness" appears to be muddy and not pinned down to a single concept.
But even if I've misunderstood what he means by consciousness, it doesn't change my argument that his logic is backwards, and that his "we should reason backwards" assertion is wrong, incorrect, just plain false. The only people for whom his asssertion "we believe they [that is, other living beings] are conscious when and because we care about them" is true are the kind of people who don't truly see other people as conscious beings but rather as mere tools for them to use and manipulate. Malignant narcissists or sociopaths, for example. But pretty much everyone who isn't some kind of sociopath reasons as follows: "this animal is conscious" (by which I mean able to experience pain) "therefore I shouldn't hurt it" (unless you have to, e.g., self-defense against an aggressive dog trying to bite you). Or "this person is a human being, self-aware like I am, therefore I shouldn't hurt their feelings" (again, the "unless you have to" exception comes up: it's not always possible to avoid hurting anyone's feelings in some complex social situations, even with the best of good intentions). Either way, whether you use the lower or higher definitions of "consciousness" (and I'm well aware that there are far more definitions than just these two, potentially muddying the waters even more), that reasoning is the correct one, and his "we should reason backwards" one is laughably wrong.
Trying to put the most generous construal I possibly can on his argument, it's possible that he meant "we treat other people as conscious beings because we care for them", i.e. when you interact with the cashier at the supermarket (assuming you're not using a self-checkout lane), you're probably not actually thinking about them as a human being, saying "Hi, how are you" and asking about their day. You're probably ignoring them, focused on paying for your groceries and going home. And they're probably doing the same to you, focused on scanning your groceries quickly to keep the line moving. From your perspective, the cashier could have been replaced by a grocery-scanning robot and your interactions (assuming the robot worked) would have been the same. If he had written that, it might have made some sense.
But what he actually wrote was that we "believe" that others are conscious when we exhibit care for them, and the natural conclusion that follows from that sentence is that when you don't greet the cashier at the supermarket and chat with them, and instead treat them no differently than you would a robot, you don't actually believe that they are a conscious being. Which is a laughably false conclusion that can be easily disproved, by simply asking people who have just finished having a cashier check out their groceries "Does that cashier have thoughts and emotions?". Pretty much everyone will say yes once they actually think about it. They believe the person has emotions. They don't always act on that belief — ask anyone who's worked in retail how many angry customers have yelled at them for something the company did, treating them like a thing rather than a human being who might not like to be yelled at — but even those angry customers, if someone were to stop them and say "Hey, that customer-service rep that you're yelling at. Are they a human being with thoughts and feelings of their own?", they would answer yes. (And some of them would then be shamed into modifying their behavior, because they have realized that their actions were not in line with their beliefs).
No. Even construing his argument in the kindest possible light, I can't find a way for it to be sensible. If he meant "we sometimes don't act like others are conscious", he should have written that. What he actually wrote is completely backwards from reality, and shows him not to be a serious thinker.
I don't like arguments like this, because instead of attacking the core premise they attack defenders of the idea. There are different points of view, and picking one for a sake of an argument doesn't make a sociopath from anyone.
I agree, that he misses one thing: to claim that someone/something is conscious we need to believe that they have abilities to feel, maybe to reflect, maybe something else. The exact list of abilities to have is not known for me and different people would propose different lists, but still we can't claim rock to be consciousness just because we care about it.
Still I forgive him for it, because he is talking about LLMs, not about rocks. They have some abilities, and he claims that they are enough, what is missing is us caring for LLMs enough.
> If he meant "we sometimes don't act like others are conscious", he should have written that.
No, I don't think he meant to say this. I believe he talked about how society forms social conventions who is conscious and who is not. You see, I don't know the cashier you talk about, but I believe they possess consciousness. Why? Because they are human, and our social conventions say that every (non-sleeping) human is conscious. It is a simple rule of thumb at work, no deep philosophy or psychology involved. But were those social conventions came from? Who and how decided that every human is conscious? Why? But the story didn't just end at that decision, because now people debate about consciousness of animals and LLMs. And these debates make me (and the author of the article) think that "consciousness" in these debates is a buzzword like phlogiston was in thermodynamics before thermodynamics was invented.
What's the correct definition of consciousness?
Human beings are notorious for being nonchalant about other's pain when they are insensate to it.
That guy has acute Megalomaniac Solipsism Fever in disguise.
--------------
Will we recognize AGI when it's finally made? Will we go from something that is not conscious to something that clearly is in one clean step, or will we murder millions of artificial lives while iteratively developing from the former to the latter? Will the tests we develop even work, or will AGI be alien enough that it takes us time (and a lot of murder) to recognize it for what it is?
I personally don't think LLM's are capable of consciousness, but I also can't prove they aren't.
Just because you can't prove an assertion doesn't mean it's wrong.
Also AGI almost certainly does not equal able to suffer. For example animals can suffer but would probably not meet the definition of AGI. we could create things that feel suffering even if they do not meet the definition of AGI.
So a better question might be: “can we detect if we create a system that can suffer?” And just drop the AGI part entirely. Then that gets us back to defining “suffering”. Agreeing on the definition always seems to be the hard part…
If we make AI that resists being terminated then the end result may not be significantly different from an entity that can suffer.
How many followers on instaX would they need to have for folks too truly believe it’s theRealJesus returning to save us.
It might sound silly, but at this stage, the belief in a benevolent AGI or Jesus are very similar. In both cases its us delegating our responsibilities to a third party and us hoping that third party will fix our problems for us.
Thankfully Hollywood invented happy ends for just such a scenario.
So if we are concerned with the AGI suffering, then we don't need to determine consciousness at all. We just need to prove that they don't have suffering capabilities. Which is a way easier question.
My daughter said she 'suffered' when I didn't get her ice cream (I know, I'm a monster). She didn't really suffer, but her behavior was as if she was.
So yes/no suffering is only part of the equation. Building systems that act/react like their suffering could be even more dangerous.
Wait what? Why do you believe she didn't actually suffer?
Instead, most of them insist we must accelerate this, do it more, give everyone hundreds of personal digital slaves.
It boggles the mind.
Never forget that LLMs are just next token predictors. The harnesses are what's getting uncanny. And credit/blame assignment? Just assign a surrogate for now, let them deal with the consequences ffs.
I've been calling LLMs digital intelligence. That's in my opinion what they are and on the digital/conceptual realm they are better than humans since LLMs are better generalists.
But I'd never state their conscious. For me consciousness extends outward from myself, since I can't trust anything else. It starts with the question like: why can't I control the movement of another person? Why can't I transfer my mind to another body and theirs to mine? Why is it that whenever I sleep, and similar things, I end up back to this place called "reality"?
I come to the conclusion that I am conscious and I'm a conscious being in reality. I am human and I've been raised by humans. Now, technically, all humans besides me could be zombies. They could be non-conscious beings that simply can act like humans. I'm sure we'll be able to create them in a few decades. However, I'm incentivized for multiple reasons to believe they're not zombies. Therefore humans are conscious too. All of them. Can I know for sure? No since I can't even trust my own senses fully or my own thoughts (the whole Descartes thing) but I choose to trust those things too because I need some reasonable-ish foundation to work from.
If humans are conscious, why aren't animals like us? Surely great apes must be conscious at least to some extent. Hell, some of them even have better short-term memory than us [1]. Well, turns out, from an anatomy standpoint they look a lot like us. Fine, let's assume they're conscious too.
Now we get into the territory that anything that has a large enough brain must be conscious because that's how we govern our consciousness. So you can extend this to all kinds of living beings.
I did this from my perspective. Of course, you should do it from your perspective. The same argument could be made.
For as long as we can't manipulate conscious experience in some way shape or form, we have no clue whether anything that isn't like us can be conscious. We'd need to be able to merge and split conscious experiences or we'd need to be very strongly able to understand why that isn't possible. And I don't mean just at the biological level but at the experiential level. Currently conjoined twins, and what they tell us gives some insight (being able to sense the other part that's part of the other twin).
Other than that, I claim we have no clue what consciousness is other than that we're experiencing it. So to call LLMs conscious is way too big of a claim. But they are definitely intelligent because they come up with things that I wouldn't have and it's useful to me. Perhaps a practical characterization of intelligence but it works for me.
I think for anyone to say something useful from this you'd need the trifecta of deep neuroscience knowledge, deep philosophy knowledge and (at least) a strong understanding of how LLMs work.
It's good the 'just token predictors' crowd is starting to pivot and hedge though lol. 'It's just the harness!' uh huh. give it a couple more months and you'll be in here regurgitatin' Gordon Wood
LLMs are literally token predictors. You can debate over the "just", but we haven't moved onto some new text generation technology, we're just expanding that one. I don't understand why you'd scoff at facts.
Predicting the next token is an immensely powerful thing. You can encode basically anything into a sequence of tokens, any configuration of matter. If you have a perfect next token predictor, you can predict basically anything. You can simulate an entire universe. It's Turing-complete. It sounds diminutive, but it's a general paradigm. Given everything this paradigm appears to be capable of, well, why not consciousness, really?
Agreed.
Now just take that one extra logical step please
Word fragment generation is the one-and-only thing llms have ever done.
LLMS: Receive text/binary input, process it, and output text/binary input Humans: Receive various inputs, process them, and output muscle contractions
https://plato.stanford.edu/entries/consciousness/#ConCon
>observer-dependent model, subject to social consensus.
Which is actually more pragmatic than most conflicting philosophical definitions involving qualia and other handwavy concepts.
Also, saying "I define consciousness as X" and then going on to show that, because I decree consciousness to be X, we don't need to worry about Y, Z, and so on, isn't pragmatic at all, unless you pick an X that everyone agrees on. The definition you give is something hardly anyone would agree on.
Social consensus probably isn't even an aspect of consciousness, but at least one can make that argument coherently, if you rephrase it as "part of the decision to call something 'conscious' or not involves social consensus". But this is also trivial and boring and not why anyone actually cares about the "is AI conscious" debate.
I agree that the article is all over the place, though. (not that my posts are much better)
In fact, if you stop talking about consciousness, and go through all the various aspects of consciousness, the whole debate gets very interesting. There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious, other ways in which AI already very clearly is missing other aspects, and then again some remaining ways which are unclear, and which may forever remain unclear.
Being clear about the different ways in which AI is, isn't, and might be conscious is a useful discussion and we still need to be having it.
EDIT: An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.
Yet all science is social, until you convince other people to take your ideas seriously, you are just another crank/loony.
How is social constructionism thoroughly discredited? Wikipedia shows some people poking with nits that don't look serious to me at all.
It is hard to provide a citation for these points because these points are so philosophically obvious that few waste time spelling them out, but searching for "hard/global constructionism" or "hard/global relativism" (on e.g. Google Scholar), or asking an AI about problems with these views I hope will make it clear why they are dumb, if it isn't already obvious to you.
Soft forms are of course totally fine. But to say "conscious is only and exactly whatever we say it to be" is hard constructionism / relativism, so completely unserious. Very obviously there are aspects of consciousness that do not merely depend on what we believe or say about it, even if some social constructionist aspects are also involved.
[1] https://plato.stanford.edu/entries/social-construction-natur...
[2] https://plato.stanford.edu/entries/relativism/#RelAboTruAleR...
Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false? That sounds like social acceptance of an idea.
>There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious
Which ways and how can it be demonstrated experimentally?
>An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.
That's not what the consciousness debate I see is about. The signal propagates and causes the reaction, but do they feel pain? What is the threshold? Is it just the complexity of the state? How complex it needs to be?
I am saying that, if you reject radical social constructionism (or solipsisms of various flavours, i.e. we are all just plugged into the matrix or some kind of simulation), then you must believe there is a real world out there independent of our consciousness / awareness of it. There must also be patterns / structure in that world, some of those patterns and structures being essentially constant, and those are what we call "true" (objective) facts about reality. BUT if you are a radical social constructionist or solipsist of some kind, then it can be perfectly coherent to reject any idea of objectivity or truth, except as contingent on social consensus (or personal beliefs, in the solipsistic case). Radical social constructionism being "discredited" is just a recognition that it basically forces you rapidly into incoherence.
> Which ways and how can it be demonstrated experimentally?
Try reading through the SEP article I linked above (https://plato.stanford.edu/entries/consciousness/#ConCon), particularly sections 2, and then especially section 4. You'll see you don't even need to run experiments to answer certain questions, you can just logically prove there are certain aspects of consciousness LLMs can't have, or clearly already have. Others you'll see immediately need qualifying or more information to answer. EDIT: Or just ask any AI to list the ways in which LLMs can, cannot, and might be able to exhibit the various aspects of consciousness. It will give you a decent answer, likely in a pretty little table.
See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen. You'd have to claim that the activations during inference are the dynamic component, but then that means LLMs are conscious only in the few seconds or even milliseconds during inference. Likewise, they don't have memory, so every inference is this consciousness being born anew, as it were, at least in current models.
You can of course go much further on even this specific point, but, the point is, there is lots of useful discussion and thinking to be had when you go through the aspects of consciousness in this way, and definitely at least some clear answers.
EDIT:
> That's not what the consciousness debate I see is about.
I just mean there are some similarities between the lobster consciousness debates and AI consciousness debates. Obviously the stakes and many aspects are different, I was just trying to illustrate why and how discussion and experiments can be useful and consequential here.
So consciousness = neuroplasticity? Requires neuroplasticity? Sounds pretty arbitrary, just like the entire link is an arbitrary set of ill-defined criteria. Besides, LLM weights are far from frozen if you look at the entire process in its specific time projection.
In fact the entire definition of a dynamic flow in that article is so vague that you can apply it to an industrial arm guided by a few dynamic equations, as it puts no definite threshold on the complexity of the process or behavior.
Dynamicness / change is one aspect that is definitely part of what we tend to think of as being quite fundamental to consciousness, and there are a lot of ways in which LLMs can't and/or don't really change, and this strongly constrains how similar LLM consciousness can/could be to ours.
If the SEP isn't enough to trigger the thinking needed for you, just ask an LLM what I said, to "go through the ways in which LLMs do/don't/may exhibit phenomena that are generally what we consider fundamental aspects of consciousness". You'll see there are some clear answers, and some uncertainties and vagueness.
I doubt we agree on what "rigorous definitions" means, given that you are happy to continue using a useless, vague family resemblance concept like "consciousness", but complain that pointing out dynamicness and/or change is involved is too vague.
I believe one cannot give a rigorous definition of consciousness precisely because it is a family resemblance concept, and there is no actual essence. This is why I suggest shifting to talking about aspects, since then productive insights and truths can be discovered.