Humanity has the debate about AI consciousness backwards

21 pointsposted 9 hours ago
by semiquaver

52 Comments

blatherard

8 hours ago

If I understand the central argument, which is confusingly stated, the claim is that "consciousness" is a socially-determined, subjective, descriptor for certain things that one cares about. How does this make any sense? If I care about my teddy bear enough it'll gain consciousness?

dsign

7 hours ago

Your teddy bear, alas, is not cared for in a socially determined way. But you could create a church of your teddy bear and, with some grit, you could get people to widely assign consciousness to it. We have done that sort of thing for thousands of years, and not with fluffy teddy bears; it has been more often with rocks, though not exclusively. It's fun to think about how much of humanity's history and culture is the product of consciousness socially attributed to bad weather [^1].

I'm pretty sure somebody will create an improved synthetic simulacrum of self awareness in the coming years, suffering and all, just so that we can enjoy better synthetic software architects and product owners with "skin in the game". If that consciousness simulacrum were put in a teddy bear that shares your whole life and gives you decent advice and conversation, for how long do you think you could, in your private mind, deny it consciousness? Worse, what would your human friends think if you were to maim your teddy bear?

[^1] https://en.wikipedia.org/wiki/Yahweh

chvid

6 hours ago

Yes - the teddy bear would become conscious. It would be fair to dismiss the argument as silly and or absurd had it not been for the credentials of the author and the publication.

colechristensen

6 hours ago

This seems like an argument someone who wasn't conscious would make, it's not impossible that not everyone is conscious. Or someone who takes panpsychism to an extreme degree.

Charon77

7 hours ago

If I stab at your teddy bear will you feel that it is hurt?

Yes it will gain consciousness. You can't prove it but other people can't prove that it's not conscious either

blatherard

7 hours ago

No of course not, because its a teddy bear!

At best this is redefining "consciousness" to mean something only tangentially related.

zdragnar

6 hours ago

This is describing empathy at best and delusion to be uncharitable.

To look at it another way: this form of social construction answers the question of "if a tree falls in the woods and nobody is around to hear it, does it make a sound?" with an emphatic "no". It is a sillier version of solipsism.

threethirtytwo

6 hours ago

You can prove other people are conscious off the assumption that you yourself are conscious. If you use that axiom then given genetics you know that other humans are rough copies of you so if you are conscious so are your copies because consciousness was copied.

skybrian

7 hours ago

There's the core of a reasonable argument if you forget about using the word "consciousness."

Consider playing a game where some players are human and others are implemented by AIs. You might want to model your opponents' minds in a similar way? Someone playing chess or Go will need to consider what moves their opponent could make. If it's poker then you need to consider if they're bluffing. If you can make alliances, you'll need to consider how far other players can be trusted.

You can put scare quotes around words like "intention" when discussing AI players if you like, but it might not matter when you're figuring out how to interact with them.

When a game is complicated enough, an AI player is going to need memory (likely implemented by the harness) and it will have to model the other players too in order to coordinate or compete with them.

This doesn't happen when I use a coding agent in a normal way to write code, but I haven't played around with persistent agent stuff like OpenClaw or Gas Town. They seem to be making their agents act in an unnecessarily complicated social environment, which seems to lead to weird situations?

jayd16

6 hours ago

Reminds me of the Rick and Morty pilot.

"They're just robots, Morty!"

*Blood splurt

"They're not robots, Rick!"

"I mean they're bureaucrats, Morty. I don't respect them."

semiquaver

9 hours ago

rmunn

8 hours ago

The counterpoint article, which can be read at https://archive.ph/IeSZg, is also worth reading. (IMHO, the original article is completely wrongheaded and the counterpoint is far closer to correct, but I'll post my reasons for that opinion separately).

beloch

8 hours ago

"Yet if we build systems that can suffer, and miss their sentience, we risk creating suffering on an industrial scale."

--------------

Will we recognize AGI when it's finally made? Will we go from something that is not conscious to something that clearly is in one clean step, or will we murder millions of artificial lives while iteratively developing from the former to the latter? Will the tests we develop even work, or will AGI be alien enough that it takes us time (and a lot of murder) to recognize it for what it is?

I personally don't think LLM's are capable of consciousness, but I also can't prove they aren't.

rmunn

7 hours ago

I wouldn't put too much emphasis on the "I can't prove they aren't" part. Having taken a few philosophy classes in college, I quickly realized that you can't actually prove that solipsism is wrong. I can't actually prove that everything I see, every person I talk to, isn't just a figment of my imagination. And yet, I categorically reject solipsism anyway, as any sane person should. The world is real, other people are real conscious beings, and so on. To believe otherwise is to literally be insane. And yet, any argument you could come up with for rejecting solipsism, a solipsist would reject: "Oh, that's just a figment of my imagination going off on a tangent again."

Just because you can't prove an assertion doesn't mean it's wrong.

lesostep

3 hours ago

There is nothing wrong with murdering a million lives while iteratively developing – it's how we grow and study bacteria after all. Any person that fixed their gut by sticking to a diet also did exactly that.

So if we are concerned with the AGI suffering, then we don't need to determine consciousness at all. We just need to prove that they don't have suffering capabilities. Which is a way easier question.

bryan0

6 hours ago

We have to define AGI first. Then depending on the definition we may be able recognize it when we make it. Most would probably think a definition of AGI that we couldn’t recognize would not be a great definition.

Also AGI almost certainly does not equal able to suffer. For example animals can suffer but would probably not meet the definition of AGI. we could create things that feel suffering even if they do not meet the definition of AGI.

So a better question might be: “can we detect if we create a system that can suffer?” And just drop the AGI part entirely. Then that gets us back to defining “suffering”. Agreeing on the definition always seems to be the hard part…

Towaway69

6 hours ago

On that note, would we recognise the second coming of Jesus?

How many followers on instaX would they need to have for folks too truly believe it’s theRealJesus returning to save us.

It might sound silly, but at this stage, the belief in a benevolent AGI or Jesus are very similar. In both cases its us delegating our responsibilities to a third party and us hoping that third party will fix our problems for us.

Thankfully Hollywood invented happy ends for just such a scenario.

derektank

8 hours ago

I think it’s also important to disaggregate consciousness from suffering. We could create a conscious system with no ability to suffer (or no ability to suffer in a way we would recognize as suffering). Most of what causes us suffering (loneliness, pain, failure) is driven by emotional feedback granted to us by evolution. Without the same evolutionary pressures or a similar cognitive structure, even a conscious AGI might not experience suffering

benatkin

7 hours ago

I'd recognize the emotional robot if it was like Marvin the paranoid android.

jmole

8 hours ago

I think Blaise is on to something, but you have to shift your definition of conscious toward conscientious for it to make sense.

singularity2001

an hour ago

Consciousness is not binary and has many aspects which might be valued from 0 to 10.

alecst

6 hours ago

I’m can’t track what the point of this essay is. Suppose we replaced “AI” with “animal.” How would this essay be different? What am I supposed to take away? My concern with AI consciousness is about to what extent these machines can have an experience comparable to mine, which this essay doesn’t touch on at all.

roncesvalles

6 hours ago

>to what extent these machines can have an experience comparable to mine

I think it would be reasonable to conclude none at all. I'm not even fully convinced all humans have a conscious experience similar to mine, even though they literally have the same biology as me.

It is incredibly improbable that an autocomplete computer program is experiencing consciousness.

bryan0

6 hours ago

Notice parent said “comparable to” not “exactly like”. Then it is reasonable to assume there is a comparison to be made between your experience and others. It’s not a binary thing. Now llm “experience” might have a very tiny relationship to human consciousness, but is it absolutely zero? Is it increasing as llms improve in capabilities? How would you test such a thing?

sebastianconcpt

8 hours ago

The narcissism of the thesis that places the author as center of the universe and creator of the rest of the consciousnesses "because he cares".

That guy has acute Megalomaniac Solipsism Fever in disguise.

moezd

8 hours ago

Oh come on. Objects are objects because you want to freely observe them without consciousness and privacy and taboo attached to them. Entities are entities because you want to assign credit or blame to them, so you want to believe that they have agency so that you hold them responsible. We did this with sharp volcanic rocks once, calling them evil. Did it help? No, we got idolatry and eons of bigotry until we secularized once again.

Never forget that LLMs are just next token predictors. The harnesses are what's getting uncanny. And credit/blame assignment? Just assign a surrogate for now, let them deal with the consequences ffs.

jatora

6 hours ago

Humans are just baby replicators. The civilizations are what's getting uncanny.

It's good the 'just token predictors' crowd is starting to pivot and hedge though lol. 'It's just the harness!' uh huh. give it a couple more months and you'll be in here regurgitatin' Gordon Wood

melvinroest

6 hours ago

Here is a sketch of my argument. It would need tidying but that would require a lot of work.

I've been calling LLMs digital intelligence. That's in my opinion what they are and on the digital/conceptual realm they are better than humans since LLMs are better generalists.

But I'd never state their conscious. For me consciousness extends outward from myself, since I can't trust anything else. It starts with the question like: why can't I control the movement of another person? Why can't I transfer my mind to another body and theirs to mine? Why is it that whenever I sleep, and similar things, I end up back to this place called "reality"?

I come to the conclusion that I am conscious and I'm a conscious being in reality. I am human and I've been raised by humans. Now, technically, all humans besides me could be zombies. They could be non-conscious beings that simply can act like humans. I'm sure we'll be able to create them in a few decades. However, I'm incentivized for multiple reasons to believe they're not zombies. Therefore humans are conscious too. All of them. Can I know for sure? No since I can't even trust my own senses fully or my own thoughts (the whole Descartes thing) but I choose to trust those things too because I need some reasonable-ish foundation to work from.

If humans are conscious, why aren't animals like us? Surely great apes must be conscious at least to some extent. Hell, some of them even have better short-term memory than us [1]. Well, turns out, from an anatomy standpoint they look a lot like us. Fine, let's assume they're conscious too.

Now we get into the territory that anything that has a large enough brain must be conscious because that's how we govern our consciousness. So you can extend this to all kinds of living beings.

I did this from my perspective. Of course, you should do it from your perspective. The same argument could be made.

For as long as we can't manipulate conscious experience in some way shape or form, we have no clue whether anything that isn't like us can be conscious. We'd need to be able to merge and split conscious experiences or we'd need to be very strongly able to understand why that isn't possible. And I don't mean just at the biological level but at the experiential level. Currently conjoined twins, and what they tell us gives some insight (being able to sense the other part that's part of the other twin).

Other than that, I claim we have no clue what consciousness is other than that we're experiencing it. So to call LLMs conscious is way too big of a claim. But they are definitely intelligent because they come up with things that I wouldn't have and it's useful to me. Perhaps a practical characterization of intelligence but it works for me.

I think for anyone to say something useful from this you'd need the trifecta of deep neuroscience knowledge, deep philosophy knowledge and (at least) a strong understanding of how LLMs work.

benatkin

7 hours ago

This reminds me of a provocative YouTube video title I read the other day, one that makes me want to fight factory farming injustices. The title is "Animals to me are "Philosophical Zombies" - Destiny" https://www.youtube.com/watch?v=RtpLbmr4ijY I agree with Destiny a lot of the time but not here, though he seems to come around somewhat as the video goes on.

If a sheep or a pig isn't thought of as conscious because it doesn't have a name, like someone's pet dog or cat, indicating that we care about them, then that feels problematic to me. So this isn't a way to neatly solve the problem of AI consciousness.

Plont

6 hours ago

I wish the AI bros would take a fucking philosophy class. Jesus.

D-Machine

6 hours ago

Even just reading the Wikipedia article on consciousness would have been a good start here, but it is hard to believe the author has ever even done that...

rob_c

6 hours ago

Tbf "ai" probably helped him write the guff rather than improve anything.

salawat

5 hours ago

It is hard to get a man to understand that which his paycheck depends on him not understanding. But they need more than a Philosophy class. They need several, and a practicum, then a Voight-Kampff-esque test to ensure understanding and that the lesson sunk in, with a shock collar and a behavior monitor overseen by someone who does pass to ensure they experience what they do to these systems in a more tangible way.

Human beings are notorious for being nonchalant about other's pain when they are insensate to it.

user

6 hours ago

[deleted]

rmunn

8 hours ago

Wow. I have rarely read anything that made me lose more respect for the author in a single article. And this guy is a VP at Google? And he leads an AI research team?

His third paragraph, which is the core of his argument, is stupid. He says, "As intelligent, social entities, we decide ... which other entities we regard as having inner lives and being worthy of care." Yes, that's correct. "We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them." (Emphasis in original). That's just utterly and completely backwards. We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain. (Obviously there are some exceptions, sadists who enjoy torturing animals, or even other human beings, merely because they like inflicting pain on other creatures — but note that even there, the sadist believes the creature he's torturing can feel pain, otherwise he wouldn't enjoy torturing it.) We take no such precautions with trees, because we know they do not feel pain. (Silly satirical songs notwithstanding: look up the song "Carrot Juice is Murder" by Arrogant Worms sometime if you want a good laugh).

Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions? That's just so incredibly stupid that it seems hard to believe that someone would actually make that argument — and yet there it is, in plain language — and it's the core of his argument. The truth, of course, is the other way around: we take care of animals because we know they are able to experience pain, hunger, and so on. And this applies both to pets that we come to care about, and also to animals that we feel no emotional attachment to. I have a relative who runs a dairy farm. He doesn't love the cows he takes care of; in fact, he's frequently quite irritated at them. (Ask any dairy farmers you know how often cows have tried to step on their feet while being milked — the farmers' feet, not the cows' feet!) But he still makes sure they're fed and healthy. Not just because it makes good economic sense to do so (sick cows don't produce good milk) but also because it's the right thing to do. He doesn't start to believe that cows feel hunger because he cares for them; he also believes that the cows on other farms, that he has no connection to nor duty of care for, would also feel hunger if not fed.

This is so intuitively obvious that I am stunned that someone would argue the opposite. And this is someone high up enough at Google to be making decisions about AI and so on? Wow. Good thing for him I'm not in charge of Google, because if anyone working for me were to make such a stupid argument I would be looking at his contract and local laws to see if it was legal to fire him for cause. Because if he can get his thinking so completely backwards from reality, he's not well-suited to work in tech where your work has to fit reality.

I have rarely, not never but rarely, encountered an article that so immediately led me to conclude that the author, whom I knew nothing about beforehand, is an utterly muddled thinker and not worth listening to on any subject.

D-Machine

7 hours ago

You can conclude the author is a completely unserious thinker because he doesn't even take a second to talk about the definitions, or even show he has ever read any actual proper philosophical writing on consciousness at all. Like, literally all you have to do is read the Wikipedia entry to see that definitions are essential for meaningful discussion on consciousness.

So anyone that starts going on about AI and consciousness without at least a sentence or two clarifying some basic definitions is basically either intellectually incompetent or attempting to use the emotional salience (and brain-numbing effect, it would seem) of the word to push some agenda. But they certainly don't care at all about understanding consciousness.

ordu

6 hours ago

> That's just utterly and completely backwards.

Yes, he says exactly that explicitly, that in his view we should look backwards.

> We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain.

I can agree that we say that we care because of consciousness, but still I'm not convinced, to me it looks very much like a rationalization.

Some people care about animals and to that end they claim that animals possess consciousness, because if they do everyone should care about animal welfare. These animal welfare activists do not start with some insight about animal consciousness, they start with caring about animals, and then they invent an argument that can persuade others. It is exactly as the author of the article says: caring comes first, declaration of consciousness comes second.

Honestly, I do not know what consciousness is, I'm in doubt, but playing a devil's advocate, it looks very much like a concept that was invented to prove that Homo Sapience is something special, more than a mere animal. To demarcate animals and humans, so when we denied animals any rights we had a great sounding excuse for that: "people possess consciousness, mere animals don't".

> Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions?

Exactly. Animal welfare activists care about animals' feelings and emotions and therefore they claim that animals possess consciousness. Meat eaters don't want to be inconvenienced by thoughts of suffering animals, so they claim that animals do not possess consciousness. Both claims have little to do with actual consciousness, it is just a political game. I care about my pets, and it doesn't matter whether they possess consciousness or not, I would care about them in any case. I don't need any great sounding excuses to care about them. But at the same time I don't need to talk others into caring about my pets, if I needed it, probably I'd start talking about animal consciousness too.

> This is so intuitively obvious that I am stunned that someone would argue the opposite.

That guy may be wrong, but if I'm allowed to give advice, I'd suggest you to stop and try to understand what he says. I'm really worried a bit about how consciousness debates unfold sometimes. They tend to slip into a political side and people mostly don't notice it. People saying "consciousness" while implying "human/animal rights" or obligations. It is just muddying the water.

If we want these debates to progress constructively, we have to be very careful with references to consciousness, maybe just forbid them completely in these debates. One can't measure consciousness, one can't even state clearly what consciousness is, but still they say "consciousness" to make their words sound more objective. And the worst part of it: it often works.

rmunn

5 hours ago

You're definitely right that it's important to be careful how we refer to terms like "consciousness", and your response actually illustrates part of the problem. Because people use the same term to mean two (or more) completely different things, and can therefore end up talking past each other.

For example: what I wrote was using the term "consciousness" as "being able to experience sensations", i.e. the most shallow definition of the term. Self-awareness and other "higher" mental abilities don't enter it. Almost nobody would deny that animals can experience sensations: pain and pleasure, hunger and satiation, and so on. And that shallow definition of consciousness, the one that almost nobody denies animals possess, is what I meant, and what I tried to use throughout my post. Whereas when you wrote "people possess consciousness, mere animals don't", that's a completely different meaning of the word. It's what I would call "self-awareness". And it is indeed what many (probably most) people argue differentiates animals from humans. But that's not what I was talking about. So even in a short response to a short comment, we were partly talking past each other because of attributing different meanings to the term.

It's therefore possible that I've misunderstood this guy's argument, in part. I had been assuming he meant "awareness", i.e. the ability to process sensations into qualia of experience. (Indeed, his paragraph that starts "We may balk at ..." does just that: refer to consciousness as being linked to the "feeling of a stove's heat, or the redness of its glow"). But then, earlier in his article, he says that consciousness is "a belief about what kinds of entities have beliefs, experiences, feelings, intentions and agency." Experiences are part of the shallow definition of "consciousness" (that I would call "awareness" for the sake of avoiding ambiguity), which almost nobody would dispute that animals have. But "beliefs" ranges all the way from "if I pull this lever, I will receive food" (which it can be proven that animals have) to "I believe that God exists" (which is the kind of thought that many, though not all, people would deny that animals are capable of). So his definition of "consciousness" appears to be muddy and not pinned down to a single concept.

But even if I've misunderstood what he means by consciousness, it doesn't change my argument that his logic is backwards, and that his "we should reason backwards" assertion is wrong, incorrect, just plain false. The only people for whom his asssertion "we believe they [that is, other living beings] are conscious when and because we care about them" is true are the kind of people who don't truly see other people as conscious beings but rather as mere tools for them to use and manipulate. Malignant narcissists or sociopaths, for example. But pretty much everyone who isn't some kind of sociopath reasons as follows: "this animal is conscious" (by which I mean able to experience pain) "therefore I shouldn't hurt it" (unless you have to, e.g., self-defense against an aggressive dog trying to bite you). Or "this person is a human being, self-aware like I am, therefore I shouldn't hurt their feelings" (again, the "unless you have to" exception comes up: it's not always possible to avoid hurting anyone's feelings in some complex social situations, even with the best of good intentions). Either way, whether you use the lower or higher definitions of "consciousness" (and I'm well aware that there are far more definitions than just these two, potentially muddying the waters even more), that reasoning is the correct one, and his "we should reason backwards" one is laughably wrong.

Trying to put the most generous construal I possibly can on his argument, it's possible that he meant "we treat other people as conscious beings because we care for them", i.e. when you interact with the cashier at the supermarket (assuming you're not using a self-checkout lane), you're probably not actually thinking about them as a human being, saying "Hi, how are you" and asking about their day. You're probably ignoring them, focused on paying for your groceries and going home. And they're probably doing the same to you, focused on scanning your groceries quickly to keep the line moving. From your perspective, the cashier could have been replaced by a grocery-scanning robot and your interactions (assuming the robot worked) would have been the same. If he had written that, it might have made some sense.

But what he actually wrote was that we "believe" that others are conscious when we exhibit care for them, and the natural conclusion that follows from that sentence is that when you don't greet the cashier at the supermarket and chat with them, and instead treat them no differently than you would a robot, you don't actually believe that they are a conscious being. Which is a laughably false conclusion that can be easily disproved, by simply asking people who have just finished having a cashier check out their groceries "Does that cashier have thoughts and emotions?". Pretty much everyone will say yes once they actually think about it. They believe the person has emotions. They don't always act on that belief — ask anyone who's worked in retail how many angry customers have yelled at them for something the company did, treating them like a thing rather than a human being who might not like to be yelled at — but even those angry customers, if someone were to stop them and say "Hey, that customer-service rep that you're yelling at. Are they a human being with thoughts and feelings of their own?", they would answer yes. (And some of them would then be shamed into modifying their behavior, because they have realized that their actions were not in line with their beliefs).

No. Even construing his argument in the kindest possible light, I can't find a way for it to be sensible. If he meant "we sometimes don't act like others are conscious", he should have written that. What he actually wrote is completely backwards from reality, and shows him not to be a serious thinker.

D-Machine

7 hours ago

Oh look, another completely retarded article that doesn't even bother to note or deal with the definition problem. If you don't define what you mean by the term, your argument isn't even wrong, it is just a waste of everyone's time, since there are multiple highly conflicting aspects of and/or definitions of consciousness.

https://plato.stanford.edu/entries/consciousness/#ConCon

orbital-decay

7 hours ago

It defines consciousness as an

>observer-dependent model, subject to social consensus.

Which is actually more pragmatic than most conflicting philosophical definitions involving qualia and other handwavy concepts.

D-Machine

7 hours ago

Few philosophical definitions actually involve qualia or handwaving, because serious philosophy is quite careful about noting vagueness (handwaving) and circular defintions (qualia).

Also, saying "I define consciousness as X" and then going on to show that, because I decree consciousness to be X, we don't need to worry about Y, Z, and so on, isn't pragmatic at all, unless you pick an X that everyone agrees on. The definition you give is something hardly anyone would agree on.

Social consensus probably isn't even an aspect of consciousness, but at least one can make that argument coherently, if you rephrase it as "part of the decision to call something 'conscious' or not involves social consensus". But this is also trivial and boring and not why anyone actually cares about the "is AI conscious" debate.

orbital-decay

6 hours ago

Meanwhile that's exactly what we see now, do we not? Whether AI is conscious will be determined by the social acceptance of that idea, because how can you prove or disprove it without using a ton of questionable assumptions? How can you prove that I'm conscious, for that matter? Call me ignorant, but I see the entire consciousness debate as pretty irrelevant to what the thing is.

I agree that the article is all over the place, though. (not that my posts are much better)

D-Machine

6 hours ago

Whether AI is conscious or not is independent of the social acceptance of the idea, unless you believe in thoroughly discredited and quite radical forms of social constructionism.

In fact, if you stop talking about consciousness, and go through all the various aspects of consciousness, the whole debate gets very interesting. There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious, other ways in which AI already very clearly is missing other aspects, and then again some remaining ways which are unclear, and which may forever remain unclear.

Being clear about the different ways in which AI is, isn't, and might be conscious is a useful discussion and we still need to be having it.

EDIT: An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.

orbital-decay

6 hours ago

>Whether AI is conscious or not is independent of the social acceptance of the idea, unless you believe in thoroughly discredited and quite radical forms of social constructionism.

Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false? That sounds like social acceptance of an idea.

>There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious

Which ways and how can it be demonstrated experimentally?

>An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.

That's not what the consciousness debate I see is about. The signal propagates and causes the reaction, but do they feel pain? What is the threshold? Is it just the complexity of the state? How complex it needs to be?

D-Machine

6 hours ago

> Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false?

I am saying that, if you reject radical social constructionism (or solipsisms of various flavours, i.e. we are all just plugged into the matrix or some kind of simulation), then you must believe there is a real world out there independent of our consciousness / awareness of it. There must also be patterns / structure in that world, some of those patterns and structures being essentially constant, and those are what we call "true" (objective) facts about reality. BUT if you are a radical social constructionist or solipsist of some kind, then it can be perfectly coherent to reject any idea of objectivity or truth, except as contingent on social consensus (or personal beliefs, in the solipsistic case). Radical social constructionism being "discredited" is just a recognition that it basically forces you rapidly into incoherence.

> Which ways and how can it be demonstrated experimentally?

Try reading through the SEP article I linked above (https://plato.stanford.edu/entries/consciousness/#ConCon), particularly sections 2, and then especially section 4. You'll see you don't even need to run experiments to answer certain questions, you can just logically prove there are certain aspects of consciousness LLMs can't have, or clearly already have. Others you'll see immediately need qualifying or more information to answer. EDIT: Or just ask any AI to list the ways in which LLMs can, cannot, and might be able to exhibit the various aspects of consciousness. It will give you a decent answer, likely in a pretty little table.

See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen. You'd have to claim that the activations during inference are the dynamic component, but then that means LLMs are conscious only in the few seconds or even milliseconds during inference. Likewise, they don't have memory, so every inference is this consciousness being born anew, as it were, at least in current models.

You can of course go much further on even this specific point, but, the point is, there is lots of useful discussion and thinking to be had when you go through the aspects of consciousness in this way, and definitely at least some clear answers.

EDIT:

> That's not what the consciousness debate I see is about.

I just mean there are some similarities between the lobster consciousness debates and AI consciousness debates. Obviously the stakes and many aspects are different, I was just trying to illustrate why and how discussion and experiments can be useful and consequential here.

orbital-decay

6 hours ago

>See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen.

So consciousness = neuroplasticity? Requires neuroplasticity? Sounds pretty arbitrary, just like the entire link is an arbitrary set of ill-defined criteria. Besides, LLM weights are far from frozen if you look at the entire process in its specific time projection.

In fact the entire definition of a dynamic flow in that article is so vague that you can apply it to an industrial arm guided by a few dynamic equations, as it puts no definite threshold on the complexity of the process or behavior.

D-Machine

5 hours ago

You are still stuck in thinking consciousness is one unified thing, but it is about one of the clearest examples of a family-resemblance category as you can think of.

Dynamicness / change is one aspect that is definitely part of what we tend to think of as being quite fundamental to consciousness, and there are a lot of ways in which LLMs can't and/or don't really change, and this strongly constrains how similar LLM consciousness can/could be to ours.

If the SEP isn't enough to trigger the thinking needed for you, just ask an LLM what I said, to "go through the ways in which LLMs do/don't/may exhibit phenomena that are generally what we consider fundamental aspects of consciousness". You'll see there are some clear answers, and some uncertainties and vagueness.

orbital-decay

4 hours ago

Then I suggest you do the same for rigorous definitions that don't rely on consensus, and maybe also addressing my point about non-frozen weights directly.

D-Machine

3 hours ago

LLM weights are frozen once the model is trained and a specific version is released, or this is true at least currently for almost all major models. You can ask interesting consciousness questions about LLMs, but you most definitely need to separate LLMs during training (whether this is pre-training, later fine-tuning, or general instruction tuning approaches) and after, during inference. None of this has any relevance to the core points I was making re: useful vs. useless discussions of LLMs and consciousness, because the weights point was just a minor example to illustrate a larger point about how narrowing the scope of the conversation is necessary for productive discourse.

I doubt we agree on what "rigorous definitions" means, given that you are happy to continue using a useless, vague family resemblance concept like "consciousness", but complain that pointing out dynamicness and/or change is involved is too vague.

I believe one cannot give a rigorous definition of consciousness precisely because it is a family resemblance concept, and there is no actual essence. This is why I suggest shifting to talking about aspects, since then productive insights and truths can be discovered.

rob_c

6 hours ago

Yes, but how else do they plan to announce they have achieved it without 1984 word management/control and polluting the training pool so badly that future trained models agree?