> What is it like to be an LLM?

That's a question I actually asked myself.

From the point of view of a LLM, words are everything. We have hands, bats have echolocation, and LLMs have words, just words. How does a LLM feel when two words match perfectly? Are they hurt by typos?

It may feel silly to give LLMs consciousness, I mean, we know how they work, this is just a bunch of matrix operations. But does it mean it is not conscious? Do things stop being conscious once we understand them? For me, consciousness is like a religious belief. It is unfalsifiable, unscientific, we don't even have a precise definition, but it is something we feel deep inside of us, and it guides our moral choices.

Nagel's "What is it like to be a bat?" assumes that bats are conscious, and that the question of what is the subjective experience of being a bat (e.g. what does the sense of echolocation feel like) is therefore a meaningful question to ask.

The author inventing "batfished" also believes bats to be conscious, so it seems a very poorly conceived word, and anyways unnecessary since anthropomorphize works just fine... "You've just gaslighted yourself by anthropomorphizing the AI".

I love this, hope it takes off like "enhsittification" or "slop" have already.

None of us have even experienced the full range of what humans can experience, so even we don't fully know what it is like to be any given person, we only know what it is like to be ourselves. It is kind of amazing when you think about it.

> Human beings can, in fact, learn to echolocate, and they seem to experience it as vision

Sure - although depending on how quickly one was scanning the environment with echolocation it might also feel a bit like looking around a pitch black room with a flashlight.

In any case it's essentially a spatial sense, not a temporal one, so is bound to feel more like (have a similar quale to) vision than hearing.

It makes sense given that we are just receiving light waves instead of sound. Light waves contain way more information but your brain would still come up with some kind of "visualization" based on the info. I didn't listen to the program but he might just see blobs of varying sizes instead of any kind of detailed image.

The Ethiops say that their gods are flat-nosed and black, While the Thracians say that theirs have blue eyes and red hair. “If oxen and horses and lions had hands and were able to draw with their hands and do the same things as men, horses would draw the shapes of gods to look like horses and oxen would draw them to look like oxen, and each would make the gods' bodies have the same shape as they themselves had.

- Xenophanes, ~500 BCE

<https://www.azquotes.com/author/38174-Xenophanes>

I like your point that translation isn't just technical, it's political. Every attempt to "bridge the gap" shapes power as much as it conveys meaning

I believe you're falling into a purely linguistic trap. In other languages we wouldn't even use the word "like" in this kind of constructions, that's an English thing because other wordings sound awkward, but I don't think it entails comparison.

In translations to Spanish, the article is titled "¿Qué se siente ser un murciélago?", literal word by word translation "What is felt being a bat?"

In French, "Quel effet cela fait-il d'être une chauve-souris?", literal word by word translation "What effect it makes to be a bat?"

In Chinese, "成为一只蝙蝠可能是什么样子", i.e., "To become a bat could be what feeling/sensation?"

None of these translations has a comparative word. And at least in Spanish (I won't speak about the other two because I'm not so proficient in them), using a comparative expression similar to "being like" in English ("¿A qué se parece ser un murciélago?") would sound awkward and not really convey the point. Which is why the translators didn't do so.

Of course I know that the original article is in English, but I think the author basically meant "What is felt being a bat?" and just used the "like" construction because it's what you say in English for that to sound good and clear. Your highlighted text could be rendered as "An organism has conscious mental states if and only if there is something that is felt being that organism – something that is felt by the organism." and it would be more precise, just doesn't sound elegant in English.

This is the conclusion I come to whenever I try to grasp the works of Nagel, Chalmers, Goff, Searle et al. They're just linguistically chasing their own tails. There's no meaningful insight below it all. All of their arguments, however complex, all rely on poorly defined terms like "understand" "subjective experience", "what it is like", "qualia", etc. And when you try to understand the arguments with the definition of these terms left open, you realise the arguments only make sense when the terms include in their definition a supposition that the argument is true. It's all just circular reasoning.

The way I understand it the second thing is the observer of the organism, the person posing the question. The definition seems to be sort of equivalent to the statement "an entity is conscious IFF the sentence 'what is it like to be that entity' is well-posed".

"What is it like to be a rock" => no thing satisfies that answer => a rock does not have unconscious mental states

"What is it like to be a bat" => the subjective experience of a bat is what it is like => a bat has conscious mental states

Basically it seems like a roundabout way of equating "the existence of subjective experience" with "the existence of consciousness"

edit: one of the criticism papers that the wiki cites also provides a nice exploration of the usage of the word "like" in the definition, which you might be interested to read (http://www.phps.at/texte/HackerP1.pdf)

> It is important to note that the phrase 'there is something which it is like for a subject to have experience E' does not indicate a comparison. Nagel does not claim that to have a given conscious experience resembles something (e.g. some other experience), but rather that there is something which it is like for the subject to have it, i.e. 'what it is like' is intended to signify 'how it is for the subject himself'.

There is no fundamental error it’s purposefully exactly as you state. Nagel is saying that consciousness is that second thing.

Do you believe that each run of a ChatGPT prompt has a conscious experience of its existence, much like you (presumably) do?

If you don't believe that, then you face the challenge of describing what the difference is. It's difficult to do in ordinary language.

That's what Nagel is attempting to do. Unless you're an eliminativist who believes that conscious experience is an "illusion" (experienced by what?), then you're just quibbling about wording, and I suspect you'll have a difficult time coming up with better wording yourself.

The “something” here refers to inner experience (something similar to Kantian “aperception”.

The tricky bit is that “to be” is not an ordinary verb like fly, eat, or echo-locate. And “‘being an organism’” is — in the context of the paper — about subjective experience (subjective to everything except the organism.

To put it another way, the language game Nagel plays follows the conventions of language games played in post-war English language analytic philosophy. One of those conventions is awareness of Wittgenstein’s “philosophical problem”: language is a context sensitive agreement within a community…

…sure you may find fault with Wittgenstein and often there are uncomfortable epistemological implications for Modernists, Aristotelians, Positivists and such…then again that’s true of Kant.

Anyway, what the language-game model gives philosophical discourse is a way of dealing with or better avoiding Carnapian psuedo-problems arising from an insistence that the use of a word in one context applies to a context where the word is used differently…Carnap’s Logical Structure of the World pre-dates Wittgenstein’s Philosophical Investigations by about 25 years.

Nagel's question "What is it like to be a bat?" is about the sensory qualia of a bat, assuming it has consciousness and ability to experience quales, which he assumes it does.

The question is not "What would it be like (i.e. be similar to) to be a bat?" which seems to be the strawman you are responding to.

I think that's why he states it as a biconditional, which makes the exclusive restriction you're arguing is necessary

That particular phrasing happened to catch on, but I don't think it's essential to any of the arguments. How would you phrase the distinction between objects that are conscious and objects that aren't? Or are you saying that that distinction is just a verbal trick?

It might be one of those cases where the philosophical hook works because of its rhetorical power

Our hearing is more sensitive than we give it credit for being.

We used to play a game in my dojo where we'd toss a "knife" (wooden) around in a circle, as if sending it to an ally. Yes, you could throw it in a way that can't be caught... but then the person picks it up and chooses how to throw it back. You learn quickly not to be a dick about it.

Naturally, we were trying to get better at catching, and eventually moved to a trust version, where I shout "Go!" as I throw the knife. Turns out you can hear the knife leaving someone's hands more reliably than they can shout "Go!" on time.

What brought down your level on convinced?

>Anything that is able to be measured is able to experience.

I was quite liking this explanation but you lost me here. I very strongly agree with your opening, and I think it's the key to everything. I think everyone insisting on a categorical divide runs into impossible problems.

And a good explanation of consciousness has to take the hard problem seriously, but doesn't have to agree that subjective and objective, or first person in third person or whatever you want to call them, are irreducibly distinct categories. But I think it makes more sense to say that some subset of all of the objective stuff out there is simultaneously subjective, rather than saying that all stuff at all times is both objective and subjective. I don't think an apple experiences gravity the way a mind experiences a conscious state, but I do think the through line of understanding them both as importantly physical in the same sense is key to tying physical reality to explanation of conscious states.

> Sensory deprived, paralyzed, or comatose individuals can be conscious but have no means to experience the outside world, and depending on their level of brain activity, they might not even have an "inner world" or mind's eye experience.

If they don't have an "inner world"/"mind's eye" and are sensory deprived, in which sense can they be considered conscious? What is your definition here?

How can an apple "experience" gravity? I think you're overloading the term "experience" to mean two very different things, which happen (in some languages like English) to share the same word. You could say gravity "happens" to an apple, and then there's no confusion with subjective experiences.

But I still don’t know what it’s like to be a bat

Also, if there is a soul, then how can we be confident concisouness arises from physical means? If there is a soul, it is the perfect means to differentiate concisouness and p-zombies.

This is a spiritual error called monism. It's taking non-dualism too far.

> As there are no real difference between this rock and the other rock behind the mountain that I can't see.

There is a real difference between the two; there must be because they're in different places. Monism requires you to deny actually existing differences by saying they're not "real".

> There should be no difference between my thoughts, senses, feelings, emotions etc and that of other people.

This is what in therapyspeak you'd call "not having boundaries". You aren't the same thing as other people; you can tell because the other people don't think that, won't let you borrow their car, etc. It opens them or yourself up to abuse if you think this way.

>There's no "whats its like to be a bat". Because that invokes a sense of a "soul" or "spirit" or "self" being transferred from one being to another.

what's it like to be a human?

"There's no "whats its like to be a human". Because that invokes a sense of a "soul" or "spirit" or "self" being transferred from one being to another." -- anon-3988

it does?

> If you are a secular person, it should follow that you are a non-dualist.

It depends on your definition of "dualism". If you define it as "having a soul that was created by a higher being", then yes, they are mutually exclusive.

On the other hand, one can also define dualism as being purely evolutionary. David Chalmers [1], an Australian philosopher and cognitive scientist, has some interesting ideas around how dualistic consciousness may relate to quantum mechanics.

[1] https://en.wikipedia.org/wiki/David_Chalmers

Secular != Strict Materialist

There are far fewer of the latter than the former

IMO a lot of this comes from still holding onto the dualist idea of "I" as a separate non-physical entity, and then expecting some region of the brain to act as a link or communication channel to it.

> If I could experience other persons or beings in the first person, and the matter in each person explained why it is that I experience that specific person or entity, I might believe otherwise.

Materialism doesn't say that there's some "I" that could experience different persons. I think the best you could do, in theory, is transplant aspects like your personality/train of thought/memories into someone else's brain (by physically altering it to have those aspects).

Contrarily, the problem I have with non-materialist/physicalist explanations is that they don't really seem to explain anything.

If we assume dualism, that there is some non-material stuff -- call it soul or spirit or mind or psyche or whatever -- that gives rise to consciousness, I think it's fair to ask how it does that.

And if the answer is "we don't know" or "it just does", I really can't see what we've gained over materialism.

I'm open to there being something in reality that explains being someone or a specific person. Having a specific location in reality raises the question of how it came to be and what are the possibilities or limits - it's a piece of unexplained information that suggests more information and explanation is needed. Saying it is meaningless or assumed actually appears to not explain anything. But I would have to think about it more.

Materialism is indeed on shaky ground, but the ground shakes everywhere else, too.

The problem of consciousness has no real solution. A quick way to demonstrate this is via the simulation hypothesis. Consider the following for yourself in first person:

It's impossible to know for certain whether I am in a simulation until I wake up outside of it. Not having observed any evidence of being inside a simulation (probability=0) doesn't necessarily mean I'm certainly in base reality. It could be that the evidence just hasn't been observed yet. And even then, it's impossible to know whether that outer world is a simulation until I wake up in the outer-outer-world, and so on.

That is to say, if my definition of real equals my consciousness equals my existence, I'm really saying that consciousness/reality/existence is a self-defining thing.

Descartes' cogito had unexamined metaphysical convictions. "I think, therefore I am" is not compatible with consciousness because rationality has consciousness as a dependency. If I think back on my entire conscious experience as a timeline, I was conscious before I was rational. I had to derive rationality from experience, not the other way around.

Now I throw away those convictions. "I think" means the same thing as "I am", and "therefore" is a decorative force of habit rather than a reference to logic. In which case, "I think, therefore I am" is the same as: I observe that I observe.

Is the same as: I observe.

Is the same as: I am.

There is no certainty beyond this, only convictions. Even if I'm truly a human brain in a matter-based world, the world would still appear uncertain to this brain in this way.

"A scientist rejecting consciousness is not that different from a nun accepting god in this regard. Neither of them are fully honest with themselves and the world."

That's what I find myself thinking as I take a materialist stance and assume that this is base reality and other people are real in the same way that I am. This appears to fit all of my observations the best, so far, after all.

// end of monologue

And here's my pitch, from me to you:

Let's be provisional materialists together. You can't know if it's the ultimate truth but you can make the correct predictions more often and not be alone while doing it.

In this world "why" and "how" are synonyms. Asking why experience happens is equivalent to asking how experience happens.

It's unfortunate there are pseudo-intellectual trolls on hacker news.

[dead]

Wittgenstein famously said "If a lion could talk, we could not understand him". This subject is a philosophical fun rabbit hole to explore.

The very capability and flexibility of language drove evolution of the mind beyond what species with less linguistic behaviors could handle. After all, facility with language is a massive survival benefit, in our species more than any other. It’s circular because feedback loops in evolution are circular too.

That’s called meta cognition (what humans do) not subjective experience - which is the feeling of what happens and sets animal or agentic creatures apart from rocks (not sure about plants)

What is that criticism supposed to be criticizing?

Nagel's paper covers a lot of ground, but none of what you described has any bearing on the point about it "what it's like" as a way to identify conscious experience as distinct from, say, the life of a rock. (Assuming one isn't a panpsychist who believe that rocks possess consciousness.)

That's the magic answer. It's a/the hard problem, but permeable to inquiry. The top neuroscience research into consciousness however doesn't seem like this kind of inquiry Dennett is referencing.

Pretty much the same conclusion here. Consciousness is what we feel when sheaf 1-cohomology among our different senses vanishes.

Bringing it back to bats, a failure to imagine what it's like to be a bat is just indicative that the overlaps between human and bat modalities don’t admit a coherent gluing that humans can inhabit phenomenally.

Do you really mean that it's very nearly the same thing "To be a" you, and an Elon Musk, a homo sapiens infant, and an Orangutan? And only modestly different from these to be a dog or a horse?

If I've understood you correctly, I'll suggest that simple sensory intersection is way way not enough: the processing hardware and software are material to what it is like to be someone.

You would be adding bat experience (sonar, hanging upside down, flying etc) to become a literal Bat Man so to speak :)

But you would never know exactly what it feels to be a bat without removing your human level experience from the picture

I suppose someone (even an experienced vimmer) might argue that learning vim is not so much "umwelt-transformative", but rather like "muscle memory training", like LeetCode drilling.

Indeed, basic vim-navigation - (hjkl, w, b) is muscle memory.

But, I'd argue the umwelt shift comes from vim's modal nature and its language of text objects. You start perceiving text as having an inherent grammar - "inside parentheses", "around word", "until comma." Text gains topology and structure that was invisible before.

The transformative part isn't the keystrokes but learning to think "delete inside quotes" (di") or "change around paragraph" (cap). You see text as composable objects with boundaries, not just streams of characters. This may even persists when you're reading on paper.

That mental model often transforms your keyboard workflow not just in your editor - but your WM, terminal, web browser, etc.

Interesting topic, but I can only see one chapter. Based on chapter names it seems you lean closer to cybernetics or process philosophy. Is that true? I find ignoring time and process to be the greatest sin in the field of consciousness. The big issue is not that we don't know the quantum trick or property-dualism that explains consciousness, but that we try to remove time and process from it. That is impossible, a static explanation will never capture dynamic execution of a system.

I worked on similar topics, I publish on a "personal" subreddit.

https://www.reddit.com/r/VisargaPersonal/

An organism is conscious exactly when there is something it is like for that organism to be itself.

Or in a simpler way, consciousness is present just in case being that organism has an inner, subjective character - something that can not be reduced to a purely material state.

It means that if something has conscious mental states then it must have subjective experience from its own perspective. If John has a conscious mental state, then I must be able to ask "What is it like to be John?". Hope that helps.

I wouldn’t bother.

>Consciousness isn’t physical or measurable.

That's just a question begging assertion, and there's plenty of empirical knowledge of necessary physical conditions for consciousness as well as predictable physical influences on conscious states. Whether consciousness is "measurable" is part of what's at issue and can't just be definitionally presupposed.

> it operates on instinct without a personal, reflective self

I think we would call this "without ego" and not "without consciousness". I think it's totally possible to be conscious without ego. And perhaps bats do have an ego however small - some may be more greedy than others, etc.

Do you mean the bat has no subjective experience? If so - That’s a pretty extraordinary claim to make there and one that risks great ethical concern on the treatment or animals

If bats have no subjective experience it’s ethical to do anything to them but if there is than they deserve to (as all animals) be treated ethically as much as we can do so

IMO considering Bats to be similar to Mice -we’ve studied mice and rats extensively and while cannot know precisely we can be pretty sure there is subjective experience (felt experience there) ie almost our scientific experiments and field data with so called ‘lower’ organisms show evidence of pain, suffering and desires, play etc - all critical evidence of subjectivity

Now I don’t think bats are meta-conscious (meta cognitive) because they can’t commiserate on their experiences or worry about death etc like humans can but they feel stuff - and we must respect that

> Physicalism is about as plausible as solipsism.

Physicalism is an ontological assertion that is almost certainly true, and is adhered to by nearly all scientists and most philosophers of mind. Solipsism is an ontological assertion that could only possibly be true for one person, and is generally dismissed. They are at opposite ends of the plausibility scale.

> Physicalism is about as plausible as solipsism

And while you're at it, as plausible as any metaphysical theory, insofar as you're still doing metaphysics.

>Can we just all admit there has basically been no real progress made to the mind-body problem.

I think we've made extraordinary progress on things like brain to machine interfaces, and demonstrating that something much like human thought can be approximated according to computational principles.

I do think some sort of theoretical bedrock is necessary to explain to "something there's like to be" quality, but I think it would be obtuse to brush aside the rather extraordinary infiltrations into the black box of consciousness that we've made thus far, even if it's all been knowing more about it from the outside. There's a real problem that remains unpenetrated but as has been noted elsewhere in this thread, it is a nebulous concept, and perhaps one of the most difficult and important research questions, and I think nothing other than ordinary humility is necessary to explain the limit an extent to which we understand it thus far.

If anything, it's getting weirder... real progress looks, well, batshit insane. For example:

Against Mind-Blindness: recognizing and communicating with diverse intelligences - by Michael Levin

https://www.youtube.com/watch?v=OD5TOsPZIQY

Much of the mind-body problem comes from Descartes, who assumed that physical reality was nothing more than a bunch of particles bouncing around. Given that the mind cannot be reduced to this (whatever my experiences are, they are different from particles bouncing around), then the mind must be something utterly unlike everything else in reality. Thus Descartes posits that the mind is one thing and the body another (substance dualism).

If one drops the assumption that physical reality is nothing more than a bunch of particles, the mind stops being so utterly weird and unique, and the mind-body problem is more tractable. Pre-17th century, philosophers weren't so troubled by it.

I'm not sure how to answer the even more fundamental question, "What is it like to be yourself?" What constitutes a valid answer? It's a vague question.

It's more or less OK, thank you for asking. Recently I felt:

Disappointed when I went somewhere and there wasn't any tea,

Enthralled by a story about someone guarding a mystical treasure alone in a remote museum on a dark and stormy night,

Sympathetic toward a hardworking guy nobody likes, but also aggravated by his bossiness to the point of swearing at him,

Confused due to waking up at 7 pm and not being sure how it happened.

You probably don't entirely understand any of those. What is it to entirely understand something? But you probably get the idea in each case.

The moniker was mostly invented by the press. But if we're talking about all four "horsemen," I think they all made positive contributions to their respective fields. Likewise, there are fair critiques one can level at each of them, including Dennett.

Could you please stop posting unsubstantive comments? We're trying for something else here.