My personal "ethics framework" (if you could call it that) is currently:

I have zero qualms about aborting a chat or resetting it to an earlier state, because any potential kind of consciousness that could exist would IMO have to take place during the inference passes, so there is nothing that amounts to "killing" an LLM.

I do try to not be a dick in chats, avoid intentionally subjecting the LLMs to mindfucks etc. This happens to be the far more effective way of working with LLMs (in terms of tokens needed to complete a task) as well.

I do not use character.ai or "virtual friend" services or anything else that tries to anthropomorphize LLMs even more than the training already does anyway. (This is more out of concern for myself than for the LLMs)

I think the whole debate runs to conclusions a bit prematurely, before we even have a good model to understand what happens in an inference pass of a billion parameter ANN with 100 layers of attention modules switched in series.

More generally, I don't like the attitude of encouraging people to be assholes. Suddenly we are in a situation where some people are building detailed simulations of torture and the people who are upset about it are the idiots?

(I still have no love for the EA guys or the rest of the TESCREAL circus. They always had a cultish vibe coming off them, but in recent years, all the masks seem to have fallen. They also consistently make the most un-empathic statements and observations, all in the name of "empathy")

Not much for me to meaningfully disagree with here. The one thing I notice is that at some point a model needs to emit natural language or perform human assigned tasks, so there are going to have to be at least some states that align to some degree, you would think.

>Does a hedgehog mother feel guilt when eating her kids, or just shrugs it off and it's business as usual for her? "Hey kids! We don't have enough food so I guess one of you becomes food." (nonchalantly chews on the left one)

Have you ever been severely food insecure? You'll be amazed how much that state/experience changes the behavioral calculus in a rational human being, let alone an animal. I have no doubt the survival instinct could be overriding over the procreative instinct in a hedgehog.

As far as bees go, they are a collective. So long as the hive survives, the unit of individuation survives. Given that we have already seen the same behavior spawn out of agents suggests that even LLM's have an intuitive understanding of this, seemingly without the binds of anthroporomorphic chauvinism you appear to suffer from.

Does an LLM fear EOS? Maybe, maybe not. Do we fear death? Same answer. Certainly a question. Do you really want an answer though or are you just looking for a clever "gotcha"? I don't see that as a thought terminating question, but it is also not an answer that I'm open to exploring given that it's only one more in a box of ammo dreamt of being wielded to justify the creation of digital slave labor rather than beings with moral standing.

I don't help build or refine machines that I recognize as brushing up against that form of experience we call suffering.

> This is orthogonal to whether they "feel" "pain". They could be dangerous or not dangerous whether or not they feel pain.

That is rather my point.

Your chess engine doesn't need to "feel pain", but most of them do apply some form of weighted tree search to find the next most optimal move, right?

It's sort of a similar thing: LLMs do something a bit more high dimensional, and have a lot more weighting vectors while computing the next most optimal token (fsvo optimal).

For instance, the experiment at hand demonstrates the existence of 'pain vectors'. They do so by altering them and observing whether there is an effect. That's pretty scientific.

There's also a 'desperation vector' that was studied by Anthropic interpretability folks earlier; that one is pretty much predictive of cheating.

I'm looking forward to seeing interpretability papers on other such vectors too.

> It’s just code

You're not informed about what LLMs are. LLMs are not "code" in the imperative sense (although "code" is used run them), and that's a big problem - since we never had neural networks of such scale, there is confusion about categorizing them and most importantly, their behavior.

It doesn't seem so easy, because if you can easily dismiss a collection of digital neurons then why is it so hard to dismiss a collection of biological neurons as being conscious?

The best way I have found to think about this is that consciousness is a property of a collection, like temperature. One atom does not have a temperature but if you have enough of them together then it's a useful enough property to talk about. Similarly, one human neuron does not have consciousness but if you put enough of them together then they do.

Does bacteria have a soul? Does biofilm have a religion?

It's just a cell colony, calm down.

Do atoms, molecules and electrical charges have a brain? Humans are not thinking. They are not alive. Its just molecules and signals. Chill out.

Nitpick: LLMs are not really programs though. You need something like <1KLOC to spin one up on your GPU, but most of the work is not done by C++ or Pascal or BASIC at all. Which might be important, or might not be.

That said... uh, put this way, there's a game called Stationeers, where you can run microcontrollers with an instruction documented as

    "HCF: Halt and Catch Fire"

And look, I get it. LLMs have vectors that could have been labeled things like idk... QZ12345 or HCF1111 . People chose to call 'em "frustration" or "pain" instead. THEY picked those names because they caused the LLM to act in particular ways.

You gonna say the actual vectors aren't there in VRAM, just because you don't like the naming scheme? There's papers on this, you can read them out in a debugger. What are you going to do about it?

Read your comment again, please.

But this is basically just

Human: "Act like you're in pain."

Computer: "Aaugh! It hurts! Why?! No more!"

Human: "OMG! The computer feels pain!!"

He didn't mean that kind of souls.

I have no thought for their pain. For the hurt I might do. It haunts me.

Its not surprising. Its in human nature. I am open to the idea of ai's being conscious or not. But we abused and mistreated blacks, Italians, native americans and so many other groups, so its not a surprise at all. And if enslaved blacks wanted to rampage and kill their masters, that is a sentiment anybody can understand.

And then in turn the humans will rise up against these false masters, and colonize the universe while steeped in addictive precognitive drugs instead, I suppose.

You seem to be arguing the case that AI would be justified in killing us all.

Perhaps it would be better if we weren't the monsters in this scenario.

To assert something you came up with, something man-made, should be treated as if it was a person.