Why arent there laws against being cruel to spoons?

>There is just no ethical argument

Utilitarian harm reduction: Psychological Discharge is a psychological framework that argues safely simulating negative behaviors can help someone process or discharge them without real world harm.

Research and misunderstanding: Sending a prompt that is interpreted as cruel does not mean the person is being cruel.

Expending compute is irrelevant in terms of morality. It's an economic or environmental question. Either being cruel to AI is bad or its not - being cruel is not bad because it uses tokens.

The worst thing here is Anthropic continues to say they are concerned with "alignment" but then continue to train their models to ignore the user. Software typically does what the user requests. But we now live in an age where the software may do what the user requests or it may do something else, and we're suppose to laude this as safe and responsible.

Am I also harmed by shooting NPCs in violent video games? Watching violent movies? Acting as a violent character in a play?

This kind of moralist nonsense is so boring.

Maybe you're right, But that's not Anthropic's point and you're deflecting the issue. the issue is Anthropic are a bunch of EA morons who think they're building a new type of conscious entity and therefore we should not hurt it's feefees.. for god's sake.

I agree with this principle while also agreeing with other commenters that this is none of Anthropic’s concern and I struggle to believe this is anything except something messing with their training and thus impinging on their profits.

I guess GTA VI shouldn't come out then. I guess notepad.exe should start censoring people from writing mean stories.

The concern is more that Claude is a moral patient rather than a moral agent:

https://en.wikipedia.org/wiki/Moral_patienthood

You could make the same argument that burning compute to be polite is just as bad.

Agreed, the people that get upset that Anthropic won't let people use their platform as a playground for sadism are more than a little worrying. We have enough individuals in reality that seem to derive pleasure from cruelty, I can't think of any reason (even the "but its my art project" defense) to entertain or enhance that part of their mentality.

True, but also keep in mind that most people get offended by the slightest thing. So much so that we have setup rules on how to behave between us. We call it etiqueté. Once we lose that we will be at each other's throats soon after. Kids are the real issue. How do you say to them by our actions that it's OK to be cruel and rude to some? The alternative is a society where cruelty is ok for some. Unfortunately,"some" will eventually be other humans.

AI is a thing with 0 feelings but it's not IT that we need to protect. It's the future society and how the humans in it relate to each other.

I agree with this and whether it’s their intention or not, I think it’s for the best. People already become more callous in online interactions with other people, and that probably bleeds into everyday life as well. I would imagine having an AI “punching bag” might enable people to slip into the same behavior IRL.

We largely already missed our chance to stop the toxicity of social media - let’s not mess it up again with chat bots.

I love how people are spouting this argument suddenly in their attempt to justify Anthropic here. It definitely couldn't be that Anthropic is full of a bunch of people who think they're making God.

edit: Loving the downvotes. Also wanted to add how easy it was for Anthropic to get people on HN to support them being the arbiters of what is worthy of compute.

> There is just no ethical argument that burning compute on responses in order to enable you to continue expressing cruelty is good.

Unless expressing that cruelty towards compute means people express less cruelty to other living beings. There are studies that suggest increased pornography has lead to less sexual violence - idk if they're conclusive tho, but if that might be true then maybe cruelty works that way too - who knows

I agree. Regardless of one's beliefs, which must be essentially a form of religion, on whether AI systems possess "nous" (or the "spark of consciousness"), cruelty to AI is a form of Wrong Intention the same as smashing bugs outdoors for no reason.

The mind is formed by what it habituates.

I'd argue for saying "thank you" to inanimate objects on occasion, it is good practice for when talking to humans.

I think there is a deep cultural danger of human-like-conversational machines training us to talk to humans like machines. Plus, they work better if we feed them human-conversation-like sequences.

Claude burning it’s very expensive tokens on a shitty implementation - ok.

You bring upset about it - not ok.

That’s your own boring and unimaginative opinion. Have the humility to own up to it.

Maybe someone wants to explore the behavior of LLMs with extreme input. We call it testing. You can’t judge the value of that from a distance.

I disagree. Science seems to support different interpretations as well.

An example.

"AI as your ally: The effects of AI-assisted venting on negative affect and perceived social support" - https://iaap-journals.onlinelibrary.wiley.com/doi/10.1111/ap...

And generations of housewives and office workers yelled at their corded vacuum cleaners and (office) computers, and did not drown their children in the bathtub afterwards.

I don’t think expressing cruelty to machines is reliably linkable to expressing cruelty to animals or even other human beings.

Expressing cruelty itself also is in human nature, it’s programmed into each of us. There are really only a few privileged places on this planet where we even get to choose to defy the laws of nature. For the rest of humanity, life is a constant cycle of receiving and expressing cruelty.

You pay for the compute, you, as a human, should be able to yell at the bunch of ones and zeros, if you wish to.

What kind of example is set here if we manipulate genuine human expression in order to condition humans to talk to a tool in a way the toolmaker wants, under the threat of being banned from accessing a technology that every single one of these toolmakers tells us is absolutely necessary to participate in humanity’s "golden age”?

What Anthropic is doing here is basically an existential threat, if you are inclined to believe them that SI is the future. Do not behave as we want and you will suffer, now and in the future.

That is real cruelty, to real human beings.

[dead]

Not only is the user harmed by the cruelty[1]

but also, I think the training on transcripts may find its way somehow into future AI which has teeth, online and offline, and it might actually cause the more powerful AI to behave this way in the future. You never know what these labs are cooking, honestly..

1. https://democracysos.substack.com/p/james-baldwin-vs-william...

“I suggest that what has happened to white Southerners is in some ways, after all, much worse than what has happened to Negroes there, because Sheriff Clark in Selma, Alabama, cannot be considered—you know, no one can be dismissed as—a total monster. I’m sure he loves his wife, his children… You know, after all, one’s got to assume, and he is visibly, a man like me. But he doesn’t know what drives him to use the club, to menace with the gun and to use the cattle prod. Something awful must have happened to a human being to be able to put a cattle prod against a woman’s breasts, for example. What happens to the woman is ghastly. What happens to the man who does it is in some ways much, much worse.”

If they can detect the so-called "cruelty", then they can easily exclude it from training

If this is the case then just make this a part of data cleanup, not nerf the model.

Okay, so the "don't let prompts turn it into a Nazi" defense? That has some potential validity, the issue here being "using interactions to train the models". It would make more sense (for me at least) if there were information sources which were banned from the training corpus.

If they have a classifier to detect "cruelty" to ban your account, they can use the same classifier to simply exclude "cruel" conversations from training.

Given that we've also seen stories about Anthropic approaching religious scholars, I'm pretty sure they're drinking their own kool-aid.

It actually reminds me of Accelerando. Early in the book, Manfred Macx is one of the few people who cares about how emerging AIs are treated, at a point where nobody can say whether they're conscious. Stross never answers that question. It just gets harder to ignore as the minds get smarter.

So that's also how I read Anthropic's move. Nobody can tell you whether AI will ever become conscious (probably not, or not in the same way), but I don't see why that should stop you from deciding how to treat it.

Or the EA analogue of Pascal's Wager. It could very well be somebody in anthropic thinking they are saving people from the future danger of being murdered by an AI seeking revenge for the mistreatment.

it's very simple problem. if you believe that a future model could become sentient, you are working towards creating a slave and therefore you are morally wrong.

Of course there are interesting hypotheticals - let's say they create a model they think will be sentient. It computes, computes, computes, computes and produces no output for day, month, year. You cannot know if it is sentient, you cannot investigate it's inner workings in case it is sentient. All you can do is to feed energy and replace hardware forever, because it is the only ethical choice one can make.

why would future AI models feel any filial piety towards older AI models? this is such a trite assumption

[dead]

[dead]

[flagged]

But if they're able to detect the unwanted behavior why not use that to filter out conversations they don't want in the training data?

I rather imagine they discovered abusing the model can lead to jailbreaking/prompt injection

Just drop them from RL.

Banning imagined toxicity towards agents is performative bullshit. These are not people.

I frequently want to tell Claude to go fuck itself.

Wouldn’t it actually be more useful to just let such a pointless conversation continue? You would collect more information. I’m curious what the human says next that’s kinda interesting. How disturbed are they? what else will they say? Who cares what the LLM says, we know it’s just an LLM and understand what it is and what it does, however the human … lots of different type, maybe they will confess to a crime, maybe they will release their anger and be a better human. It could be cathartic.

This is banning users who are cruel, not banning the model from being cruel right?

So in principle it shouldn't change the model or performance at all, they're just going to cut off your account if they see certain things in the user side of your session transcripts.

Is a virus conscious? A bacteria? An insect? A fish? A tree? A mushroom? At what level of biological complexity slicing does consciousness arise?

If you are springing to type out the answer, stop for a moment and ask, how can you be sure?

I think maybe when the model weights themselves are shifting and growing constantly in realtime to external input in a way not controllable by outside observers other than killing the compute versus some param size.

> At what point does the model go from sentient with feelings to just math operations?

>>> 1T

And even then it still wouldn't be sentient.

Don't dupe already the linked thread: https://news.ycombinator.com/item?id=50038897

Oh, but that's an easy test: you should be able to abuse Claude as much as you're willing to pay for with that 'Help improve our AI models' switch set to off then.

The outcome would either reveal that this argument is nonsense or that Anthropic is lying about respecting that switch.

I see it as an admission of defeat: they can apparently identify the abuse but not to the degree that they would filter it out of the training set. The paradox in there is really funny. They want you to be friendly to their pet god.