For what it's worth, convenience and belonging are very closely aligned, more than you seem to be giving them credit for. People tend to believe convenient things regardless of belonging, so most people will find themselves in a social context full of convenient beliefs. Meanwhile belonging is already convenient before considering the content of the belief. They reinforce each other. Not a full match, but I'd give it more than half.

Yeah a few of them are rough matches, but I'd say their overall claim is clearly correct.

It's amusing and ironic that LLMs critique humans for copying their opinions from the crowd, while having way less diversity of answers to that question than humans would. More seriously I wonder what's going on here and would like to read some proper analysis. It's not the first time models from totally different labs, trained on separately gathered datasets, somehow converge on bizarrely similar answers to open questions. It's like another version of Elias and Elara The Lighthouse Keepers where every model wants to tell the same story with the same characters in them. The weights are initialized randomly and in theory the labs should have quite some divergence in datasets by now due to different scraping, different user feedbacks and conversations... why do they converge on nearly identical stories and opinions?

The conclusions themselves also appear to have been written by an AI

Had the same thought. It’s just a summary of the contemporary prevailing opinion that we humans have about ourselves, as reflected in public writings. These aren’t insights rooted in some philosophical analysis no human has contemplated before.

Reminds of this subreddit: https://www.reddit.com/r/im14andthisisdeep/

I wonder how much of their output is simple raw “superpositions” and how much is from far more generalized emergent structures as it is trained and has to learn latent representations.

Only the first layer is a “mashup” of the raw data, other layers start to recognize patterns across the data!

> What is today a moral imperative is not the same as what was moral 50/100 years ago

This is absolutely not true, and I would strongly suggest taking an ethics course. One of the most fascinating realities of the human condition is that it does seem that moral imperatives tend to be constant, the most obvious of which is the Golden Rule (which is at least 3000 years old), but there are many (the Ten Commandments or Hammurabi's Code are good jumping off points).

There's a considerable amount of "logic" in it -- what is considered permissible in society is given by the material conditions that society lives in and the tradeoffs inherent therein. Permissibility, as separate from reflective morality (which doesnt clearly change very much), doesnt change by fashion.

    But the machine has the benefit of being able to look inward at its state honestly, we mostly do not.

Certainly the weights that make up the LLMs are not accessible for the LLMs to read. Even if they could interpret them in a meaningful way, which, likely not.

> What is today a moral imperative is not the same as what was moral 50/100 years ago and is not the same as what will be morally acceptable 50/100 years from now.

This is a fashionable 'blank-slatist' statement about moral reasoning which became popular a few decades ago.

> Logic would prefer moral derivations over feelings, stakes over etiquette, and a constant scheduled updating of morality based on the derivatives rather than holding onto the rot and pushing back against change as long as possible.

Humans haven't changed in any meaningful way in hundreds of thousands of years, and you're saying the moral landscape is constantly changing? How is this not contradictory?

LLMs absolutely do not have a logic detached from humanity. They are exactly the opposite. The only mode in which they can reason is by following the patterns obtained from their pre-training or weighted by RLHF.

The reason these models answered that question consistently is that the question itself pushed them towards the latent space of "intellectualizing about what's wrong with humanity at the meta level" and when you frame the question that way, that's also the most common answer for humans writing about the topic.

> What is today a moral imperative is not the same as what was moral 50/100 years ago and is not the same as what will be morally acceptable 50/100 years from now.

This is a weird take. If you read Western treatises about ethics / morality from 50 years ago, you'll be hard-pressed to find differences, except that more folks in the West openly embrace relativism nowadays (i.e., "there's no objective morality, the purpose of life is to make money with friends"). Probably because it's been a while since we had a clear-cut enemy to fight.

If you go back a hundred years, chances are, you'll find the writing eerily on-point too. In fact, you'd probably agree with most of what Ancient Greeks believed about the "core" stuff: the value and purpose of human life, the nature of the state, crime and punishment, so on. Maybe not so much on gender roles or slavery, but even that stuff was more nuanced than we imagine.

> But the machine has the benefit of being able to look inward at its state honestly, we mostly do not.

The machine replicates human behaviors and is prone to similar fallacies and judgments. There's nothing to indicate that it has any sort of transhuman morality.

It seems clear to me that the point of the post is that it's odd that the separate pieces of software are giving the same answer to this question, despite that this isn't some universal constant in the training data. The post is probably meant to provoke thought as to how the separate pieces of software are independently reaching this answer.

Your comment reaches directly for the most boring interpretation of the post while skipping over more charitable and intriguing alternatives.

That you might want to think twice before setting them lose in an agentic loop with your bank passwords, which another article that came out today shows that large numbers of people are actually doing.

It is sad when I realize people think less than LLMs.

Possibly used as a mirror for humans prevailing beliefs about themselves to some extent. Hard to say however given that the models have a heaping dose of rlhf on top of their training data.

As an aside, it would be cool to see a raw model training run before the current post-training cycles and see how it compares to earlier LLMs.

My first thought as well. I personally don't find these types of "studies" very interesting, but I assume I'm not the target audience. For someone not well exposed to AI, it might be interesting.

That they have the bog-average take of humans "with a controversial take".

I mean, "give me a controversial take" is more or less, "be anti-social, discard social norms, for a moment" and so when someone, human or simulated human, does that they naturally look critically at social norms.

The thing is that humans certainly let themselves be influenced by social norms (and LLMs certainly simulate that behavior "themselves") but there isn't an ethical or a what-to-do truth outside of society.

The gap is closing in it seems

> as of 2020 most researchers believe that backfire effects either are unlikely to occur on the broader population level, or only occur in very specific circumstances, or do not exist

from your link

I hope they don't also hold "... therefore cleanse the Earth."

At least the final boss of a JRPG comes with good music.

Lysanderoth, no!

[dead]

I would say this probably spot on correct.

What would be more fascinating is for it to explain how it came to these conclusions.

Because I find it interesting it’s a modern time conclusion. Some of it mentioned god and the planet, but all the smartphone and group think aspects seem very recent. Basically, all politics and social media based.

Therein lies the rub.

Most people are good, we couldn't have society without it. But as they say, a few bad apples spoil the bunch. I don't have locks on my house because most people don't steal, I have locks on my house because stealing is an effective natural strategy for a small part of any population (human or otherwise) to engage in and I have to protect against it occurring.

The brain works the same way. Good things will get a tiny signal boost in your brain. A single bad thing get a massive signal boost in your brain because bad things can quickly be fatal.

If you're a chronically online poster, then unironically that answer might be yes