> The obvious conservative response to a technology whose downsides are unfathomable is suppression, not encouragement. “Move fast and break things” might be an ethos conservatism can accept when the object being broken is the B2B SaaS status quo. But it cannot do so when the objects in question are human intelligence, sociability, agency, and existence.

Hold on.

I accept all the rethorical apocalyptic stuff, it comes with the title and each of us can have an opinion.

But “move fast and break things” is Facebook motto, and when it come to sociability, society at large, and politics, I you cannot say that.

Facebook/Meta merely breaking B2B SaaS?

It’s not really about “Meta is bad”. It’s recognising the fact that social media, spearheaded by Facebook has changed society and politics.

Some like it some don’t. I don’t want to enter a culture war, but can we agree that with Meta the “object in question” is not just a SaaS? It is sociability, it is society, it is politics?

This guy is an associate professor of politics at Princeton University.

How can you advocate for future policies over AI while at the same time misleading on the tech revolution that has shaped the past, what, 20 years?

What is being taught in those classrooms?

> Given what we have already seen of AI, we can be confident that its widespread adoption will create a less free society, a less virtuous people, and a degraded culture.

I think this claim needs stronger evidence than "we can be confident".

The question to Sam and Dario would be: if the thing you are building is going do displace humanity, then why are you building it?

Didn’t read the whole thing, but it starts out a lot better than I expected. Most critics are in denial about the state and trajectory of the technology. This critic is not, he just thinks it’s bad.

The word Apocalypse is on the table and if you agree that ignorance of History invites Doom then my brief perspective here may appeal to you (otherwise tl;dnr) ...

From the AI Apocalypse Is Here essay:

  ... the truth that widespread interaction with machines endowed with 
  conversational language abilities and superhuman cognitive power is 
  the greatest social-psychological experiment the species has ever run.

As a boy I had conversations with my grandfather who was born in 1888. If my parents were still alive they would be 104 years old. My older sister (deceased) did Duck & Cover drills in school (but I did not, as I am only 64).

I have lived my whole life with always a small number of men able to (roughly) push a button and end our whole world. Like most of you, I don't think about that much.

But the other day I parked outside a small town USA Post Office and noticed the faded yellow & black triangles icon on a sign reading: Bomb Shelter.

The MAD reason this doom has not happened, despite apparently a number of close calls, is not that our past and present leaders are always wise good people, but that everyone in the high tech multi-trillion dollar nuclear weapons universe has, so far, plausibly believed that if they nuke, everyone else will nuke, inevitably being a Mutually Assured Destruction.

The societies and voters and economies of the world's post WWII "great powers" had to be convinced and corralled (for way more than my whole 64 year lifetime) that this MAD experiment was going to work.

And nuclear weapons pose a verifiable Apocalypse, not a maybe~AGI + maybe~Apocalypse.

Have we been lucky? Or is MAD, genius? Both?

I do see we are not in the mood/mode to be very MAD at exploding AI.

So it will launch, whatever IT turns out to be.

> China has not known freedom for some time, but citizens of the United States have a long experience of it.

Lol?

> China has not known freedom for some time, but citizens of the United States have a long experience of it.

This sort of statements are too broad and impossible to falsify. Freedom in what context? Political? Societal? Also can you have freedom or not have freedom? Is freedom a binary state?

Freedom in many countries is a scale, you are free to do this or that but not too much of it and not to everyone.

Furthermore, even in countries that are deemed to have greater freedom than others, they are still countless things that are banned so again freedom is really in the eye of the beholder.

> the escalating calls for an international treaty to ban superintelligence

There is 0 chance of that happening since we can't even define what intelligence is never mind superintelligence. A model can be great at coding software while hopelessly incapable at solving trivial things like when there was the whole "strawberry" spelling debacle not long ago.

> In other words, the present AI resistance speaks almost entirely of preventing risks still to come.

And that is exactly why this sort of effort is not going to succeed. If you are looking to ban or regulate things that don't pose a threat then what are we even doing here?

We have always started by allowing things and then restricting them when harm had been proven. Not the other way around.

Changing how things work and preemptively banning or regulating things in advance because of the harm that could result without clear evidence is opening the Pandora's box where a bunch of bureaucrats decide what citizens should or should not do based on vibes. Hardly a model to follow.

> Given what we have already seen of AI, we can be confident that its widespread adoption will create a less free society, a less virtuous people, and a degraded culture.

And this is supported by evidence I imagine? "we can be confident" is nothing but vibes. No sources, no peer-reviewed studies, just opinions. Is this supposed to be a good argument in favor of regulating AI because if it is then it's preposterous.

If you think AI is bad and will create chaos, fine, prove it then. And even if you could prove it, it certainly would not be a binary thing. AI is a tool that can be used in many ways. Some bad, some good.

> So far, opponents of AI have overwhelmingly focused on predicting its economic effects.

And we are supposed once again to trust these people because of what? Their track records? Experienced economists have trouble predicting what the global economy will look like 5 years from now but we are supposed to simply accept that the conclusion of the people who are for regulating and/or banning AI are infallible. That makes 0 sense.

At the moment the effect of AI is a known unknown. We do not know what it will do to the economy 5 years from now. Maybe it will destroy entire parts of the global economy, maybe it will create more opportunities in many industries. If we do not know then the best course of action and is to wait and see what happens before jumping the gun.

> It is—already—a moral and cosmological cataclysm

And the superlatives keep on coming...

I stopped reading after that. This article is not serious. We can criticize AI without using terms such as the ones above.

The author also never presents who "the people who oppose AI " are and what they really want. Are they governments? NGOs? Groups or citizens? We don't know.

I don't like the people who preach that AI is going to solve all our problems but the author is going in the complete opposite direction and using the same vibes to support their argument and that really does not help his case.

>China has not known freedom for some time, but citizens of the United States have a long experience of it.

One of those American beliefs presented without evidence because it’s just so obviously true, but what the fuck does this even mean? Exactly how are Americans more free?

So...

AI and cannabis bad...

Tobacco and alcohol good...

It's the American Way???

Personally, I think he nailed it with the first paragraph and went downhill from there. If humanity needs an intellectual codpiece to stay relevant, humanity IMO has failed their own Turing test. To truly believe the only two outcomes are we stop AI before it's too late or humanity is replaced by it goes against the past 10,000 years of technological progress (and yes, it came with a body count, we are still poo flinging monkeys with opposable thumbs after all despite all the hubbub about fire, the wheel, and everything else).

The problems with our experiment in democracy are broadly political and not technological. By all means, regulate the deployment of AI and crack down on surveillance to boost privacy and freedom, also known as "wouldn't be prudent" or "not gonna happen" because capitalism and freedom. For example, if I were to suggest that the left and the right do not need to agree on renewables and nuclear, but there's no reason they couldn't agree that coal is bad, that's not gonna happen either. A friend recently stated that most of these problems will not be addressed until enough people have died in the west because of them as he pointed to the recent European heat wave. That's my prediction for everything else that comes to mind, two podcasts from a revolution and all that.

IMO we should work towards AI offloading reasoning and drudgery to enable and assist humans instead of replacing them. Even the best two models out there frequently hallucinate and downright lie to the user, they are essentially Rainman and we need to be Tom Cruise to their Dustin Hoffman for the foreseeable future. But such a viewpoint doesn't hit the engagement numbers of going Yudkowsky light as this article does so we don't get pieces like it or if we did, they would never see enough eyeballs to get traction.

Western corporate socialism is the equivalent of the big bad malevolent AGI we all feared, and we're all still here, we could all do something about it, but my money is sadly still on the opiates of the masses keeping us in check for the near-term. And relying on the opinions of a small cabal of billionaires who likely couldn't bathe themselves without assistance at this point in their antihero's journey to predict what's coming with AI is folly, but it is a cautionary tale to not let that happen in the first place by staying intellectually engaged and mastering the technology instead of the exact opposite that most seem to prefer.

My tradespeople friends continue to do just fine. Good luck building robots that can replace them (Sorry Karl). Too bad they're only 15% or so of the work force. But I don't think anyone gets how hard it's going to be to give LLMs and their ilk the common sense they need to really really really replace 60% of the workforce with white collar jobs. And while self-checkout kiosks do replace cashiers, I don't get why anyone misses them. Ditto for the accidents waiting to happen roaming the aisles of Amazon fulfillment centers. Where is the quiet dignity of such work that must be preserved? Also, by all means give me the robots to work in the mines. See "not gonna happen" above for the same reason we won't be replacing tradespeople any time soon. It's up there in my mind with the smaller cost of giving everyone over 50 in coal early social security to retire the segment down to the point of industrial coal, but see "wouldn't be prudent" above.

And then there's the barb about individualism. Sure, LLMs were trained just like Freakazoid to have the powers of the Internet. But anyone trying to use one straight out of pre-training is gonna have a bad time. There is significant post-training to shape the user experience that can be individualize and fine-tuned to any task for which it will continue to hallucinate and lie because trying to control hyperdimensional silly putty like LLMs is hard, but it's not 100% mean reversion.

In the end, I'm probably as deeply pessimistic about the future as the author of this article, but for completely different reasons. America, the country that went to the moon in 10 years and walked away from global conquest after WWII stands poised to make itself irrelevant and cave to the reckless agenda of pleasing the billionaire class. AI is a sideshow in comparison IMO and we absolutely have the power and capability to deploy it responsibly, it's just that we won't because that would make the billionaires sad.

[flagged]

I'm really disappointed when these blog posts insist on political navelgazing.

There's no political angle worth discussing because none of them touch the ground. We have to look at what we have with sober eyes and ignore the grifters who insist "AGI is coming".

What we have is a copypasta fraud machine. There's no discussion to be had beyond that. It's illegal in most use cases, and a source of implementation uncertainty in all others. No new capabilities have been unlocked from this, nor will ever be. We just found a way to cut cost where the cost was already minimal.

Hopefully the naked, unabashed, sociopathic greed of the intent to replace people with a system that "outperforms humans at most economically valuable work" will be their downfall.