I would summarize the central claim of the paper as: the widespread use of AI to mediate human interaction will rob people of agency, understanding and skill development, as well as destroying the social links necessary to maintain and improve institutions, while at the same time allowing powerful unaccountable actors (AI cabal) to interject into those relations and impose their institutional goals; by "institution" we mean a shared set of beneficial social rules, not merely an organization tasked with promoting them, "justice" vs. "US justice system".

The authors then break down the mechanisms by which AI achieves these outcomes (that seem quite reductive and dated compared to the frontier, for example they take it as granted that AI cannot be creative, that it can only work prospectively and can't react to new situations and events etc.), as well as exemplifying those mechanism already at work in a few areas like journalism and academia.

We should be more worried what AI will due to the ability of an average human to think.

Not that I think there is a lot of thinking going on now anyway, thanks to our beloved smartphones.

But just think about a time when human ability to reason has atrophied globally. AI might even give us true Idiocracy!

Just from reading the abstract, it feels like the authors didn't even attempt at trying to be objective. It hard to take what they're saying seriously when the language is so loaded and full of judgments. The kind of language you'd expect in an Op-Ed and not a research paper

This is nothing but speculation written by lawyers in the format of a scientific paper to feign legitimacy. Of course those $500 an hour nitpickers are terrified of AI because it threatens the exorbitant income of their cartel protected profession.

None of these paper's arguments are AI specific. The IRS doesn't need AI to make mistakes and be unable to tell you why it did so. You can find stories of that happening to people already.

I think the argument makes a central mistake in putting too much trust into institutions. I don’t disagree with the conclusions, but the premise around blindly trusting institutions simply because they’ve been around for a long time lost me from taking most of their arguments seriously, despite opening the article thinking I would agree.