I still don't understand how to interpret these conclusions. This is all based on a report from a 'third party' 'evaluating' Anthropic, but it's mostly all the same EA people on both sides of it saying the sky is falling. These are the same people that did the junk investigation into the 'HF incident'. How do we even know that METR is being honest, or even objective? How do we know AECI is a good enough benchmark to make this kind of claim, or that the benchmarks that make it up are any good on their own? A lot of Fable 5.2 seems to be a regression in terms of cost and a potential sign of diminishing returns on compute scaling, so to have this come out at the same time claiming we'll see the explosion next year (according to the original article, basically starting in January) is curious.

The authors said we'd be halfway there by now, but most of the political turmoil stuff hasn't followed because the capabilities have not made it as far enough into most workplaces. The public's view of these systems' capabilities and reliability is closer to GPT-3 than FOOM.

I'm not sure that they're wrong, because they have followed AI capabilities somewhat closely. It's the downstream effects that seem to be all over the place by comparison. I don't know what to believe, honestly.

Interesting to revisit the comments on the original AI 2027 submission:

https://news.ycombinator.com/item?id=43571851

For what it's worth I don't think there can be an explosion as the rate of AI doing stuff is limited by compute which is limited by the power supply which is maxed out. I think it will be more rapid intelligence progress similar to what we've been seeing.

This is concerning. It's clear we need some level of societal backlash to halt and reverse this.

Society has no true frame of reference for such exponential growth. I see a parallel to COVID. It was not a big deal on March 1, by April 1 society was halted. This is worse.

Its a reasonable opinion to hold people working on such breakneck AI progress to the standard we would hold if your friend told you they were making a nuclear warhead in their basement. It should be stopped.

I'd really love to call this end-times millenarianism with no factual basis, but Kokotajlo et al. have had a much better track record at making predictions than I am, and I've always underestimated the rate of AI progress. This is much both much more reasonable and much more terrifying than I'd like to admit.

Seems like this should be bigger news. But maybe that’s a sign that HN doesn’t buy these scenarios. What is it?

[dead]