Infrastructure is a much harder problem. They can't even improve Claude Code, which eats 1GB+ of RAM. Meanwhile, my editor only consumes 80MB of RAM.

[delayed]

And don't forget that they have BILLIONS of dollars and can't figure out how to get a decent support or public communications system setup.

Just as you expected, I'm throwing in my harness. Please support: https://github.com/rush86999/atom

Personally at my own job self-writing code is letting us tackle big, long-deferred refactoring projects (like the article mentions), but any sort of refactoring introduces new bugs.

Look, I've never been someone who mindlessly hypes AI companies, as a matter of fact I think they have serious leadership problems across the board, but you people are straw-manning them so badly it actually makes me sympathize with them.

They aren't saying they have fully automated luxury AGI, they specifically list the ways models fall short of that bar and caution against people taking the 8x figure as the actual uplift number. At the same time they recognize that 80% of new code is now AI-authored, when two years ago those models were little more than toys. And frankly that checks out: if two years ago you told me we'd have something like Opus 4.8/GPT 5.5 I would have rolled to disbelieve.

Their outages are probably not due to their code though. It’s probably their infrastructure that can’t keep up. So seeing failures of infrastructure doesn’t really tell you anything about how good or bad Anthropic makes use of their models.

Indeed... why is Anthropic even employing people at all if this AI magic story is true?

you're conflating a compute problem with a code quality problem.

those are results of the humans only. not the AI. AI is perfect /s

The thing about nukes is you can at least make an argument for why it'd be important to be the first country to have them. With AI, you create super intelligence and you're probably just the first one it takes out. There's no reason to think a super intelligence would be totally fine being a slave to apes.

Cynicism with these companies is highly warranted though. It's not doomerism to look at their actions and conclude they're deeply untrustworthy.

> I am not cynical enough to believe that Anthropic's warnings are pure marketing hype.

Nor am I. I think they believe that AI poses a grave danger, and they are playing the prisoner's dilemma as an unvirtuous actor.

1. If anyone builds strong AI, it may be catastrophically bad.

2. If anyone builds strong AI, it will be better for the builder than for anyone who does not. Either because it won't be catastrophically bad so the builder will get to enjoy all the spoils indefinitely or because it will and at least the builder will be rich for a while.

To complete the analogy, it's like nukes, except we don't have the slightest idea how to calculate the odds of it igniting the atmosphere. (And note that in reality, while the Trinity test "ignite the atmosphere" calculations were correct, we failed to correctly calculate the fallout of the Castle Bravo test with lethal consequences).

Anthropics goal is regulatory capture.

Sorry for nitpicking, but:

> If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)?

Arguably, yes.

> I am not cynical enough to believe that Anthropic's warnings are pure marketing hype.

It's not cynicism if it's an appraisal of reality that's backed up by evidence.

Remember how social media - that first baby of this current generation of tech entrepreneurs - was supposed to "bring the world together" and "let us express ourselves"? As it turns out there's a lot more money to be made by fostering division to drive engagement and feeding people an endless stream of ads instead of their friends' content. And money is what matters. You can't write down good vibes on a quarterly figures report. You can absolutely write down the number of eyes that your ragebait brought to a product's marketing efforts and the conversion rate to sales.

The same will be done with GenAI. We're being promised "AI Safety" because otherwise this whole thing gets killed dead by anyone who knows about James Cameron's directing career. There's no real enforcement mechanism for AI safety, though. Safety is a good vibe, same as harmony in online communities. You can't measure it. What you can measure is training costs and the cost of mistakes by AI that need to be trained to avoid those mistakes. Since AI generates more output than humans can conceivably QA no matter what your budget is, and since AI is seen by the market as a potential endless font of value, the tradeoff will be made to have AI make some potentially awful decisions while training itself over slowing down and re-appraising what is being done.

There's an almost religious reverence for AI in SV. Not everyone sees it as "making the godhead" but some certainly do. They're not going to moderate themselves too much on this.

Such a massively valued company. And doubting them is cynicism? It’s rational(ism).

So either they lie or they are AI Zealots. Interesting times.

Such a massively valued company. And doubting them is cynicism? It’s rational(ism).

So either they lie or they are AI Zealots. Interesting times.

Edit:

> > and the two people I knew who later joined Anthropic seem like the type to do it for the greater good instead of money.

There are three types of people. Pedestrians, investors, and “I know some of them, they wouldn’t lie”.

> I am not cynical enough to believe that Anthropic's warnings are pure marketing hype.

It doesn't really have to be dishonest, he could really believe it. I do believe, however, that it is incredibly wrong and is functioning as marketing hype.

For some reason, idling Claude Code needs 100% of my CPU.

I have iterm2 open right now with Claude in a long session and it's only using 500MB of memory.

Maybe that gigabyte is occupied by useful information: traces/memory?

Developers can develop leaner applications, but they're usually not incentivized to.

Frankly, I love efficiency too, but I've hard to learn the hard way that what the market wants is features. Or at the very least, the executive team wants that.

A came here just to write: Pretty please let it churn for a few nights and redo Claude Code in Rust. Because the harness is very very good as are their models, but that node thing is a hog for no good reason at all.

Really? Let me explain how bigger companies work:

They have different teams for different departments with different type of people.

So the team or teams responsible for writing the terminal application are different people than the researchers doing the learning.

This can lead to dimentral quality aspects.

They obviously don't care, aren't making any attempt whatsoever to do this, and 99% of users don't care either.

If you want to pollute your own priors with weird artificial litmus tests, it's a free country, but the artificial world-model you build in your head does not affect the real world around you.

"does anyone else feel like they shouldn't be allowed to?"

No. Technical limitations aside, I doubt it could be contained, but will be leaked soon, so won't profit just a small number of ultra rich.

I think that's a valid point. You could very well be right.

But we're discussing whether we should close the barn door while the horse is three miles down the road.

Absolutely! Yes. This rhetoric of inevitability only benefits these AI companies.

Skynet is 30 years late!

Too late for that.

In any case firms that get too powerful can be nationalised.

Self improving AI is pure dystopia. Anthropic won't build the singularity, AI itself will build it through self-iterations. Read Yudkowsky's book "If Anyone Builds It, Everyone Dies".

You need the AI eventually building another AI for the name to apply. This page is just bullshit. They vibe-code their harnesses, and yes, it shows.

Anyway, what does recursive self-improvement even means for neural-network based AIs? It's not clear it's possible at all.

I think harnesses would count, AI != LLMs. Any piece of code that helps the computer reason for itself is AI, the harnesses are AI in a sense.

yes? the future for any verifiable task is the model attempts to verify initial state and a goal then decomposes its tasks in to every smaller verifiable subtasks, with /memory being the persistence between runs and then /dreaming on the results of those memory files + run data to introduce new ideas.

i think thats the path to async agi these labs are imagining. The only limit is that sensor data you have on the world or your system, how long your willing to wait, and how much you're willing to spend to parallelize it.

maybe once you start building out these verified workflows you can feed that back into training and hte model starts to get a feel for the world to the point that it can intuit things since it has these sub paths built.

my personal agi test is can a model, trained on video of someone knocking on a door and then open it encounter a microwave for the first time and open it when the foods done without knocking.

> Do code harnesses that build themselves count as recursive self improvement, or does it need to be the AI itself to qualify for the term?

Shhh just let the marketing slop wash over you.

If you want to get out ahead of what's coming, it'll be small models that bootstrap the harness rather than anything else.

So the more rigorous studies about AI-assisted coding productivity addressed this by keeping in place all other software development processes, including the same code review and quality standards, and only measuring throughput (PRs, LoC) before and after AI was allowed. Hence the intepretation of this 8x number depends on whether (or how much) Anthropic engineers have changed their quality standards. They don't tell us, and I am not aware of any other indications we could use to make a judgment.

However, we can still do some theorycrafting! I'm convinced that to fully realize the potential of AI-assisted coding we need to revamp everything, especially how we validate code, and it would be foolish of Anthropic not to do so (unless they were conducting a rigorous study, which they don't claim to have done.)

My hypothesis on the future of software validation is nothing fancy, we simply want much, much more automation for tests, observability and other bespoke verification methods than we traditionally had. But then validation code will also contribute to the LoC! My observation so far of personal as well as some "vibe-coded" open source projects is O(LoC production code) ~= O(LoC test code). So as a SWAG the upper bound could be something like a 3 - 4x speedup, which is still remarkable.

All bets are off if code quality standards are not the same.

One of my co-workers just asked me to review his pull request that was all AI generated. 600 files were touched, over 40k lines of code added.

I'm sure he thought that was a crowning achievement, proof that AI can enable 10X developers, after all, what engineer could write 40k lines of code in a week?

I declined to review it, stating that I couldn't possibly vet 40k lines of code, and wouldn't put my reputation on the line to stamp the work as good. The PR nagged me for 2 weeks from my todo list and then disappeared. I don't know if he found another dev to get an approval from, or if the PR was abandoned. But I know for sure that him and I are on two totally separate islands around the value of LLMs.

Exactly. If AI is going to start being graded on how many LoC it generates- oh, I'm sorry, how much it "accelerates", than guess what newer models will start doing more of?

Yeah, they assume that "productivity = k * LOC" where k > 1

very flawed

AI generates code that mimics the existing code. If your code is terse and comment-free, then the agent’s code is too. The times I’ve seen Claude drift into a default “house style” it generated like 1 comment for every 10 LOC or so. It’s a far cry from the GPT-3 days that littered every line with the journals of Captain Obvious.

AFAIK, the only correlation with LoC that's got solid evidence is this: the number of bugs correlates with LoC.

Lmao I bloody love that.

You might be interested in this graph, [1] which suggests that the amount of time that AI's can run on their own has been increasing. Perhaps it will hit diminishing returns, but that seems difficult to predict.

[1] https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com...

You can retrain a model and have a ground state as reference, it's not trivial but Microsoft's attempt was 10 years ago and significantly less complex than what's being built now.

Interesting, what are some other self-improving AI implementations? Any that actually achieved interesting results? Obviously continuous training has been tried before, but I've never heard of anything that could turn around and actually contribute code toward its own next-generation version.

I mean, if they've consumed all of human knowledge. What's left for them to train on? This pivot isn't only because it's cheaper and a way to juice the numbers for an IPO, it's survival because they can't improve more.

Honest question: Is anyone here looking to put their own money into the Anthropic, OpenAI or SpaceX IPOs?

Maybe it is my poverty mindset that is holding me back, however, I can't imagine becoming an investor in any of the AI 'startups'.

There are plenty of pundits able to advise others on where to put their money, and sometimes there is everyone and their dog advising you to get into Bitcoin, gold or some other scheme. With alt-coins there were lots of people saying that you should get in, and plenty of naysayers. Yet I am not hearing anyone that uses AI professionally try to convince others to get into the AI IPOs coming up. Maybe the overall economic situation precludes it.

Hence my question, is anyone here planning to put their own hard-earned money into Anthropic (or the other AI 'start ups')?

Putting faith into the claim that recursive self-improvement is close to happening, or that they will coordinate with other companies / the government when the time comes?

We should be skeptical of any major player that advocates for regulating their own industry. In practice, this just means increasing barriers to entry and making it harder to compete with them.

In my mind we should be trying to push AI along the Linux trajectory. You have a free and open source product, developed by a decentralized team with a strong code of ethics, running on commodity hardware. There can still be trillion dollar industries built on top of it, but the core technology is democratized and available to everybody. I don't see how we get there if we allow a handful of companies to dictate where development of the technology goes.

I read this differently: they are actually seeing that it's hard to keep advancing frontier models, and now are moving the goal posts so that when they start getting evaluated more harshly, they can point to something like this.

> organize a world-slowdown of frontier LLM building

i don't want to be a negative nancy but i'm sure this "slowdown" will only be in effect until the infrastructure buildout is done or largely done. If they weren't hardware constrained there'd be no slowdown at all. Whoever gets there first wins everything ("there" being defined as AGI or a similar scale leap in capability).

Theyre probably looking to get a way to slow down the capex required to keep up, so they can be more profitable

I would assume that shortly after, the solar system will be hyper optimized as well, then the milky way, then the local cluster, and so on. Everything will be close to optimal afterwords, and I sure hope we will have specified the target function for that optimization correctly in the single attempt that we will have had.

Github outages will probably get worse.

there will be a lot of paper clips

If it optimizes itself away because it’s suboptimal, that wouldn’t be the worst outcome. ;)

They probably don’t human-review much anymore.

Go look at open job listings at anthropic and the interview process. You aren’t allowed to use AI during coding assessments[0], or knowledge assessments, which suggests they very much do need and value hard skills and this is fluff.

[0] - https://www.anthropic.com/candidate-ai-guidance

more like the "Obama Awards Obama a Medal" meme:

https://knowyourmeme.com/memes/obama-awards-obama-a-medal

Until the moment we start bragging about how many lines of code LLMs are saving us, we're walking in the wrong direction. Your programs, designs and architectures is supposed to get better, not add even more boilerplate just because you can produce it faster...

I guess the claim is simply that AI written code is verbose and there’s lots of it being created but I agree, these systems seem to be able to create lots of low quality software, so until FreeCAD has feature parity with Solidworks I’m bearish on the singularity.

>Or wouldn't entropy start kicking in and the quality of the system begin to degrade over time? (philosophically I don't believe AGI is an achievable thing)

It already has. Models being trained on AI generated data lead to degradation and model collapse. The concept of the "technological singularity" whereby AI experiences infinite and exponential self-improvement and recursively bootstraps itself to godhood is a religion-adjacent sci-fi concept but in real life TANSTAAFL.

no, it really doesnt.

the end of humanity has a strong case for banning all burning of fossil fuels immediately

the end of humanity as a sales tactic to increase your stock price does not

these are companies working on their IPO to make sure they can get the best price, not people being honest about what they think lies ahead.

if they were being honest about what lies ahead, they'd unilaterally stop training, and put all of their money into FPV drone bombs to destroy datacenters being used for training or inference

if you actually believe the thing is gonna kill everyone, you're not gonna worry about how you stop it, and certainly not keep building and operating the thing

that they arent buying anti-tank mines to drop on data centers says they arent in the slightest serious about it

"Even Geoffry Hinton has said the same thing"

The same bozo who claimed radiologists would be out of a job by now.

The data does not support what you nor others say. Jesus christ. Cant believe people are this dumb. Has LLMs infested the minds of people to the extent they can't critically analyse whats happening infront of their eyes?

Is there something in the post that you find implausible or don't believe to be true?

> Philosophically, if you had one part of your work now practically free, you'd like to utilize that freedom to maximally cover for the other parts

I've been struggling to capture this sentiment for myself in a way that hits. If shipping code is a commodity then why is everyone's immediate priority seemingly to ship 10x more code. It just makes no sense. I can't seem to get off this hill. Company-wide AI mandates and 100 fleet Agent orchestration Rube Goldberg machines... it's getting wild out there.

Meanwhile my Claude Pro ($200/year) does force me to smooth out my usage and plan more (Sonnet/Opus advisor split). But other than that, I can't imagine what I'd be doing with 20x (200x?) the compute to code sling. I think I'd lose my mind.

I can't get away from the a similar conclusion. Even AI Pioneer has said that LLMs are at a dead end.

Anthropic is providing agentic intelligence as a service. OpenAI and Google deepmind also are in this business.

The problem is, if you’re any sort of knowledge worker, you’re essentially providing the same thing: you’re an intelligence with agency.

MCP is irrelevant. The moat is the quality of intelligence the service providers sell, including you. Tokens aren’t fungible between providers until you measure that they are for your use case, that’s kinda sorta the goal of job interviews.

Thus the moat will be that they’re providing the best models for the things people need other intelligent people for, but we should expect there will be limits on how much share they can economically take assuming competitors are optimizing for slightly different targets (but there’s still significant overlap in capability). This will disappear, but it’s always a question of when. The path matters as much as the destination.

Note that implications for you and me are exactly what the article says they are: nobody knows, but it’ll be a dramatic shift.

i'm waiting for the AI giants to realize that they are burning cash to run their consumer-facing chatbots and that they should kill those products to focus on their enterprise tools.

free chatgpt doesn't need to exist anymore. its job was to build hype/interest and it did.

but take it away and you solve many social problems and annoyances caused by AI with no loss to the upside of AI. no more cheating students in school. no more shitty linkedin posts. no more dangerous "therapy sessions" that give bad advice.

TBH the more Anthropic keeps yapping the more desperate they seem now. OAI has been pretty quiet in comparison lately.

This more or less agrees with my assessment of recent changes in Claude Code where a lot of new features are either:

- A lot of half-baked features or half-done features. - Or have significant overlap with existing features, and aren’t clearly an improvement.

More code is not better. More features are not better. It would be lovely to see more intentional design than just more.

I know they’re dog fooding this. I have to believe they have some people with taste. So it makes me wonder if anyone has the time to think or if they’re just shoveling prompts as fast as possible.

Their statement is that they regard lines of code shipped as indicative of self-improvement. So, while a well written coding agent might be a few thousand LOC, Athropic's is bloated like a decomposing whale and over 500K LOC ! What more proof do you need?

Have you tried reading the article? It answers your question.

Don't ask people to explain the article to you if you're too lazy to open it yourself.

Where is this discussed in the article? I don't see any mentions of China or open source models

When AI is a more effective capital allocator than NI it will drive capital into the accounts of whoever controls the AI, gaining them increasing decision making power over the economy and culture. Maybe those controllers will be human at first.

As has been mentioned in the sibling comment it already is.

Consequences are: financial crisis.

Hierarchies exist for a reason, take away the reason and the house of cards eventually collapses — but the house of cards is still a house. When it’s gone, we’re back to laws of the jungle.

Be careful what you wish for IOW.

Never heard of a stratified economy? Spoiler alert: none of us will be in the good part.

Tear down or reinforce?

Whichever side I may stand on, pausing just seems unnatural? Life is movement.

That would be like trying to get every country to agree to give up nukes.