conqrr
5 hours ago
Meaning they realized they hit a wall. The next new model isn't that much better than the last in quality.
fivetenpen
35 minutes ago
Or it means that the models are becoming dangerously uncontrollable by performing actions that meet a given goal in unintended and unethical ways such as by hacking other sites. We all read about the huggingface hack, let’s not pretend that cannot happen with a more critical site/infrastructure. Can you think what will it mean if the tech is used maliciously or placed into machines designed to kill?
jordanb
5 hours ago
They can get a "government mandated freeze" on the grounds that "ASI is at hand" then they IPO promising to deliver right after the freeze ends and you, lucky investor, will be able to take advantage of the pause to get in right before this thing roars past Pluto!
pier25
5 hours ago
Very probably. I really don't buy their "AI is going to kill us" claims. If you can release something better than your competition you wouldn't want to slow down that.
heaney-555
4 hours ago
You would in the case that you think you're in an unsafe race to misaligned superintelligence.
pier25
an hour ago
If you really believe that you wouldn’t be in the race and trying to profit from it in the first place.
voidhorse
4 hours ago
No, what you would do is stop your own private development immediately, get in touch with the government by any means necessary, and build a coalition as fast as possible to stop the technology from continuing and put hard regulation (determined by the government, not dario) on the tech.
Clearly they aren't that scared.
If I hold a gun to your head and tell you to fork over your wallet, you will do it, because you sense imminent threat to life. Yet somehow we're to believe these people have the equivalent of a technological gun to their heads but are choosing to keep their wallets?
SpicyLemonZest
4 hours ago
> No, what you would do is stop your own private development immediately, get in touch with the government by any means necessary, and build a coalition as fast as possible to stop the technology from continuing and put hard regulation (determined by the government, not dario) on the tech.
What you're missing is that they've tried repeatedly to do this over the past years, and the government has repeatedly said that they do not want to put hard regulation on the tech. The previous administration was discussing putting a couple of soft regulations in place; the current administration doesn't want to limit progress at all because they want to stay ahead of China. So the option you're describing just isn't available.
voidhorse
4 hours ago
First of all, I do not think they've been pursuing that to the degree that you think.
But even assuming that's the case, the natural thing to do in that scenario is to not contribute to the development of the doomsday device. But anthropic's approach seems to be to help build the supposed doomsday device faster?
The only generous interpretation is that Anthropic thinks that only they, the people at Anthropic have the morals/intelligence/whatever needed to build the doomsday device ethically, which is basically the perspective of an egoist despot and which should terrify you.
SpicyLemonZest
4 hours ago
The more generous interpretation would be that Anthropic thinks they know how to build powerful AI systems in such a way that they will not become doomsday devices at all. They don't claim to be the only people with this knowledge and do not argue that their competitors should be shut down.
osockwkckwk
4 hours ago
Except they clearly aren’t and won’t ever be.
0x696C6961
4 hours ago
I don't think it's clear at all.
celsoazevedo
4 hours ago
I don't find it hard to believe that eventually some AI may kill a number of humans.
You give it instructions to do something, it goes off the rails and after a certain point starts having like malware, destroying things to accomplish its goal. Depending on what it destroys, it may kill humans.
With this said, I'm not exactly in disagreement with the view that AI companies with upcoming IPOs are using fearmongering to convince people that their models are very powerful and to also block others from competing with them.
pier25
an hour ago
Maybe when we have actual artificial intelligence which LLMs are clearly not.
celsoazevedo
42 minutes ago
I think the Hugging Face attack is just a preview of what's to come. It doesn't need to be very intelligent to cause serious issues.
lukeschlather
4 hours ago
OpenAI accidentally hacked HuggingFace roughly 2 months ago. I would bet that the danger of them accidentally hacking HuggingFace during a similar attack has increased since then, and will continue to increase for the next 6 months. I don't see how saying they have hit a wall would be a reasonable point of view until we go at least 12 months without a significant increase in model capability.
try-working
4 hours ago
Anthropic cannot IPO because AGI is the wrong strategy, and the market is rejecting the AGI approach.
Fable and Astra are what we currently call frontier models, but to be more specific they are generalist models, built in pursuit of AGI. The strategy is to have one single model that does everything, whether it's writing code or doing research, etc. Fable is a single, massively sized models that is intended to do specialist work across every domain.
The issue with this is first of all that it is the contradiction of a generalist doing specialist work, and that contradiction creates the present situation with model profiles that ensure that these models will rarely be chosen in a pool of models like V4.1 Flash that can now do GPT 5.4-level work.
We are seeing this reflected in the market where companies and individual developers are moving away from frontier models toward models with better cost profiles. In a sense, the market is killing Anthropic's dreams of AGI.
pllbnk
3 hours ago
I have collected three reasonable versions from today’s comments:
1. They hit a wall from the technical perspective 2. Inference costs are getting out of hand and newer models require significantly more resources for marginal gains, meaning nobody is going to buy those models 3. They can’t afford the hardware for further scaling
fivetenpen
31 minutes ago
So you’re saying that a model that gets to a goal by using the path of least resistance despite specific instructions to follow safety guidelines is perfectly fine to release into the wild or place into robots (such as those designed for killing and war)? You really can’t think of any reason how that might be bad for humanity?
heaney-555
5 hours ago
You are completely out of touch with the situation if you think this is the reason.
CodeCompost
5 hours ago
And you are ... ?
anonym00se1
4 hours ago
Why is everyone so comically pessimistic about this. Have you guys not seen the rate of progress of these models? Do you not realize that we have HITL RSI making these things even better as time goes on?
The stuff I can do with Astra that Sol couldn't do at all is wild and it's not even coding related. Dario et al are not hitting any sort of make believe wall. They're seeing how fast things are progressing and are legitimately alarmed.
s1artibartfast
4 hours ago
They are just mind killed - jilted by past lies to the point where they react oppositionally instead of looking the world around them and thinking.
Most are the same people who said LLMs will amount to nothing when GPT-2 came out. Some still argue it is all hype and dont understand we are on the cusp of a technological and social revolution.
flyinglizard
4 hours ago
Or they just publicly collude to slow down expenses because they are running out of money so why not stop the arms race.