How will that protect it?

The laws they are arguing for are not going to protect anything, they essentially are waging war on open models and since most open models that are 'pretty good' to 'good enough' hail from China they have chosen to play the China-fear card.

But they won't be able to stop anybody from doing this if they want to continue to operate Claude as a business going forward, and that is assuming that is what is actually happening. Chinese labs have been extremely open about their developments, they have shipped working models to the likes of me, which is more than I can say about Anthropic. On top of that they don't sermonize about how they are going to improve the world and never mind the accidental destruction, that's just collateral damage.

This is basically like the RIAA/MPAA in the early 2000s trying to lobby politicians to crack down on torrents. It won’t work either.

Today, movies make money from in person theater tickets and records make money from tours. I don’t know what the equivalent would be for AI (maybe IRL robots?) but it does not look promising.

I'll admit it is likely somewhat reductive, but it's an incredibly nuanced area, touching on both the copyright debate and the now-seemingly-ubiquitous behaviour of scraping the entire internet for content.

Probably stamping heavily on much of the nuance, there's a certain irony in any of the frontier models complaining about others stealing their work. The secret sauce for the frontier labs may be the reasoning layer, but the secret sauce of any piece of literature is the specific ordering of the words. I hope the parallel is obvious.

The frontier labs are spending record-breaking amounts of money on hardware (to the extent that they're locking out anyone else from the market, but that's beside this particular point, kinda) in order to create their service, and another company that is 'distilling' has not paid the same hardware toll. How much did it cost an author to create their literary work, which could be read and ingested for a miniscule percentage of the cost it took to create? For the benefit of the service provided by a (wannabe) multi-trillion dollar company.

If the frontier labs can prevent or slow down 'distillation' actions that customers are using their service for, then that's all well and good. Maybe they write an acceptable use policy that says it's not allowed (and they probably have). Good on them, it's their service. It's the way this is framed that I have a problem with. Referring to it as an "attack" and lines like this:

Second, illicitly distilled models present a national security risk.

It's bloviation. It's fear mongering. It's being one of the biggest, most powerful and influential companies in the world at the same time as playing the victim. Feel free to share the methods being used to prevent what is considered unacceptable use, but let's not pretend these are the only companies having to expend effort to protect trade secrets and company value. Let's also not pretend this post is all that much more than "lead-up to IPO" hype.

Yes, it is reductive. HN is unreasonable when it comes to defending distillation, the billions labs pour into transforming raw information into a model is transformative work that they should reasonably try to protect. Folks just want to have free stuff and jump through hoops to defend that view.

[dead]

> It probably scales with model capability, distilled models are inherently less capable and therefore less able to commit a crime

Not according to Anthropic:

> We find that GLM-5.3 develops end-to-end exploits in 50 of 410 attempts. Claude Mythos Preview did so at a similar rate—in 56 of 410 attempts.

Another quote:

> [..] a researcher used GLM-5.3 on a sandboxed machine with a local Linux build of a popular web browser. Over the course of a day (and with limited human attention), GLM-5.3 found several previously unknown vulnerabilities in the browser’s JavaScript engine, and chained them together into a working exploit: a webpage that, when visited, reads arbitrary files from the visitor’s computer

Source: https://www.anthropic.com/research/glm-5-3-and-the-spread-of...

But if we follow that to the logical conclusion then Anthropic should switch off their service immediately. That is the single most effective thing they could do to ensure those evil Chinese who are categorically incapable of coming up with something original (/s; just in case) would not be able to make further progress.

Nobody mentioned hundreds of organizations with powerful AIs.

I can't tell you about 'we'. I can tell you about 'I' or rather 'we' but there we is me and a couple of my friends. We've invested a bunch of money into the hardware required to run sizeable models locally and that seems to work just fine. Yes, it uses energy and sure, it takes up space. But at least there are no gatekeepers.

Plausible, but even then: they really should provide some proof of this alleged distillation. Right now all we have is a mixture of FUD and hearsay. All of the juicy claims in the first part of the article are utterly bereft of evidence.

And who is to say how much of this stuff has been going on between OpenAI and Anthropic, because whoever thought of it first must have been using it already.

I'm going to interpret your comment on the assumption that you did not first steal the raw materials for you physical products somewhere else. Unlike Anthropic.

It's a shitty practice for you, but not for the consumers if the quality is still the same. Yes, you should be mad and try to block that from happening, but that doesn't mean that is a bad thing for everyone

"altruists"

I might frame this comment from you. I'm honored.

And literally the whole reason to offer an AI service is...

The comment to which you're responding isn't righteous indignation, it's a reaction to the righteous indignation within the article. ie. the same as your reaction, but applied one level up.

Additionally, there's a vast (repeated loudly with cave acoustics: VAST! Vast! vast! vast ...) difference in scale between one person reading a fishing book and teaching a handful of people how to apply it and what the frontier AIs do. Vast! With a coupla trillion dollars behind 'em. It's a silly analogy if it's an analogy at all.

The problem with all this is that techbros can't spell 'consent'. But when it is 'their' (for small values of their) added value then suddenly consent is all they see.

Did you steal this metaphor from a written work without compensating the writers?

yeah, and if the resoonse is off the rails you cant catch it earlier and reduce the cost. i genuinely think this shoukd be illegal

recent claude models smear away personal taste when its in a continuum of vocal opinions and not corrected frequently. this drift is a cross domain transfer of the stupid political neutrality when facts are “contended” publicly. it results in some nauseating opinions about current us government wrt culpability for dead children. you can replicate this yourself, just ask any >= 4.8 model about moral culpability of wh for usaid termination consequences.

also my setups for making models reliable requires reified jointly inspectable deduction traces. its also fristrating because aside from context limits ive gotten incredible work out of claude 4.5 in the past, despite its context limits being a serious bottleneck. but those reliability things interact terrible with all the safety filter bs.

to be clear this is a customer/consumer rights issue. and the real safety issue. if they are so concerned about unsafe distillation, make it easier to fucking do alignment respecting distillation. no amount of any distillation benefits any fucking paying user whatsoever. its literally a tragedy of firewalling a compilation of public commons because you raided all the book stores and libraries first

edit: im also on the market atm so if anyone wants some smart ocd focused on actual interactive agent reliability, while all my black box tooling isnt public, aside from my lack of public marketing im hard to beat

The prompt and most reasoning is implicit inside the codebase. From every subsequent commit one can easily infer what the bugfix prompt was supposed to be or reasoning failures are.

All these repositories can easily be a part of the pre-training / post-training datasets. Perhaps the distillation the blog is speaking about is for text-generation not the code generation.