But the analogy still holds.

The original authors of all the text, creators of the media and developers of the software did far more work than Anthropic.

I get that angle but it’s a weak argument as Anthropic is doing the same to others. Also while there’s certainly a lot of computing power needed to do what Anthropic does, it’s increasingly clear there isn’t much secret sauce involved. Everyone knows how do to the core work it’s just a question of who wants to burn billions on compute to do it.

Anthropic’s anger here seems mostly rooted in their annoyance that this exposes they don’t really have core IP that’s not just easily replicated. And that’s clearly a problem for a deeply unprofitable company trying to convince people they’re worth $2 trillion.

> What Anthropic is doing requires way more resources than what the Chinese labs are doing.

Oh that’s very sad.

Meanwhile Anthropic made a product from the work effort of millions of people without compensating them, sell that product on tap and unless I am mistaken do not even have their competitors’ cover of having released any sort of meaningful open weights model.

They have taken from culture (including very specifically their most direct customers’ specific culture — our culture), turned it into a machine to make themselves rich, appear likely to predicate their valuation on permanently removing people from the workforce, then want to dump themselves onto pensions funds and ordinary savers to carry the bag.

It is, I agree, philosophical, because karma is a philosophy as well as a bitch.

And the communal work of humanity is orders of magnitude more work than what anthropic pays for their scraping of content. I got no check from them for my contributions

Isn't it better for planet? By not doing the wasteful transformation work again

Distillation doesn't "grab 95% of lab's work", that's ridiculous. At best it's tiny icing on top of the cake that's already there. It's not even necessarily done on a better model (e.g. GLM 4.7 distilled Gemini 2.5, a weaker model), I'm pretty sure A\ and OAI could do (or even do) the same with greater efficiency since they have access to logits, weights, and internal state of open models.

>An argument can be made that Anthropic is also only doing the last 5% of the work (because the content they are training on was the other 95%) but that's a bit more philosophical.

How is this philosophical? They should release the unsupervised pretrains, at the very least.

But that’s not more philosophical. It’s a perfect parallel! Enormous amounts of work, vacuumed up and resold. What’s the difference? If it’s ok to vacuum up all the knowledge in the world, then that includes knowledge of how to use all that to power an LLM.

Wait, wasn’t 95% of the work creating the content in the first place?

Yeah, you need a lot of resources in order to waste a lot of resources. Look at codex and Claude code - these trillion dollar companies 'with top talent' cannot build what open code and pi/oh-my-pi have built in the open for free? Both codex and Claude code, are slow, buggy pieces of shit (and I say that as someone who still heavily uses both for work, moved to open code and pi for personal stuff). The reality is these companies mostly focus on marketing and market capture through, non-competitive means - their services and software are unreliable, buggy trash.

Can you elaborate on why the second is “a bit more philosophical”?

I see absolutely no distinction between the two, aside from minor technical approaches to gathering the content.

> but that's a bit more philosophical

It sounds exactly the same, not more philosophical to me, except one is more inconvenient.

Conventional wisdom is Google did all the groundwork with LLMs...

It is kind of ironic that they scraped the web for publicly available data and used it freely to train their models and now their freely available models are being used to train other models.

I'm overall pro-Anthropic and pro-banning open-weights AI, but I agree with the parent commenter; distilling Claude models is not that different from pretraining on web data. It's all basically the same sort of thing.

I think a good litmus test here would be if Anthropic were to not care about distilling their models when the distillers keep the resulting models closed-source and sell tokens via an API. If they cared only about security concerns and not about people profiting off of their work, then they should be publicly fine with this and only protest against it going into open-weights models.

Sounds more like a Western vs. Eastern outlook on innovation and how you accomplish it.

Is hypocrisy a philosophy?

They spend 95% of the money, perhaps, but burning compute is not the same as doing the work.

> An argument can be made that Anthropic is also only doing the last 5% of the work (because the content they are training on was the other 95%) but that's a bit more philosophical.

Actually, it's an interesting argument to make. How many labour-hours went into creating the training data Anthropic has collected? Probably multiple billions of hours. How many labour-hours did it take them to setup the datacenters, scrapers, and training algorithms? A few thousands hours?

I mean, 95% of the work if you don't factor the work to create the training data in the first place...

You are going to be surprised to hear how many resources were necessary to create all the data Anthropic is digesting

Lol, no. The original authors of all the text Anthropic took in did 95% of the work, Anthropic did 4% of the work and the Chinese labs do the last 1%.

So Anthropic and other US AI companies... stole harder, and therefore deserve more?

> What Anthropic is doing requires way more resources than what the Chinese labs are doing.

And writing a book requires many more resources than what anthropic does

“We’re both thieves, Steve. We both stole from xerox. You’re just mad I got there first.”

[flagged]

AI training, if viewed through the capitalist mindset, is plain theft. Anthropic can't morally defend copying someone else's IP, but denouncing others copying Anthropic's stolen IP.

That doesn't mean they won't try, and that also doesn't mean they won't succeed.

Anthropic and OpenAI are very happy to ignore the IP rights of others, so I'm not sure how they can ask for any kind of IP protection themselves. Live by the sword, die by the sword.

Capitalism: moral rights exist when they give us a moat.

Model weights wouldn't be covered in a patent. You could patent a method of creating weights in a model, but you couldn't patent the weights themselves.

I wish people here could at least bother to inform themselves about the IP rights they are so quick to insist are abhorrent, when they seem to not even have a first clue as to what they actually cover.

Caveat: cost of creating human knowledge/art >>>>>>>>>> cost of original training >> cost of distilling

You could say Anthropic distilled human knowledge and art.

It's "free" as in beer, not free from copyright. LLMs are free from copyright on the other hand. So which is more "free"?

Sounds like Anthropic should close up shop then, those chumps are offering their product on the internet for any random loser to distill

It's not that the Chinese companies are right, it's that Anthropic has no place to complain about stealing.

> They probably all think about Mash and a vodka still, and a completely unrelated association.

I don't know anyone with even a passing understanding of how LLM training works that thinks that is the appropriate analogy.

Then Anthropic should stop saying it. So long as they try to play victim here folks are going to call out their BS.

It ain't gonna happen. At work I have a dropdown menu in vscode with a dozen models to use interchangeably. They're all essentially commodities and will compete on price and squash almost all profit margin.

Sorry but it’s looking more and more like the top American labs won’t have any kind of moat.

They meant distilling in a more original sense, not per se in the LLM-era meaning of the word sense.

Oh no, they might stop doing the thing that benefits me and that they were never required to do in the first place.

As opposed to what? The US? You think the US is any different? Literally our pedo president publicly admits to insider trading. You think the US government gives a rat's ass about the American people?

There's no "pulling back" things that have already been open sourced.

Every workplace is an authoritarian regime where workers don't have a say and grovel with "please don't replace me" like that isn't the entire point.

The whole point of AI is to get rid of you so rich people can play with the planet like it's minecraft. Software engineers just think they're special because they're the ones building it like they'll get a pat on the head for being good little servants to the investor class. Or worse, that their portfolios will let them be the gods who rule over ashes.

Yes, we already discussed the US.

It is in the interest of everybody-except-OpenAI and Anthropic that those companies have capable competitors; better still for the competitors to share their advances.

They don't have to be our friends to act in our interest.

Agreed, the US is not your friend and will pull the rug from under you at the first convenient opportunity.

Just like how Anthropic and OpenAI is already doing?

China is much more authoritarian than the US, but at this point it's like comparing two types of metastatic cancer.

The point is get what you can from both to develop open models, data and tools.

You can be grateful to individual things an enemy does.

Ironically the US is the only country that has done that

While your friend is Sam Altman, or US megacorps ?

Or did they not pull back when their models allegedly became highly capable, with the whole mythos debacle ?

> An authoritarian regime is not your friend

Absolutely. In fact, the authoritarian regime already did try to force export controls on major frontier labs not long go so this isn't a theoretical.

This is equally true for US AI

Chill, Buddy. Why are you so anti-American?

There are several? And there exists companies whose entire business is just providing them? iirc

It's particularly important we all scan and destroy our own unique books!

Given the amount of people complaining about crawlers in the past 2 years, I think it's the latter

That's ok. They took a few trillion worth of content and burned a couple of hundred billion of their own money. I really don't see the problem, if they didn't think about this long and hard before they went down that part it should not be on the rest of us to bail them out. The 'frontier' is less important than the democratization process.

The amount of progress that came out of academia and other public sources should not be underestimated either and without all that OpenAI and Anthropic wouldn't even exist.

If food has harmful uses, and I think any one sensible would have to agree that it does, then giving everyone unrestricted food is likely to make it worse.

You can do this with cars, tools, computers, ... whatever you want. So, no, I think your point is wrong.

I simply don't trust the people who would decide who gets AI (or guns) to make good choices.

All concerns balance against competing concerns, and in this case freedom of computing and knowledge wins over safety. Especially since it’s trivial to copy and share open models.

Gun owner enters the conversation. In high trust societies, armed people are very polite people. I don’t want the bad guys out there being the only ones with guns. Besides I like to shoot just like you like (whatever you like to do that is legal).

Replace “gun” with anything and you will see how your comment falls apart.

What’s next? A registry for food purchases? Your beer gut is starting to show.

> giving everyone unrestricted AI is likely to make it worse

Worse for whom?

The only effective defense against predatory corporate and government AI is personal protective AI.

Anything else is unilateral disarmament. It's the only way individuals can survive in the worse case scenario.

> gun nuts

Guns are different. They can't protect you against the government, contrary to gun nut claims.

I would say that there are very few people on earth that I trust less than Sam Altman, Dario Amodei and Elon Musk. Also my own government claims to have used Anthropic models to bomb a girls' school in Iran. If you combine US regulations with sociopathic private companies, you get into a worst case scenario for humanity imho. Again, I'm thankful to China or any other entity pushing open models, local models and even distribution of this technology.

You are aware of the rule that any company has to have a CCP representative in the leadership?

(Similar to how U.S. labs have NSA leadership)

https://www.anthropic.com/research/glm-5-3-and-the-spread-of...

> If China ever gets ahead, they're going closed source and weights immediately.

Anthropic itself admits that Chinese models are merely months behind. Your argument does not make sense, because Chinese labs are contributing massive optimizations like the one this post is about.

>People are so ready to praise China

Please quote people you appear to be patronizing. China can't do anything about previous released self-hosted Chinese models. If you can show that local Chinese models funnel vast amounts us data home I'm sure you can move a lot of people to your side.

Comments like this also always fail to address why there aren't Western AI companies doing the same thing. Is it because they might get sued into oblivion by Big AI in the US?

It might be better for all of us if you solve that first instead of repeating something the government has been repeating for the last decade or more. It does this, mind you, while sabotaging itself in countless high-tech fields and leaving it all to China for the taking.

> For example Google published the transformer architecture and released BERT as an open weight model, but doesn't really even talk in public about the (presumanbly) specialized internal models behind revenue-generating products.

That's giving a lot of credit to Google's organizational ability to productize Google's research...

From a diplomatic perspective, the US is quite busy alienating itself from all its allies, just as it is trying to lock down and totalize a major breakthrough technology. What better way to cement the alienation of the US than to show both its hubris and its selfishness at once by demonstrating how anyone can do what they do?

So much that you can instantly tell if a model is Chinese by asking it about Tiananmen Square.

I’m pretty sure they meant they believe in the 10% risk of everybody dying but are ok for all of us to take that risk without our consent because they saw a 4year old tragically die. Which sounds completely unhinged, to say the least

Most people aren’t okay with that, but the blame lies in the people who deployed the AI tool in that situation, not the makers of said tool.

This really is a place where the guns-don't-kill-people argument applies, even moreso than guns themselves. The U.S. government massacred school children in Iran. Why does it matter how they targetted them?

Well, I have a $750 card that runs at about 50-60% of that token rate :)

In all fairness there are probably a lot of folks who picked one up for around MSRP (even if one of the board partner cards with an MSRP 10-15% over the FE).

Local inference will have a boom of cheap, powerful, and available cards at some point (even if it isn’t until 2028/2029). At some point the hyperscalers, and frontier labs, will face the capex problems that everyone talks about, and NVidia, AMD, Apple, and Intel will want to keep selling products.

Powerful, by today’s standard, local inference needs to be accessible to really unlock the “AI” economy long term. It’s just like how the move from mainframes to the PC 40ish years ago unlocked the “computer revolution.”

$9k is a small price to pay to experience the rapturous glory of AGI. I'd easily pay up to 3 times that to comfortably run the superintelligent models released in this post RSI world.

Indistinguishable or very mildly better. But it's considerably faster. Some portion of that is also probably down to improvements in model harnesses, I've been using opencode.

I’m not an AI researcher, but it seems like there’s a ton of waste having a universal model that knows everything when any individuals use case requires like generously 10% of what’s stored in the model. Does it even need to have memorized knowledge stored in the model or could it just look up info and docs like humans do? If all you need is the language and intelligence, I think Opus5.5 equivalent intelligence will run on an iPhone within 5 years.

[dead]

unless the major players collude they don’t get decide if they are in a race to the bottom. The best model was compelling 6 months ago when everyone was too impressed to care about price but that has worn off now and clients are paying attention to price. The best model is no longer a license to charge any amount.

Costs dropping opens you up to competition on price.

There isnt a lot of money in enterprise ai. Also my enterprise company gives me glm.

They’re not contradictory positions. Inference is too expensive now to make money because the industry is immature and hasn’t yet optimized for financial success while customers don’t care much about price because they’re more concerned about not missing out.

Inference will be too cheap long term to make money because it is being commoditized and customers will start to care about results and not just be wowed by impressive technology.

And of course this technology will continue to exist but that is irrelevant to the business. OpenAI investors don’t care if LLMs exist in 10 years, they care if their investment in OpenAI has made money.

Right now you are right -- even if I ran my local LLM all day, the quality is not nearly as great, and it runs slowly -- so I use the tier one AI subscription services as they are faster and smarter. But that might only be true for a limited amount of time, and a limited number of circumstances.

To borrow your steam engine analogy, if local LLMs get as good as a Toyota Prius, even if OpenAI / Anthropic offer Ferraris, most people will be happy with their Prius as their daily driver.

Similarly, if the big labs start raising prices or cutting usage, you won't be able to use it as much as you want -- whereas a local LLM will run all day every day without costing you any extra money.

So right now you are right, but who knows how long that will last.

I drive my 2018 car to work these days. It is good enough.

This will change when out of control billing gets noticed. Then us devs will, I presume, get token rations.

I can see my new Thursday afternoon "oh chit" moment being that I didn't complete my weekly task because i torched all of those tokens M-W doing task/ticket grooming using the hot hot model instead of the dodo model with jira mcp connector :D

> For nearly all tasks, I want the fastest and smartest AI model.

This is not nearly true for everyone else in the world.

For example, think about the world in ~2021 pre-LLM. Would anyone say the sentence "I only want the fastest and smartest humans working on my project"?

No of course not. Most people don't want to pay $10 million dollar salary to the best programmers in the world. They prefer to pay $200k salary to a median programmer and that's good enough for their ecommerce website.

I think a scooter or a bus is a more apt comparison than a steam engine. The scooter and bus can both get the job done with some acceptable trade-offs, depending on your circumstances and what you’re willing to accept.

However, there are many use cases where they aren’t the right tool for the job.

I find this too. In fact, recently I've been pushing more and more to the latest and greatest model every time there is an update. It just saves me so much headache.

Genuine ELI5 question: in this metaphor, which "old" models are steam engines at this point? Which are Model Ts, etc.

The margins on NVidia datacenter hardware are ... high. At least one order of magnitude larger than a consumer chip.

Given the recent deepseekv4.1 advances - how good of a 3B model can we make to run on an iphone natively? is it good enough to match common muse/dot use cases for consumers? the phone is already always on.. no need for a cloud server.

Yes, competition is great for us.

I wonder if margins on GPT-6.1 Sol and Opus 5.5 are now 75% or 90%.

What a strange way to put it. Market forces are a good thing in a market economy. Only someone who secretly wants or hopes for monopolies would you say something to the contrary (a VC).

I'd call it overcapacity instead. A lot of investment without capital discipline making sure there is actually return of investment leading to too much supply.

Not that OpenAI, Anthropic or SpaceX aren't doing the same.

Nice read. American media does a terrible job of covering this.

>you can't actually make money in China doing China things

Do you do business in China?

I'm curious what you mean by this? Because in my experience, you can only do business in China by doing "China" things.

I'd be interested in picking your brain as to how you get around those issues?

> they cannot coordinate and the state won't let them coordinate

That is how actual capitalist market should work and what anti-monopoly legislation should ensure.

LMAO recession, China? You've been reading too much Brad Setser

This looks like a speed run of the history of analytics DBMS’s.

Once upon a time, everyone had a secret sauce in network or data encoding or query optimization, but in the last ~10 years computational physics and economics have basically decided the “correct” architecture and everyone (including OSS) has converged.

Commoditizing their complement. Makes sense.

They certainly thought. But were they able to do it now without DeepSeek papers?

timeline suggests not.

I dunno, they've been competitive in the lies market lately. It's just a matter of ambition.

Did you not see the meeting with the President yesterday? All of the “safety discourse” was kayfabe.

> It's not that niche, if you've been online a little bit you'd know this expression.

No way. You'd need to be pretty well versed in gamer lingo. Even more specifically, combative, likely FPS gamer lingo.

I dunno, man, I first got on the Internet in 1986 and was Cloudflare's CTO for years. I've been online quite a bit.

Also, it's possible that jgrahamc is the former CTO of cloudflare

> if you've been online a little bit you'd know this expression

I've been online since circa 1995 (earlier if you count BBSs), and I can't say I did. It's possible to infer its meaning but assuming everyone is on the same circles as one is, is silly.

Yes, but to expand on my flippant response, the opening of this post is a sign of bad writing. And I don't want to waste my time on bad writing.

It shows that the author hasn't thought about their audience and has assumed that everyone knew and used the same terminology as them. And, worse, assumed that they'd understand immediately why they were using that terminology.

The opening is If you read the news headlines these days, you would be forgiven for thinking that the Western labs are getting spawn-camped by Chinese labs en masse. If you don't know what spawn-camping is you're lost; if you do it's not obvious what that means in this context. Good writing brings the reader along with the writer.

It would have been clearer if they'd written: If you read the news headlines these days, you would be forgiven for thinking that Western AI labs are being outplayed by Chinese AI labs using something similar to the gamer technique of "spawn-camping". The Chinese appear to be waiting for each Western release and then instantly distilling it. A little like gamers waiting for their opponents to reappear (from the dead) at their camp and then kill them off immediately.

This is better because a reader unfamiliar with the idea of spawn-camping learns something and it explains the metaphor. But I could be wrong in my interpretation of why they are using spawn-camping since they fail to explain it.

I am talking about correlating Anthropic/OpenAI cache prices going down with Deepseek publication - neither of those labs have said that's what they used for example.

And the only data they are showing is that cache prices went down for new Claude/OpenAI models but that's proving nothing, IMO.

This is the exact definition of rumor.

[dead]