Step 2.7’s MTP seems broken (at least for ik_llama.cpp) where the draft model starts and ends in block 3 but ik_llama bails out looking for block 0 :(

No more revealing than OpenAI, Anthropic and Google always having some new model that just so happens to be waiting in the wings whenever their competitors announce their own model bump.

I think you are reading a lot into this.

There's always an announcement by one of the frontier labs.

> They're backed by a quantitative hedge fund that views AI as infrastructure, not as a product to monetize directly. The ROI for them comes from trading alpha, not API revenue.

That used to be true, but now they've raised ~7B$, so we'll see how / if that changes.

Also, we’re seeing a classic commoditization spiral with open models rapidly closing the gap and driving prices towards the marginal cost of inference. The reality is that models themselves are general commodities and there's just not enough difference between them. A company can get ahead of others by a few months, but then the rest quickly close the gap. It's a really low margin business because there's no way to differentiate yourself.

Chinese companies understand this and they're treating models as shared infrastructure akin to Linux. The money is going to be in customization niches. Companies will charge to tune models for specific use cases and charge support for that. There's also going to be money at the bottom for hardware vendors making chips and memory. But the middle tier of generic LLMs is seeing involution where there's relentless competition driving profits towards the bottom.

And OpenAI has the audacity to call themselves a non profit. All leading US models are closed source for one purpose, money.

Nope. It is purely a marketing and distribution strategy. Without open sourcing their models, their businesses would have never gotten off the ground. I've written about this here: https://try.works/writing-1#why-chinese-ai-labs-went-open-an...

[dead]

Open-source is also altruistic. If DeepSeek does become self-serving once they get the top spot, it doesn’t take away from the altruistic contributions that they made towards open models.

Is it though? A large number of people get to experience a lot of power over others because they moderate Wikipedia. That's certainly why some of them do it, just like on Reddit

This statement is factually true and you are voted down because many people lack knowledge.

Any individual that provides free labor cannot survive off of said free labor. He must work for money to survive or get donations from someone who earned that money from incentive based labor in order to even buy the food he needs to exist as a living human being. Much of the time that labor is actually closed source.

This is a logistical reality. A lot of open source advocates are unable to get their brains out of the whole mentality that open source literally cannot exist without incentive based software supporting it. Who pays for GitHub to exist? Who pays for the food swes eat? I just code for open source all day and money falls out of the sky.

My smart friend says there are jobs that pay you to work on open source exclusively. Smart guy. In this case you follow the money trail. How does that company get enough money to pay a guy to work exclusively on open source?

I hate to quote pithy proverbs, but "the road to hell is paved with good intentions." One can have an altruistic goal which ends up harming people too, which is where that proverb comes from. Prohibition and The War on Drugs in the US are two good examples of something that had altruistic origins[†] but ended up doing way more harm than good.

[†] Another problem with altruism: we don't all agree on whether a goal is altruistic, and what's altruistic in the enactor's eyes might not be in yours. Curating a fountain of human knowledge like Wikipedia? Probably altruistic. Protecting humanity from itself by installing your company as the stewards of frontier LLMs? Not so altruistic in my view.

Go read Max Stirner. True "Alturism" doesn't exist. It's all egoism, even if and especially if you think it's not.

Look at how far OpenAI has drifted from their original mission. Everything comes back to greed, so it's ideal for the world if selfish motives happen to coincide with what's good for the world, like advancements in open models

It's a standard take since it is how markets tend to work. They aren't powered by altruism, it is a big system for turning greed into good results. We don't have all this stuff because people suddenly woke up one morning and decided to be nice.

I don't understand what is interesting about it: it's the default.

Markets don't run on altruism.

Isn't it the entire basis of capitalism?

I don't think so. I can confidently predict that altruism will give you a very unreliable income stream in the vast majority of cases.

Disagree. It’s More reliable.

Could you explain? (asking in good faith)

Extremely interesting comment, thank you. Got some links where I can download this source material? I don't read or speak the language, but will try interrogating it with an LLM

I 100% agree with you and want to add something.

If you simply take what the Chinese government says at face value, you will be correct way more often than 95% of Western policy wonks, media talking heads, "analysts" and so forth. Because, like you say, they tell you everything they're doing.

In the recent US-China summit, Xi Jinping just came out and used the Thucydides Trap metaphor, which tells you everything about where China thinks it is and where it sees the US going, which is to become increasingly belligerent as their power declines. Now whether or not you agree with that assessment (I do agree), it still tells you China wants to avoid open hostilities, it sees itself as continuing to rise and it fears what a declining US might do.

> And they seem mostly happy with sticking to speculations that match preconceived notions

...you've almost achieved enlightenment on the nature of the majority of HN comments: vibes-based, off the cuff braindumps, where an idea is examined as it is being typed. It's great for tech and software discussions where many commenters have good knowledge or even mastery, but on "exotic" topics, an incorrect take can be voted to the top because it sounds right by affirming the biases of the majority. If you're a practitioner in a non-tech field and a topic in your field comes up for discussion on HN, be ready to be disappointed - and be ready to question all the other correct-sounding comments in areas unfamiliar to you.

Aspiration for the masses and policy reality are different things. They will say monopolies are bad, but what they mean is only if the state doesn't control them. The CCP has huge state run monopolies and there's nothing you can do about it. You can't understand their policy without understanding Marxism-Leninism. When pesky reality gets in the way of policy, it's called corruption.

In communist or post-communist countries like China and Russia, the percentage of government workers is extreme. To them, all social action is political action, and that includes economic. Since there is only one party allowed, any economic action (and thus political action) which threatens the CCP is unacceptable. They leave other private companies alone.

The CCP is bad and this conclusion is justified not only in theory as a distant observation of ideological concepts, but from their behaviors around the world which echo some of the causes of World War 2. That doesn't mean Chinese AI companies are bad, but the CCP will certainly find ways of using it for its purposes. For now, they're quite far behind on AI, but they deserve credit for optimizing for some use cases which masks poorer generalization.

I had the most ironic rollercoaster ride thanks to your comment.

I copied it into DeepSeek because I figured who's better to teach me about greatness of Chinese government policies if not the most popular Chinese LLM?

Anyhow, it must have detected _something_ in your comment because Chinese censorship policy kicked in and DeepSeek refused to talk about it. Funny because I would wager the overall sentiment about China's abilities to govern in your comment was positive but okay.

I literally asked it "expand on it so I can learn more about Chinese policies" and that was enough to get censored!

Anyhow, after saying few times "huh? but I want to learn what's great about Chinese policies!" it finally gave me response in... Mandarin. So I asked it to provide me that information in English and... it refused. Talk about difficulty of finding any materials in English ;-)

After starting a new chat and using plenty of positive adjectives to make sure I don't want to learn a single bad thing about China, I finally got a list. Looks like China is 100% successful in everything they do! How neat!

So I said "That's cool. Does this set of policies have one name? Can you recommend any books about it? In English of course" and...

...request denied again.

I mean, maybe it says more about how horrible DeepSeek is for this kind of research but boy, it was so ironic I now have stack of iron at home.

Peace for who? Like just look at the last 50-70 years of US intervention in LatAm. Backing a series of coups and extremely violent right wing dictatorships.

The issue is one of incentives. The US needs cheap foreign labor because of deindustrialization policies in the 60s and 70s. These were arguably passed as a check on labor power since socialism was still looking potentially ascendant at the time. Whatever the reason though, the contemporary US is reliant on keeping foreign wages down and domination of the oil trade. Imperialism grows out of this need for external resources to maintain economic growth. It would be less relevant to us if we had better fundamentals, but we traded those away to avoid letting certain demos get wealthy and powerful.

China's play is more mercantile. They benefit most from stable trade conditions. They get richer the more customers they have. They benefitted massively from becoming an industrial trade partner with the US in the 60s and 70s. Because of this, they have completely different foreign policy objectives. All they need to do to win is normalize relations and build trade infrastructure. Its way cheaper than imperialism.

https://en.wikipedia.org/wiki/United_States_involvement_in_r...

As good as Gemini's visual intelligence is, it's a terrible agent.

Thank Apple?

Those are mostly for embedded devices and the current "sponsor" is Apple.

Aren't they only open weights, not true open source?

It'll be their inability to build coherent products that dies them in, not their models.

You ask me what I think. So far deepseek has been very consistently trying to advance state of the art research in a transplant and public way by writing papers and publishing working code. They are also not at the mercy of the stock market in the same way many Americans companies are. Before anyone assumes too much, I live in Europe.

You mean open weights, I guess? There are as far as I know very few open source models, the training data is seldom released. Sadly.

For real. Reading old comment threads makes me sad, because the level of discourse was so much higher in the past. Although this place is still deeply appreciated, it’s clear that its culture is going monotonically towards reddit.

Is there anywhere public anymore that isn’t being overrun by lobotomized p-zombies (partisan zombies)? Is it even possible to make such a public space? Ressentiment consumes all discourse.

Yet accumulation of power by a very small elite through state and selected corporations happens to be a defining characteristic of that political regime.

you're right, full of corporate sock puppets shilling their vapor wares, idly dreaming that the world isn't what it is.

The investment round only closed in the last few weeks, it would have had zero influence on anything up to & including DeepSeek V4.

Whether that now changes, who knows. It appears the CEO will remain the largest investor/shareholder, though, so I'm hoping not much changes.

Access to everything every American company feeds into the AI is well worth it to the CCP.

According to EU statistics, yeah

Does that figure hold up when we look at Silicon Valley financing? Uber alone was subsidized to the tune of billions. Let alone the recent batch where we're into hundreds of billions.

Even the latest World Bank report, the defacto neoliberal institution, recognized a couple of months ago that leaving the industries focus be dictaed by purely capital decisions was bad, as in _really_ bad.

You can give them money by using their api. Just because their model is open, doesn’t mean they are a non profit.

Not everyone has the American “fuck you got mine” zero sum game attitude. Also they’re making some of the American and European AI companies look bad which they can leverage with their trades if they wanted to.

I think there are some sockpuppet accounts active but what also contributes is that many people are absolutely fed up with US technological hegemony and welcome alternatives to core technologies from elsewhere.

Hasn't that been the mantra of open source for 40 years. Armies of companies, trillions of valuation, or even just Wayland, suggest that isn't always the case.

> This will result in the Chinese companies making progress faster. Full stop.

Is this happening? These open models have been a generation or two behind the closed models for quite a while now. They've been keeping pace but clearly behind.

Why would they collaborate? Why not defect and just keep theirs private and implement the open ones?

Just a crazy catch 22, it seems

> They don't develop them because they don't collaborate publicly anymore.

How do you know they aren't doing this stuff? Something has to account for them leading the industry.

Yes - I meant as applied to LLMs/Transformers.

You know this from? Any sources? I'd love to learn more because it would be one of the very few industries that still write assembly by hand extensively enough to warrant hiring experts on just that.

So I was curious about how A100s would do running DeepSeek v4. I can't find any instances of running v4 Pro on even an 8xA100 cluster. So you need to run Flash at ~284B params. A100s don't support FP8 so you're running FP16 so you're taking a hit that way. But I see estimates of 30-50tok/s for an 8xA100 cluster. They're drawing 300-400W each so you're looking at probably 3500+ Watts, which is roughly 0.01tok/W.

Now jump ahead 2 years and you seem to have a massive jump in performance [1]. The tokens/Watt goes up by at least 2 orders of magnitude. And the B100 is 3-4x that. And we're about to hit the R100 (Rubin) cliff.

That's what this is going to come down. When hyperscalar DCs are getting to Gigawatt power usage, it all comes down to power efficiency. Those A100s aren't far from being sold for scrap.

I've been looking into how different companies are handling depreciation for this. Amazon seems to be saying the life is 3-4 years, Google 4-5 and Meta is saying 8+, which I think is wildly optimistic.

[1]: https://lambda.ai/inference-models/deepseek-ai/deepseek-v4-f...

It wasn't a claim about "how China works" it was a humanistic point that 350 million people should not envy the progress of 1 billion people on the other side of the world.

It's not in the interests of the vast majority of either country to engage in conflict. American citizens should therefore resist the professed bellicosity of leaders who will never see any consequences, on a battlefield or otherwise. The average American will not be so lucky. Chinese innovation in a world of cooperation need not be less beneficial to Americans than the reverse.

I see. Fair point

Self-distillation is also a technique.

Long ago, OK.

Recently, maybe. I see a lot more copycat behavior however.

True. But at some point you got to close your eyes and take a step forward.

It’s like with VPN providers. Is Mullvad actually collaborating with law enforcement? They very well could be. It is a calculated risk.

Is DeepInfra actually logging and training or selling the logs? They could be.