Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

335 pointsposted 13 hours ago
by hd4

209 Comments

kilotaras

11 hours ago

Alibaba Cloud claims to reduce Nvidia GPU used for serving unpopular models by 82% (emphasis mine)

> 17.7 per cent of GPUs allocated to serve only 1.35 per cent of requests in Alibaba Cloud’s marketplace, the researchers found

Instead of 1192 GPUs they now use 213 for serving those requests.

bee_rider

6 hours ago

I’m slightly confuse as to how all this works. Do the GPUs just sit there with the models on them when the models are not in use?

I guess I’d assumed this sort of thing would be allocated dynamically. Of course, there’s a benefit to minimizing the number of times you load a model. But surely if a GPU+model is idle for more than a couple minutes it could be freed?

(I’m not an AI guy, though—actually I’m used to asking SLURM for new nodes with every run I do!)

svachalek

5 hours ago

Models take a lot of VRAM which is tightly coupled to the GPU so yeah, it's basically sitting there with the model waiting for use. I'm sure they probably do idle out but a few minutes of idle time is a lot of waste--possibly the full 82% mentioned. In this case they optimized by letting the GPUs load multiple models and sharing the load out by token.

jychang

an hour ago

They definitely won't idle out- if they idle out, it'll take on the order of up to 60 seconds to load the model back into VRAM, depending on the model.

That's an eternity for a request. I highly doubt they will timeout any model they serve.

andy_ppp

2 hours ago

How does this work with anything but trivially small context sizes!?

jychang

an hour ago

Tensor parallelism, so you only need to store a fraction of kv cache per gpu.

smallnix

6 hours ago

> I guess I’d assumed this sort of thing would be allocated dynamically

At the scale of a hyperscaler I think Alibaba is the one that would be doing that. AWS, Azure and I assume Alibaba do lease/rent data centers, but someone has to own the servers / GPU racks. I know there are specialized companies like nscale (and more further down the chain) in the mix, but I always assumed they only lease out fixed capacity.

citizenpaul

3 hours ago

>Do the GPUs just sit there with the models on them when the models are not in use

I've assumed that as well. It makes sense to me since loading up a model locally takes a while. I wonder if there is some sort of better way I'm not in the know about. That or too GPU poor to know about.

yorwba

5 hours ago

The paper is about techniques to do that dynamic allocation to maximize utilization without incurring unacceptable latencies. If you let a GPU sit idle for several minutes after serving a single request, you're setting money on fire. So they reuse it for a different model as soon as possible, starting even before the first request is finished, because: If you don't have a dedicated GPU for a model, are you going to wait for a multi-gigabyte transfer before each request? So they have a dedicated GPU (or two, one for prefill, one for decode) for a group of models that are processed in an interleaved fashion, scheduled such that they stay within the latency budget.

make3

6 hours ago

the models are huge, so not a single (latest gen) one can fit on a single GPU.

It's likely that these are small unpopular (non flagship) models, or that they only pack eg one layer of each model.

svachalek

5 hours ago

Per the very short article, the solution was to pack multiple models per GPU.

make3

3 hours ago

yes but that could mean a layer per model

yorwba

10 hours ago

Not really, Figure 1(a) of the paper says that the 17.7% are relative to a total of 30k GPUs (i.e. 5310 GPUs for handling those 1.35% of requests) and the reduction is measured in a smaller beta deployment with only 47 different models (vs. the 733 "cold" models overall.) Naïve extrapolation by model count suggests they would need 3321 GPUs to serve all cold models, a 37.5% reduction to before. (Or 6.6% reduction of the full 30k-GPU cluster.)

somerandomdude2

8 hours ago

Really:

"A paper presented at SOSP 2025 details how token-level scheduling helped one GPU serve multiple LLMs, reducing demand from 1,192 to 213 H20s."

Which, if you scale it, matches the GPs statement.

yorwba

6 hours ago

From the SCMP article you might get the impression that the various figures all refer to the same GPU cluster, but in the paper itself it's very clear that this is not the case, i.e. the 213 GPUs in the smaller cluster are not serving 1.35% of the requests in the larger cluster. Then if you want to scale it, you have a choice of different numbers you could scale, and each would get different results. Since they're constrained by the limited number of different models a single GPU can serve, I think scaling by the number of models is the most realistic option.

MangoCoffee

7 hours ago

In the past, software and computer engineers would tackle problems head-on, designing algorithms and finding creative solutions.

thanks to the US restrictions on semiconductor industry (Chinese), Chinese engineers are being forced to innovate and find their own ways to overcome challenges like the old school engineers (What Silicon Valley used to be)

_heimdall

6 hours ago

If you're one who sees progress as an end goal unto itself, what you describe is a good thing. When one party is attempting novel solutions to outcompete the competition we will be faster to whatever the next change is.

That said, I'm not sure what the US policies specifically have to do with this. Countries are always in competition with one another, and if one industry or technology is considered a national security threat they will guard it.

djoldman

12 hours ago

Key paragraph:

> However, a small handful of models such as Alibaba’s Qwen and DeepSeek are most popular for inference, with most other models only sporadically called upon. This leads to resource inefficiency, with 17.7 per cent of GPUs allocated to serve only 1.35 per cent of requests in Alibaba Cloud’s marketplace, the researchers found.

make3

6 hours ago

these other models are likely much smaller

hunglee2

13 hours ago

The US attempt to slow down China's technological development succeeds on the basis of preventing China from directly following the same path, but may backfire in the sense it forces innovation by China in a different direction. The overall outcome for us all may be increase efficiency as a result of this forced innovation, especially if Chinese companies continue to open source their advances, so we may in the end have reason to thank the US for their civilisational gate keeping

dlisboa

12 hours ago

History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years.

In many senses there's hubris in the western* view of China accomplishments: most of what western companies have created has had significant contribution by Chinese scientists or manufacturing, without which those companies would have nothing. If you look at the names of AI researchers there's a strong pattern even if some are currently plying their trade in the west.

---

* I hate the term "western" because some "westeners" use it to separated what they think are "civilized" from "uncivilized", hence for them LATAM is not "western" even though everything about LATAM countries is western.

achierius

12 hours ago

> most of what western companies have created has had significant contribution by Chinese scientists or manufacturing, without which those companies would have nothing. If you look at the names of AI researchers there's a strong pattern even if some are currently plying their trade in the west.

While I don't disagree with your overall point, it's important to recognize that this is only a phenomenon of the last ~30 years, and to avoid falling into the trapn of Han racial chauvinism. E.g. there were ~no Chinese scientists in Germany in the 70s but they were heavily innovating nevertheless.

dlisboa

12 hours ago

Absolutely. China obviously has a longer history with innovation but they like to make it seem everything was invented by them at some point in the past. I'd say newer technology is where China has had a bigger impact.

Consequently newer tech is precisely where global cooperation is most required so no country can really do it by themselves. We could even say no country, western or otherwise, has been doing it on their own for the past 500 years or so but alas...

tsunamifury

10 hours ago

It’s more helpful to think of China as an accelerant rather than an innovator in this position.

huntertwo

7 hours ago

The whole “China copies everything” narrative is becoming less and less true.

It’s funny - it’s at the point with Chinese manufacturing for niche electronic goods (e.g rooftop van air conditioner) where some Chinese brands are more trustworthy - more value for your money and sometimes even better overall quality. With American brands you gotta make sure you’re not overpaying for dated tech that is inefficient. Maybe the same will happen with LLMs.

xbar

5 hours ago

It is less exclusively true.

hshdhdhj4444

8 hours ago

Ironically, the best way America could have prevented China’s rise in tech was by stapling green cards to diplomas of Chinese citizens who completed their higher education in the U.S. like the plan in the early 2010s.

ahmeneeroe-v2

8 hours ago

Is that the best way? China's rise had already happened by the 2010s

Preventing that could have been prevented in the 70s, 80s, 90s by stopping offshoring, blocking student visas, and prosecuting IP theft.

brookst

8 hours ago

Those students would have just gone to other countries, written their PhD dissertations there, advanced another country’s tech sector, and the US would have found the pain of isolationism that much sooner.

It is not possible to keep core IP secret. HN folks, of all people, should know this. Anything that thousands of people know is de facto public knowledge.

corimaith

5 hours ago

How are you going to gain core IP research if you don't have experience or access to leading edge researchers to pass you knowledge in the first place?

Talent is proportional to population, but that only matters if society and state has the infrastructure to raise that talent up. Otherwise Nigeria or Indonesia would be scientific powerhouses, and Iran would have modern fighter jets.

Cyph0n

4 hours ago

> Otherwise Nigeria or Indonesia would be scientific powerhouses, and Iran would have modern fighter jets.

The reason why these statements are not true is because of colonization, delayed industrialization, and Western intervention post-independence. Getting out of this “quicksand” is exceedingly difficult.

China did well to industrialize quickly and keep intervention at bay - in fact, you could argue that it making the rest of the world reliant on its industrial capacity helped address the intervention problem.

wagwang

3 hours ago

This world view is just wrong from top to bottom. Between 1945-1970, China was in a much worser state that any of those 3rd world countries in many aspects. Are you saying post 1970, the reason why these countries did not develop as fast as china is because of colonialism and intervention? You can apply the same arguments to Japan in 45 and Korea in 50. But but but aid? Ok and we send billions to countries all around the world every year.

Cyph0n

3 hours ago

But I just pointed out that China is an exception? Also, China had access to human capital at a scale that no African country had.

Japan pre-45 was a world power, and had industrialized by the early 1900s. WW2 was a mere setback.

Korea is more of a “miracle” than Japan was, but they also did well to industrialize ASAP. They also didn’t face the brunt of European colonialism.

markdown

2 hours ago

Aid is the single largest barrier to development and progress over the long term.

heavyset_go

3 hours ago

China wasn't beholden to neoliberalism and the World Bank and IMF.

China made the right choice to dump a ton of resources into different industries without the expectation of immediate RoI or any RoI at all. Anyone or anything that got in the way of their goals were dealt with.

ahmeneeroe-v2

8 hours ago

Keeping core IP secret is not the issue. The issue is allowing the Chinese to steal IP and then compete in our markets with that stolen IP.

>students would have just gone to other countries, written their PhD dissertations there, advanced another country’s tech sector,

which other countries specifically? No other country has a tech sector. It's the US hegemony or the China hegemony.

FpUser

7 hours ago

>"...The issue is allowing the Chinese to steal IP..."

I do not think they need permission. There is no force that could order country to recognize IP. Do you really expect all world forever pay rent to few giant corps?

ahmeneeroe-v2

6 hours ago

Bad reading comprehension.

You're talking about recognizing IP, I am talking about stealing IP AND selling stolen IP in our markets.

1st: yes force can be used to discourage the theft of IP. This is merely an obstacle, not a total blocker 2nd: yes force can be used to block IP from our markets. This is actually incredibly trivial and would have been very easy 40-years ago.

FpUser

6 hours ago

>You're talking about recognizing IP, I am talking about stealing IP ..."

If country does not recognize IP then "stealing" is not a theft in their eyes.

As for using force to prevent "theft": what force? Military? You might get burned really bad.

ahmeneeroe-v2

3 hours ago

Sorry no. If you can't imagine how one country couldn't use force to stop another country from stealing IP then you are either not intellectually up to this conversation or not arguing in good faith.

corimaith

5 hours ago

A military controlling the flow of goods into its own borders isn't going to "burn bad", it's called customs controls.

FpUser

3 hours ago

>"The issue is allowing the Chinese to steal IP and then compete in our markets with that stolen IP"

That was the original message. My understanding of "our markets" was customers of the US which include the US itself, China and many other countries. Sure the US can prohibit importing of China's goods. It can not control what happens in the rest of the world to the degree that it once could.

adventured

8 hours ago

The inclusion of China in the WTO is what changed everything.

The elites thought they'd set up shop in a new, gigantic consumer market and reap the rewards. So they got Clinton to spend his last days in office lobbying very aggressively for China's inclusion into the WTO.

China had different plans. Keeping the plunderers out (this time) was one of the smartest moves any nation has made in recorded history. Then the same elites slowly pivoted against China, post realizing they wouldn't be allowed to own China. If we can own you, you're our friend; if we can't own you, you're our enemy. And this is quite obviously not a defense of China's human rights record or anything else, that's not the point. China only mattered (in the enemy sense) when the elites realized they were going to be locked on the outside of the rise.

marcosdumay

2 hours ago

> The elites thought they'd set up shop in a new, gigantic consumer market and reap the rewards.

So... Seems that's exactly what they are getting.

delfinom

8 hours ago

Eh? They are a country with a culture that values education like a few other asian countries. They were set back by the colonial bullshit of Europe, devastation of WW2 and communist revolutions and so. They would have gotten to the same point as they are today, just longer if anything if any attempt was made to hamper it.

ahmeneeroe-v2

6 hours ago

You're 100% correct. "Prevented" is too strong of a word (that I borrowed from GP).

I would characterize my recommendations as things that could have been done for the US to not fund or encourage the re-rise of China.

lurk2

6 hours ago

Massive vector for theft of trade secrets and intellectual property.

It’s notable that China did not adopt the same policy during the period you are associating with their rise. Indeed, they’ve taken the opposite stance in recent years and (now that they have stolen American IP) have moved to seize control of assets and expel the superfluous foreigners.

There is a lesson to be learned there, but it’s contrary to the argument you are trying to make.

deadbabe

7 hours ago

But they didn’t do it, because the current administration can’t get it through their thick skulls that the key advantage the US can have in this world is a monopoly on all the really smart people.

heavyset_go

5 hours ago

It's just straight up low expectations and underestimation derived from racism in the assumption that Americans are smarter and more capable, and Chinese are only good for copying designs and making things we come up with. The idea that they can't do that like we can is pervasive.

sokoloff

2 hours ago

It’s absolutely fascinating and mystifying (and dismaying) to see educated and otherwise smart people in engineering in the US hold an opinion that two countries with over a billion people each can’t accomplish anything in engineering or science without “us”.

Despite ample and repeated evidence that they can and, in China’s case, that they’re the best in the world in several areas of manufacturing.

onlyrealcuzzo

11 hours ago

> History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years.

It's worked for a very long time for aircraft.

China has been pushing to build its own aircraft for >23 years. It took 14 years for COMAC to get its first regional jet flying commercial flights on a Chinese airline, and 21 years to get a narrow-body plane flying a commercial flight on a Chinese airline.

If for no technical reasons and purely political, COMAC may still be decades away from being able to fly to most of the world.

Likewise, in ~5 years, China may be able to build Chips that are as good as Nvidia after Nvidia's 90% profit margin - i.e. they are 1/10th as good for the price - but since they can buy them for cost - they're they same price for performance and good enough.

If for purely political reasons, China may never be able to export these chips to most of the world - which limits their scale - which makes it harder to make them cost effective compared to Western chips.

fooker

6 hours ago

> China may never be able to export these chips

While you type this, the rest of the world is already using Chinese cars, something that was unthinkable a year or two ago.

The US has closed the market off from this for its auto industry to survive.

Alupis

an hour ago

This statement doesn't make a lot of sense. 40-50% of vehicles are foreign-made already[1]. I would strongly wager it's vastly more likely that these Chinese vehicles do not meet US safety standards - which are quite high.

[1] https://www.whitehouse.gov/fact-sheets/2025/03/fact-sheet-pr...

lmz

39 minutes ago

Much higher standards than European ones? Because the cars do sell there as well.

Yoric

11 hours ago

> If for purely political reasons, China may never be able to export these chips to most of the world - which limits their scale - which makes it harder to make them cost effective compared to Western chips.

Note that this happens at the same time the US is breaking up its own alliances, so as of this writing, there's no such thing as certainty about politics.

ahmeneeroe-v2

8 hours ago

>the same time the US is breaking up its own alliances

This isn't happening. The US is driving a harder bargain with our allies. No one serious thinks anyone is walking away from alliances with the US.

redserk

7 hours ago

Why the framing of alliances like it’s a boolean?

The question of “can we trust the American government” is now being asked more often. Existing alliances and new potential alliances face that question, whether or not you personally believe that they should trust America.

Even if no concrete actions are being performed with asking that question, the fact that question is even being asked is a major drop from where we were.

ahmeneeroe-v2

6 hours ago

You're right, alliances are not boolean.

From the US perspective, we have been asking ourselves "can we trust Europe's military capacity" for a very long time and the answer (prior to 2025) was: NO.

With Trump on one side and Russia on the other, it seems like the answer has shifted to: MAYBE.

organsnyder

5 hours ago

> From the US perspective, we have been asking ourselves "can we trust Europe's military capacity" for a very long time and the answer (prior to 2025) was: NO.

NATO's mutual defense clause has only been activated once: after 9/11, when the United States declared war on the Taliban in Afghanistan.

Out of the 3621 deaths of coalition soldiers, 1160 of them were from nations other than the United States, including 457 from the UK, 159 from Canada, and 90 from France.

https://en.wikipedia.org/wiki/Coalition_casualties_in_Afghan...

ahmeneeroe-v2

3 hours ago

Yes I served with those people so I am not denigrating their sacrifice when I say:

2001 was 24 years ago. A lot has changed since then. Europe's militaries are much degraded and the threats are much enhanced.

Yoric

6 hours ago

Frankly, this sounds like you're repeating propaganda.

When the US called its allies to its wars, NATO responded. Now that the rest of NATO is being threatened, the US is playing neutral, trying to see which side will bid highest for their help.

brookst

8 hours ago

The US has become unreliable and erratic. Countries aren’t cutting ties or anything, but certainly investing to reduce exposure to capricious US leadership. Much of Europe is increasing domestic military production rather than just buying more from the US precisely because the US has publicly discussed leaving NATO and/or not honoring its guarantees.

ahmeneeroe-v2

6 hours ago

Increasing domestic military production is actually a great outcome.

Obviously Wall Street would have preferred purchasing from US-listed/owned arms companies, but from the perspective of a military alliance, having well-armed allies is the main point.

It's really hard to argue with Trump's methods if they led to Europe finally spending on their own defense.

markdown

2 hours ago

You've moved the goalposts

fxtentacle

6 hours ago

I observe serious financial commitments towards walking away from US tech:

The EU is pumping money into what they call "digital sovereignty" left and right. Germany just cancelled their Microsoft subscriptions and replaced them with self-funded Open Source for Schleswig-Holstein, which is roughly 5% of all government employees. That's one hell of a trial run. Germany's "OpenDesk" and France’s "La Suite numérique" even made into the new "Franco-German Economic Agenda 2025", which self-describes as "bilateral coordination to full swing for a more sovereign Europe".

ahmeneeroe-v2

3 hours ago

You think our alliances are Microsoft Office licenses?

traverseda

6 hours ago

No one is going to walk away from that kind of alliance tomorrow, sure. Stuff like "we're going to remotely disable military equipment we've sold you" is going to have consequences though. It's not walking away from alliances, it's just focusing on more stable countries.

Yoric

6 hours ago

To clarify: it's not exactly "remotely disable".

It's "block everything that depends on US clouds", which is a considerable downgrade (because you can't upload all mission parameters to an airplane without going through the cloud, and you can't use self-diagnosis features), but not entirely a kill switch. Close enough, though.

Our_Benefactors

5 hours ago

> Stuff like "we're going to remotely disable military equipment we've sold you" is going to have consequences though.

Proof of this happening or even having the capability of happening? There is none.

x1ph0z

an hour ago

lol this is a joke right? Ask any Canadian if they think America is driving a hard bargain or pushing away its closest ally.

Yoric

6 hours ago

Are you sure?

I mean, the current administration has repeatedly threatened to invade militarily two of its allies. Also, it has repeatedly threatened to not honor military agreements with most others, and both the current president and vice-president have insulted the leaders of several allied countries to their face.

Oh, and if that weren't sufficient, the current admin has unilaterally broken all trade treaties (alongside most intellectual property treaties) it held with its commercial partners.

The EU is slow at it, but it's no accident that everybody is doing their best to move away from US tech and military dependencies.

wood_spirit

9 hours ago

But at the same time they are fielding multiple new stealth aircraft and their jets and missiles outperformed western aircraft in the recent Pakistan India flare-up.

ambicapter

9 hours ago

So you'd think they'd be able to build a commercial jet liner, no?

samus

7 hours ago

Roughly speaking, an aircraft must fulfill a certain amount of economy (cheap, low cost to operate), safety, and performance.

If you compromise on safety, you get something that is still suitable for the military. If you don't care about economics you can participate in the space race.

But for commercial air travel, you don't have the luxury to pick just two; a competitive commercial airliner has to perform exceedingly well in all three regards.

If you're an airline using expensive aircraft you will go bankrupt. If your aircraft is too slow then your competitors will eat your lunch, and if you have a reputation of being unsafe then your customers will run away or the government will pull the plug (likely both).

IMHO affordable commercial air travel is one of the biggest marvels of 20th century engineering.

garblegarble

4 hours ago

China currently can't make the high-performance, efficient, long-life jet engines that US & Europe make. The commercial market is heavily cost-sensitive, so they can't compete there currently as a result.

This doesn't matter so much for military purposes: they can easily eat the cost of a higher maintenance and replacement schedule on a smaller number of military jets with fewer hours on them.

This gives them more iteration cycles, speeding their building up of experience. They're catching up. Industrial espionage will help them along too, but not as much as the experience from engineering their own designs.

kelipso

8 hours ago

It’s would be a result of where the money and resources go, I assume. Apparently they haven’t felt a need to manufacture their own commercial jets but they did for military jets. They definitely feel the need in the case of chips.

hmm37

8 hours ago

But commercial jet liners aren't as important to China for security. They have high speed railroads for that.

ta20240528

8 hours ago

So… you'd think the USA would be able to built a nationwide, high-speed rail network?

See what I did there?

sofixa

11 hours ago

> China has been pushing to build its own aircraft for >23 years. It took 14 years for COMAC to get its first regional jet flying commercial flights on a Chinese airline, and 21 years to get a narrow-body plane flying a commercial flight on a Chinese airline

And both those planes have a strong dependency on "western" components that won't be overcome before the 2030s, and even then, they're around a generation behind.

huntertwo

7 hours ago

5 years behind becomes 3 years behind. China is expanding their manufacturing abilities faster than the US. Soon they will surpass the US. Look no further than their generic consumer electronics manufacturing.

zawaideh

12 hours ago

Re: Western. A similar thing plays out when the term "international community" is used in news. It refers to the US and its major allies which means US, Canada, Western Europe, Japan, Australia and New Zealand more or less.

newyankee

10 hours ago

Essentially countries that were developed prior to 1990 or so , although South Korea is a tricky case today going by this definition, as are Taiwan, Hongkong and Singapore

nicoburns

9 hours ago

> A similar thing plays out when the term "international community" is used in news. It refers to the US and its major allies which means US, Canada, Western Europe, Japan, Australia and New Zealand more or less.

Wait, really? I thought "international community" meant all countries.

tsimionescu

6 hours ago

There was a particularly memorable use of this sense some time ago, when the UK representative to the UN explained that they abstained from a vote in the General Council that passed with something like 200+ members voting for it because "the international community is still divided on the topic".

acaloiar

8 hours ago

That's because you're reasonable.

Sometimes it's used in the expected way, but (more?) often, "international community" euphemistically refers to whomever is currently one of, or an ally of the above mentioned countries.

tsunamifury

10 hours ago

Yes community refers to whose who participate in community.

How is this hard to understand?

Broadly speaking coast de ivory and the like is not a participant in the international community.

tsimionescu

6 hours ago

China, Russia, India, Pakistan, Iran, Saudi Arabia, and many many other countries that are very active members of the international community are not counted among members of THE "international community". Hell, much of Europe isn't either, including some of the former colonial empires, on some topics.

tsunamifury

5 hours ago

China, Russia, India are certainly referred to when using this term. Iran and Saudi Arbia may or may not be. Usually not Pakistan, so I really dont know what in the hell you are saying.

raincole

9 hours ago

> look at the names

Why would I do that tho? If we look at the names of scientists/researchers/engineers/businessmen, the conclusion would be that the US has contributed nothing to the world. Europeans did all the hard work!

thesmtsolver

9 hours ago

Another equivalent way to look at that:

Historically, top scientists/researchers/engineers/businessmen migrate from rest of the world to the US rather than to Europe or China.

Imagine if Europe or China were a bit more open with immigration and equally attractive, we would see the same pattern there too.

caycep

10 hours ago

this is true for anyone - create challenges, and you optimize efficiency elsewhere.

Also, isn't this the usual path to better computer science? Reducing computation needs by making better/more efficient algorithms? The whole "trillions of dollars of brute force GPU strength" proposed by Altman, Nadella, Musk et al just seems to reinforce that these are business people at heart, not engineers/computer scientists...

rayiner

11 hours ago

Nobody thinks the Japanese aren’t “civilized.” “Western” is just a euphemism for “rich and orderly.”

switchbak

9 hours ago

It is an odd category, and Japan is often considered to be "Western" - these days at least. That certainly wasn't the case even a few generations ago.

I think it's ostensibly supposed to be more about shared cultural values, but even that is a pretty weak way to divide countries. Perhaps "an ally of the United States" is a little more accurate?

Any societal dividing line like this is bound to hit on problems once subjected to the real world.

huntertwo

7 hours ago

I think Japan likely shares more values culturally with China than USA outside of political systems. In any aspect outside of that Japan is not western.

hollerith

3 hours ago

The leaders of Japan during the Meiji era (starting 1868) will be sad to learn that their attempt to thoroughly Westernize Japanese society failed even though it enabled Japan to dominate China (and most of the Western Pacific) for decades.

bad_haircut72

11 hours ago

Its more about democracy and adhering to the global (set up by America post WW2) system of laws and trade.

rayiner

11 hours ago

I think most people considered Spain a "western" country even in 1970 when it was controlled by Franco.

hopelite

8 hours ago

I am not sure exactly to what degree, but "I hate the term 'western' because some 'weste[r]ners' use it to separated what they think are 'civilized' from 'uncivilized'" is definitely a bit of an antiquated perspective at this point; almost like a justification to hold on to other older perspectives about "racism". I have started resorting to using terms like European Cultural Block because of it in certain communities that understand contemporary topics and have an advanced understanding of the world.

Your first statement is not likely unique to China though, even though they have demonstrated that in about the last 40 years, which I don't really think qualifies as "history". What it does demonstrate is that societies that have a certain kind of ethnic self-respect and can cast off the detrimental influences of foreign, hostile, and even enemy elements to pursue their own self-interest and survival will succeed, regardless of hurdles placed before them.

It's really just a story of personal development and either escaping, evading, and avoiding detrimental, toxic people and their behaviors. All of humanity that all has to currently still share a single planet with ZERO save spots, would be better off if we all not just allowed each other to be ourselves in our won places without others subverting, subjugating, infiltrating, dominating, poisoning, or polluting any other people on the planet. Then everyone can decide if we want to be friends or not friends with each other, collaborate and be friendly or simply avoid each other. We do not have to like each other to get along if everyone agrees on a base understanding that no people can parasitize and abuse and manipulate any others.

corimaith

5 hours ago

>History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years.

I don't think you can really produce a definite counterfactual that they would or wouldn't have taken longer or shorter without it, but certainly they were pushing for self sufficiency long before technology restrictions. But we're not going to be handing our technologies to our competitors on a silver platter, and it's also best for businesses to start weaning themselves off the Chiinese market. Virtually every market reliant on them today is in big trouble.

As for hubris, I think that's more a projection of your part if you want to start bringing up race cards with regards to contributions, that kind of argument would be applicable to everyone. And AI research is highly diverse and international, Chinese names don't dominate the list more than Turks, Greeks, Malaysians, etc.

MSFT_Edging

7 hours ago

The whole "western" or "the west" always makes me laugh. Half the time it's a dog whistle for "white". Like many right-wing commentators love saying "Western Values" to avoid saying "white, Euro-centric, Christian values".

Mexico is a modern country, an industrialized country, a country that is exactly as "western" as the US or Canada. They have the same religious beliefs, speak a dialect of a European language. They have European style cities, a long history of cultural contributions. Yet they're not white enough to be part of "The West".

I think at this point we should be honest with ourselves in it's usage. 90% of the time it's a racist dog whistle.

skinnymuch

2 hours ago

Mexico isn’t as “western” as the west. Mexico isn’t the first world. It’s not part of the exploiting Global North.

The stuff you bring up ignore the power dynamics which are arguably the most important part.

lazide

7 hours ago

But why would anyone want to do that?

You do realize that antagonizing people with nuclear weapons and the largest economies in the world rarely results in positive results, right?

hopelite

2 hours ago

Arguably the sample size for that is 0 though; but even if we are generous, it’s only about 1 so far.

lazide

an hour ago

The number of nuclear armed countries with large economies that will get worked up if you call them racists for being ‘the west’ is waaaay larger than zero.

tw04

7 hours ago

> History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years.

Really? How long has China been attempting to build their own jet engines? How long have they been attempting to build competitive CPUs?

History has shown withholding tech successfully keeps them at least a generation behind the west.

In some fields like CPUs they “make up for it” by just building larger clusters, but ultimately history does not show what you’re claiming. The only thing it shows is that we need to be even more diligent in protecting IP because a large portion of their catching up is a direct result of stealing the tech they were cut off from.

slaw

4 hours ago

tw04

4 hours ago

>Jet engine since at least 2011

Huh? Did you read your own link? The jet engine that was shown at an aviation show as a non-functioning prototype in 2011, with hopes they'd have a functioning version by 2016, and in service by 2020 (it wasn't in service in 2020). Notice at the very top of your own article it says "still in development".

>CPU since 2000

That isn't remotely competitive, and at least a full generation behind.

slaw

3 hours ago

What 'since at least 2011' you don't understand?

tw04

3 hours ago

What part of "a non-functioning prototype" don't you understand?

Literally anyone can make a prototype jet engine. The metallurgy and process to make a functioning one is several orders of magnitude more difficult. Which is why... China still buys the vast, vast majority of their jet engines from Russia for military use. And their commercial passenger jets use engines from CFM.

notepad0x90

12 hours ago

western is a cultural term derived from a geographic one. The US is also not 'western' strictly geographically as it is not in western europe, neither is australia. But they both originated from Britain's empire and share in it's cultural ancestry. It means "western europe and it's cultural derivatives". Spain and Portugal's empire fell away long before britain and france's and they don't have similar geopolitical relations like NATO, so it's hard to consider their former colonies/upstarts part of the same sphere of cultural influence.

China for sure will catch up, the question is what they will do with it. They're not ambitious like the US/West. The US wanted influence all over the world as an extension of the cold war and to keep economic interests safeguarded. But China just doesn't operate that way. They're more hands-off. They could be opening up alibaba cloud datacenters all over the US, offering it as an AWS/Azure alternative, funding tons of startups all over europe, the US,etc... to exert their influence, but they won't. They have a more long-term low-and-slow approach to global domination. The "100 year marathon" as they called it, which they'll win for sure.

China's greatest weakness is not just their lack of ambition,but their command-economy. They're doing capitalism but with central control of the economy. It intertwines government policy with corporate policy, making it harder to do business overseas (like with bytedance/tiktok).

tsunamifury

10 hours ago

False.

Westernism is broadly an extension of the academic notion of classicism, starting in Egypt and then Greece Rome and into Europe and the Americas.

notepad0x90

7 hours ago

It's not an academic notion (at least not strictly), since virtually everyone uses it routinely.

tsunamifury

6 hours ago

Oddly since I got many downvotes with this statement, it's clear the average hacker news reader knows very little about world history or common knowledge

notepad0x90

5 hours ago

That's very reductive of you. "western" like "3rd world" is used very differently than what you or academics would like.

tsunamifury

an hour ago

I feel like you didn't even grasp the base sentence meaning. "Classicism" is an academic concept

lawlessone

8 hours ago

In a way withholding a tech becomes a signal saying "Hey this is important" so the result is China dedicates more resources to researching it lol.

tsunamifury

11 hours ago

It’s helpful to think of westernism as a platonic ideal. Individually derived reason and virtue, superior to state and sometimes ‘gods’ as a tradition to drive up the total survivability, richness, and stability of the community.

Concepts that enable the individual should empower a chosen configuration of society not the other way around.

Contrast this with non westernism where either education of the individual is not valued or the state is the primary goal over the individual.

I’ve worked with states governments and individuals around the world for 20 years and find this very useful definition. What’s confusing is the nations who have half adopted westernism but don’t fully due to either caste systems or government dominated thinking.

It’s an arrow towards rationalism over tradition, individualism over collectivism, flatness over hierarchy, and future over past. But only the limit of the resources any given society has.

nextworddev

9 hours ago

Name one thing China has invented first in LLMs that the “west” adopted as a standard

nextworddev

9 hours ago

Your silence is deafening, qwen bots

chuckadams

11 hours ago

I find "western" is often used to disparage "western thought", as in it can't grasp the deep wisdom of those mysterious orientals that transcends normal logic and reason. Declaring such a split is the underpinning of a whole lot of woo-woo beliefs.

hbarka

9 hours ago

Disparage or exalt? It can also be used in an objective sense without conjuring insult.

switchbak

9 hours ago

It can be both. From different people, and it often is.

notepad0x90

12 hours ago

I think anti-immigrant rhetoric will have the most impact against the US. A lot of the people innovating on this stuff are being maligned and leaving in droves.

Aside from geography, attracting talent from all over the world is the one edge the US has a nation over countries like China. But now the US is trying to be xenophobic like China, restrict tech import/export like China but compete against 10x population and lack of similar levels of internal strife and fissures.

The world, even Europe is looking for a new country to take on a leader/superpower role. China isn't there yet, but it might get there in a few years after their next-gen fighter jets and catching up to ASML.

But, China's greatest weakness is their lack of ambition and focus on regional matters like Taiwan and south china sea, instead of winning over western europe and india.

dlisboa

11 hours ago

> But, China's greatest weakness is their lack of ambition and focus on regional matters like Taiwan and south china sea, instead of winning over western europe and india.

That's a strength. Them not having interest in global domination and regime change other than their backyard is what allows them to easily make partners in Africa and LATAM, the most important regions for raw materials.

notepad0x90

11 hours ago

You would think so, but historically that's why they never became more than a regional power. Empires for millennia craved trade with China but only the mongols from that region made it all the way to western europe in their invasions.

It is a strength, if their goal is to have a stable and prosperous country long term, and that seems to be what they want. good for them. But nature abhors a vacuum, so there will always be an empire at the top of the food chain. Such empires want to maximize wealth for their people and secure them against threats, that's why invasions and exploitation of weaker countries happens. That game hasn't changed. Friendly relations work, until you need a lot of resources from a country that doesn't want to give it up. Or, like with the US, when they're opening up military bases next to your borders and you need a buffer state. Or, when naval blockades and sanctions are being enforced against your country for not complying with extra-sovereign demands.

History shows that countries content with what they have collapse or weaken very quickly.

China will have a population crisis in a few decades for example, and it won't have the large manufacturing base and its people will be too used to luxuries to go back to slaving for western countries for pennies. Keep in mind that the current china itself is so great and prosperous because of all the invasions it did against western china and satellite states like Vietnam and north Korea (the US isn't special in this regard).

lossolo

10 hours ago

> But nature abhors a vacuum, so there will always be an empire at the top of the food chain

The world has been bipolar and multipolar before in history, and it can be again. The unipolar period of American dominance is ending.

notepad0x90

7 hours ago

Yes, it can but those poles are expansionist/influential empires not isolationist states. For example, China wants involvement in African development but they don't want any say or interference in local affairs, they can exert influence but they don't want to.

ikidd

10 hours ago

>Them not having interest in global domination and regime change

I don't even know where to begin with that one.

Jackpillar

7 hours ago

Yeah waiting to see historical examples of contemporary China being interested in global domination and regime change, especially in contrast to the US.

OrvalWintermute

11 hours ago

if you've been tracking the shark deals they give countries for loans, I think you'd recant what you just said.

"while the CCP accuses the West of predatory interest rates, the average Chinese rescue loan carries an interest rate of about 5 percent, more than double the IMF’s standard 2 percent. As of Oct. 1, 2025, despite higher U.S. interest rates, the IMF’s Special Drawing Rights lending rate stands at only 3.41 percent, still significantly lower than what China charges struggling nations for so-called relief."

These countries paying these loans are the ones least able to pay them back, and at more than double IMF loans, they are really putting them in a vise.

skinnymuch

an hour ago

China’s retribution or punishment against loan issues is nowhere close to IMF and the west. Same with them not wanting the state to do what they want.

You did the equivalent of showing some stat showing black and brown people do violence and crimes and saying “see how uncivilized they are” ignoring everything else.

bkandel

11 hours ago

China's greatest weakness is that their working-age population has already peaked and is in the process of plummeting, which will continue over the coming decades.

notepad0x90

11 hours ago

Yes, and being content and lacking ambition isn't good. Expansionism and immigration can solve that, but they're culturally stagnant in that regard.

Without immigration, the US would have faced the same problems.

rayiner

10 hours ago

> But now the US is trying to … compete against 10x population and lack of similar levels of internal strife and fissures.

I can’t tell whether you think the anti-immigration stance is a good thing or bad thing.

notepad0x90

7 hours ago

it's bad for the US, because China has 10x population. the US can't make up in quality, what it lacks in quantity without immigration and attracting foreigners.

hollerith

an hour ago

If you go by official figures, China has 4.2 the population of the US, but some experts believe that China's official figure is drastically exaggerated.

rayiner

6 hours ago

The U.S. actually has more people age 25-64 with a college degree (about 90 million) than China (about 80 million).

notepad0x90

5 hours ago

Yeah, and lots more immigration too. I don't know if those are the ones with college degree, but you can see who's writing all the academic papers. Look up stats like the 3rd most popular language by state and you'll be surprised. There has been a huge effort to import doctors from all over for example, due to the shortage in the US. Even in our tech industry, you don't need to look far to see all the H-1B's. I can't think of a single industry requiring skilled and educated workers that isn't relying on immigration significantly. Even our schools are relying heavily on Chinese and other foreign students for revenue lol.

My point was, the non-immigrant birth-rate is very low, so arguably the US should have arrived at the same demographic crisis as japan, china and south korea. Not only that, immigrants attend college at a much higher rate than native-born too.

rayiner

3 hours ago

Your argument was that the U.S. needs immigration to compete with China’s population advantage. But China doesn’t have a population advantage in college educated workers. China has to scale its educated workforce by sending more of its population to college. Which is certainly doable, but they are currently where the U.S. was 60 years ago.

Also, the U.S. has a fertility edge over China, which skilled immigrants do not contribute to. The birth rate of the groups comprising most skilled immigrants (Asians) is very low, much lower than for other Americans.

marknutter

5 hours ago

Nobody is anti-immigrant outside of a small pocket of anti-H1B folks in the tech community. People are, however, anti-illegal-immigrant, which is completely different.

hollerith

7 hours ago

>But now the US is trying to be xenophobic like China, restrict tech import/export like China but compete against 10x population and lack of similar levels of internal strife and fissures.

Do I infer correctly that you believe that China has less internal strife and fissures than the US has?

csomar

11 hours ago

> But, China's greatest weakness is their lack of ambition and focus on regional matters like Taiwan and south china sea, instead of winning over western europe and india.

How can they have international hegemony before they clear their regional order? China is more interested in aligning Taiwan than invading; though it’ll probably invade if it can’t align it diplomatically.

China is probably not interested in continuing the current Western-style order but to implement their own sino-stuff. At least with the CCP at the helm.

notepad0x90

5 hours ago

They've dominated their region for a long time. Vietnam and NK are their sattellites basically. Russia is their close ally. The only regional opposition they have is India. Taiwan is a small bug to them, the only reason they haven't invaded it is because of TSMC and ASML, mainland China hasn't caught up with them and Nvidia yet.

They're all over Africa and south Asia. But unlike the US/West they don't exert political influence. When they build infrastructure for example, they set up worker camps, isolated from the local population. they only employ their own imported people and clean up and leave quietly afterwards.

They're acting like good business partners, instead of a superpower wielding it's might and extending its influence. it's good for business for all involved parties for sure, and smart too. But not having strong influence means for example, the US can come in, outbid them, bail out african loans to China and they lose that source of commerce.

lesuorac

9 hours ago

The US isn't slowing China anymore.

China has an import ban on chips [1] so its irrelevant what the US does.

[1]: https://www.cnbc.com/2025/09/17/nvidia-ceo-disappointed-afte...

xadhominemx

9 hours ago

The US is certainly slowing down China considerably. China would certainly not have an import ban on Blackwell GPUs if they were made available. And upstream, the ban on EUV and other high end semiconductor production equipment has severely limited china’s capacity to produce logic and DRAM (including HBM).

overfeed

9 hours ago

> China has an import ban on chips

Only in response to the US banning the export of the high-end GPUs China wanted. The import ban is the Chinese government burning the the landing ships, it clearly communicates to everyone that there is no going back, and total commitment is expected.

unethical_ban

9 hours ago

Would they have done that if the US had been more "reliable" in providing the chips and didn't cut them off in the first place?

The point still stands that the US instigated the split.

downrightmike

9 minutes ago

That's how it usually goes, fully expected

reliabilityguy

12 hours ago

Tbh this whole situation reminds of how Japan excelled in making a lot more with a lot less after WW2, e.g., fuel-efficient engines, light cars, etc. these constraints were not present in the US (and to some extent in Europe), and resulted in US cars being completely not competitive in non-US markets.

dataviz1000

12 hours ago

I've been in Chile, Peru, Colombia, Panama, and Costa Rica.

The streets are flooded with cheap Chinese cars and I see more BYD than American cars. If the car wasn't made in Japan or Korea which probably account for most of the cars, it was likely made in China. Moreover, I haven't been in countries with the closest ties to China.

sofixa

11 hours ago

> The streets are flooded with cheap Chinese cars and I see more BYD than American cars

This isn't surprising in any way, American "cars" (quotes because the vast majority of what American manufacturers pump out isn't cars, it's trucks) haven't been competitive in decades. The only globally competitive vehicles were developed in Europe by GM Europe (Opel, since sold to PSA now Stellantis) or Ford Europe (which axed all models bar the Puma). The rest is too big, expensive and inefficient from the vast majority of uses. Tariffs and good marketing keep American car manufacturers in business in the US, but those don't work in most other markets.

The more appropriate comparison is with European automakers such as VW Group, Stellantis (Peugeot, Citroën, DS, Fiat, Chrysler, Dodge, Ram), Renault. And there too BYD is winning as well in mosy countries, but at least there's a comparison possible.

Workaccount2

8 hours ago

The US couldn't compete anyway. The US is an advanced economy and will struggle to compete in non-advanced categories.

It's like trying to level your MMORPG character to 100 by only farming in lvl 30-40 mob areas. It's really not worth it and mostly forced.

sofixa

8 hours ago

Cars are relatively advanced and more importantly, seen as a status symbol by many. There is, IMO, plenty of space for not-the-cheapest, quality cars.

Take Renault for example, their Renault 5 and 4 EVs are good looking, not luxury but definitely premium, and the 5 sedan starts at 30k€; the 4 crossover starts at 29k€. This is before a 5k€ government subsidy. Their boring, fewer bells and whistles, low cost model, the Dacia Spring, starts at 17k€. The Renault 5 and 4 are made almost entirely in France, while the Dacia is made in Romania - a lower cost country, but still an EU member state.

The comparable in size and autonomy BYD Dolphin starts at 20k€. Both for cheapness and quality/design, Renault are competitive.

rkomorn

8 hours ago

The new 5 is one of the first cars I've really liked in a while.

They really nailed the modern-with-subtle-calls-to-retro look.

sofixa

8 hours ago

Agreed, I'm not a car person and too young to remember the original 5, but it's really good and fun and modern and slightly retro looking. It's almost making me want to buy it, and I have absolutely no use for it.

rkomorn

8 hours ago

If you haven't already (which seems unlikely), look into the Renault 5 Turbo.

Bit of an absurd car, but the modern (non-turbo) 5's slight bumps over the rear wheels are such a good callback to the Turbo (the original Renault 5 were basically all flat).

Really fine design stuff IMO.

tsunamifury

10 hours ago

The premature optimizer is never the innovator.

Japan eventually stopped that role and their products improved greatly.

djmips

8 hours ago

I like your point but I think it's a little too harsh to call it premature. Sometimes you're forced into that position and it makes sense to do so. But it's a good take if you're busying yourself with optimizations of old tech you'll never innovate or be a leader. It's still a good preoccupation in many constrained situations.

segmondy

12 hours ago

may backfire? it's a bit too late for that.

go to 2024, western labs were crushing it.

it's now 2025, and from china, we have deepseek, qwen, kimi, glm, ernie and many more capable models keeping up with western labs. there are actually now more chinese labs releasing sota models than western labs.

Workaccount2

8 hours ago

But they aren't keeping up

They are lauded for the ability to cost ratio, or their ability to parameter ratio, but virtually everyone using LLMs for productive work are using ChatGPT/Gemini/Claude.

They are kind of like Huffy bicycles. Good value, work well, but if you go to any serious event, no one will be riding one.

segmondy

8 hours ago

they are keeping up. i have been using just chinese models for the last 2 years. chatgpt/gemini/claude have marketing. there's nothing that you can do with those models that can't be done with deepseek, glm or kimi. if there is, do let us know.

Workaccount2

5 hours ago

They can't attract a large contingent of users. Because despite being able to do everything the big name models can do, they cannot do it as well.

This aligns with the benchmarks as well; they benchmark great for what they are, but still bottom of the barrel when competing for "state of the art."

And yes, it's great you daily Chinese models, but the vast majority of people try them, say "impressive", then go back to the most performant models.

MSFT_Edging

7 hours ago

The downside of their efficiency and cost-ratio is that they undermine the circular economy of massive data centers, GPU sales, and VC money that is constructing an extremely wasteful bubble.

Workaccount2

5 hours ago

The bubble is there in China too, it's just on the governments books instead of private investors books.

NSPG911

2 hours ago

way too early to say that

while qwen, deepseek and kimi are opensourced and good, they are preferred because of their insane token ratio, they use a lot less for more, but a by product is that they are less accurate it is amazing progress by the chinese companies, but they definitely can improve a lot more

rasz

2 hours ago

Have you tried using those models? qwen for example cant even do something as basic as clustering analysis on a list of integers, hell it goes off the rails when just reading said integers from a file - starts babbling about determining number of digits, indexes, tries concatenating numbers together into one big string, no idea wtf is going on with that model.

hunglee2

12 hours ago

too early to call a winner, though it is disappointing to see US withdrawal from open source. Still the main outcome of open source is distribution / diffusion of the idea, so it will inevitably mean US open source will come back, hopefully via some grass roots maniac, there will be a Linus-like character emerge at some point

segmondy

12 hours ago

i'm not calling a winner, i'm just saying that the chinese have caught up despite the embargo. google, openai & anthrophic have phenomenal models. i stopped using openai & anthropic after they called for open weight/source regulation. i use google because they offer gemma and i got a year gemini-pro subscription for free, use openai gpt-oss-120b since i can run it at home, and the only model i currently pay for is a chinese model.

mixologist

11 hours ago

user growth has slowed. the technology that should help users is only being pushed from the top, while users refuse to use it. openai pivoted to porn.

does it really feel like they have a chance to recover all the expenses in the future?

crypto grifters pivoted to ai and, same as last time, normal people don’t want to have anything to do with them.

considering the amount of money burned on this garbage, i think we can at least declare a looser.

rzerowan

12 hours ago

Fingers crossed for convergence rather than divergence in the technical standards.Although the way hings are going it looks like the 2 stacks will diverge sooner rather than later , with the US+ banning the use of CHN models while simultaneosly banning the export of it quasi-open models. We may very well end up in a situation like the old PAL vs NTSC video standard where the PAL(EU/Asia/AFrica) and NTSC(America's/Japan) gradually converged with the adoption of digital formats. Instead here would be a divergence based on geopolitical considerations.

hunglee2

12 hours ago

positive take: a bifurcated tech tree might give us (humanity) a better chance of faster advancement, as it would be a persistent A/B test in live environment. Where I would join you in the crossing of fingers is to ensure such A/B testing is competitive but not destructive. We may even evolve to a situation of complementarity, an American Ying vs the Chinese Yang. Lets hope so!

sspiff

9 hours ago

Their are signs that China is not open sourcing their SOTA models anymore. Both Huawei and Qwen (Qwen-Max, WAN 2.5) and have launched flagship models which are yet to be opensourced.

natrys

8 hours ago

Qwen's max series had always been closed weight, it's not a policy change like you are alluding.

What exactly is Huawei's flagship series anyway? Because their PanGu line is open-weight, but Huawei is as of yet not in the LLM making business, their models are only meant to signal that it's possible to do training and inference on their hardware, that's all. No one actually uses those models.

camel_Snake

9 hours ago

Small counterpoint but there are also 2 new players putting out SOTA open source models (Moonshots Kimi and zhipus GLM) so we're still seeing the same number of models overall, just via newer entrants.

amelius

11 hours ago

Another outcome may be that we now have to learn Chinese to understand their datasheets ...

anonzzzies

9 hours ago

I was doing this in the 70-80s with electronics from Hong Kong and Japan. The nice cheap stuff ( I was very young ) was all sheets in things I basically had to pattern match against notes of others on BBS and meetups.

knowitnone3

9 hours ago

It's much easier to copy what others are doing instead of spending the time and money for research and engineering. It's also much easier if you steal the tech. I could never have invented a bicycle but I can sure make a copy of one.

FpUser

6 hours ago

"... instead of spending the time and money for research and engineering..."

China has plenty of R&D and science now.

archerx

10 hours ago

I want China to release GPUs with a ton of VRAM, 128gb - 256gb. It doesn’t matter if they are half as fast as Nvidia because having a big model at a reasonable speed is better than not being to run them at all. AMD could have done this and have had a massive impact on nvidia’s market share but they choose not to because reasons.

myth_drannon

11 hours ago

China's innovation relies on the stolen western IP, without it, China is nothing. Also USSR/Russia is no longer a scientific powerhouse that can supply China with some military innovation. A dictatorship combined with cheap labour it 100% guarantees that the country's innovation is stunted, no matter what the Chinese propaganda claims.

Hikikomori

10 hours ago

And the US has never stolen IP?

myth_drannon

10 hours ago

Corporate espionage is ever present but it is criminalized. The only time US as a country did that you can say "stole IP" was after WII when it took Nazi rocket scientists and technology. China is the opposite; stealing tech is done by the state apparatus (same was done by USSR and reverse engineering computers for example).

Frankly I'm not surprised that this is done, probably if US was so behind it would have done the same to reduce the gap. Everyone is trying to survive and outsmart and outwit the other, instead of collaborating.

g8oz

10 hours ago

You are mistaken about American intellectual property theft. They engaged in extensive IP theft from Britain in the 18th and 19th century with the encouragement of the government. See https://interestingengineering.com/innovation/the-spies-who-...

Also during World War I the American government seized German chemical patents thereby launching the American chemical industry. So that is an example of theft by the state apparatus.

myth_drannon

4 hours ago

I'm talking about modern times, after WW2. Not when US had slavery and lynching, blacks was a form of entertainment.

IT4MD

11 hours ago

I believe this is an Pollyanna take on AI. There is nothing about humans that tells us humans will bring AI to fruition for the other humans and a mountain of evidence showing how it will be used to abuse humans instead....for profits/power/whatever horse shit the masters of the universe have decided upon.

belter

10 hours ago

China is a nation of engineers...The US has been relying in on H-1B immigrants. Science is under attack. The truth is the US already lost: https://youtu.be/whVlI6H4d-4

braza

12 hours ago

Does someone know if there's some equivalent of those engineering/research blogs for Chinese companies?

I used to follow the ones from Western companies, but honestly, after some point in time, I would like to see some cases from what I consider is a good benchmark for everyone that does not work in FAANG in terms of engineering.

supriyo-biswas

9 hours ago

The company blogs of Chinese companies will often do articles like this[1] talking about a new innovation or optimization that they did, but this will be often just mixed in with marketing articles too.

I would also assume there's a lot of content in the native Chinese forums, which unfortunately, as an English-speaking person, I wouldn't be able to easily refer to :(

[1] https://www.alibabacloud.com/blog/how-does-alibaba-ensure-th...

checker659

9 hours ago

They are working with tiny models. Not sure how well it'd scale to bigger models (if at all).

CaptainOfCoit

9 hours ago

They're all LLMs, so no, not tiny, but not exactly huge either:

> Our current deployment runs in a cross-region cluster comprising 213 H20 GPUs, serving twenty-eight 1.8–7B models (TP=1) and nineteen 32–72B models (TP=4).

ibejoeb

8 hours ago

Sounds like this virtual GPU is a separate scheduler. I wonder what kind of latency is introduced by marshaling all that data around.

mighmi

8 hours ago

To what extent is this practice applicable to other loads?

catigula

3 hours ago

Sounds like they stopped doing something stupid.

throwaway48476

11 hours ago

Its easy enough for a a well resourced entity to take a pre trained model and deploy it on new hardware to save on the NVDA tax. It's far less likely for research and model training to happen outside the mature NVDA ecosystem.

lnxg33k1

6 hours ago

Lots of shareholders here, move along, there is nothing to read

wslh

8 hours ago

How feasible is that in an horizon of 5 years new optimized "equations" will cut the need for more GPUs?