How would we do that? There's no historic precedent for that. Its fundamentally an information theory thing: what's the max amount of intelligence you can get out of 1 KB/MB/GB? There has to be a limit and I'm not convinced that it's far off.

And by the time the models that require 1TB memory will be better.

It's like people saying mobile chips are going to be better than PC chips... until they realize PC can be made of mobile chips too if that comes true.

And besides llm's memory requirements:

-HBM has a lot of room to grow(bandwidth/power/density). And an HBM memory takes 3x the amount of wafers of regular DRAM.

-HBM + better DRAM designs could increase random memory access speed by alot. I've seen research talking about 7x.

At what point it makes sense that half of consumer GPUs and most of the server GPU/CPUs start using HBM?

I think the chipmakers are pretty securely set up here, the usual cyclical issues aside. Between the stubborn resistance of the data to being compressed without significant loss, and to a lesser extent the ability to do more with more memory (we’re surely at diminishing returns for LLMs already, but diminishing doesn’t mean zero) it seems like a fairly safe guess (I am not an expert on anything) that for the next decade at least you’ll still want at least 128-256GiB of video RAM to have a comfortably “SOTA-enough” experience. Then the other side of the rectangle comes from putting that VRAM in every consumer and office-worker device. Selling everything at a big premium to a few efficiency-obsessed SaaS providers is what you do when you’re temporarily supply-constrained. Sidelining the SaaS middlemen and selling to gloriously inefficient consumers who will leave their laptops closed for most of the day is the dream. This is probably not unrelated to the fact that the country which is currently making a big strategic push into memory manufacturing is also championing open models.

The fundamental issue is that for these systems bigger is essentially always better (if affordable). So if we can squash something like Kimi K3 down to run on a 'normal' system, that just incentivizes devs to increase the model size until once again we're at the limit of what can be run.

What do you base this prediction on? It seems very unlikely, unless by SOTA you mean the current SOTA.

SoTA AI will move well past 1TB+ memory requirements in 1 year or so.

Also, you will be able to play with much more competent models locally. They will still feel like children compared to the adults living in the SoTA region.

Are you thinking of fundamental architectural changes (like the Transformer)? Or incremental (like MoE or GQA)? Are there specific neolabs or techs you are following that lead you to this prediction?

Indeed it seems quite possible we are one architecture breakthrough away from existing chip stock driving us all the way to ASI.

Nobody has done AFAIK the information theory to prove it’s not possible.

Yeah, you can directly print the SOTA AI model on the chip. I also believe this can be done on older process nodes. The most important factor would be speed. How fast can you go from new model to new chip?

Even if we do, we are going to need a lot of RAM for the billions of agents running everywhere.

"we" being China?

NOPE. The opposite will happen. CXMT will flood the market thus making 1 TB models affordable.

that would be perfect timing

Hyperscalers are screwed, data center mania couldn’t even be completed during this massive spending spree, all while the people seemingly sitting on the sidelines are working on getting these things baked into the OS and chipsets in a way consumers wouldn’t notice

EU chip industry: best I can do is supply chips that are in 99% of industrial equipment and vehicles. Also supply the only machines in the world that enable leading edge lithography processes.

EU: we should create a law regulating the viscosity of the glue being eaten

CXMT uses old Qimonda patents (even though Qimonda's tech is now outdated), and the former CEO of Qimonda is currently building a fab for an FeRAM-based alternative to DRAM in Magdeburg. It's a gamble as FeRAM still suffers from a more limited lifetime than conventional DRAM at this moment but I think there are still somewhat interesting developments happening in Europe.

EU chip industry: best I can do is power electronics.

To be fair, all high end chips are made in Asia with EU tooling. US happens to be the place where the managers are and the trade is facilitated. There has been some good progress to push Asia make cips in USA with EU machines however IIRC significant part of the chain is still in Asia and chips made in USA are shipped to Asia for further processing.

Edit: changed China with Asia to distance the comment from Taiwan-China based politics.

HN continues its transmogrification in X.

Don't make me tap the sign that says "ASML".

Difference between Europe and the others is that it doesn't brag nearly as much.

You're forgetting about ASML, which is European and makes the actual machines to produce a huge portion of the world's high-end chips.

EU bourgeoisie invests in US tech

> EU chip industry: best I can do is eat glue in the corner.

Aren't they the only ones able to supply the chip-making machines?

EU: More windmills!

Oh Jon

If the chinese government feels its critical to china's success then it will be backed like alibaba and wechat and others were

Tesla is kind the same. Everybody bought it, because it went up so much, less because of the product. I guess every AI would do the same.

South Korea has opened its financial doors to the outside world, and since SK Hynix is a publicly traded company, that's allowed capital to flow in and take control of the company.

> the investment mania in SK and it's really scary.

i mean, it shouldn't be scary, because their failure is their own, and i doubt there would be collateral damage outside of those high risk investors.

It's scary for _them_, but it's an interesting case study for an outsider.

It's driven by the supreme leader's grand plan, like everything else there.

> 8% to 9% of the worldwide DRAM market share in 2025

how much more do they need to ship to satisfy you?

The price is similar, as they all rely on same upstream suppliers. The quality is kinda meh.

Look at DDR5-6000, it clearly is rising again since June.

The short answer is VW doesn't have the ability to compete with China's best EV car company on EV price, factory automation, or raw input (labor, electricity, metals) economies of scale. The complaining is to get politicians / regulators to prevent the Chinese EVs from being imported into their domestic markets.

Chinese structural advantages: monetary policy, lower cost of living, better infrastructure (electrical and transportation are all new within the last 20 years), lower cost of {concrete, factory machinery, metals, electrical} due to economies of scale.

Chinese EV manufacturers advantages: they don't sell ICE cars, so they don't have any legacy customers, factories to retool, or brand migration issues. And they mostly stole their technology from hiring Tesla factory workers to moonlight for them and share the design, technology, tooling, and RE the Tesla finished product.

Major Western legacy car company disadvantages: Ford, Toyota, Volkswagen, etc... all have significant reasons why they can't or won't make the super risky attempt to go all-EV. They all have $10s of billions in debt they are still paying off for their current capital improvements. Those debts assume they contribute selling ICE because EVs last longer so they would sell fewer units per year. It's simply less risk to avoid making any EVs and pay politicians to protect their businesses than to try to compete with 10 year old Chinese phone-companies-turned-EV companies. On top of that, the worldwide demand for EVs is only about the volume of the single most popular model of Toyota. They would have to go into massive debt on a giant risk that consumers will want more EVs (and that the EVs they design/build will sell better than the current ICE cars they have meticulously crafted for 50-100 years).

50 years ago you could ask the same question about Europe, the USA and Japan: why are they so much more productive than everyone else?

So others did spend energy on becoming better and now you see a different picture when you look at the world.

Because the capitalist west doesn't have strategic view, beside maybe on the defense sector. Everything else must produce profits next quarter not in the next 3 years.

It is because in the west we suffer from self inflicted obstacles that make it impossible to get out of our own way. The Chinese gov must think they are playing on easy mode.

> Would that energy not be better spent to try to become better?

The problem is, shareholders have bled Western companies dry in the chase for returns on investments. There is no money left in the companies to retool them to be competitive with China.

Meanwhile, the CCP just pours dollars into their companies.

Unfortunately, there is no punishment in Western societies if you run a company into the ground as long as you pay your debts on the way out.

The rest of the world is chasing models of investment in procurement of goods and services that get stuck on neoliberal capitalist antipatterns where every action demands a short-term profit motive. These are now very large-scale, highly-complex-supplychain manufacturing projects, and you cannot remain competitive if you are just Putting Faith In The Market to solve your problems. There's too much planning and sustained, structured goal-orientation required for investors or for the sort of contract-bid, public-private actions Western politicals are used to engaging with.

It takes 2 years in EU to get approval to build a shed. Imagine how long it takes to get approvals for a chip factory which uses a lot of water and very toxic chemicals.

UK has been trying to get approvals for a new runway at Heathrow airport for over 20 years !!! now.

Mostly self-inflicted wounds in form of climate pledges, climate taxes and such. Because of the goal to achieve zero emissions goal and climate neutrality, we abandoned cheap energy sources such as coal and pushed our heavy industries out of Europe to China. We still need those products, so we import them from China while wondering why European industrial segment isn't growing. Even in the clean energy sector, most solar panels come from China.