China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply.

But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer.

I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

This is wrong. It is NOT in their nature to keep the market under-supplied -- eg, Samsung, the industry's largest company, was notorious for expanding their capacity during the industry downturn to gain market share while everyone else was cutting back to minimize loss.

I'm guessing you are also probably unfamiliar with the terms like "chicken game" which refers to the cutthroat, high-stakes price wars where dominant semiconductor manufacturers intentionally overproduce and slash prices. This is literally how the industry went from dozens to just three majors today since the 80's.

If the existing memory makers retains control of the market and don't defect from the optimal-long-term equilibrium for themselves, that's true. It just takes one player to defect for short term gains as we've seen with some past boom-and-bust cycles. Alternatively, it takes a sufficiently-resourced player with enough incentive to enter the market themselves (NVidia, Google, Amazon, the PRC government through one of many companies...)

Relevant article posted on HN about this a few days ago: https://davidoks.blog/p/ai-is-killing-the-cheap-smartphone

Reminds me of how Samsung is giving out $340,000 per person bonuses. Shows you how much of a stronghold they have in market.

Supply and demand always balance out. There is no way manufacturers aren’t going to compete away these inflated margins, as long as they feel like this demand is sustainable.

CXMT is scaling up incredibly fast, they are on a clock (south koreans) their monopoly will end relatively soon, although I'm guessing that the AI companies will crash before that anyways.

Apple could always decide to build their own fab or some such thing.

Is any of this actually unforeseen? Buying the vast majority of the world’s supply of something does have mostly predictable consequences.

That's already the case with say bf16

For lifespan, AWS is still running a ton of T4 GPUs from 2018, that power a lot of computer vision models. A ton of these will have a long life, not all ML is about frontier LLMs.

Patents is not the issue here. Not even close.

The up-front investment of a memory fab is measured in billions, and takes years to construct and get running. The margin on the chips themselves is terrible, so without scale its not worth even trying. DDR5 is a industry standard that takes some effort to conform to, but the licence fees is a drop in the bucket to the cost of creating a fab.

The fabricators were cautious about increasing production, and slow to start planning. It takes further time to build up capacity, and if the demand drops down, they may end up producing dram at a loss when the market flips over to oversupply. The demand whiplash could kill any company that dared betting on increasing production. See the "bullwhip effect" https://en.wikipedia.org/wiki/Bullwhip_effect which has killed semiconductor fabricators before.

There is a discussion to be had about how to maintain national semiconductor production in Europe and US as a strategic industry, but historic attempts have all failed.

I have fairly simplistic view of the economics involved here. Could you explain why the ability to sell more chips wouldn't be sufficient enough incentive to increase supply?

For supply to meet demand. Depends very much on how aggressively producers scale and on how demand grows or shrinks.

Safe to say at least a year or two. It'd be shocking if it took a decade.

Probably, but at some point we're very likely to run out of significant training improvements and it's not clear that we'll see that point coming from a long way out.

Likewise it's probably dwarfed by improvements in how we make dram - continuing the roughly exponential (maybe a bit less recently) scaling of chips - but not necessarily.

The 2x from returning to previous costs is interesting because it's practically guaranteed, and it's on top of everything else. We're just currently "overpaying" (relative to the stable market price) for the manufacture of dram because of a sudden increase in demand.

Unless there's a new paradigm, scaling up is all they can do to improve performance. They've shrunk down all the way to 1-bit models and all the low-hanging fruit is gone. There's no way for them to get much smaller, so they have to get bigger and faster to meet expectations.

Nice, you were lucky. =3

Used enterprise HDD’s also jacked up now. It’s absurd lol

People spend that much a month on restaurants

Demand for DDR3 is up because people who want DDR5 or DDR4 but can't afford either any more are choosing DDR3 and old DDR3-compatible systems to put it in, instead of what they really want.

All memory products use many shared resources in the supply chain, so if there is high demand in one product line, others have to raise prices to compete for the resources or stop making those lines altogether.

That is to say at least you were able to buy them at $350 today, with the current trajectory there will be no supply at all in few months.

You could set up swap space on Intel Optane media, it'll be about the same performance as DDR3 and sells for ~$1/GB on the secondary market. Though it will be a lot more power hungry than Flash, let alone DRAM - so not suitable for all uses.

there's an economic term for this: substitute good. https://en.wikipedia.org/wiki/Substitute_good

Being desperate?

The price of SSDs is similarly depressing.

Someone was selling an Epyc machine with 512GB RAM @ 500 EUR last year. I regret not buying it now ...

> Its price just goes up and down in cycles.

Historically the price has always trended downward. When I first got into computing $200 could buy you 128 MB (yes M) of ram. Really nice systems had 512 MB.

That's obviously changed over the decades as process shrinks have lead to higher memory density. We should generally expect that ram will cheaper up and until the point where process shrinks stop happening. They've definitely slowed, but they haven't stopped.

> Crucial are already selling DRAM made by CXMT.

Crucial was disestablished this year.

Looking at the history of the memory industry the biggest risk is that a firm would over produce and go bankrupt. Maybe this time is different but so far no memory chip maker has gone under because their competition increased capacity.

Is there any indication research is being focused on reducing menory footprint of inference for frontier class models? Is the low hanging fruit already gone there?

What is the risk? Competition is good for consumers.

These are fundamentally different points in design space though, hbm doesn’t have a 10mw idle draw like lpddr does.

Can’t put HBM in smartphones and laptops. The power drain is too great.

Not many workloads are RAM bandwidth limited. Power and latency are much more common bottlenecks, and HBM loses on both of those.

As Yogi Berra famously said, "It's tough to make predictions, especially about the future." But based on historical tech industry trends, a price increase in one component that's this rapid and extreme, is likely to eventually regress somewhat toward the long-term trend line - even if that trend line experiences a longer-term shift upward.

As always, some interpret certain recent events as reason to conclude "but this time it's different." Occasionally they are correct. But that doesn't change the fact that it's reasonable to assume some of the recent extreme, rapid price inflation is due to shorter term market distortion. It's also pretty clear that some of the recent increase in demand represents a stable increase in the long-term trendline. The question is how much is long-term stable and how much is short-term distortion.

Then I will make my build last as long as it can, in protest of that. I do expect at least a performative price drop in the coming years, though.

This has already happened. Home PC market is practically dead already due to memory, ssd and graphics card price inflation. Makers of components like PC cases and power supplies etc. are seeing demand down 30-40% year over year and this is going to put many suppliers out of business. NVDIA has stopped even listing gaming revenue on their earnings reports. Both NVDIA and AMD are not seriously interested in supplying the consumer GPU market anymore either.

The only hope left is really Apple, but even apple has conspicuously delayed the launch of M5-gen mac minis and mac studio. Mostly because even Apple can't source enough DRAM to fully supply all their product lines.

I don't understand the threat to the PC market.

Prices haven't risen THAT much and are quite affordable. And if you look at the improved quality of upscalers (DLLS 4.5 for example), gaming is now more affordable than ever, despite the increased cost of components.

Of course, the 5090 prices are insane, as are for SOME memory models, but that's nothing new and represents a fairly small market share.

> When I started building gaming pcs, the top top card was 750$ (NZD)

When I started building gaming PC, the top $700 cards didn't even provide comfortable performance or graphics. Back then, you were supposed to have several of this connected SLI or somethin. And even then, it wasn't always reliable, and it resulted in stuttering, lags, and graphical artifacts (in cases when it worked). Today, even $700 graphics cards are a much better product from a user perspective than the high-end cards of that time (and that's not even taking into account that $700 cards back then were much more expensive).

there's much more than triple A video-games running at 240 Hz on Ultra settings... a 200 USD laptop/computer has enough power to run hundreds of interesting indie games and AAA from the past

Yes, this will definitely renew interest in Stadia type products.

Indeed, Gamers Nexus is doing interviews with PC component manufacturers, and some are hurting bad right now. The PC market is no longer in competition, but rather survival mode. =3

https://www.youtube.com/@GamersNexus/videos

Don’t you worry - Microsoft and Amazon will have you covered with cloud streaming.

Can’t afford a computer because they bought up all the supply? They’ll conveniently sell it back to you with a subscription!

You’ll own nothing and be happy.

It's more likely to kill the AI market. They're overbuilding capacity and most of it is going unused. The upcoming haircut is going to kill a lot of the major players.

They've intentionally crafted an unsustainable business model in an effort to get users in the front door and raise their MAUs. We've seen this story before. We should know precisely where it's headed.

If it crashes after it kills the PC we’ll be left with… nothing? Path matters as much as destination

I fully agree, the billion dollar question is when it will come.

Oracle is getting sold because of how much capex they're spending on new data centers in the middle of a high rates environment. It's not like they're stockpiling cash due to doubting AI.

> [...] better idea to directly fund the expansion of capacity [...] > > However, that the hyperscalers and AI companies aren't doing this says a lot about their true beliefs about how much future demand AI will have.

With what money? They have to spend the money they get on hardware ASAP else they are left behind.

Intel struggled because they bet the company that Moore's law was over back in ~2014, and instead of upgrading their fabs to EUV they sent the money back to shareholders.

They forgot Moore's main lesson: only the paranoid survive. They thought they could coast, and it nearly killed them.

It makes me sad for the Neo 2.0. More ram is the only thing stopping me switching to it from a Pro.

They’re going to pivot to you renting desktop cloud compute instead of owning anything.

“Bubble”

Because RAM isn’t in PCs only. It’s in tablets, phones, laptops, DIY computers like the Raspberry, mini PCs, watches, smart TVs, game consoles, cars, routers, cameras, all smart appliances from refrigerators to washing machines, fitness trackers, printers etc. Cloud services are irrelevant to most of these categories.

Possibly the best deal there is

I really need to shut up, or bite the bullet and by one.

If you graph the tokens per second on the 5090, your jaw will hit the floor at how cheap it is

Which surely is the highest it'll ever be! You're suggesting that the price will go down in the future? Would love to hear more about your thought process!

I believe msrp is $2000 right?

There was only a very brief time it was selling for MSRP (last fall for $2000). Even if you use that as the previous data point, it's only 200% increased.

> The RTX 5090 is faster than an H200. It just has less ram

H200 has HBM and much more 64-bit compute

From my point of view, I suppose we will enter a "Let AI generate entertainment" era. In which you just might rent everything, including games. No need for a beefy computer at home, you just need a slim endpoint:

"Order yours now, for just $99.99 per month, hardware included! Order today, and you will get three months of 'Office Suite' for free, with a small additional cost of $49.99 after month 4. On a tight budget? Switch to the yearly subscription, and pay comfortably in 18 installments."

If DLSS 5 becomes the norm it's possible that just makes things worse. The DLSS 5 demos required an entire separate card to run the model, though IIRC NVIDIA did claim it would eventually work on a single card. Given what the model is doing (yassifying the whole scene instead of just upscaling/reconstructing) it makes sense to me that it would increase compute demand instead of reduce it like previous versions of DLSS.

I know the differences between SRAM, DRAM, ...

I asked for evidence different people keep feeding me opposite stories: one insists its not fab capacity but wafer competition, with a recent article claiming HBM3E takes 3 times as much wafer area per bit than LPDDR5X. Others tell me the complete opposite: its fab capacity, not wafer shortage.

Do we have citable references to ground either set of claims?

And that article is contradicting other voices. If that article were correctly identifying the bottleneck as wafer shortage due to switching to HBM, why is everybody discussing the memory makers instead of the boule growers. Memory makers can expand operations all they can, which makes no sense if wafer supply doesn't follow, and the article is suspicously light on semiconductor boule / wafer mfr's.

So which is the bottleneck: fabs or boule growing?

also consider how most solar panels are monocrystalline silicon, how credible is silicon wafer shortage ... really? there is so much disinformation in this market...

Is there an understanding of what OpenAI intends to do with that memory?

Surely they need GPU capacity and would need memory for those GPUs but OpenAI doesn't build GPUs or any hardware, right? So did they pay to keep the supply locked up, or do they have the ability to put that ram into use?

Every time we add capacity with almost anything, we find ways to saturate it.

A lot of words to say that Sam Altman bought up the worlds total supply of ram chips for the next few years.

"That leaves small SAAS companies exposed to incoming inflation in the cost of hosting".

Which they will pass on to their customers. If their product provides enough value the customers will pay.....

Capex expenditure start exploding after covid with the chart going hockey stick at the end of 23/start of 24, almost 2.5 years ago.

A lot of capex is supposed to go into the datacentres, didn't they know that datacentres need to be filled among other stuff with RAM? I wonder if at some point we will discover that there is a shortage of fibre optic cables of SFPs ...

PS: Obviously armchair economist here too ... but for it doesn't seem too difficult to foresee the increase of the demand.

//Making memory isn’t super hard. That’s why it is a commodity.

These two aren't related.

Dram is a commodity because the you can replace a chip from hynix with a chip from micron, the have the same behaviour.

And a price competitive Dram isn't easy manufacture, or China would have made it already.

> Making memory isn’t super hard.

Then why do only 3 companies make it?

> up and down cycles have bankrupted the vast majority of players in the past

Exactly, so what’s the incentive for anyone to sink half a billy into building out more capacity.

The existing players get to rest on their laurels and succeed whether or not the AI bubble busts.

They won’t.

>I for one will only buy Chinese RAM from now on, no matter the prices.

Memory is a commodity, so I think you will be very lonely in your quest.

I heard Greg Brockman on a podcast saying they are limited by computer and memory. They have line of sight in solving many different kinds of problems. But they also have to survive in the meantime. Hence the focus on enterprise recently. They could just ask Government to fund them doing other research areas

Better algos = more demand

Memory squeeze will get worse before it gets better.

isn't there a law for that? as things become cheaper you consume more?