throwawayffffas
a month ago
A thought occurs. GPUs have a limited lifespan. Gpus die after 1-3 years of use.[1] Just in time to train an LLM or two. The data centers themselves without the GPUS is like 40% of the cost from what I hear. In 3 years from now when the boom ends, they are going to be empty warehouses, with very good networking and cooling.
[1]. https://ithy.com/article/data-center-gpu-lifespan-explained-...
adityaathalye
a month ago
True, they burn through GPUs. However, I wonder what the actual curve looks like... what fraction of total GPU capacity is getting maxed out, to the 3 year "burned to crisp" threshold. Training is harsher than inference is harsher than speculative capacity-hoarding (because, competition).
Even after that, what does a "burned out" GPU look like. Is it a total bust, or is still usable at... say, 25% capacity for "consumer type applications"?
Thank you for that GPU lifespan explanation... taught me a thing or two today.
throwawayffffas
a month ago
> Even after that, what does a "burned out" GPU look like. Is it a total bust, or is still usable at... say, 25% capacity for "consumer type applications"?
From what a hear it's a mix, of completely dead to degraded performance.
> Training is harsher than inference is harsher than speculative capacity-hoarding (because, competition).
I have heard over 70% quoted used for training, and like 5% for general purpose inference and the rest for code generation. But don't quote me on these numbers, I don't recall the sources. One has to assume that some capacity is also used for traditional high performance computing.
adityaathalye
a month ago
Still, I do wonder about the GPU manufacturing capacity upstream of datacenters, even though it grows relatively slowly. I suppose NVIDIA's order book is booked solid a few years out. However, capacity that they add can't just be repurposed / retooled for other use cases.
What could substitute LLM demand, if the LLM/AI business contracts rapidly?
adityaathalye
a month ago
Educated guesstimates are worth a lot. Thank you for the stats!
adityaathalye
a month ago
Another factor... They built it, and we didn't come.
Groq investor sounds alarm on data centers (axios.com)
32 points by giuliomagnifico 2 hours ago | 21 comments
> Venture capitalist Alex Davis is "deeply concerned" that too many data centers are being built without guaranteed tenants, according to a letter being sent this morning to his investors.
> [snip]
> What he's saying: "The 'build it and they will come' strategy is a trap. If you are a hyperscaler, you will own your own data centers. We foresee a significant financing crisis in 2027–2028 for speculative landlords."