nabla9
8 hours ago
With CMOS the heat puts hard limits for every type of stacking (transistor layers in a chip, or stacking CMOS chips).
In DRAM tech you can stack several (4-8 at least) HMC memory layers on top of one logic layer.
SSD is moving from 100+ towards 1000 layer memory chips.
londons_explore
6 hours ago
If you know you are thermally limited, there are a bunch of designadjustments you can use to produce less heat. Lower clock speed, more regions powered off at any given time, clock gating, etc.
By being able to stack logic, it will just change the way designs are made. for example, custom silicon to accelerate a specific operation, for example gzip compression, becomes far more attractive because now there is lots of 'spare' silicon area as long as it remains powered down 99% of the time.
nabla9
5 hours ago
Silicon area is expensive. And the logic becomes more expensive per area when you stack it. The whole purpose is to extract performance from the chip by making it dense and distances short.
amirhirsch
4 hours ago
Maximum performance is only one corner for optimization. We do not operate chips at high temperatures in our pocket; the goal is to get low power for the same functionality to have a long battery life, in which case parallelization means doubling the number of transistors while halving clock rates lowering the total power quadratically. It is also not more expensive per area to stack logic: by lowering the area of each die, you increase the yield of each in the stack.
wtallis
an hour ago
> We do not operate chips at high temperatures in our pocket;
Sure we do. When you actually hit your phone's processor with a non-trivial workload, the die temperature very quickly spikes to near the safe limits for silicon (ie. 90+ °C). It's only if the workload is sustained for a relatively long time (eg. when gaming) that the heat starts to be conducted to the outside of the phone and drive further throttling of the processor to prevent those surfaces from reaching temperatures that aren't safe for human hands. Even when a phone is heat-soaked enough that it is enforcing skin temperature limits, there's still a much higher temperature at the die.
> It is also not more expensive per area to stack logic: by lowering the area of each die, you increase the yield of each in the stack.
The stacking process itself does not have perfect yield, even when assembling known-good dies. Every layer added increases the risk of the stack as a whole becoming non-functional. 3D stacked DRAM has mostly been limited to single-digit stack heights, despite being the easy case for die stacking: every die is the same, has a more or less identical workload (and thus similar thermal expansion) and uses the same interface with the same TSV locations. Most of that goes out the window if you're trying to make a large stack of various co-processors.
d_tr
4 hours ago
You also save quite a bit of routing, right?
nabla9
3 hours ago
> by lowering the area of each die, you increase the yield of each in the stack.
Nope. The yield is relative to the surface area in layers, not the die surface area.
d_tr
2 hours ago
You mean you have to stack the chips first to test them, in which case the whole stack would have to be discarded?
mapt
3 hours ago
The upper limit of heat extraction is an interesting problem. Intel certainly seems to have bumped into a region with the i7-12900k and later where the chip is heat limited under all load tests even on watercooling, and improving significantly on the best air-coolers requires taking off the protective 'heatspreader' for closer contact with a heatpipe. Those heatpipes seem to be mostly watercooled.
Some kind of supercritical CO2 heatspreader with direct die fluid contact perhaps?
gdiamos
8 hours ago
Is this why there is more attention on PIM - processing in memory?
You don’t burn active power?
petra
7 hours ago
It's mainly as a way to overcome the bandwidth limits between memory and CPU.
It will increase the heat density of the memory, but there's room to play there. Plus there are some designs that use analog electronics for compute, and it reduces power.
CyberDildonics
4 hours ago
PIM - processing in memory?
I don't think this is something that actually exists in a working competitive state. I also don't think there is "more attention" by anyone actually working on cpus.
You don’t burn active power?
What does this mean?
gdiamos
3 hours ago
SSDs are 3D with high stacking. They retain data without burning power, unlike SRAM.
What if you put some of these 3D transistors tightly integrated with a 3D SSD and power gated them?
It seems like you would beat the memory wall.
Obviously manufacturing that is beyond us today.
CyberDildonics
3 hours ago
It seems like you would beat the memory wall.
I don't know what this is supposed to mean.
Why would putting coprocessing somewhere outside of the CPU be better than just sending data to storage?
gdiamos
3 hours ago
It takes a lot of energy to move data
CyberDildonics
3 hours ago
Says who?
Also wouldn't this idea just be moving data before it's fully processed? Why wouldn't the intermediate data be bigger?
Also "a lot" isn't a number and what are you comparing it to?
gdiamos
2 hours ago
Says the hotchips keynote - https://www.youtube.com/watch?v=rsxCZAE8QNA&t=2480s
- data that is moved is about 100x higher power than data that is local
You have a good point about intermediate data. This would not apply to all algorithms.
CyberDildonics
2 hours ago
You're talking about processing data, then moving data and processing it somewhere else, which is the opposite of the locality that presentation is saying valuable.
giancarlostoro
5 hours ago
> SSD is moving from 100+ towards 1000 layer memory chips.
Does this increase capacity, performance or both? I'm by no means a hardware guru.
apples_oranges
8 hours ago
SSD layers: Do you mean each cell is storing around 100 different values or are cells layered on top of each other (each addressed separately etc.)?
throwaway48476
8 hours ago
The cells are layered but each cell stores 4 bits in QLC.
AnimalMuppet
5 hours ago
QLC = "Quad Level Cell", if you're like me and didn't know that.
d_tr
4 hours ago
Which is a strange name and underselling it btw, since there are 16 levels...