In-DRAM goes back a long time. There were plenty of papers in the 90s about various ideas for turning a bank of DRAM into a SIMD machine. They weren't as clever as some of these ideas or as well developed but these papers are just the latest versions of an old idea.

One day, I'm going to credit my entire department, deli and everyone in the park at 2pm as contributors too.

This seems to be a formatting error. For such a huge author list, you usually write only the first name and then "et al." for "others".

This behaviour has been around since the earliest DRAMs with multiplexed row/column addresses. The Mostek MK4096 of 1973 could probably do this. Only took about half a century for someone to figure it out.

Yup, and incredibly they are able to do this on standard RAM by "intentionally violating the timing parameters":

Processing-Using-DRAM (PUD) leverages the inherent analog operational characteristics of DRAM to enable highly parallel bit-serial computations directly within memory arrays. Prior research has demonstrated that commercial off-the-shelf DRAM can achieve PUD functionality without hardware modifications by intentionally violating the timing parameters.

These studies have established two fundamental PUD operations: RowCopy and majority-of-X (MAJX) (Fig. 1). The RowCopy operation facilitates data movement between different rows within a subarray by issuing a PRE command followed immediately by an ACT command before bitline precharging completes, enabling data transfer through the bitlines. This operation affects all cells along a row simultaneously, making it approximately 100 times faster than processor-mediated data movement. The MAJX operation performs a majority vote among X cells sharing the same bitline that are activated simultaneously, implemented in commercial DRAM by issuing ACT, PRE, and ACT commands in rapid succession without delays. This allows concurrent activation of 2∼32 rows. MAJX enables bit-serial computations that leverage the parallelism of subarrays with 65,536 columns, serving as the fundamental computational unit for PUD.

Getting LLM inference running on any thing is going to be the next “it runs Doom”

This pops up in low latency HFT specifically with networking cards.

Certain network cards have either a bug or combination of features that work in an interesting way to the benefit of the trading firm.

These bugs (and features too) sometimes get removed in favor of either getting rid of the bug or those features are seen as not needed for the larger market etc. Therefore, firms will sometimes attempt to buy up all available supply of certain models.

This usually falls under "interoperability testing", but usually mitigated through your firmware rather than hardware. In the worst types of cases, you need to make sure your hardware works with some popular defunct vendor from 15 years ago since some big customers have used that hardware for 15 years, without issue, and will see your hardware as the problem when they plug it in.

For communication equipment, this is super important, with all sorts of "quirks" put in for vendors that didn't follow the spec. And, that includes keeping quirks in your firmware, so you don't break anyone else's. Imagine entire walls of legacy and long-gone and current competitor equipment, with robot arms to plug things in, and you have an idea of what some hardware validation labs look like.

Motherboard manufacturer firmware is also filled with quirks for specific CPUs, chipsets, etc.

Undefined behavior in C/C++ has been discussed for a very very long time. I'd say the impact of it when combined with optimizing compilers first came to broader public awareness around the 2010ish time frame (maybe 2013?) which is now about 12+ years old.

As for this paper, it's not about relying on a bug but rather presenting what might be possible with DRAM in the hopes of standardizing capabilities.

A matrix is a representation of a linear function (e.g. a function that plays nice with + and scalar multiplication). A specific subset can be used to describe rotations in 3D space. Quaternions can (arguably) do this better. But quaternions cannot be used to describe any linear function. So I do not think this makes sense for LLMs.

Quaternions only have four fixed dimensions. For neural networks you need many, many more dimensions.

I think you are mixing things. Quaternions are in the same category as complex numbers. They can be represented as matrices, and there are probably nice uses of matrices where the element is a quaternion (such as QDNNs) instead of a real number. My experience is that in massive architectures such as LLMs, simpler forms are more successful unless there is a true benefit to representing things with more elaborate types of scalars (such as in physics, or 3d graphics).

My understanding was that the main benefit of quaternions in computer graphics was representing rotations in a way that doesn't result in gimble lock.

And beyond that, for those rotations, a quaternion doesn't scale nearly as well as you add dimensions. Complex numbers are a complex representation of two space, quaternions are a complex representation of three space, and to go to four space you have octonions, which have eight elements.

I also strongly suspect that there are earlier sources.

However, IRAM looks like compute near memory where they will add an ALU to the memory chip. compute in memory is about using the memory array itself.

To be fair, CIM looked much less appealing before the advent of deep-learning with crazy vector lengths. So people rather tried to build something that allows more fine grained control of the operations.

Your questions are kind of amusing since Apple will use LPDDR6-PIM on the next generation of iPhones.

https://www.patentlyapple.com/2024/12/apple-plans-to-transit...

Yes, but only a little. Read the paper (or just search for "temperature") to see details.