GPUs have wavefronts so I assume it is similar? Here is a page that explains it:

https://gpuopen.com/learn/occupancy-explained/

We're starting to see "AI chips" in that space.

"Positronic" came to my mind.

The reason problems are hard to fit into most of what's tried is that everyone is trying to save precious silicon space and fit a specific problem, adding special purpose blocks, etc. It's my belief that this is an extremely premature optimization to make.

Why not break it apart into homogeneous bitwise operations? That way everything will always fit. It would also simplify compilation.

[dead]

> Reminds me slightly of the Cell processor, with its dedicated SPUs for fast processing, orchestrated by a traditional CPU. But we all saw how successful that was (:

The success of the Cell is more subtle:

- (It seems many) game developers hated it because it is so different to program than other CPUs of other game consoles of its time (in particular the CPU of the Xbox 360). For game studios, time to market and portability of the game to other consoles is important.

- On the other hand, scientists who ported their high-performance numerical computations to the Cell seem to have loved it. Such software is often custom-built for the underlying hardware, and here cost (of the hardware) and possible speed are the measures on which to evaluate the hardware. Here the Cell processor of a PS3 cluster was much more competitive than other available solutions (GPGU did not really exist at this time).

> Reminds me slightly of the Cell processor

I was thinking the same.

Also the Tilera CPU with many cores and mesh network (back in mid 2000's, eventually bought by Nvidia and used in something, don't remember).

Tangent:

Back in early 2000's I had a hobby project (ALife with ANN brain) and I was looking for more computation. Multiple CPU's was not ideal, GPU wasn't ideal because the read/write/computation model only matched 1/2 of my ANN's flow and was a mismatch for the other half.

I read about a new cpu and I ended up talking to one of the key guys from Tilera, I was pretty impressed they would take the time to talk to some random guy working on a hobby project.

I asked about the performance of individual computational units (assuming custom could beat the industry) and he surprised me when he responded "nobody is going to beat Intel at integer, you won't get an increase from that perspective"

Also thinking machines corporation

https://en.wikipedia.org/wiki/Thinking_Machines_Corporation

I'd believe more in a heterogenous chip (e.g. MI300X, Apple M series, or even APUs) than in completely new chip tech.

What they say is far too vague, so it is impossible to know whether they have any new and original idea.

It is well known that the CPU cores that are optimized for high single-threaded performance are bad for multithreaded tasks, because they have very poor performance per power and per area, so you cannot put many of them in a single package, because there are limits both for the die area and for the power dissipation.

There are 3 solutions for this problem, all of which are used in many currently existing computers.

1. A hybrid CPU can be used, which has a few cores optimized for single-threaded performance and many cores optimized for multithreaded performance, like the Intel E-cores or the AMD compact cores.

2. One can have one or more accelerators for array operations, which are shared by the CPU cores and whose instruction streams are extracted from the instruction streams of the CPU cores (like in the CPUs from many decades ago the floating-point instructions were extracted from the CPU intruction streams and they were executed by floating-point coprocessors).

The instructions executed by such accelerators must be defined in the ISA of the corresponding CPUs. Examples are the Arm SME/SME2 (Scalable Matrix Extension) and the Arm SSVE (Streaming Scalable Vector Extension) instruction sets. These ISA extensions are optional starting from Armv9.2-A or Armv8.7-A. AFAIK, for now only the recent Apple CPUs support them, but in the future the support for them might become widespread.

3. The last solution is to have an accelerator for array operations that has a mechanism independent from the CPU cores for fetching and decoding its own instruction stream. The CPU cores have to launch programs on such accelerators and get results when they are ready. Such completely independent accelerators are either parts of GPUs or they may be completely dedicated for computing tasks, when they no longer include the special-function graphics hardware.

Any up-to-date laptop CPU already includes inside its package at least 2, if not all 3 of these solutions, to provide a good multithreaded performance.

For servers, it is much less useful to have all these variants in a single package, because one can mix for instance one server with big cores with high single-threaded performance with many servers using much more compact cores per socket, for good multithreaded performance, and the servers can use multiple discrete GPUs per server.

It is not clear with whom this "Flow Computing" wants to compete. They certainly cannot make better compact cores than Intel, AMD or Arm. They cannot make something like a SME accelerator, because that must be tightly integrated with the cores for which it functions as a coprocessor.

So their "parallel processing units" can be only competitors for the existing GPUs or NPUs. Due to their origins in execution units for shader programs the current GPUs are not versatile enough. There still are programs that are easy to run on CPU cores but it is difficult to convert them to a form that can be executed by GPUs. So there would be a place for someone that could design an architecture more convenient than that of the current GPUs.

However there is no indication in that article that there exists any problem for which the "Flow Computing" PPUs are better than the current GPUs or NPUs. If the PPUs have some kind of dataflow structure, then their application domain would be even more restricted than for the current GPUs and NPUs.

EDIT:

Now I have read their whitepaper "Design goals, advantages and benefits of Flow Computing", from HotChips.

However, what that paper says about their patented architecture raises more questions than provides any answers.

Their description of the PPUs is very similar to the description of Denelcor HEP from 1979. HEP (Heterogeneous Element Processor) was an experimental computer designed by Denelcor, Inc., which was intended to be a competitor for the supercomputers like Cray-1 (1976).

While HEP was based on very good ideas, its practical implementation was very poor, using non-optimized and obsolete technology in comparison with Cray, so it has never demonstrated a competitive performance. The lead architect of HEP has later founded "Tera Computer Company", in 1987, which has designed computers based on the same ideas with HEP. Tera Computer had very modest results, but somehow it has succeeded in 2000 to buy the Cray Research division of Silicon Graphics, then it was renamed as Cray, Inc. (now a subsidiary of HPE).

While Cray-1 and its predecessors (TI ASC and CDC STAR) were based on exploiting the parallelism of hardware pipelines with array operations, which can provide independent operations on distinct array elements, which can be executed in parallel in different pipeline stages, HEP was based on exploiting the parallelism of hardware pipelines with fine-grained multithreading, where independent instructions from distinct threads can be executed in parallel in different pipeline stages.

HEP had multiple CPU cores ("core" was not a term used at that time). Each CPU core was a FGMT core, which could switch at each clock cycle between an extremely large number of threads. (FGMT is a term that has been introduced only much later, in 1996, with its abbreviation only in 1997; at the time of HEP, they used the term "fine-grained multiprogramming")

The very large number of threads executed by each FGMT core (e.g. hundreds) can hide the latencies of data availability.

The description of the "Flow Computing" PPUs is about the same as for HEP (1979), i.e. they appear to depend on FGMT with a very large number of threads (called "fibers" by Flow Computing) to hide the latencies. Unlike GPUs and NPUs, but like HEP, it seems the "Flow Computing" PPUs rely mainly on multithreading (a.k.a. TLP) to provide parallelism, and not on array operations (a.k.a. DLP).

The revival of this old idea could actually be good, but the whitepaper does not provide any detail that would indicate whether they have found a better way to implement this.

Isn't this where "NPUs" are going now?

SIMD units already fit somewhere in the middle.

And it always gets prematurely optimized to fit a specific problem instead of being made general purpose compute engine.

FPGAs are essentially horrible systolic arrays. They're lumpy, and have weird routing hardware that isn't easy to abstract out. Those lead to multiple day compile times in some cases.

They don't pipeline things by default. The programming languages used are nowhere near a good fit to the hardware.

It's just a mess. It happens every time.

Systolic arrays are only used for fixed function computing within the context of individual instructions, e.g. a matrix multiplication or convolution unit. In practice however, most of these arrays are wave front processors, because programmable processing elements can take variable amounts of time per stage and therefore become asynchronous.

AMD's XDNA NPU is based around a wave front array of 32 compute tiles, each of which can either perform a 4x8 X 8x4 matrix multiplication in float16 or a 1024 bit vector operation per cycle.

How is it the same? The Xeon Phi was basically just smaller/weaker cores but more of them and with their SIMD units still there.

I've been following that saga for a long time. Seems mostly like vapourware sadly.

At this point, probably never it seems...

GPUs work on massive amounts of data in parallel, but they execute basically the same operations every step, maybe skipping or slightly varying some steps depending on the data seen by a particular processing unit. But processing units cannot execute independent streams of instructions.

GPUs of course have several parts that can work in parallel, but they are few, and every part consists of large amounts that execute the same instruction stream simultaneously over a large chunk of data.

Because GPU are physically built to manage parallel task, but only a few kinds

They are very specialized

CPU are generics, they have lots of transistors to handle a lot of different instructions

www.greenarraychips.com

Considering place and routing of even synchronous digital logic is at the edge of what we can do computationally I really don't see this happening anytime soon.

I’ve not yet had this problem but I surely will now! Thanks I guess.

The word "fintech" is disappointing, most because everyone calls their classical old school finance company a fintech that has a minor tech component. E.g. something like Solaris Bank is a fintech, but BaaS providers are a rarity and not the average fintech.

While the Itanium failed the Ageia PPU did succeed with its compiler. It was acquired by NVIDIA and became CUDA.

https://en.wikipedia.org/wiki/Ageia

Is this like the Itanium architecture with its compiler challenges?

Indeed, a very smart compiler would be necessary, perhaps too much for the current compiler art, like the itaniun.

But...how about specializing to problems with inherent paralelism? LLMs maybe?

Something similar to CUDA or OpenCl should do it, right?

This is a duplicate of https://news.ycombinator.com/item?id=40650662, and this article has nothing new in it.