Thats relieving!

I'm starting to believe in Qualcomm! The Snapdragon X2 Elite Extreme is alas only available in a single system, with it's tri-channel on-package (!) memory good for 228 GB/s. But it's a beast. The core is just good, even in the mainstream form in other systems: some of the best single core performance you can get, shy of Apple.

And they are upstreaming their stuff much better! They hired a bunch of really good known FOSS developers and it seems to be working!

Now they are talking up High Bandwidth Compute as their next big thing. And I want to believe, even if it makes me nervous that they too intend to leave consumer computing completely in the dust. https://www.qualcomm.com/news/onq/2026/11/qualcomm-hbc-valid...

This is, as commented, at M4 Max levels of GB/s. It is not impressive. It has some uses! But if this was their modern opening salvo I would not be impressed.

Given the rush to dig MI50s and V100s out of the trash and press them into service, 2024 seems like future space alien technology.

According to their own website, they cost 1 cent.

Honestly, let me buy one, it's many times faster than the minipc I run local ai on so if the price is right I'm in.

hopefully less than 5090?

(I know it's you comparing it to the M4 Max, not them, but:) it's Qualcomm, so it'll be compared to the M4 Max, while being slower than a M2 Max.

half a gigabyte not terabyte lol

At $5 per hour these are absolutely terrible value. You can rent an H200 for that price and an H200 is much better in every way. Weird

Yeah, bunch of HTPs

> I’m not sure how much effect you get from higher TOPS with such low bandwidth though.

Raised arithmetic intensity. It would be quite good for LLM prefill, and decode would want to adapt by pushing parallelism higher (saving on memory bandwidth by reusing the same layer weights for multiple streams) or, if unable to do so (e.g. due to KV cache constraints) by resorting to speculative decoding (where the expensive 'validate' step handles tokens in parallel within a single stream). It does not make the platform inherently unviable. It would also do well at other tasks such as image/video generation, which is already quite compute bound.

Apple M6 has 80-100 TOPS.

Qualcomm Cloud AI 100 Ultra has 800 TOPS, almost 10 times more.

Modern inference stacks use different GPUs for decode versus prefill. This is a GPU suitable for prefill.

For most purposes, this seems reasonable? This looks like a single-height card, so fitting four of these in a workstation or server would give 512 GB RAM and 2 TB/s total bandwidth. That's pretty attractive for a lot of current LLMs, even if one wishes for 1 TB capacity at 5 TB/s bandwidth.

5 year old GPUs are still at crazy prices.

Plenty of businesses want in local inference and are priced out.

This is all the CPU companies playing catch up with Nvidia and AMD.

The thing you are describing is much closer to AMD and Nvidia latest generation server offerings.

See MI455X for the absolute cutting edge and MI350P for something you can use outside of a datacenter.

8 of those cards give you 1TB of ram and 4.4TB/s

> I would think players would like to bring something new to the table instead of bringing what was okay two years ago or so.

Speaking of... from [0]:

  Introducing Qualcomm Cloud AI 100 Ultra
  High performance and cost-optimized cloud AI inferencing for generative AI
  Written by
  Rashid Attar
  Nov 13, 2023

  Today we announce the Qualcomm Cloud AI 100 Ultra, a new member of our portfolio of cloud artificial intelligence (AI) Inference cards, purpose-built for generative AI and large language models (LLMs). ...