pama
3 days ago
Having worked with people doing bringup of specialized chips, I am awed at how the world has changed.
> When the first chips came back from the foundry in May, the team pointed its internal AI models at designing software to run benchmarks such as SemiAnalysis’s InferenceX. On DeepSeek’s multi-head latent attention kernel benchmark, performance climbed from 0.31 percent of the theoretical ceiling (set by the chip’s compute and memory bandwidth) to 88.94 percent in roughly 40 hours. Ho says this result is repeatable, so the time between when foundries deliver the first chips and when production ramps up can be reduced. “All our schedule assumptions are going to be based on the fact we have this capability now,” he says.
wmf
3 days ago
Back in the day you'd write the code before the chip came back but I guess today it's faster to wait.
brookst
3 days ago
Makes me wonder about AI and FPGAs. If the cost and effort to (re)program them goes to zero, maybe interesting new applications?
UncleOxidant
10 hours ago
I've got some Tang nano and Tang primer boards (9k, 20k and 25k LUTs). The 9K has a cute little SPI LCD display. Claude has been able to interface with the boards via USB (it setup a UART on the FPGA to do so). We've gotten a GoL as well as a Pong game (where paddles are controlled by Tsetlin machines) and a MicroCNN that does MNIST classification. For the latter we were having trouble getting correct answers back from the CNN (accuracy was very poor which didn't match simulation). Claude created an on-chip logic analyzer and doggedly troubleshot the problem (turned out to be some kind of problem in the yosys synthesis tool - we had to use -noalu to work around). Anyway, I was blown away by this. Claude was communicating with the FPGA board and analyzing the data coming back from the internal logic analyzer.
nz
2 days ago
Many languages can compile a subset of their code to FPGA HDLs. Back in the 80s Harel's group had statecharts that were compilable to C, C++, and FPGA HDLs. Not sure that LLMs brings anything substantially new to this.
gregsadetsky
2 days ago
I’m also very curious! I got a pair of icebreaker boards [0] and they’ve been great to toy around with.
The tooling is open source, and Fable in a loop - especially when paired with a digital scope that Fable interfaces with (the Saleae’s [1] are great) - gives you a level of verifiability that feels like beyond what software typically gives you. ie it feels more like Lean than code with tests.
I had ai implement a few toy circuits (sha hashing, 8088 emulation, a tiny llm) but yeah. Still looking for fun applications.
There have been a few recent fpga threads on hn, check them out. [2][3]
[0] https://1bitsquared.com/products/icebreaker
brookst
2 days ago
Thanks for sharing experience and links. Think I’m going to have to do this.
Eridrus
2 days ago
I had this same thought and think this is a generally interesting direction, but I think we're in a bit of a weird spot where the compute heavy stuff is on GPUs already and most infra stuff is not compute bound (it's often I/O bound or memory bound in some way).
It doesn't help that FPGAs are not made at the same scale as CPUs so don't benefit from the economies of scale.
I'm super curious if you have thoughts on specific pieces of software that would be economically better because I've thought about this in my niche and sort of come to the conclusion that it won't help.
I do think things like SIMD in CPUs will get more use and maybe we will get more difficult to program for CPU features, but I haven't found a use case where off the shelf FPGA components would help with typical software.
brookst
2 days ago
I’m looking at realtime mechanical processes, like shaping extrusion beads from a clay 3d printer. Clay is heterogenous and pressure takes time, so hand tuning is never just right. But put an fpga with vision processing? Seems promising, with millisecond-level latency that I’d never get pushing to a remote system for processing.
Eridrus
a day ago
Unclear that this sort of thing wouldn't already be quite well served by the existing gpu/npu hardware optimized for neural nets. If you're doing traditional CV, you can run it on the CPU just fine.
drob518
3 days ago
Hm. An interesting thought. Paired with RSI loops, that would allow rapid iteration in the hardware domain as well.
brookst
2 days ago
Yeah. Just spent an hour planning a closed-loop vision-based extrusion modulation system for 3d printing, with extensive telemetry and offline processing to iterate on the realtime system. Great, like I need another side project.
threatripper
3 days ago
The longer you wait the faster you will go.
LoganDark
3 days ago
Back when teams proved their designs and actually understood them...
RussianBot9580
3 days ago
Haha - understood. Good one!
They'd write a limited test for a feature based on an ask from the software team garbled by a five layer game of telephone. Claim that the module passed validation. A few months later the software folks would have to pull a few all nighters to figure out how to work around the resulting turd during bringup.
saidnooneever
3 days ago
cant wait for no one to really know whats in chips i mean, even intel hardly knows what all their reserved mem ranges are for. who will decap the chip and see if the docs were right? xD
marcelo-earth
3 days ago
> “All our schedule assumptions are going to be based on the fact we have this capability now”
is the world we live in, planning things while waiting for a more powerful LLM
cindyllm
3 days ago
[dead]
_zoltan_
2 days ago
I optimize a lot of CUDA and it got really, really good at it.