The first time I powered on a Dell Poweredge R420 without the lid on

> How loud can they be?

LOL.. for home use I go either fanless (very low power), or 4U chassis even if I don't need the space, just to fit large quiet fans.

1U or 2U at home with fans is not what you want.

this is when you start thinking... well if a 40mm fan is jet-engine-loud, and an 80mm fan is just leaf-blower loud, and 140mm still spins too fast... maybe I can use a box fan, cardboard, duct tape...

(personally, I have wondered why people didn't do pc system cooling with a car radiator with huge slow fan, a giant reservoir like 5-gal homer bucket with lid or a 55gal drum, and maybe an submersible aquarium pump or two.)

I'm not aware of any motherboards, even server boards, that have more then 7 slots. Ditto for rackmount chassis. I'm sure they exist. If any do that are standard [E-]ATX, I'd love a link.

We've all been there. I don't think I've ever owned a home where I didn't have to cut holes in the drywall to run cables.

I mean, this is cool and all, but 2 DGX Sparks is like $10k, right? That's a huge upfront cost to run DS4 Flash 0731 which is definitely not comparable to higher-end Claude models. It's extra egregious when you consider how much it'd cost to run DS4 Flash 0731 via an API (how long would it take to run up a $10k bill doing what you're doing?).

>and from someone that has been using Claude for a long time, I can definitely say I don't need it anymore. Not for the stuff I'm doing.

I'm curious what this implies. It suggests that you were using Claude prior to this setup, so it doesn't seem like you were limited by security or local features? I can understand that someone not wanting to send their data to Anthropic (etc.) might be willing to pay for this kind of setup to accomplish that, but what was your motivation for doing this?

If it matters, if I had enough money that I could blow $10k on 2 DGX Sparks without it being a significant cost I probably would for the hell of it, so I'm not digging on you if this is ultimately what this comes down to. But, as an investment, this doesn't seem like a very good deal.

a heat stack chimney would make it silent

"used to" in your sentence already answers your curiosity.

ROCm runs most things just fine without any ceremony or difficulty. I have a couple of the same cards as covered in the article, bought before they got expensive (I'd recommend current Radeon AI Pro 9700 cards over the V620 now, though, as they have increased in price since I bought mine), and they're at the long end of the supported chart for ROCm...nearly EOL. But, they currently work great with llama.cpp and current ROCm 7.14. They're pretty fast and stable. I also have a Strix Halo 128GB, and while the Strix Halo can run bigger models, the dedicated GPUs are quite a bit faster. And, since the best models you can run at home are probably Qwen 3.6 27B and Gemma 4 31B, and those both fit comfortably on dual 32GB GPUs, I find I use the desktop more often than the Strix Halo.

Anyway, there's very little reason to spend 3x or more for the Nvidia ecosystem these days. The only exception is the Strix Halo vs the Nvidia GB10 platform. Strix Halo was a no-brainer when it was half the price, but it's risen in price to be almost the same price as an Asus GX10. In that one instance, I think the Nvidia based Asus is a better choice. The Strix Halo is almost too slow to make use of 128GB. Some MoE models are comfortable, like Laguna S2.1, so it's probable that there will someday be an MoE model that is better then Qwen 3.6 27B or Gemma 4 31B that won't run on 64GB but will run on 128GB. I don't know of one, yet, though.

AMD also used to focus almost entirely on their data center line and not pay any attention to their lower end stuff for more advanced AI features, but that's been changing, and ROCm support has broadened to include almost everything AMD ships now, including the smaller embedded stuff. I think they've realized that as long as the only way to develop AI for AMD was to have a quarter million dollars worth of hardware, they would always fall behind a platform that can be developed for and tested on consumer hardware.

For shits and giggles I had Claude build out a full voice cloning pipeline that runs completely locally on my Steam Deck. I’ve got Gemma e4b, Qwen and Chatterbox all running on the AMD.

lemon-server is amazing and gets you the right models for your hardware, along with a model router

Add openwebui and/or pi and away you go.

https://lemonade-server.ai/

Uhh.... I have a box with two AMD Radeon AI PRO R9700 32GB.

It was the most trouble-free AI setup that I have done. The driver worked right out of the box with the stock Fedora kernel, ROCM can be installed from the regular repo, and most AI software has ROCM builds by now.

The fan shrouds likely would have been fine. The hot spot on electronics is generally very localised to the chips and drops off sharply as you get further from them. I've printed a 10" home server rack out of PLA which is warm and under constant load, and it's been running for around a year now with no deformation.

A good rule of thumb is if touching it would cause burns, it's too hot for PLA, otherwise it's likely ok.

But, we know 100% that cloud-provided anything can and will get nerfed, broken, removed, or in some other way rug-pulled. It already happens all the time, and companies are getting more and more aggressive/stingy about what counts as "yours" and what counts as "bought" and what counts as "acceptable use".

Yes, bringing everything local still means you need to download things, maybe over and over. But once it is on your machine, nobody can yoink it from you just because they want more money or they don't like what you're doing with it.

Is there any hope that something like unsloth studio will let people retrain models continually so that, when the day comes that there are no good open models being released, something like the final generation of qwen whatever can continue being relevant into the future?

Hermes injects the date/time into the stream, and you tell it to web search for anything newer than the training cutoff date. Works like a charm with deepseek flash.

Updated models isn’t the same thing as hosting them somewhere else

We will try anyway. No need to try to FUD people into passive acceptance of the cloud.

I got mine below $400.