How much intelligence can you run locally?

1 pointsposted 7 hours ago
by DangerousYams

2 Comments

DangerousYams

7 hours ago

I wanted to understand where local models stand today compared to frontier models. What kind of hardware is needed? What are inference options? And what does local vs open-weight hosted on data centers look like?

spottedmarley

4 hours ago

According to my own benchmarks running local models in my custom harness, Qwen3.5:122b was already better than Haiku and about equal with Sonnet 4.5 on coding tasks. Qwen3.8:27b-gsq-iq3s is better than both of them and also pretty dang fast on local hardware (~50-60tk/s on GB10 128Gb VRAM).

The harness is just as important as the model.