Ask HN: How is everyone using Local LLMs?

2 pointsposted 6 hours ago
by mraza007

Item id: 49211806

3 Comments

joe-dakroub

5 hours ago

I use Bionic (https://lmstudio.ai) and various local models like Qwen Coder Next to work on various tasks. Honestly, none of the models that are runnable on a typical MacBook Pro come close to the cloud-based frontier models (open or closed) but Bionic provides a nice experience and makes trying new models out trivial.

verdverm

5 hours ago

I'm running ~6 small models on an oem spark to back the various features of my personal knowledge base tool. A couple of LLMs and some non-llms too

https://github.com/verdverm/gmd

I run a "bigger" model on a second spark for coding tasks with an OpenCode Go sub for big models. It's nice to have access to the best open weight as well.

conartist6

6 hours ago

Well i have a really powerful model implanted right into my skull. When a model is so convenient, and even free to use, you'll find that you use it all the time I think