kennywinker
4 hours ago
This feels like using a harrier jet to take in your groceries. Seems like something that could be built using one of the very small edge models - a 0.3 or 0.5b maybe with some fine tuning, and run on cpu on device for free without any real quality loss.
iamsky
3 hours ago
It definitely can, it just depends on the complexity of the commands you want it to generate. Most of the time a small local model will do what you want. If you know of a small local model fine tuned to generate shell commands I'm all ears. I usually run it with a 9B qwen model.
Pairing this with dictation is fun though, and speeds things up a lot for terminal heavy work.