Show HN: Local LLM Notepad – run a GPT-style model from a USB stick

39 pointsposted 6 days ago
by davidye324
(github.com)

8 Comments

gxonatano

6 days ago

> walk up to any computer

Windows users seem to think their OS is ubiquitous. But in fact for most hackers reading this site, using Windows is a huge step backwards in productivity and capability.

Zetaphor

6 days ago

Surely you're hinting at Linux, in which case this runs fine with WINE

exe34

6 days ago

Why not llamafile? Runs on everything from toothbrushes to toasters...

romperstomper

5 days ago

Seconded for Llamafile, here is a link for references https://github.com/Mozilla-Ocho/llamafile . It indeed is working on all major platforms and its tooling allows easy creating of new llamafiles with new models. The only caveat is Windows where there is a limit 4Gb for executable files so just a llamafile launcher and the gguf file itself must be used. But this approach will work anywhere anyway.

ensocode

6 days ago

Interesting, will definitely try it. What can be expected? What other models do perform ok with this?

ge96

6 days ago

Wonder if you can use/interface with those coral accelerator boards