Show HN: Local LLM Notepad – run a GPT-style model from a USB stick

40 pointsposted 7 months ago
by davidye324
(github.com)

9 Comments

gxonatano

7 months ago

> walk up to any computer

Windows users seem to think their OS is ubiquitous. But in fact for most hackers reading this site, using Windows is a huge step backwards in productivity and capability.

hereme888

7 months ago

idk... I gave up years of trying to switch to Linux as my main OS after the obvious difference in stability, support, ecosystem, and...yes even responsiveness in many apps.

Zetaphor

7 months ago

Surely you're hinting at Linux, in which case this runs fine with WINE

exe34

7 months ago

Why not llamafile? Runs on everything from toothbrushes to toasters...

romperstomper

7 months ago

Seconded for Llamafile, here is a link for references https://github.com/Mozilla-Ocho/llamafile . It indeed is working on all major platforms and its tooling allows easy creating of new llamafiles with new models. The only caveat is Windows where there is a limit 4Gb for executable files so just a llamafile launcher and the gguf file itself must be used. But this approach will work anywhere anyway.

ensocode

7 months ago

Interesting, will definitely try it. What can be expected? What other models do perform ok with this?

ge96

7 months ago

Wonder if you can use/interface with those coral accelerator boards