Ask HN: What you using local models for?

2 pointsposted 9 hours ago
by kandros

Item id: 49230235

3 Comments

cobrabts

7 hours ago

Searching my own notes meaning instead of keywords. I use a small model to do the search and a mid-sized model (around 7-8B) to answer, pointed at a few years of my notes. It's not as smart as a cloud model, but it's private. I though things at it which I would never paste into ChatGPT. And its quick enough that 'search what I have thought about before' is now a habit instead of hassle. The hard part is getting it to surface the right notes, not the answering itself.