Show HN: I'm tired of my LLM bullshitting. So I fixed it

3 pointsposted 4 hours ago
by BobbyLLM

Item id: 46721773

6 Comments

gogopromptless

2 hours ago

How many facts do you have after using your tool for 2 weeks?

BobbyLLM

2 hours ago

Well, I've been using it more than two weeks (though I did just spend a month in Tokyo) so ... not sure how to answer that :)

Do you mean generally though - how many facts does it extract from a typical document of X length? Or do you mean what my own personal corpus currently is?

gogopromptless

2 hours ago

Size of your personal corpus is a rough signal of usefulness. If you've been using it a while and you have 500 facts, thats very different from 5000 facts.

My crappy personal system is a telegram channel "Today I Learned" that grows at roughly 5 messages per day. The search is obviously much worse than your tool.

The design of no rewrites for facts storage was an approach I have not seen from any of my friends' agent memory setups, so that difference struck my curiosity for how well its working in practice.

BobbyLLM

42 minutes ago

Got it. Answer in 3 parts.

If you're talking about the "Fastrecall" part of it, it's clamped to around 3000 entries. Beyond that...I mean...it's just a JSON file. How much do you hate yourself? lol :)

I use mine as intended (well, by me) as a short term memory store with a TTL of 14 days. Right now it has...350ish stored facts with different expiration dates. You can play around with the settings and max resets (touches), but if ctx > max_items, janitor rolls in and the oldest get pruned anyway.

If you're talking about the chat bloat control part (cut the crap), I set mine to keep the last 2 user/assistant pairs and a soft cap of 2000 characters. That middle part obviously decays during the chat. I like to keep my chatty fast.

If you're asking about the KB (attach) system: generally speaking, the SUMM function seems to generate a summary of upto about 1500 characters. That's what...250 words? Short and sweet...because again, potato pc + smart grep = don't be stupid.

In theory, the KB thing is limited by you SSD and pain tolerance. I have about 1500 items I query against and it's still in the sub-secondish range.

Mentats / vault? Scales via Qdrant.

Does that answer it? None of this shit is enterprise grade; it's purely for personal use.

reify

2 hours ago

gooncoding

"bro I spent all weekend in claud code. Its incredible"

"Oh nice, what did you build?"

"Dude my set up is crazy. I've got all the vercel skills. plus custom hooks for every project"

"Sick, what are you building?"

"My set up is so optimized. I'm using, like 5 instances at once"

"Hello bro, stop playing with your dick, answer my questions bro!"

BobbyLLM

an hour ago

Bro, what you get up to with your bro, is between you and your bro, bro.

Also, Claude said he likes me better than you and he'd know because he's French and old soul and shit.

PS: The concrete thing is a local control layer that makes my LLM shut up unless it can cite my docs. Plus I can run in on my p.o.s P1000 and not line NVIDIA shareholders pockets by buying more VRAM.

Repo’s over there if you want to kick the tires.