If you don’t want to reinvent all of this yourself, this is exactly the problem we’re solving at Ailog.

Most local LLM setups break down because people try to use the model as both the reasoning engine and the memory store. That doesn’t scale. What works in production is a layered approach: external long-term memory (vector DB + metadata), short-term working state, aggressive summarization, and strict retrieval and evaluation loops.

That’s what we built at https://www.ailog.fr . We provide a production-ready RAG stack with persistent memory, retrieval controls, grounding checks, and evaluation tooling so models can handle long-horizon, multi-step tasks without blowing up the context window. It works with local or hosted models and keeps memory editable, auditable, and observable over time.

You can still build this yourself with Ollama, Chroma/Qdrant, and a custom orchestrator, but if you want something already wired, tested, and scalable, that’s the niche we’re filling.

Happy to answer questions or share architecture details if useful.

I built an agent that has access to my diary, it has the ability to build hierarchical summaries of my diary, which help to compress context, I gave it tools to read pages, search using full text indexes and RAG (the former worked better, but I think it's largely because of limitation in my RAG implementation), it also has the ability to record memories (append to a specific markdown page). The latter are automatically included in the system prompt, when I invoke chat.

https://github.com/robertolupi/augmented-awareness/blob/main...

I use it mostly non-interactively, to summarize my past diary entries and to create a Message Of The Day (MOTD) shown when I launch a terminal.