I Cut 80%+ of Context Overhead in My Coding Agent

12 pointsposted 6 hours ago
by asm3r96
(m-reschreiter.at)

7 Comments

k9294

4 hours ago

I'm pretty sure it's a bad idea because each time you connect and disconnect tools, you are hitting a full cache miss on the full context, that will be probably more expensive than having these tools in place from the very beginning.

tym0

3 hours ago

Yeah adding tools dynamically seems good but removal is a terrible idea. Instant cache nuke.

I also really don't see how the hacky command interceptor is that much better than a dedicated tool. You're one which away from a very confused LLM.

mnahkies

4 hours ago

Does anyone know what the impact on prompt caching is?

I like the idea of dynamic tool/skill sets, but wonder if it needs to be part of the new session workflow where you would choose a group to activate rather than this dynamic.