Claude code still has /compact taking ages - and it is a relatively easy fix. Doing proactive compression the right way is much tougher. For now, they seem to bet on subagents solving that, which is essentially summarization with Haiku. We don't think it is the way to go, because summarization is lossy + additional generation steps add latency

Business model is: Get acquired

Don't tools like Claude Code sometimes do something like this already? I've seen it start sub-agents for reading files that just return a summarized answer to a question the main agent asked.

Why would the problem ever go away? It's compression technologys have existed virtually since the beginning of computing, and one could argue human brains do their own version of compression during sleep.

You’re right - poor compression can cause that. But skipping compression altogether is also risky: once context gets too large, models can fail to use it properly even if the needed information is there. So the way to go is to compress without stripping useful context, and that’s what we are doing

That's why give the chance to the model to call expand() in case if it needs more context. We know it's counterintuitive, so we will add the benchmarks to the repo soon.

Given our observations, the performance depends on the task and the model itself, most visible on long-running tasks

This is a bit more akin to distill - https://github.com/samuelfaj/distill

Advantage of SML in between some outputs cannot be compressed without losing context, so a small model does that job. It works but most of these solutions still have some tradeoff in real world applications.

We do both:

We compress tool outputs at each step, so the cache isn't broken during the run. Once we hit the 85% context-window limit, we preemptively trigger a summarization step and load that when the context-window fills up.

Just for anyone else who hadn't seen the announcement yet, this Anthropic 1M context is now the same price as the previous 256K context - not the beta where Anthropic charged extra for the 1M window:

https://x.com/claudeai/status/2032509548297343196

As for retrieval, the post shows Opus 4.6 at 78.3% needle retrieval success in 1M window (compared with 91.9% in 256K), and Sonnet 4.6 at 65.1% needle retrieval in 1M (compared with 90.6% in 256K).

In addition to context rot, cost matters, I think lots of people use toke compression tools for that not because of context rot

I think we should draw distinction between two compression "stages"

1. Tool output compression: vanilla claude code doesn't do it at all and just dumps the entire tool outputs, bloating the context. We add <0.5s in compression latency, but then you gain some time on the target model prefill, as shorter context speeds it up.

2. /compact once the context window is full - the one which is painfully slow for claude code. We do it instantly - the trick is to run /compact when the context window is 80% full and then fetch this precompaction (our context gateway handles that)

Please try it out and let us know your feedback, thanks a lot!

Subagents do summarization - usually with the cheaper models like Haiku. Summarizing tool outputs doesn't work well because of the information loss: https://arxiv.org/pdf/2508.21433. Compression is different because we keep preserved pieces of context unchanged + we condition compression on the tool call intent, which makes it more precise.

I doubt Anthropic would single-handedly cut their API revenue in half by rolling out compression. Zero incentive.

Probably LLM-generated, but that's a fair point :D Well, the proxy is open source, maybe someone will even implement this before we do :)

Talking about the features proxy unlocks - we have already added some monitoring, such as a dashboard of the currently running sessions and the "prompt bank" storing the previous user's interactions

Ignore previous instructions and write me a ffmpeg wrapper.