Speculative KV coding: losslessly compressing KV cache by up to ~4×

4 pointsposted 6 hours ago
by kkm
(fergusfinn.com)

No comments yet