Predictive Speculative KV Replication for Bursty LLM Inference

33 pointsposted 9 hours ago
by shreybirmiwal
(jwlabs.vercel.app)

3 Comments

msp26

6 hours ago

the hand drawn diagrams and highlights are charming