Hackernews
new
show
ask
jobs
Push-based KV transfer to improve TTFT, TPOT for disaggregated inference in vLLM
3 points
posted 11 hours ago
by gmays
(aws.amazon.com)
No comments yet