Cutting p99 latency in Go by 74% with adaptive hedged requests

1 pointsposted 8 hours ago
by soniccontroller

3 Comments

soniccontroller

8 hours ago

To clarify the 74% number - that’s relative to no hedging (or static hedging) in a straggler-heavy setup.

The interesting part here is less the absolute gain and more that adaptive hedging matches the best hand-tuned static threshold without requiring manual tuning, even as latency distributions shift.

In real systems where latency isn’t stationary, that’s where this approach should have more advantage.

soniccontroller

8 hours ago

One thing I found interesting is that even same-target hedging helps behind load balancers, since you’re effectively racing different backend replicas.

But I’m still thinking through cases where this could backfire (overload amplification, etc.)