Scaling and benchmarking a critical message bus using a new indexing strategy

52 pointsposted 3 days ago
by eatonphil

5 Comments

jgalt212

3 days ago

To me, buses seem like a single point of failure your app / service would be do best to avoid depending on. I won't argue you shouldn't publish everything to the bus, but I will argue you should minimize the number of subscribers.

rtpg

3 days ago

This is pretty funny to see, because every financial firm interview process I've seen involves some variant of solving a bunch of stuff with min heaps.

Never before has a set of engineers been more primed to solve a problem

user

3 days ago

[deleted]

soltanov

3 days ago

[flagged]

hansvm

3 days ago

Yes, they do, but the extent to which that matters varies greatly. Accuracy/time tradeoffs exist (and are moderately common at $WORK right now). If you provably can't do better than a linear scan (and benefit from the increased accuracy from doing so at a business level), you might as well lean into it and choose a dead-simple, CPU-friendly algorithm for your problem.