25 minutes ago
Take a look at https://github.com/concurrencykit/ck and https://github.com/microsoft/mimalloc, it will fit well for a zero-copy and mem aligned reverse proxy. Also, if you want to add a DDoS protection and more advanced L4 stuff check out https://docs.ebpf.io/ebpf-library/libxdp/libxdp/
an hour ago
Boost asio if you love C++ and asynchronous networking.
an hour ago
I switched out asio's epoll backend for its io_uring in a database server and CPU utilization shot up. Probably depends on usage and the specifics of how it's integrated into the event code.
35 minutes ago
That’s paradoxically what you can expect on a busy server - your CPU can spend time doing work that would have been previously IO wait time. Of course, it could be a bug in the implementation where you’re spinning doing no work erroneously, but depends on the details.
a minute ago
Yeah, the explanation that I've usually heard for this sort of thing is that it's intended to get back CPU time that's lost when too many system threads are blocking to keep something on every core even during I/O (or pay for it in terms of the context switching overhead if you compensate for this with an extremely large number of system threads). The theory is that you'll avoid idle CPU compared to the common "one thread per core" way of doing things due to some of them being idle during I/O, at the cost of using some extra CPU to handle more things in user space. Obviously how much this helps can vary between use cases, but the measure of how much it's helping (or if it's maybe not helping at all!) is throughput, not CPU utilization.
an hour ago
Yes, io_uring is significantly faster than epoll (I think I had like 20% faster req/s with io_uring.) The catch is that its kernel opt-in and disabled just about everywhere for security reasons. I think that it has direct memory sharing between the kernel and user-land which is kind of yikes. There's been multiple exploits that hit io_uring in recent times. It's because of this that even engineering projects that try to reach the highest performance possible (like Go) don't really bake io_uring in as a sane default. Though if you want to take the risk you can always run it yourself for your favourite language. It is faster but the cost is possible exploits.
22 minutes ago
Any kind of poll mode networking:
Rdma, dpdk, io_uring it’s really kind of up to the user to do the memory isolation
In io_urings case tho, you can’t do much because the rings are in the kernel.
I’m hopeful though that with Llm things will get better.
But it’s just hard problem to solve . Very difficult to do in the kernel itself, and folks don’t really even understand tuning for it.