Safe Lock-free Primitives with iceoryx2's ByteAtomic

14 pointsposted a day ago
by elfenpiff

8 Comments

danbruc

a day ago

A common approach to mitigating the described data race without using blocking locks is to utilize a sequence lock.

A sequence lock is a blocking lock. If the writer dies between the two increment operations, then the readers will spin forever waiting for the counter to become even again.

rbranson

a day ago

This is why wait-free and lock-free are separate concepts. Author is not claiming wait-free.

jnwatson

a day ago

Also, there's no guarantee of progress. The writer can starve the reader forever.

jimaway123

a day ago

From the article: >>>The Problem: Even if the reader detects that the data was modified and discards the copy before use, the act of copying the non-atomic data itself still triggers undefined behavior. While a sequence lock can detect that a data race occurred, it does not prevent it.

Why is this a problem? Isn't the correct way to deal with a sequence lock failure to just retry? A torn read yes means you get undefined behavior as far as the result of your read, but you throw it all away and start again anyway so what is this solving?

adrian_b

a day ago

The problem is that a reader should not do anything with the data that is read, which depends on its value, before validating the copy, by counter comparison.

As another poster has said, the high-level languages leave undefined what happens when you copy non-atomically data that is written concurrently, but in fact the computer cannot catch fire when you do that, and the only thing that can happen is that the data may have values that are invalid for its type, e.g. an integer that is defined to belong in a range may have a value outside that range.

A much more serious problem that is not mentioned in TFA is that on computers that do not use Intel/AMD CPUs, this algorithm needs write barriers and read barriers. The writer must use 2 write barriers, after incrementing the counter before accessing the shared data, and before incrementing the counter after finishing with the shared data. Similarly the reader needs read barriers after the first reading of the counter and before the final reading of the counter.

From a hardware perspective this is correct. From a language perspective it's UB, and unless your compiler has defined that UB, it doesn't matter what the hardware's behavior is unless you're writing assembly.

elfenpiff

a day ago

iceoryx2 provides zero-copy inter-process communication mechanisms based on shared memory and data structures that are modified concurrently by multiple processes.

One of the key operations in these algorithms is a memory copy using core::ptr::copy. However, this results in undefined behavior if one process reads the data while another process writes to it concurrently. Even if our lock-free algorithm reliably detects such a race, iceoryx2 cannot depend on undefined behavior in a safety-critical system. This blog post introduces our solution: a byte-wise atomic wrapper that enables well-defined concurrent copy operations. It also shows how it can be used to implement a simple sequence lock.

eqvinox

a day ago

You're fixing a theoretical problem (mismatch between CPU and compiler memory models, the CPU is perfectly fine doing these reads and writes, it's only the compiler declaring them "UB") by throwing away a shitton of performance, forcing everything into bytewise accesses. Considering this is Rust, I would at minimum expect this be written to be generic over access size to allow using 64-bit reads/writes.

I'm also missing any acquire/release barrier annotations in your code snippets. If you're using sequentially consistent accesses you might as well just single thread your code, performance wise.

Lastly, in almost all cases it's way more efficient and appropriate to shuffle things on the whole-object level, posting and retrieving pointers, and not poke around inside objects (especially on the byte level). Check how rare the use of seqlocks in the Linux kernel is, compared to other RCU primitives. (and regarding "appropriate", cf. top-level comment by danbruc https://news.ycombinator.com/item?id=49168283 )