p1necone
9 months ago
"Torvalds said that it is not necessary to understand Rust to let it into a subsystem; after all, he said, nobody understands the memory-management subsystem, but everybody is able to work with it."
Chuckled a bit at this line, anyone have context on how true this is?
j16sdiz
9 months ago
"Torvalds said that, for now, breaking the Rust code is permissible, but that will change at some point in the future. Kroah-Hartman said that the Rust developers can take responsibility for the maintenance of the abstractions they add."
This need some very good expectation management.
j16sdiz
9 months ago
For most driver or subsystem, maybe you don't need to know how mm works.
Rust is different. The kernel Rust teams are trying to encode some safety invariant. If any of those mismatch with the C side, it breaks. Those invariant need some non trivial knowledge of rust to understand
nine_k
9 months ago
Is there an example of what you're describing?
steveklabnik
9 months ago
There’s a recent drama where the Rust folks asked some people to clarify some of the semantics of some of the filesystem APIs, and this request wasn’t taken well. There’s been a bunch of hn threads about it.
asne11
9 months ago
like which?
xgstation
9 months ago
not sure if this is what the op referred but like this one https://news.ycombinator.com/item?id=41450347
didn't find threads that regarding "clarify APIs semantics", but kernel docs are indeed not in a very good condition. Since C does not provide same level of soundness that Rust does, there are many hidden traps.
asahi developer had a good discuss about this https://threadreaderapp.com/thread/1829852697107055047.html
steveklabnik
9 months ago
This overall situation is, yes. And the stuff from Lina is related, thanks for also pointing that out.
steveklabnik
9 months ago
I apologize, I am on my phone, so rather than curating links, check out https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu..., most of which are about this situation.
pornel
9 months ago
About invariants generally: Rust wants to know when memory behind each pointer is immutable, or mutable by Rust only, or could be mutated by something else while Rust has a pointer to it. Rust also encodes which types can't be moved to another thread, and which pointers own their memory and need to be freed by Rust.
These are part of the type system, so they need to be defined precisely. The answer to these questions can't be just "it depends". If there are run-time or config-dependent conditions when the same data is owned or not or immutable or not, Rust has enums, unions and unsafe escape hatches to guard access to it.
dangitman
9 months ago
[dead]
vlovich123
9 months ago
By definition the undefined memory behavior is only in the undefined parts of the spec modulo bugs. The spec is written against an abstract virtual machine and C was one of the first to pioneer such a concept and why it was so successful at getting ported everywhere.
dangitman
9 months ago
[dead]
steveklabnik
9 months ago
> I was under the impression most of how c interacts with memory is part of the undefined part of the spec
For a long time, the memory model, formally speaking, was underspecified. Both C and C++ agreed on and added a memory model in C11 and C++11.
> fencing
You can add a fence via this API: https://en.cppreference.com/w/c/atomic/atomic_thread_fence
> This varies per arch
Right, so what assembly this API will emit depends on the underlying architecture details.
Notably, the Linux kernel does not use the standard C memory model, it uses its own.
vlovich123
9 months ago
To be clear, that’s the definition of the memory model under concurrent execution. I’m pretty sure the single threaded version was well defined
steveklabnik
9 months ago
My understanding is that “memory model” is definitionaly about what happens when you add parallelism and/or concurrency.
Regardless, this isn’t a slight against C. Most languages don’t have an explicit memory model at all.
vlovich123
9 months ago
I introduced the word memory model so that’s on me for that error - definitionally memory model does refer to concurrency.
However Op used “memory behavior”. The C standard definitely has defined and defined and undefined memory behaviors even pre C11. For example, dereferencing null is an undefined single threaded memory behavior that bit the kernel when the compilers started optimizing from assumptions about never reaching UB code. But for example there’s lots of defined behaviors like what happens when you dereference an aliased pointer.
And the Linux kernel definitely defined a memory model for itself before C and made all architectures conform to that.
steveklabnik
9 months ago
Yeah that's fair. As I said, I'm not trying to imply something was bad here before C11. I thought "underspecified" instead of "unspecified" would communicate "some of it was defined but some of it wasn't but it's been fully so for a while now" but maybe that missed the mark.
dangitman
9 months ago
[dead]
syndicatedjelly
9 months ago
It’s an opinion, but it sounds very good from the perspective of treating the relationships between system and subsystems as an interface to be managed.
raggi
9 months ago
It's very probably true in the totality of "as expressed in a real build for all configurations and architectures", too much variation of behavior to have the whole map in mind at once. You can work through it potentially, and I'm sure a few come close, but others will have things top of mind that experts don't.
klysm
9 months ago
It's true in the sense that nobody understands it well enough to avoid writing memory safety bugs.
AlotOfReading
9 months ago
A lot of kernel resources are managed through infrastructure like devres:
https://docs.kernel.org/6.0/driver-api/driver-model/devres.h...
These days it's entirely possible to write a decent driver with only the foggiest idea of how memory management happens in the kernel.