edolstra
10 hours ago
The deriver field in Nix has always been a misfeature. It was intended to provide traceability back to the Nix expression used to create the derivation, but it doesn't actually do that (since that wasn't really possible in the pre-flakes world, without hermetic evaluation). So instead it just causes a lot of confusion when the deriver recorded in the binary cache doesn't match the local evaluation result, due to fixed-output derivations changing.
In the future, Nix will hopefully gain proper provenance tracking that will tell you exactly where a store path came from: https://github.com/NixOS/nix/pull/11749
Ericson2314
8 hours ago
The biggest problem of all is that derivers are not unique! A separate "build trace" map will solve this.
tomberek
9 hours ago
Presumably this would support a big improvement to both SBOM generation as well as various UX features and workflow improvements.
setheron
10 hours ago
is that the 'build-trace' feature I saw John write about ? (I want to explore that more)
Ericson2314
8 hours ago
I think Eelco has in mind a separate thing that would still be a store object field. But IMO we should not do that since derives are unique, and we should instead use the "build trace" instead, which properly handles that.
As Martin Schwaighofer has discussed, it is fine and in fact good for build traces entries to have arbitrary meta data, so the "claims" being cryptographically signed are more precise. (This is good for auditing, and if something looks suspicious, having full accountability.)
So on that grounds, if eelco would like to include some "this came from this flake" information as informal metadata. (formally the key must still the resolved derivation.) That is fine with me.
---
As I linked in my other reply, see my fast-growing https://github.com/NixOS/nix/pull/14408 docs PR where I try to formally nail all this stuff down for the first time.
mschwaig
8 hours ago
I mentioned another alternative to adding flake-specific metadata to data structures that are transferred over the network, as part of the signed traces or otherwise, in a comment on that PR Eelco linked.
It's keeping flake-specific data locally, to guarantee that it matches how the user ended up with the data, not how the builder produced it. I think otherwise from the user POV such data could again look misleading.
Ericson2314
8 hours ago
Good point. It is misleading if different flakes end up producing the same derivation, and we don't want to resign our build trace entry to account for that (which would amplify reads into writes). Separate indirection for this eval->store layer accounting sounds good.