The write last, read first rule

102 pointsposted 3 months ago
by vismit2000
(tigerbeetle.com)

31 Comments

lintfordpickle

3 months ago

I found the article interesting, but I don't think I understand what is meant by 'Write Last, Read first' rule - even after reading it a few times. It seems to be too ambiguous a statement to be helpful.

Under the section 'Order of Operations':

> "Since the system of reference doesn’t determine existence, we can safely write to it first without committing anything. [...]"

Then the next paragraph

> "This principle—Write Last, Read First—ensures that we maintain application level consistency."

What I think it means is, 'writing-last to the system-of-record' and 'a read-first from the system of record' yields authoritative results, but I don't get that just from the title. Is my understanding correct?

jorangreef

3 months ago

Yes, write last to the system of record, read first from the system of record. Or in other words, commit to the system of record, and then read from the system of record to see what's committed.

(This is similar also to how chain replication preserves consistency.)

mecsred

3 months ago

If you read first and write last isnt that the opposite of committing and then reading to see what is comitted?

withinboredom

3 months ago

I think they just reinvented 2-phase commit? I'm not sure either tbh.

layer8

3 months ago

No, in two-phase commit all target systems perform two phases (staging the commit, which may fail, and then actually committing it in a failsafe way), which isn’t the case here.

withinboredom

3 months ago

You’re thinking too concretely. What do you think you’re doing before you write “to the source of truth”? You’re staging the commit. And then you commit.

2pc isn’t literally about transactions like you are probably thinking of in a database, its an abstract “atomic change” or “unit of work” that may or may not involve a database or a database transaction. You can do 2pc with just normal files, or APIs, or whatever.

layer8

3 months ago

I disagree. The definition of a two-phase commit protocol is that you have a number of participants in a transaction, and the first phase consists of asking each participant if they can commit, and if or when all participants affirm positively, then the second phase consists of telling all participants to perform the commit. Let’s not dilute well-established terms.

The procedure in the article is not a two-phase commit, because changes are committed to the system of reference regardless of whether the subsequent commit to the system of record succeeds or not.

In addition, half of the rule in the article is about the ordering of reads, which two-phase commit isn’t concerned about at all.

withinboredom

3 months ago

It’s just moving this into the application level, but it’s still 2pc.

Intent: Begin the durable execution (i.e. resonate.run)

Prepare: Write to the system of reference -- safe to fail here.

Commit: Write to the system of record -- the commit boundary.

Ack/Recovery: checkpointing + idempotent replays.

Abort/Compensation: panic or operator intervention.

Ordering operations has always been a thing you should do. And they’re treating this as a distributed system to simulate an ACID transaction. For example, if you ever do locks of multiple things, the order you take the locks has to be the same across the entire system so that you never deadlock. If your database is taking locks, then order matters there too. They rediscovered what us in the distributed systems world have always known and fairly well-documented: ordering is how you simulate time and prevent paradoxes.

layer8

3 months ago

Not sure why this was downvoted, the comment is completely right.

jorangreef

3 months ago

Joran from TigerBeetle!

Happy to answer any questions. And thanks to Dominik Tornow of Resonate for writing this up as a guest post! It was a little rule we had coined, to help people remember how to preserve consistency across different DBMS's, and I think Dominik gave (beautiful) voice to it.

layer8

3 months ago

I concur with https://news.ycombinator.com/item?id=45886850, the article doesn’t actually spell out what the “Write Last, Read First” rule is supposed to say. The name of the rule also isn’t great, because it leaves open what to write to/read from, and doesn’t even suggest that there might be several systems to write to/read from, which however is essential to the rule.

jorangreef

3 months ago

The what here is the “system of record”. There might also be no other systems to write to/read from—and that's fine. The important thing is the order. Hopefully "Write Last, Read First" is unforgettable in that respect!

Or, can you suggest something pithier?

layer8

3 months ago

The article mixes two things: (1) That you should (depending on requirements) have one system of records while the other systems are merely systems of reference, and (2) that in that architecture, when (*) you write to both systems or read from both systems for the same logical record, then in the writing case you have to write to the system of records last, and in the reading case you have to read from the system of records first. The article however doesn’t make it clear that the titular rule is specifically (2), and that it is conditioned on (1) and on (*).

I have no suggestion for a better name off the top of my head. The issue I see is that you already have to know well its context and when it applies, also in order to not misremember it as “Write First, Read Last”, and to not mistake it as LIFO, or to relate it to a read-modify-write scenario in which you naturally would read first and write last anyway, though in a different sense. You see how the name is confusing?

jorangreef

3 months ago

I’m not convinced that it’s confusing, or that there’s a better alternative—but I appreciate your critique.

Do you not think if someone can remember those four words, they’re less likely to get it wrong?

If you could contribute some better suggestions we could consider them!

layer8

3 months ago

> Do you not think if someone can remember those four words, they’re less likely to get it wrong?

Only if they remember the words in the right order, which isn’t a given. There is nothing in the phrase that helps remembering the order, unless you already know how it has to work. What it probably does at least is remind you that the order matters.

Maybe find something that emphasizes that the master data (the data in the system of record) has the shortest storage duration (since its writes happen last and its reads happen first). Something like “keep central data central in time” (and peripheral data peripheral in time).

As a side note, while I know that “system of record” and “system of reference” are existing terms, they are a bit unfortunate in that they sound very similar, both abbreviating to SOR, and “reference data” having semantic overlap with “master data”, so “system of reference” could be taken to mean the system to be used as the reference, i.e. master data.

xmcqdpt2

3 months ago

IMO the example about checkpointing is not that demonstrative because creating entities is easy to make idempotent. What about transactions? Do they need idempotency and if so, how does it work?

bigcojosh

3 months ago

This was a lovely piece. I'm only surprised by paucity of interesting comments here. If Joran or anybody sees this late comment: I'd appreciate any other references to the approach. Surely it wasn't invented by TigerBeetle?

dtornow

2 months ago

Hey thanks for the kind words, I am happy you liked the article. You are correct, we did not come up with the rule (but Joran did come up with the name) but drew from the rich history of transaction management systems.

A wonderful paper that I highly recommend is A Transaction Model from Jim Gray, that illustrates like no other than transactions are all about coordinating actions and when to externalize their results to a user. The mental model presented in this paper frequently informs my thinking when designing systems

https://jimgray.azurewebsites.net/papers/a%20transaction%20m...

aaroninsf

3 months ago

One of the real pleasures of HN is that it provides for a clockwork-like enthusiastic rediscovery and re-articulation of basic computer science and systems concepts.

It's almost enough to make me believe in the independent existence of Platonic truths. Almost.

jorangreef

3 months ago

Of course, there’s nothing new except shining a spotlight (and coining the rule!).

tclancy

3 months ago

I was hoping this was going to be advice for being a good citizen of the ‘net, but this week seems to be Brought to You by Tiger Beetle, so here we are.

esperent

3 months ago

[flagged]

s-lambert

3 months ago

But it's not an AI generated image, it has the artist's signature in the bottom right corner. Tigerbeetle has a lot of custom artwork designed for their stuff that's high quality. I don't even see why you would think it was AI generated even without the artist signature, it's using the Tigerbeetle mascot and it looks drawn for the particular theme the blog is talking about.

pontsun

3 months ago

Yeah, it's clearly by Joy Machs.

jorangreef

3 months ago

Thanks, appreciate your kind words!

Yes, everything we do at TigerBeetle is handcrafted (by humans) for quality, including the art.

Joy Machs on our team is an artist, and as you spotted, he signs all his work—I always ask him to! :)

Joy also did much of the Zig artwork, which is how we met.

You can see more of his work here: https://sim.tigerbeetle.com (and see if you can find or "not find" an Easter egg in our docs).

user

3 months ago

[deleted]

MichaelGlass

3 months ago

from https://news.ycombinator.com/newsguidelines.html: "Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting."

evertedsphere

3 months ago

As a general rule when it comes to this pattern of replying to "this is AI generated" with that link: the people that write these posts often read HN and attach a certain amount of importance to the opinions presented here, and it's important that people express their opinions about trends in how the majority of technical writing submitted to this website is either generated or presented, before they become well and truly entrenched as being problems "too common to be interesting".

There's a difference between criticisms of the content or the reader's ability to view it and complaints about "tangential annoyances" surrounding it.

another_twist

3 months ago

You would be missing out. Its a very well written blog post from a project that is solving a really important problem.

jorangreef

3 months ago

Thank you for defending (handcrafted) art! :)