Sure, it logically makes no sense. But while learning a new subject, have you never made a silly mistake like:

bool getSchemaSizes(size_t * expectedBatchSize, size_t * expectedEntriesPerBlock) { ... }

size_t expectedEntriesPerBlock, expectedBatchSize;

getSchemaSizes(&expectedEntriesPerBlock, &expectedBatchSize)

initBloomFilter(expectedEntriesPerBlock)

Comments that explicitly call out an article as slop tend to get downvoted (or disagreed with), it's best to guide the reader towards their own conclusions.

A lot of people know the basic rule of thumb that a byte per element gives you a bit more than a 1% false positive rate.

But even just thinking about it for half a second from a balls and bins perspective, 100k items into 100 binary bins is obviously gonna saturate.

I make all sorts of silly mistakes, but I'd rarely say that running the code is the only way to detect issues.

I also don't think the author wrote much of their codebase, or much of their blog post, but that's the brave new world we're living in.

The author didn't write the blog post so my default assumption is they didn't write the code either.

I'm called in to consult on a performance problem on a scaled service. Team was load testing their code and seeing low throughput:

Me: so you have an in-memory cache, right?

Them: yes!

Me: what is the TTL?

Them: Oh, it's not set, oops. Here, let's set it to 1 minute. Hey look, the performance went way up!

Me: okay, great. When you say 1 minute, do you mean 60 seconds?

Them: uh...wait...uh....oh, the unit is seconds. Wait, why is the performance so good with a 1 second TTL?

Me: What's your load test?

Them: We crank 1M TPS fetching the same 30 items over and over.

Me: ....

I totally agree about the power of profiling but profiling without understanding would not have helped this team.

No, that's not the point. This isn't a situation where you need to "guess"; bloom filters should be sized according to their capacity. This is akin to having a fixed 10-arg buffer for your program, getting a crash when someone passes 11, and saying "this is the kind of bug you only find by building the thing and measuring it". Yeah it happens and we all make silly mistakes, but it's just not true that this couldn't have been foreseen.

Cool! My first downvote!

If you can't at least guarantee write ordering you don't even have consistency.

Fsync is often used when the data doesn't truly need to be on disk, because there aren't very good write ordering APIs exposed, even if that's all you truly need.

Well, the thing about reliability is that you can't really guarantee it by testing one particular scenario.

It seems to me that neither the old nor the new version of the code is really "durable" as I would understand the word. The old version made a write syscall per batch, but doesn't say it also did an fsync per batch. The new version writes data to an mmap'ed file, and calls fsync in the background.

So both versions are "durable" in the sense that written data is preserved even if the process gets killed, because it's in the OS page cache. But in both versions, a write can be completed before the data actually makes it to disk, so a power failure will lose acknowledged writes.

They tested SIGKILLing the process, they didn't test a power loss situation.

"Every batch of writes called file.Write on the write-ahead log"

You don't write to the WAL on a batch.

> the author tested correctness after a crash.

You mean the LLM?