Tell that to Synapse customers. Many millions of dollars are missing.
Banks have to follow strict rules to account for where all the money goes. But the way fintechs work, they usually just have one or a couple underlying "FBO" accounts where all the pooled money is held, but then the fintech builds a ledger on top of this (and, as the article points out, to varying levels of engineering competence) to track each individual customer's balance within this big pool of money. In Synapse's case, their ledger said the total amount of all of their individual customer balances ended up being much more than the actual funds held in the underlying FBO accounts. Lots of folks are assuming fraud but I'm willing to put money that it was just a shitty, buggy ledger.
FWIW, after seeing "how the sausage is made", I would never put money into a fintech depository account. Use a real bank. Fintechs also often put out the fake promise that deposits are FDIC insured, but this only protects you if the underlying bank goes belly up, not if the fintech loses track of your money.
See https://www.forbes.com/sites/zennonkapron/2024/11/08/what-th...
I’m skittish about real banking institutions as well. Vanguard for example outsourced a bunch of their dev work to India a few years ago. Had a friend that worked as a sysadmin for BoA. They were required to keep certain logging for 7 years but he would just delete it anyway when disks were starting to get full.
At a big co I worked at, the lack of consistency between trading systems caused money to (dis)appear (into)out of thin air.
Prior to one of these hiccups, I hypothesized, given how shitty the codebase was, that they must be tracking this stuff poorly.
This led to an argument with my boss, who assumed things magically worked.
Days later, we received an email announcing an audit one one of these accounting discrepancies.
JPMC proposed using crypto, internally, to consistently manage cash flow.
Not sure if it went anywhere.
At all of the exchanges and trading firms I’ve worked with (granted none in crypto) one of the “must haves” has been a reconciliation system out of band of the trading platforms. In practice one of these almost always belongs to the risk group (this is usually dependent on drop copy), but the other is entirely based on pcaps at the point of contact with every counterparty and positions/trades reconstructed from there.
If any discrepancies are found that persist over some time horizon it can be cause to stop all activity.
Wait, pcap as in wireshark packet capture?
I'm not the commenter, but yes, often trading firms record all order gateway traffic to from brokers or exchanges at the TCP/IP packet level, in what are referred to as "pcap files". Awkwardly low-level to work with, but it means you know for sure what you sent, not what your software thought it was sending!
The ultimate source of truth about what orders you sent to the exchange is the exact set of bits sent to the exchange. This is very important because your software can have bugs (and so can theirs), so using the packet captures from that wire directly is the only real way to know what really happened.
Typically not a literal pcap. Not just wireshsrk running persistently everywhere.
There are systems you can buy (eg by Pico) that you mirror all traffic to and they store it, index it, and have pre-configured parsers for a lot of protocols to make querying easier.
Think Splunk/ELK for network traffic by packet.
I suppose Pre-Calculated Aggregated Positions, but I am not an expert in the field.
Looking at the order messages sent to and received from another trading system was not uncommon when I worked in that neck of the woods
> JPMC proposed using crypto, internally, to consistently manage cash flow.
Yikes, how hard is it to just capture an immutable event log. Way cheaper than running crypto, even if only internally.
Harder than you'd think, given a couple of requirements, but there are off the shelf products like AWS's QLDB (and self hosted alternatives). They: Merkle hash every entry with its predecessors; normalize entries so they can be consistently hashed and searched; store everything in an append-only log; then keep a searchable index on the log. So you can do bit-accurate audits going back to the first ledger entry if you want. No crypto, just common sense.
Oddly enough, I worked at a well known fintech where I advocated for this product. We were already all-in on AWS so another service was no biggie. The entrenched opinion was "just keep using Postgres" and that audits and immutability were not requirements. In fact, editing ledger entries (!?!?!?) to fix mistakes was desirable.
Theoretically they even have a better security environment (since it is internal and they control users, code base and network) so the consensus mechanism may not even require BFT.
> I hypothesized, given how shitty the codebase was, that they must be tracking this stuff poorly.
That is like half of the plot of Office Space
If its for internal why not just use a normal append only log. x amount transferred from account y to account z. A three column csv oughta do it.
it literally ledger, its only show where money went but not showing "why" the money move
double entry with strong check that ensure its always balance fix this
Any time your proposal entails a “why not just”, it is almost certainly underestimating the mental abilities of the people and teams who implemented it.
A good option is “what would happen if we” instead of anything involving the word “just”.
Counterfactuals strike me as even less useful than underestimating competency would be. Surely basic double-entry accounting (necessarily implying the use of ledgers) should be considered table stakes for fintech competency.
It's all merkle trees under the hood. I feel like the crypto coin stuff has overshadowed the useful bits.
Is a Merkle tree needed or is good old basic double ledger accounting in a central database sufficient? If a key requirement is not a distributed ledger then it seems like a waste of time.
Merkle tree is to prevent tampering, not bad accounting practices
It only prevents tampering if the cost of generating hashes is extremely high.
Internally in your company you're not going to spend millions of $'s a year in GPU compute just to replace a database.
? If I use something like Blake3 (which is super fast and emits gobs of good bits) and encode a node with say 512 bits of the hash, you are claiming that somehow I am vulnerable to tampering because the hash function is fast? What is the probable number of attempts to forge a document D' that hashes to the very same hash? And if the document in structured per a standard format, you have even less degrees of freedom in forging a fake. So yes, a Merkel tree definitely can provide very strong guarantees against tampering.
"Prevents tampering" lacks specificity. git is a blockchain that prevents tampering in some aspects, but you can still force push if you have that privilege. What is important is understand what the guarantees are.
We launched Fireproof earlier this month on HN. It’s a tamperproof Merkle CRDT in TypeScript, with an object storage backend for portability.
See our Show HN: https://news.ycombinator.com/item?id=42184362
We’ve seen interest from trading groups for edge collaboration, so multi-user apps can run on-site without cloud latency.
This doesn't address the question in any way except to note that you also use Merkle Trees. Do you reply to any comment mentioning TypeScript with a link to your Show HN post as well?
What disrespectful marketing. We don’t care that you use Merkle trees because that’s irrelevant. I guess I can add Fireproof to my big list of sketchy products to avoid. It’s embarrassing.
I figured the responses would be more interesting. Questions about CRDT guarantees etc.
Perhaps worth seeding the convo with a remark about finality.
Crypto/Blockchain makes it harder to have an incorrect state. If you fk up, you need to take down the whole operation and reverse everything back to the block in question. This ensures that everything was accounted for. On the other hand, if you fk in a traditional ledger system you might be tempted to keep things running and resolve "only" the affected accounts.
It's mostly a different approach to "editing" a transaction.
With a blockchain, you simply go back, "fork", apply a fixed transaction, and replay all the rest. The difference is that you've got a ledger that's clearly a fork because of cryptographic signing.
With a traditional ledger, you fix the wrong transaction in place. You could also cryptographically sign them, and you could make those signatures depend on previous state, where you basically get two "blockchains".
Distributed trust mechanisms, usually used with crypto and blockchain, only matter when you want to keep the entire ledger public and decentralized (as in, allow untrusted parties to modify it).
> With a traditional ledger, you fix the wrong transaction in place.
No you don’t. You reverse out the old transaction by posting journal lines for the negation. And in the same transactions you include the proper booking of the balance movements.
You never edit old transactions. It’s always the addition of new transactions so you can go back and see what was corrected.
It's a question of business case. While ensuring you are always accounted correctly seems like a plus, if errors happen too often potentially due to volume, it makes more business sense sometimes to handle it while running rather than costing the business millions per minute having a pause.
I had money disappear from my HSBC account. As in, the double entries didn't match by a small amount (it was a payment between two of my accounts that didn't match up, which I couldn't trivially reconcile in the books). I pursued this for a while but they never even properly acknowledged it let alone fix it.
I had my unfounded suspicion it was some internal subtle theft going on, but incompetence is probably a better explanation.
If you live in a developed country it should be sufficient to ask them to account for it with a note that a formal complaint will be sent to relevant authorities if nor dealt with in timely manner.
That stuff like this is in order is the foundation of kapital societies and is taken quite seriously.
You'd think so wouldn't you? But alas, the effort required to solve this was at least more than I was willing to make.
If in the US, the CFPB would handle this for you.
I had a similar situation with Santander many years ago: it was a small amount and happened when I was closing my account, bank manager couldn't explain it and escalating the problem was a pain - especially because I was about to move to another country and had more urgent things to do.
I wonder how common issues like these are...
I think it's quite common, it's just that people do not notice these things.
I also had it happen one time, the bank eventually figured it out and fixed some error on their part.
You're likely correct it was theft. I was told by a CTO there that topping up accounts with bank money where it has been hacked away was daily routine and cheaper than fixing the systems. Incompetence would not manifest on routine operations like this.
> I had my unfounded suspicion it was some internal subtle theft going on
Had you watched Office Space recently?
Thanks for posting this, I will definitely listen to it.
While I haven't listened yet, one thing I don't really buy when it comes to blaming Evolve is that it should fundamentally be Synapse's responsibility to do reconciliation. This is what completely baffled me when I first worked with another BaaS company - they weren't doing any reconciliation of their ledgered accounts with the underlying FBO balances at the partner bank! This was insane to me, and it sound likes Synapse didn't do it either.
So even if Evolve did make accounting mistakes and have missing transactions, Synapse should have caught this much earlier by having regular reconciliations and audits.
They claim they did, Evolve kept putting them off, until they ran out of money.
There's a full transcript (with some images) below the player btw.
Rambling interview. As best as I can tell Synapse said there were technical issues with Evolve the bank.
Meanwhile this article said Synapse withdrew from Evolve the end user funds. Mr. Holmes of Evolve said the bank “transferred all end user funds” to other banks at the request of Synapse, but declined to identify them
https://www.nytimes.com/2024/07/09/business/synapse-bankrupt...
I'm sure the spokesperson for Evolve who then says "“It’s complicated,” he wrote in an email Friday, declining to elaborate further."" is fully trustworthy and not eliding any important details.
Wise also recently switched their US bank provider from Evolve to Community Federal Savings Bank. Maybe they had similar issues?
I see no reason why CFSB would be in any way different from Evolve, they are just not caught up in the mess yet.
Synapse problem was fundamental and it stems from the same mistake OP is making: never ever build your own, homegrown ledger if you can avoid it.
You can't avoid this. It is either your client or bank's client. And no bank will take the burden to account every $0.2 transaction for you, spending its own computing power. It just a quite expensive thing to do. That is why banks often separate the main ledger and retail ledger[s]. Each system tuned for a different performance profile.
From a cursory look at how it describes itself (BaaS, etc), Evolve is hardly a "bank" in the traditional sense of the word.
Coincidentally written something about this yesterday [1], but the gist of my take summed up is that the nature of accounting oriented data models doesn’t help when dealing with multiple FBO accounts.
The main problem is that accounting defaults to full fungibility of monetary amounts represented in a ledger, which has the effect of losing track of the precise mapping between assets and liabilities, so you end up in a position where you simply cannot tell precisely to a bank _who_ are the actual customers they owe money to.
[1] https://www.formance.com/blog/engineering/warehousing-promis...
> Lots of folks are assuming fraud but I'm willing to put money that it was just a shitty, buggy ledger.
I'm not sure there's much difference. Intent only matters so much.
I mean... Fraud is defined by intent.
You can argue negligence over mistake. But fraud definitely requires intent.
> Fintechs also often put out the fake promise that deposits are FDIC insured
Does this still happen?
Many fintechs are not licensed to hold funds and work with bank partners who hold your actual funds. That allows them to say they're insured because they're not co-mingled with the corporate funds in the event of insolvency. This doesn't stop them from making accounting errors.
the problem is the discrepancy between what the Fintech means when they say fdic insured, and what the customer hears when they're told fdic insured. the customer (erroneously) assumes it means that if the Fintech or anyone else has problems, the customer is covered up to the 250k fdic limit. what the Fintech means, is that there's someone they're partnered with that is a bank and is fdic covered. How the money is deposited into the bank is up for interperation. if there Fintech is being dishonest, they have one bank account at a bank, and all of the customers money goes into that one shared account, they're not technically lying - the money is fdic insured. unfortunately for the customers, that's not the same as each of them being fdic insured is the Fintech goes under. fdic doesn't seem to want to clarify this issue either, which is a problem.
The fact that practically all funding most of the world runs on these days is just a bunch of variables in some shitty program never stops being weird to think about. All it takes to create or destroy trillions is one (or maybe a few) CPU ops.
It really stretches the belief into fiat money to the absolute limit.
Dade: This is every financial transaction Ellingson conducts, yeah? From million dollar deals to the ten bucks some guy pays for gas.
Kate: The worm eats a few cents from each transaction.
Dade: And no one's caught it because the money isn't really gone. It's just data being shifted around.
> In Synapse's case, their ledger said the total amount of all of their individual customer balances ended up being much more than the actual funds held in the underlying FBO accounts.
When the banks do this it's called "fractional reserve banking", and they sell it as a good thing. :)
I’m constantly amazed by how much the crypto community thinks they understand fractional reserve banking while getting it so completely wrong.
In fractional reserve banking, money that is loaned out is accounted for as liabilities. These liabilities subtract from the overall balance stored (reserved) at the bank. The bank is not printing money new money, no matter how many times this idea gets repeated by people who are, ironically, pumping crypto coins that were printed out of thin air.
I think it’s incredible that cryptocurrencies were literally manifested out of bits, but the same people try to criticize banks for doing this same thing (which they don’t).
Money that is loaned out is still accounted for as liabilities.
Sure, those liabilities are accounted for in an eventually consistent matter by reconciling imbalances on interbank lending markets at the end of the day with the government topping up any systemic shortfall rather than by counting out deposit coins in the vault
But that's fundamentally much closer to the "Old Money" view than to the OP's claim about fractional reserve being like an FBO inflating customer deposits by failing to track trades properly...
There's an inordinate amount of nonsense being espoused in this thread, when the answer is in that first link. I can only assume it's the miseducation that economics textbooks perpetuate.
The "liabilities" aren't subtracted from the deposit amount when counted as M1 supply. (Actually loans are accounted for as assets and deposits are liabilities, but that's beside the point).
If customer A deposits $100 in cash, and customer B borrows $100 from the bank and deposits it back in the bank, M1 goes up because there are now two checking accounts with $100 in it. That the bank's internal bookkeeping balances doesn't change the fact that the Fed considers that more money exists.
I agree. The main point is that if B knows that they don't have to repay the $100 until 10 years in the future, then for the 10 next years everyone can pretend there are $200 in total.
This is a good explanation, I've had to explain this topic a few times as well, it seems like it's one of those topics that is very missunderstood.
To just expand a bit, I believe some of the confusion around printing of money comes from the way some economics reports are built. As a micro example, Assume a 10% required reserve, If Alice deposits $100 and the bank lends $90 to Bob. Alice ($100 deposits) + Bob ($90 cash) think they have $190 in total.
This is mainly useful for economists to understand, study, and report on. However, when the reports get distributed to the public, it looks like the banks printed their own money, as we now see $190 on the report when there is only $100 of cash in our example system.
Whether the system should work on a fractional reserve is it's own debate, but we need to know what it is to debate the merits and risks of the system.
And how does that work when the 'required reserve' is zero as it is now, and has been in the rest of the world since time immemorial?
Nobody deposits in a bank - it's just a retag of an existing deposit. The bank Debits a loan account with the amount owed, and Credits a deposit account with the advance. It's a simple balance sheet expansion in double-entry bookkeeping.
I'm really not sure why this myth persists given that central banks debunked the concept over a decade ago.
Loans create deposits, and those deposits are then converted into bank capital when a deposit holder buys bank capital bonds or equity.
[0]: https://www.bankofengland.co.uk/-/media/boe/files/quarterly-...
Most people deposit in a bank by transferring from another bank. There is more than one bank.
>These liabilities subtract from the overall balance stored (reserved) at the bank. The bank is not printing money new money
Hi, this is factually incorrect and you should educate yourself before attempting any further condescending comments on Hacker News.
But Banks are increasing the money supply with fractional reserve bank. But that is of course on purpose and account for by the govt.
In fractional reserve banking, the total deposits at a bank can be greater than the amount of physical money it holds. Since the rest of society is willing to accept bank deposits as an alternative to physical money, this is a form of printing money. Physical currency is not printed, but bank deposit currency (which is money, by de facto agreement) is.
First of all, I take offense to being thrown in as part of the crypto community, with which I have nothing to do, and for which I do not have much hope.
So now if you are unhappy with the monetary system you are automatically a crypto bro and can be dismissed?
Secondly, the problem with fractional reserve banking is as follows: Suppose Larry makes a deposit of one dollar, which the bank guarantees can be retrieved at any time. The bank loans this dollar to Catherine, which uses it to buy something from Steve. Now Steve has one dollar, which he deposits with the bank. The bank lends this dollar to Catherine2, which uses it to buy something from Steve2. And so on, up to CatherineN and SteveN
Now, in so far as transactions can take place in the economy with bank IOUs, which are considered perfect money substitutes, the amount of money in the economy has been multiplied by a factor of N. Where before only Peter had a dollar (or a dollar IOU, which are supposedly the same), now Pere AND Steve, Steve2, up to SteveN all have a dollar IOU. This leads to an inflationary pressure.
Now it is true that upon the Catherine's repaying of the debt, these extra dollars will go away. However, in reality there is no such thing as negative dollars. The supply of money has been increased by the bank.
An objection could be raised that Catherine's extra demand for money to pay off her debt will exactly offset the extra supply of money. This is nonsense! Everyone demands money all the time. If Catherine did not demand money to pay off her loan, she would demand money in order to satisfy her next most urgent want which could be satisfied by money. The increase in the demand for money is negligible.
Your explanation of fractional reserve banking is somewhat correct, but missing the big picture
Licensed banks can and do write loans at any time without having any deposits to 'lend out'. In doing so they create both the loan (an asset) and a deposit (a liability) simultaneously from thin air. The books immediately balance.
The deposit created is then paid to the borrower and the liability vanishes. The bank is left with only the asset - the one that they created from thin air.
For short term liquidity a bank can always use the overnight lending facility at the central bank. Doing so just makes all their loans far less profitable as this is at a floating daily rate.
In reality the limit to which the money supply grows is not dictated by 'fractional reserves', but solely by interest rate policy and the commercial viability of being able to make loans and demand in the economy.
Not quite. The deposit is paid to the borrower as an advance, and the deposit is transferred to the payee (or the receiving bank if the payee is at another bank)
The liability can never vanish - balance sheets have to balance. Bank liabilities are what we call 'money'. Hence how you are 'in credit' at the bank.
And when we look at the bank assets which back those liabilities, we find that (say) 10% are government-printed money, and the remaining 90% were created by banks.
Your mistake was saying Synapse merely did what banks do. Banks don't lose track of money when they increase the money supply.
My comment was meant as a tounge-in-cheek joke, with a dig at the banking system. It was not meant as a serious equivocation between what Synapse did and what banks do.
Clarifying question:
So for every $1 deposited, I can lend $0.90 but must hold $0.10 as my reserve?
It’s a bit more complicated than that.
At the point I make a loan, 2 things happen on my balance sheet: I have a new liability to you (the increased balance in your account), and I have a new asset (the loan that you’re expected to pay back). They cancel each other out and it therefore seems as if I’m creating money out of thin air.
However, the moment you actually use that money (eg to buy something), the money leaves the bank (unless the other account is also at this bank, but let’s keep it simple). Liabilities on the balance sheet shrink, so assets need to follow. That needs to come from reserves because the loan asset keeps its original value.
The reserve comes from the bank, not from you. Added layer here: Banks can borrow money from each other or central banks if their cash reserves runs low.
Finally: it tends to be the case that the limit on lending is not the reserves, but on the capital constraints. Banks need to retain capital for each loan they make. This is weighed against the risk of these loans. For example: you could lend a lot more in mortgages than in business loans without collateral. Ask your favorite LLM to explain RWAs and Basel III for more.
> However, the moment you actually use that money (eg to buy something), the money leaves the bank (unless the other account is also at this bank, but let’s keep it simple). Liabilities on the balance sheet shrink, so assets need to follow. That needs to come from reserves because the loan asset keeps its original value.
"Everything should be made as simple as possible but no simpler."
You're omitting the thing that causes the money to be created out of thin air. If the other account is at the same bank, now that customer has money in their account that didn't previously exist. And the same thing happens even if the money goes to a customer at another bank -- then that bank's customer has money in their account that didn't previously exist. Even if some reserves are transferred from one bank to another, the total reserves across the whole banking system haven't changed, but the total amount of money on deposit has. And the transfers into and out of the average bank are going to net to zero.
The created money gets destroyed when the loan is paid back, but the total amount of debt generally increases over time so the amount of debt-created money goes up over time as banks make new loans faster than borrowers pay them back.
The bank could also sell the loan instead of borrowing if they are in need of capital.
That is exactly what happens. Reserve ratio used to be 10%, same as your example. The reserve ratio is currently zero, lowered in 2020 during pandemics. But banks still can't lend out more than deposits.
> The reserve ratio is currently zero, lowered in 2020 during pandemics.
I saw this during the pandemic, and it bewildered me how little coverage of it there was. How is this not going to cause another financial catastrophe? And if we're so sure it isn't, then what makes people think they under economics so well, given that they clearly thought a minimum was necessary just a few years ago?
> I saw this during the pandemic, and it bewildered me how little coverage of it there was. How is this not going to cause another financial catastrophe?
The banks in Australia, Canada, etc have had zero reserve requirements for thirty years:
* https://en.wikipedia.org/wiki/Reserve_requirement#Countries_...
The US had reserve requirements leading up to the 2008 GFC which started off with mortgages/loans, and yet those requirement didn't stop the disaster. Canada et al did not have requirements, and yet it didn't have a financial meltdown (not itself, only as 'collateral damage' to what happened in the US).
Many central banks like the Bank of England don't even have a reserve requirement and rely on the bank rate to control it instead.
The equivalent for the USA would be the Federal Funds Rate, I suppose. The reserve requirement is just one tool among many.
Deposits≥Loans is a tautology since every time loans increase, so do deposits. It doesn't mean anything or provide any insight.
Fortunately, loans create deposits, so they are always in balance.
The bank IS printing new money. You are ignoring the money multiplier effect where the money lent by bank 1 is deposited into bank 2, bank 2 lends 90% of that deposit, which is deposited into bank 3, ... repeating the process over and over.
With a 10% reserve requirement, a 1,000,000 USD deposit will result in up to 10 times that much money being lent out.
The formula is 1/r, where r is the reserve requirement.
That´s not correct unfortunately, although it has been widely taught in economics text books, and you can blame Keynes for that. Keynes used that example to try and explain the process to parliament, and also to argue that the system didn't expand the deposit money supply over time. Ironically even the data (in the Macmillan report) he supplied contradicted him. It´s confusing as well, because the fundamental rules have changed over time.
Banks can lend up to an allowed multiple of their cash or equivalent reserves (gold standard regulation), and in the Basel era are also regulated on the ratio of their capital reserves to their loans. This acts to stop hyperflationary expansion, but there is a feedback loop between new deposits and new capital so the system does still expand slowly over time. This may be beneficial.
In engineering terms, Banks statistically multiplex asset cash with liability deposits, using the asset cash to solve FLP consensus issues that arise when deposits are transferred between banks. It´s actually quite an elegant system.
interestingly, the Fed's page on Reserve Requirements states:
As announced on March 15, 2020, the Board reduced reserve requirement ratios to zero percent effective March 26, 2020. This action eliminated reserve requirements for all depository institutions.
So in effect, the multiplier is infinity.
https://www.federalreserve.gov/monetarypolicy/reservereq.htm
I remember this. Have they ever rolled it back?
The Basel Capital rules dominate at the moment. If that ever gets rolled back... buy gold immediately.
There's more to it than that; balances are exceeded by the sum of "assets held by the bank" and "assets owed to the bank".
The whole "move fast and do a shit job" is laughably stupid. It's pure incompetence and egotism and a product of excess salary in the market.
Doing it the right way doesn't take any longer than doing it the shitty way. Successful startups focus on extreme minimalism, not focus on doing the worst possible job.
Except if you don't actually know what to do, or even what concepts to apply. This is where I see many start ups fall down.
This reinforces for me the importance of domain knowledge in engineering leadership. i.e. if you work for a finance company you need to have a decent understanding of finance in order to make the correct technical decisions and tradeoffs, same goes for journalism, same goes for commerce etc. etc. Successful organisations I've worked for have always included domain specific, non-technical, questions to tech team interviews whilst some of the most technically acomplished teams I've worked with have floundered through a lack of domain insight.
I’m a software engineer and a CPA, so I’ve got the domain knowledge. However, I’m not sure how to find a gig where I can apply that. It seems everywhere I look vastly prefers someone with twice my experience as a software engineer and zero domain knowledge over my years of experience in accounting and fewer years of experience as a software engineer. Any insight on how to utilize it effectively?
ever since i started working for an insurance company, i realized that understanding the insurance industry is far more difficult than understanding the codebases. if the codebase is acting weird, at least i can step through it with a debugger!
I agree with your sentiment here, but I think it helps to think of those domain specific questions as technical questions, just from a different technical domain. Finance is technical, mechanical engineering is technical, even sports management and sociology have big technical components too. I think having an expansive view of what technical competence is breeds the humility necessary to work across domains.
Yes, sure, if it helps you to think in those terms I guess that makes sense. Being a bit reductive perhaps but I think what it comes down to is having an interest in the "why" as much as the "what" and the "how".
If you are trying to reinvent parts of banking, you could maybe start with what already works as a thought experiment.
Here's an example of a core banking deposit transaction schema that has been extensively battle tested in many small & mid-size US institutions:
https://jackhenry.dev/open-enterprise-api-docs/operational-d...
You may note fields like "Effective Date" & "Affects Balance/Interest", which imply doing this correctly may involve exploring some interesting edge cases. Wouldn't it be cool if you could just cheat and start with an approach that already considers them?
Most of the comments here echo what the article is criticising. I can see countless threads of back-and-forth defending single-entry bookkeeping.
Sure, single-entry bookkeeping might be easier and more normalized, but sometimes it is a good idea to just stick with the systems and abstractions that have been developed over centuries.
Just use double-entry bookkeeping unless you definitely need something else. Sure, it might be icky for the programmer inside you, but I think you'll be thankful if you ever need to get actual accountants involved to sort out a mismatch.
On a related note: does anybody know of any good resources for programmers in payments and adjacent fields? Something like an "Accounting for Programmers"?
I consider that first link, Accounting for Computer Scientists, as the canonical guide for computer scientists as to wtf double entry accounting is and why it's the right way to do it.
Pardon me an old story... I never built a double entry accounting system but decades ago I did build a billing system for a internet/telcom startup that grew to a modest 8 figures revenue.
By accident and not knowing any better as a young dev, I ended up building the billing logic from day one, and for better and worse building it in two places in the system (on a consumer-facing billing webpage, and on a separate backed process that generated invoices and charged credit cards.)
It turned out to be remarkably hard to keep them in sync. We were constantly iterating trying to get traction as we burned down our capital, releasing new products and services, new ways of discounting and pricing (per use, per month, first X free, etc), features like masterpayer/subaccounts for corporate accounts, user-assignable cost centers, tax allocation to those cost centers with penny allocation, etc such that new wrinkles and corner cases would keep popping up causing the numbers on my two screens/methods not to match.
Being personally responsible for the billing, I would go over all the invoices by hand for a couple days each month to insure they matched before we charged the credit cards and mailed out printed invoices as a final check to prevent mistakes. There was always/often some new problem I'd find affecting one or a small handful of customers which I would then fix the code before we billed. I never felt good letting go and not doublechecking everything by hand.
I thought about refactoring the billing logic to occur in one place to eliminate these mismatches and my manual crosschecking, but after a lot of thought I realized I wasn't comfortable with a single codebase and liked having two codebases as it helped me catch my own errors. I then just made it easier and easier to run automate and crosschecks between the two. The billing code was a little too gnarly to be proud of, but I was very proud of the outcome in how accurate our billing was, the lack of complaints and many near misses we avoided for many years. I do feel twinges of guilt for the complexity I left my successors but I still don't really regret it.
After that experience, the motivation for double entry bookkeeping has always made a lot of sense to me. I had sort of reinvented it in my own hacky way with double logic billing code to prevent my mistakes from causing problems for my customers...
Billing, or anything else involving money, is so easy to get wrong.
The data team I ended up leading at a previous company, had an unfortunate habit of "losing" money - it wasn't real money being lost in transit elsewhere, but records of something we should charge a customer.
Or if the team wasn't losing revenue, it was double charging, etc. etc.
Took us 3 years of hard work to regain the trust of the business leaders.
Good comment, thanks for sharing
No offense but this sounds like a nightmare. It also sounds like you did a fantastic job achieving accuracy despite the complexity of the system. That’s something to be proud of.
This is The Nightmare. Devs building systems they barely understand, complexity leaking all over the place, and someone inheriting that awful job of keeping it running without having made any bad decisions themselves.
Software is full of these systems.
I don't understand why the author chooses to bring up the mantra “make it work, make it right, make it fast“ in a negative light, perhaps he misunderstands where the "make it fast" comes in?
to clarify: "make it right" is the second step, and until you make things work correctly ("right"), that's where the work stops, you have to make the system work soundly. the "make it fast", as in, optimize, comes in only after you have got it right, that all correctness and soundness issues are resolved perfectly. then you can start optimizing it (making it run fast).
it has nothing to do with delivery speed. it has nothing to do with working quickly. it's about optimizing only as a last step.
perhaps the author is lamenting the fact that it is possible for something to sort of "work", but to be so far from being "right" that you can't go back and make it right retroactively, that it has to be "right" from the inception, even before it starts barely working?
I think your last sentence is correct. I think author is saying that you can't make it work and THEN make it right with payment systems.
This is my opinion as well and I've been involved in the audit of a fintech system where auditors had to download EVERYTHING into excel spreadsheets and make the numbers balance before they would sign off on the books. That took a lot of time and money I'm guessing made a difference of at least .1 unicorns in the liquidity even that took place 3 years later.
They said the engineering team followed this mantra, which includes the phrase "make it right". But they didn't. They didn't even bother to find out what they didn't know about fintech before they started building a product.
Given the context in which he used it, I think the misunderstanding you suggest in the first sentence is most likely. Immediately afterward he talks about the time pressure startups face.
I think author chose the wrong mantra too!
What happens in fast paced startups is that you ship what essentially is a MVP as soon as possible (i.e. you stop at the "make it work" step) because you need to build your customer base, finances, etc.
A better mantra would've been Facebook's "move fast and break things". But, that only works if you can fix the things later. You wouldn't do it if you're building an aircraft for example.
This is the type of thing where it helps to hire the right people from the start. You hire a bunch of leetcode experts, but never stop to ask if, besides conjuring data structures and algos in their minds they actually can build the thing you want to build. If your people know how to build the thing, you don't need to sacrifice growth, it gets built right from the start.
Sometimes you need engineers that have some other type of education, maybe accounting, maybe finance, maybe biology. I always thought that the most important part of my career was understanding every industry I built for, deeply, and knowing experts in those areas so you could ask the really important questions. That is problem solving and engineering. The rest is programming/coding.
I have never come across a post on HN that was so scarily describing my current day to day and with a comment I agree with so wholeheartedly.
I’ve spent the majority of my career in tech with a finance angle to it, mostly sales and use tax compliance.
What I never fully appreciated was how much those accountants, controllers, and lawyers were rubbing off on me.
I was recently advising a pretty old startup on their ledgering system and was beyond appalled at what it looks like when a bunch of engineers with no finance or accounting background build an accounting system.
We don’t have to find magical accountant engineers either, it will wholly suffice if we sit actual accountants in with our engineering team during the design process.
After my design of their complete overhaul I had a friend who is a CPA completely review my work, we found a few holes in certain scenarios, but by and large we were good.
Money is a difficult engineering problem because with money comes all the human goofery that surrounds it.
No tests either? If you lose track of enough money every transaction that you can make an example of 'Every $5 purchased resulted in $4.98 in the transaction log' I think your problem is far, far bigger than not having double entry bookkeeping.
Who builds a financial system like that an considers it normal? The compensation is one thing, but you'd flee a service like that with all possible haste.
These guys. They said it themselves. “We could have built it right, but we didn’t.” They chose not to. It was not an accident. They made jokes about “dancing cents”. They did these things because there would never be meaningful consequences for doing them, and they knew it. They move fast, they broke things - money things - and they laughed about it. And now they’re lecturing people as if having willfully made these decisions gives them both moral and technical authority. This is magnificently pompous, startup VC culture nonsense.
Sounds like they refunded anything that went wrong so it's not really as bad as you make it sound.
critically, only when customers reached out. which means tons of people that weren't eagle eyed got defrauded.
Refunds limit damages in a lawsuit, but don’t prevent legal issues.
Especially important when explicitly saying you’ve done these things.
How do you figure the story is true?
Yeah that was my thought as well. Ledgers have tons of benefits but they’re not going to fix your dancing cents problem. You’re going to have bad number on the ledger.
Sure, maybe that points you to the bugs, but so would writing basic tests.
A ledger where you insist that every entry touches exactly two accounts, in a business where transactions regularly involve various types of fees, could easily misplace or misattribute a few cents here and there.
This type of business can also have fun dealing with accruals. One can easily do a bunch of transactions, have them settle, and then get an invoice for the associated fees at variable time in the future.
> where you insist that every entry touches exactly two accounts
A ledger is where every transaction balances to 0. It can involve multiple accounts, but the sum of all transfers between all accounts in a single transaction must sum to 0. This is the property of double entry that actually matters.
Maybe I'm being naive, but this seems to be not too difficult... You have a general ledger in which the invariant that it always balances is properly enforced. The hard bit is scaling and performance. There are policies around fractional transactions, but you should never get mismatched entries.
I think the trick is to store the movement not the balances. You may cache the balance, but what matters is the journal of money moving from one account to another.
For a glimpse at the _real_, human consequences of this kind of slipshod mentality toward money shuffling software see the Post Office scandal[1].
https://en.wikipedia.org/wiki/British_Post_Office_scandal
Anything that moves money should be treated with upmost seriousness and be aware of as many historical mistakes as possible.
If told people that I was using a database system where one and 100th of the data was missing after every 10 transactions would you seriously take my advice as an engineer blog post like these that seem to be calmly focused toward an advertisement of a person’s or a group’s promotion and that it’s just introducing concepts.
The user name "zitterbewegung" sounds like a German. They build sentences like that.
You have to think about a sentence like a stack and read it slowly and carefully and take note of the push and pop operations.
Germans are like RPN where everyone else is a regular calculator.
I don't know German I was using the voice recognition on an iPad.
I'm afraid it's not just sentence structure that is the problem here:
> one and 100th of the data was missing
No idea what that means
> an advertisement of a person’s or a group’s promotion
What in god's name is an "advertisement of a group's promotion"?
I was illustrating to the concept if you have a database solution that loses data randomly it would be seen as a joke compared to a dual entry ledger seen here. It feels like they were promoting themselves instead of addressing a real problem since dual entry ledgers have been used before computers in finance.
Did you mean "one hundredth?" (i.e., 1%)? "One and one hundredth" suggests 101% to a native English speaker
I read that as "(an advertisement of a person) or (a group's promotion)".
>> one and 100th of the data was missing
>No idea what that means
1/100th of a dollar is a cent - goes towards the "missing cents" glossed over by calling that "dancing cents" in the blog post.
Yes, but why one and 1/100th?
Does anyone have a good explanation for why a ledger database should care about credits and debits and normal balances? This has always seemed more natural to me as a front-end / presentation layer concept. Your ledger entries always sum to zero for each transaction, your income account has a negative balance, and you display it in a sensible manner.
I’m also surprised that this whole article starts by discussing stock trading but has no mention of how to represent stock trades. I assume they are “Sagas” consisting of money moving from the customer to the clearinghouse (or prime broker or PFOF provider or whatever) and shares moving from that provider to the account at which the shares are held. And maybe other associated entries representing fees? It seems to me that this is multi-entry accounting, which is quite common, and that entries don’t actually come in pairs as the article would like us to think.
yes, agree, I think a 'source' 'destination' model is significantly more straight-forward. Just record the 'source' account and the destination account and you essentially end up with a ledger as a directed graph (Martin Kleppmann wrote a great post on it)
I also wrote a super short post on how to model such a system on postgres
https://blog.nxos.io/A-simple-double-entry-ledger-with-sourc...
Blockchain actually kinda nails it, that's in essence a source/destination ledger, no 'postings' or similar needed, and from a balance calculation POV has been working pretty well
One reason this model isn't applied in accounting, in my personal view :), is simply historical and the fact that the number 0 didn't exist when accounting principles were created.
Wrote another post on how to model debit/credits on a source/destination ledger here: https://blog.nxos.io/Debit-and-Credits-on-a-Source-Destinati...
It's very straight-forward, you just have to accept that asset accounts have negative balances and present the absolute amount instead of a negative amount in a view.
> you essentially end up with a ledger as a directed graph
The page contains a comment from Matheus Portela who pointed to a blogpost of his about "Double-Entry Bookkeeping as a Directed Graph" [0].
"I've also had the same problems you described here and double-entry bookkeeping is the way to go for financial accuracy. As a programmer, things clicked when I realized this system is an extended directed graph.". It turned out that: "Hi Matheus! Would you believe me if I told you that I read your post in preparation for this article?"
[0] https://matheusportela.com/double-entry-bookkeeping-as-a-dir...
It isn't always clear which is "source" and which is "destination" and now you need a bunch of new conventions about these things. Accounting already has these (admittedly arbitrary) conventions so we might as well use those.
okay but how do you model a three-party transaction? say you want to collect fees or taxes
you simple create two records linked by one 'transaction', the source in both cases is the same account, while the destination for one of those postings is the fee account and the other destination is a merchant or similar account. And you can link as many of those postings under a single transaction
You never have 3 party transactions - if you do, you would not be able to track money flow.
You can have multiple transactions. One to pay tax, one to pay fees and one to pay the actual thing.
You bundle these things in another abstraction, eg. An invoice.
It is best explained by common scenarios an Italian merchant in the Middle Ages experienced. The basic concept is Assets==Liability (plus Equity). Where positive Assets are entered on the left hand side (debit). And positive Liabilities are entered on the right hand side (credit). In accounting, debit and credit just means left and right.
1. Merchant takes out a loan for $5,000 and receives $5,000 in cash.
• Assets (Cash) increase by $5,000 (Debit).
• Liabilities (Loan Payable) increase by $5,000 (Credit).
• Equity remains unchanged.
2. Merchant buys inventory for $1,000 cash.
• Assets (Cash) decrease by $1,000 (Credit).
• Assets (Inventory) increase by $1,000 (Debit).
• Total assets remain unchanged, and liabilities and equity are unaffected.
3. Merchant sells all inventory for $1,500 cash.
• Assets (Cash) increase by $1,500 (Debit).
• Assets (Inventory) decrease by $1,000 (Credit) (recording cost of goods sold).
• Equity (Retained Earnings) increases by $500 (Credit), representing the profit ($1,500 sales - $1,000 cost).
4. Customer1 deposits $500 in cash for future delivery of goods.
• Assets (Cash) increase by $500 (Debit).
• Liabilities (Unearned Revenue) increase by $500 (Credit).
• Equity remains unchanged.
5. Customer1 transfers half of the future delivery of goods to Customer2.
• No changes to assets, liabilities, or equity occur at this point. The merchant’s obligation to deliver goods (reflected as Unearned Revenue) is still $500 but now split between two customers (Customer1 and Customer2). Internal tracking of this obligation may be updated, but the total financial liability remains the same.
Actually it is clear as long as you remember that main point you made: debit and credit just means left and right.
We are all spoiled by thinking of debit/credit as equal to decrease/increase respectively because that how we interpret our bank accounts. That understanding totally collides with formal accounting where debit/credit DON'T mean decrease/increase respectively. I think this is the root cause of all confusion about double-entry accounting. I may be wrong about this, happy to be corrected but that is the bit my brain grinds against when trying to make sense of things.
E.g. I replaced all instance of debit with "Left" and credit with "Right" in your example:
1. Merchant takes out a loan for $5,000 and receives $5,000 in cash. • Assets (Cash) increase by $5,000 (Left). • Liabilities (Loan Payable) increase by $5,000 (Right). • Equity remains unchanged.
2. Merchant buys inventory for $1,000 cash. • Assets (Cash) decrease by $1,000 (Right). • Assets (Inventory) increase by $1,000 (Left). • Total assets remain unchanged, and liabilities and equity are unaffected.
3. Merchant sells all inventory for $1,500 cash. • Assets (Cash) increase by $1,500 (Left). • Assets (Inventory) decrease by $1,000 (Right) (recording cost of goods sold). • Equity (Retained Earnings) increases by $500 (Right), representing the profit ($1,500 sales - $1,000 cost).
4. Customer1 deposits $500 in cash for future delivery of goods. • Assets (Cash) increase by $500 (Left). • Liabilities (Unearned Revenue) increase by $500 (Right). • Equity remains unchanged.
5. Customer1 transfers half of the future delivery of goods to Customer2. • No changes to assets, liabilities, or equity occur at this point. The merchant’s obligation to deliver goods (reflected as Unearned Revenue) is still $500 but now split between two customers (Customer1 and Customer2). Internal tracking of this obligation may be updated, but the total financial liability remains the same.
I find this much more easier to reason with.
Yes exactly. With assets liabilities and equity having a left and right entry, they were following the convention when posting a journal entry to the ledger, left entries must equal right entries. (Debits must equal credits). Because A=L+E, we get assets to the left and liabilities to the right.
I understand this. But we’re talking about computers, not Italian merchants. Italian merchants had actual pieces of paper. Computers have tables and views and frontends that are separate from the tables.
Any self-respecting accounting system should be able to produce a balance sheet that matches the conventions you’re describing. I don’t think it follows that the actual numbers in the database that get summed to produce the total liabilities should be positive.
I've often wondered about this in the shower. Why debits and credits, when we can just make income negative and let everything sum to 0? Then you can track the balance for each account in a single field in the database.
And the answer is that "0" first entered Europe around the time they invented double-entry bookkeeping there. Negative numbers reached Europe centuries after that.
I showed the internals of a number-line-based accounting systems to an accountant once, and he was so confused by the negative incomes.
https://en.wikipedia.org/wiki/Negative_number#History
https://en.wikipedia.org/wiki/Double-entry_bookkeeping#Histo...
I think we are talking about two different things. Yes, of course you can build an accounting system using whatever database algorithm and programming framework you like. But your users expect debits and credits and A=LE or A-L=E because that’s what their auditors expect.
In the scenario four I presented earlier, I believe it is intuitive to think of unearned revenue (liability) as a positive number. When the customer picks up the order, the unearned revenue will be transferred to equity.
Thank you for an example. But I don't see how it explains why debit/credit should be used instead of simple signed amount. Like how Transaction(from, to) where `from` and `to` are Entry(account, credit|debit, unsigned amount) make things easier than Entry(account, signed amount).
You basically used different labels for positive or negative amount in the example.
The story I was told and what I believe is that the journal entry is and always is the source of truth. A merchant may have several journals. A separate one for each line of business and maintained by separate clerks. The different journals would then be consolidated into a single ledger. So he can tell what his equity is. When transferring the journal entry to A=L+E. Those early accountants used their version of Excel. For Assets, They took a page and drew a vertical line. For Liabilities, they also drew a vertical line. Same for equity. They called the left side debit and the right side credit. We don’t know why the Italians named it this way. We can only assume the first ledgers dealt with paying down amounts of credit they owed others. Anyways this early “excel” allowed simple ledgers to have two columns. Positive asset changes go to the left and negative to the right. Positive liabilities changes to the right and negative changes to the left. Same thing for equity. I assume this was mantra they told themselves to ensure correctness or reconciliation. When transferring a journal entry to the ledger there must be a debit and a credit or there is fraud. For example an unscrupulous clerk may have taken a loan out. The journal entry may not tell where that money went. When transferring to the ledger, the loan would be entered as a credit. Because the there was not a corresponding debit, either an increase in cash assets or decrease in equity. The balance would have been off and would have told the merchant something was wrong.
As someone pointed out just below, you do not need two rows. You can have one row: (amount, credit_account, debit_account).
The article does talk about a bit why using negative amounts for this is a bad idea.
I think that's because double entry bookkeeping precedes the concept of negative numbers. To be more precise, double entry bookkeeping was invented by people who had not yet been introduced to negative numbers.
At least that's how it's been explained to me.
Double entry book keeping implements an error correction and detection algorithm.
In double-entry, a transaction is the tuple (amount, credit_account, debit_account).
In singly-entry, it is the tuple (amount, account).
> In double-entry, a transaction is the tuple (amount, credit_account, debit_account).
Every “double entry” accounting package I’ve ever used can easily handle transactions that are awkward in this schema and transactions that don’t fit at all.
Moving $1 from one current account to another? I guess you declare that the $1 needs to be a positive amount, but your two accounts have the same normal balance, and calling one a “debit account” is a bit awkward.
Adding an accounts payable entry that is split between two expense accounts? Not so easy.
it's not "debit account" but "debit an account"
I've never understood that either. As a computer guy it always struck me as a redundancy -- the kind of redundancy where one thing will always eventually be wrong.
I assumed it has to do with the fact that it was invented for bookkeeping by hand. Accountants assure me that it's even more important with computers, but I've never managed to figure out how that works
Double entry accounting is still error prone, but single entry accounting is fraud prone.
A single debit can result in many credits.
A single record can (and will) be lost. Network issue, db issue, etc., some transactions will not make it to the db.
With double entry you at least have a chance of reconciling the lost transaction.
I've been doing this so long that I've finally realized that if someone can't explain why something is the way it is, that means it's wrong, or at least some arbitrary choice among several equally good alternatives.
Being able to explain it is something different then you being able to understand it.
And double entry bookkeeping should be both easy to explain (there are countless articles for it, precisely because it is a pretty easy concept) And easy to understand if you have ever tried to keep a ledger of transactions around and wanted to audit it for errors.
I always get hung up on the different kinds of accounts and their respective definitions of "credit" and "debit". It isn't that much to memorize but it's very counter to the way I understood those terms and it keeps throwing me off.
The simplest way to memorize it is to remember the accounting formula and one simple rule.
- Assets minus Liabilities = Equity (net worth)
- Your bank account or cash balance increases on the debit side
From this you can figure out that if you borrowed money, the debt increases on the credit side and the cash influx debits your bank account. The same goes for an income.
Surely it credits my bank account. Credits make things go up, no?
Clearly not. But this is why I let an accountant do it.
The bank will tell you there's a credit because to them, it's a credit. Your bank account is a loan from you to them - they owe you that money. When your account goes up, their debt to you goes up... thus it's a credit to them, and a debit to you.
Thanks. That's actually really helpful. Of course every transaction is a credit or debit depending on your point of view.
That's probably not the way I would have designed it. I'd probably have designed it from the point of view of the account, so that we'd all agree on what addition and subtraction mean. But that's my programmery point of view. I imagine that they're more concerned with the flows -- not just the numbers, but especially the actual materials being bought and sold.
But signed amounts (instead of debit/credit) formula is a way easier.
Sum of entries of assets/liabilities accounts = Equity. Moreover assets and liabilities become one type.
That is why I sincerely hope this way takes over the debit/credit approach
And single entry bookkeeping is even "easier". Doesn't mean it's a good idea.
most accounting software chooses one convention and sticks with it on all account types, to the chagrin of accountants
> As a computer guy it always struck me as a redundancy -- the kind of redundancy where one thing will always eventually be wrong.
That's the purpose. If you have a system with no redundancy, it's equally true that something will always eventually be wrong. But in that case, you'll have no way of knowing what's wrong.
With the redundancy, you can detect problems and often determine what happened.
Neither have I. It always seems like a massive cargo culting. Human accountants are liable to make very different kinds of mistakes than computers.
That part of the article felt quite wrong to me as well. I've built accounting systems that worked well for a decade, where internally the values were a single amount column in the journal table. If it was a debit, it'd be positive, if a credit, it'd be negative.
In fact, we could call these values yin and yang, for all it mattered.
Also, I'm not able to really follow what he means by "money = assets in the future".
Money is money, but if you wanted to track the intermediate state until the customer gets receipt, you would use an In Transit account (Good In Transit / Service In Transit etc.)
Yet, it doesn't change the fundamental definition of the value in the accounting system. I think the author confuses an engineering concept (sagas, or thunks, or delayed but introspectable/cancellable actions in general) with accounting.
> Also, I'm not able to really follow what he means by "money = assets in the future".
I’m guessing it’s one of two things:
1. A transaction might fail. If you enter a transaction into your bank’s website or your credit card company’s website, you should probably record it in your ledger right away. But the transaction might get canceled for any number of reasons. And the money will not actually move instantly, at least in the US with some of the slower money moving mechanisms.
2. In stocks and other markets, settlement is not immediate. A trade is actually a promise by the parties to deliver the assets being traded at a specific time or range of times in the future. One probably could model this with “in transit” accounts, but that sounds quite unpleasant.
FWIW, I’ve never really been happy with any way that I’ve seen accounting systems model accruals and things in transit. I’ve seen actual professional accountants thoroughly lose track of balance sheet assets that are worth an exactly known amount of cash but are a little bit intangible in the sense that they’re not in a bank account with a nice monthly statement.
Money never moves instantly : light speed is a limit (and also something can always happen to the message(s).
IMO it's entirely wrong, and it also makes it a lot more difficult to programmatically create transactions with 3+ legs (For example: A payment with a line item + sales tax).
I think the author is just wrong on that point, but the rest is sound. (Source: I've built bookkeeping software)
> This has always seemed more natural to me as a front-end / presentation layer concept.
Consistence is a property of the backend, if that is wrong there is not hope for later
> Your ledger entries always sum to zero for each transaction, your income account has a negative balance, and you display it in a sensible manner.
'sensible manner' is the problem here. The data/money will be diverge with time, and without proper storage of the data it will by impossible to figure out.
The problem here is NOT store 'a transaction'. That with a RDBMs works. Is to store the FLOW of MANY transactions and the divergent ways things works.
Like, your bank is telling you has $100 and your system $120. And your system sum right, but the bank rules.
Or when you do a return and cents are lost in the interchanges and chargebacks.
Or, just wrong data entry, sync, import/export, etc.
---
The way to see this is that `double entry` is a variation of `inmutable data that don't mutate and always track the flow of it' that is golden for business apps.
If you're looking for a pre-built double entry accounting Postgresql schema that provides database-level integrity checks:
https://github.com/adamcharnock/django-hordak
I created and maintain this, along with a couple of others. It is built for Django (so great if you're using Django), but extracting the schema wouldn't be too hard. It also has MySQL support, but the integrity checks are more limited.
(Side note: I'm a freelancer and available!)
Great article. I have an observation to the "engineers should know this and do good engineering" though: I work for a payments company and there is a fundamental career problem with becoming the ledger ninja: It's not enough work, and it's eventually done!
I've seen the following major phases of this work: 1) Build the ledger (correctly), and it will work well for a while. 2) Add convenience code for the callers, assist finance in doing reports/journaling, fix some minor bugs, take care of the operational bits (keep the database up). 3) Reach the scaling limits of your initial approach, but there are some obvious (not trivial) things to do: re-implement the transaction creation directly in the database (10x perf gain), maybe sharding, maybe putting old tx into colder storage, etc.
This is spread out over a while, so I haven't seen it be a full-time job, even at real startup-level (+10% MoM) growth. Even if it was, that's one person, not a whole team. I understand engineers that instead are pulled towards projects where they are in higher demand.
In another comment somebody said ledger systems are trivial when done right and super hard when done wrong - so if you did a good job it kinda looks like you just created 3 tables and some code. That seems thankless, and job searching as this type of specialist is harder than just being a generalist.
I suspect this is a "small company" problem which don't sell too many different things. A larger enterprise might have a platform for connecting businesses with businesses and businesses with customers. They might sell services (subscriptions) but also one-time purchases which require different forms of revenue recognition. Those might even be split up across different revenue streams. You end up building sub-ledgers for each one of them because the ERP can't handle the scale. Oh, and you're a public company so make sure everything is SOX compliant and easy to audit. Ah, and you operate on a global scale so do all those things in different currencies for different legal entities.
There's a reason Stripe is as successful as it is. And then there's a world where a company outgrows Stripe.
There are worse career choices ("prompt engineer" LOL) than financial engineering.
I’ve built my career on cleaning up should-have-used-a-ledger messes. it’s hard but there’s always another company that needs it, and I get bored staying in one place.
recently I discovered that in a medical billing context the system is way, way weirder than I had seen before. I shipped the three tables, but getting everything into it seems like it might be an endless battle
There is a bigger lesson here. When you are creating something new it pays to understand what came before. Do you need to disgard everything?
I gave a boorish lecture to a junior accountant recently...
When you make things up yourself it is just you and Excel. When you use double entry you have 100s of years of history to fall back on.
https://en.m.wikipedia.org/wiki/Double-entry_bookkeeping
Though there's a weirder life lesson here : a lot (most / all ?) of successes come from founders not having enough experience to know just how difficult their enterprise is.
More experienced founders would take a look at it and immediately nope out of it, and so not achieve success (after a grueling amount of work).
It's a suicide to build a finance system or similar without double entry ledger.
Worse, I've worked at where the transaction reference id from vendor is not recorded, it's lost so the past data cannot be reconciled!
> Instead of using negative numbers, Accounts have normal balance: normal credit balance literally means that they are normal when its associated entries with type credit have a total amount that outweighs its associated entries with type debit. The reverse is true for normal debit balance.
But that is an interpretation made by the viewer. A customer typically is an asset account, whose balances are in the debit column. But if we somehow owe them money because let's say they paid us an advance, then their balance should be in the credit column. The accounting system need not bother with what the "right" place for each account is.
It is quite practical to have only a simple amount column rather than separate debit/credit columns in a database for journal entries. As long as we follow a consistent pattern in mapping user input (debit = positive, credit = negative) into the underlying tables, and the same when rendering accounting statements back, it would remain consistent and correct.
> It is quite practical to have only a simple amount column rather than separate debit/credit columns in a database for journal entries. As long as we follow a consistent pattern in mapping user input (debit = positive, credit = negative) into the underlying tables, and the same when rendering accounting statements back, it would remain consistent and correct.
Another benefit of Credit / Debit side on double-entry bookkeeping is you need to balance both side in a single transaction. Say if the user account 2003201 is in Credit and it got an addition of 1000 value, a same value need to be added on Debit side. If it's (1) a cash topup, then 1000 value need to be added to Cash account (let's say 1001001) on Debit side. Otherwise if it's a transfer (2) from another user account 203235, then the account need to be Debited 1000 value as well.
It's Asset = Liabilities + Equity, while the left equation is Debit (which increase value when a Debit transaction happen, and the right equation is Credit, which increase when a Credit transaction happen. In (1) case, the cash account increase since it's on Debit account, while in (2) case, the user account decrease because it's a debit transaction on Credit account.
With negative/positive, the invariant would be sum(amount) = 0; with your approach, it would be sum(debit-credit)=0. Both are valid, it is just two ways of expressing the same thing.
I think it is useful to think about double-entry book-keeping in two layers. One is the base primitive of the journal - where each transaction has a set of debits and credits to different accounts, which all total to 0.
Then above that there is the chart of accounts, and how real-world transactions are modelled. For an engineer, to build the base primitive, we only need a simple schema for accounts and transactions. You can use either amount (+/-ve), or debit/credit for each line item.
Then if you're building the application layer which creates entries, like your top-up example, then you also need to know how to _structure_ those entries. If you have a transfer between two customer accounts, then you debit the one who's receiving the money (because assets are marked on the debit side) and credit the other (because liabilities are on the credit side). If you receive payment, then cash is debited (due to assets), and the income account is credited (because income balances are on the credit side).
However, all of this has nothing to do with how we structure the fundamental primitive of the journalling system. It is just a list of accounts, and then a list of transactions, where each transaction has a set of accounts that get either debited/credit, with the sum of the entire transaction coming to 0. That's it -- that constraint is all there is to double-entry book-keeping from a schema point.
literally every startup I’ve worked at has had busted accounting system that had to be painfully and expensively rewritten as a proper ledger. We should be teaching “Accounting for Programmers” as a required course in CS programs, or at least spreading memes about how money data must be append-only and immutable or you are being negligent in ways far more serious than most programming errors
The Software Engineering department next door from my CS department had a mandatory course on accounting. I don't think CS needs accounting, but I did learn two skills that would have helped here: why one should never use floating point for money (which may not apply here, who knows), and how to stick to a formal specification. "Money is neither lost nor created" is the most trivial example one can think for an invariant.
Relatedly, some months ago I asked how to correctly record food in a double-entry bookkeeping system [1] and it triggered a 1300-words, 8-levels-deep discussion. We should remember more often than accounting is a degree by itself.
[1] https://news.ycombinator.com/item?id=39992107
Haha, same. https://news.ycombinator.com/item?id=39994886
I kept digging and digging on a "sell some lemonade for $5" example, and ended up at:
- $5 debit to cash (asset => debit means +5)
- $5 credit to revenue (equity => credit means + 5)
- $X debit to cost of goods sold (liability => debit means - X)
- $X credit to inventory (asset => credits mean - X)
A double-entry for the money, and a double-entry for the inventory, for a total of 4 entries.
It's too complicated for me. I'd model it as a Sale{lemonade:1,price:$5} and be done with it. Nothing sums to zero and there's no "Equity + Income + Liabilities = Assets + Expenses" in my version.
But this is HN, and I think a lot of people would call my way of doing things "double" because it has both the lemonade and the money in it. So when I say I'm not sold on doing actual double-entry [https://news.ycombinator.com/item?id=42270721] I get sweet down-votes.
I think because programmer find it confusing about translation of model in computer to real world
Accounting predates computer by hundreds of years, there are better way to do certain thing of course but we must follow convention here because that's the norm and everyone understood
is there no individual accountability regime in the US?
in the UK, as an engineer, if I'd built this I would expect the regulator to come after me personally for not ensuring the system had adequate controls to protect clients money/investments
with a potentially unlimited fine + prison time
>is there no individual accountability regime in the US?
Here in the US, programmers like to call themselves Engineers, forcing everyone else to use the term "Professional Engineer" or "Licensed Engineer" or some other modifier before their title. I hate it, I wish they would stop, but it's not going to happen.
Software here is a wild, wild, West. The motto most live by is "move fast and break things"... even when those things are people's lives.
There’s a few more professionals that are called engineers.
Railroad locomotive operators, ship engine operators.
The name precedes the creation of licensed tertiary education level engineers.
A lot of people seems to ignore the fact that licensed professions that require an accredited diploma in a tertiary level education program is a relatively recent feature of our societies.
The PE thing is more than 100 years old in the US. By 1947 every state had a PE licensure program. It has nothing to do with programmers.
In fact it was lobbied in order to disinclude software engineering from it's purview in most states.
Has that ever happened? It's incredibly hard to prosecute directors in the UK for obvious malfeasance. I have never heard of a software engineer being sanctioned for crap code.
for engineers[1] it is relatively new, having only been introduced in the last couple of years
[1]: technically those performing a Certified Function
unless the engineer was provably malicious wouldn't it be responsibility of the product owner? Ownership usually entails accountability, they could order proper QA?
It would be nearly impossible to prosecute for just bad code. It would require more and is limited very small scope of people.
I leave you with the 2008 financial crisis as exhibit A on exactly how nobody gets in trouble for lack of financial accountability
The secret is to have everyone in on it. Everyone is guilty but nobody is quite culpable enough to punish.
The low level guys were just doing their jobs, and each individual transaction was mostly legal. A few weren't but it's hard to sort out which ones. Maybe the management should be responsible for ensuring that nobody ever did anything illegal, but they can't really supervise everything all the time, can they?
Poof. Guilt is just a rounding error that all rounds down to zero. The government passes some new regulations to prevent that particular scenario from happening again, and the same people set about finding a new scam.
This kind of still works for things that aren't real like money or law because they are societal constructs, but for the cases involving the real world, there's no escape from consequences (only dumping them onto someone else to deal with).
Not for software that I'm aware of.
For certain regulated professions there is, if a building falls down due to a bad design the professional engineer (PE) that signed and sealed the plans can be held personally liable.
Can you please provide a reference to the regulators coming after an engineer in the UK.
Also you say regulations in the UK have been changed recently.
I'm not aware of regulations that apply to software engineers.
Are you a professionally qualified engineer?
I don't see how a rank and file programmer would ever be personally responsible for their code. You can blame management for forcing untested or known flawed logic, but not some shmoe that pushes an "off by 1" bug while working weekends and late nights with no testing and hard deadlines.
Pushing a bug, yeah that happens.
Deliberately implementing a financial system that ignores established (and probably legally required) accounting practices? That's kind of like a structural engineer willfully disregarding the building code because that's what management asked for.
Depends if the particular activity is regulated or not.
a stock trading platform, as described in the article?
Could well be the entity actually selling the services.
In North America, “engineer” doesn’t necessarily mean a software engineer with a professional certification. Software developers have taken to calling themselves engineers. Whether engineering professional bodies should start going after people for this or not is a different topic.
But it’s entirely possible for someone who calls themselves an engineer to not actually be a certified engineer. So the activity wouldn’t be regulated because the person isn’t part of a professional body that regulates members.
In that case, lack of competence would be a civil issue unless it resulted in something criminal.
There isn't even a way to get certified as a professional engineer for software in the US.
it's what you're doing (your "function") that's regulated
not your job title, or piece of paper that you have that says you're X, Y or Z
"Professional Engineer" is a protected title that requires licensing to be used for a discipline. That licensing process does not exist for software in the US right now.
Accounting systems are super hard when you do them wrong and kind of trivial when you do it right.
There is no in-between.
Martin fowler wrote quite a bit on the subject and it's a good match for event-sourcing.
Hey, great works
I literally build the same system and ask for HN Ask but no one answer it and already using double entry and ledger based on some research and AI advice lol
I implement all the check its mention in your page and quite satisfied using postgress constraint and trigger function to detect abnormality before bad row is inserted
but the problem now is populate the database using fake data because how rigirous my constraint and rule are, I need to simulate real world use case and duplicate it on development machine but this are hard
if you ever read this, can you give some advice please, because simply looping that function is not gonna work because constraint of double entry and ledger
The payments industry is rife with nonsense like this.
At a fintech startup I was working with, we built a "shadow ledger" because we couldn't trust a 3rd party PP to give us the correct information, which would otherwise allow double spending on accounts.
We tried 3 different (major!) PPs - they ALL had similar flaws.
Building fintech platforms are not easy, specially for large systems which not only require robustness, scale, but also traceability, daily reconciliations. We even had features to undo and redo (correct and replay) transactions from a time in history.
I did not like finance much, but building a fintech system really teaches a lot, not only from technology perspective but from managing stakeholders, processes, compliance, dealing with all kinds of finance-specific issues.
One should always work in fintech, at some point of time in career.
Could someone explain something fundamental to me about the need for double-entry accounting? Why can't your source of truth instead be proper SQL tables following these schemas (using Python tuple/list notation for convenience)?
transactions: [
(txID, timestamp, [
(accountID, delta, otherInfo),
...
], reason),
...
]
accounts: [
(accountID, routingNumber, accountNumber, ownerID),
...
]
Crucially, notice that "accounts" don't track balances here, and so I'm asking: why would I need TWO entries per transaction, and why would I need to do my own tracking of the balance of every account, when I can just keep a single entry per transaction in a proper ACID database and then build any view I need on top (such as running balances) with a battle-tested projection mechanism (like a SQL View) so that I still track a single source of truth?
what you’ve sketched out already looks like a double-entry bookkeeping system, if you have the constraint that all of the deltas in your array on your transaction sum to zero (so money always _moves_ and is not created or destroyed)
calling it “double” might be misleading - in the ledger systems I’ve worked on, a transaction that has only two entries - a single source and a single destination - is actually rare. In practice, a payment tends to have a source, the intended destination, a secondary destination for platform fees, possibly another destination for taxes.
And much more complicated situations are possible - I currently work in medical billing where there are multiple virtual internal accounts involved in each transaction for different kinds of revenue (including, for example, money we billed for but don’t expect will actually get paid)
so a transaction becomes a bundle of a set of several debits or credits that happen simultaneously and sum to zero. if you have that, you have double-entry bookkeeping, even if you schema puts it all in one table
Ahh thanks so much, that clears it up for me! Every description of double entry I'd seen so far seemed to have the notion of keeping a running balance for each account (and then having a separate transaction for each one where they all sum to zero) as a central feature, so I thought avoiding that would make it no longer double-entry, and it always puzzled me why that was so crucial. Your comment made me see that wasn't actually the point!
You transaction includes multiple entries. Your schema is multi-entry by definition :P
It's funny how many commentators here confuse debit/credit with double-entry.
SQL or not doesn't really matter. Double entry provides redundancy, and redundancy allows for auditing. It's like checksumming to detect errors.
I guess I'm asking: what would go wrong with the design in my comment, which makes transactions first-class and has a single entry for each one? What errors would double entry prevent that this doesn't?
You offered no design. Db schema has nothing to do with transaction guarantees.
Double-entry accounting is like strongly typed programming languages.
Yeah it's a real pain to get started because you need to understand the core concepts first then fight to balance the transactions ("compile errors") before you have anything useful. And when your results are wrong, you know that at least it's not because of the basic stuff.
I don't understand what the difference in modeling between:
Entry(account, direction, non-negative amount), direction is debit or credit.
vs
Entry(account, signed amount), + is debit, - is credit (for example).
It's a two way mapping and should be equivalent. Unless debit or credit amount could be negative. But as I understand it's a big NO-NO in accounting.
SV Tech Bros roasting themselves on HN.
Every engineer in London is laughing at this right now.
Wow double entry! Ledgers are hard! Wow. So true.
> It’s just that…it goes without saying that fintech companies should know better
haha, if only.
move fast, break everything if it gives profit
An incredibly depressing thing in our profession is how we collectively lack memory.
We routinely re-discover stuff that probably was already solved by a quiet lady writing a CICS transaction in a s-360 system in 1969.
Yeah, dealing with money and especially others people money without double entry bookkeeping is a bad practice.
But this particular problem is the consequence of the choice of using floating point binary math to deal with monetary quantities.
Given the fact that most modern languages have very awkward support for arbitrary precision decimal numbers the most sensible way to deal with money usually boils down to store it as an integer.
There are lots of depressing things about our profession, but also lots more room for freedom and flexibility. It has some of the old west about it.
Part of the reason is that there is so much to be familiar with. There is creativity and craft. Then, there are all the tools we have to be familiar with - languages and platforms - and these change underneath you. Witness how the arrival of cloud has up-ended traditional ideas about deployment. Then the CI stuff. Then, specifics of each business domain.
Accounting is a more stable profession, and more structured. If you delivered an accounting solution for a public company that behaved correctly and cleverly but deviated from GAAP people would rightly treat you like a crank.
There is no GAAP in our field, but I feel things moving that way. I am currently reacting to some changes in the local law. In times past you needed to work to a principle - demonstrate that your systems are secure. You could chose to do this with off-the-shelf firewalls and layer-3 controls, or by building layer-3 controls on a BSD box, or by structuring parts of your network with vlan separations, or by having isolated networks. Increasingly, things are moving towards a mode where regulators mandate a specific model with a one-size-fits-all attitude, and you are required to fit in with it. In other guidance documents, I have been surprised to see strict rules specifying the design and behaviour of CI. There are some stupid assumptions in this - for example - regulators seem to love systems like Crowdstrike. There will be ups and downs as the conseqeucens of bad decisions flow through. Still, it seems to me we are converging towards something like GAAP.
I feel there is no excuse. Startups cut all sorts of corners but why? Get rich at other people's expense?
The tech founder should know their shit prior to building it so that time and runway and deadlines are no excuse. Really if you are doing fintech you should have employment experience and understand all the operations well.
Otherwise they are no better than say home builders who mismanage and go bankrupt. Or less geneously, they are conmen.
Feeling vindicated for the double entry transaction system we built at clearvoice.com for our two-sided marketplace, leveraging the fantastic DoubleEntry Ruby Gem from Envato.
Wait, why can’t I just use a Postgres database with the Decimal type and call it a day?
Blockchains are examples worth learning from about how to build ledgers. Append-only transactions which start as pending and then eventually confirmed. Balances are cumsums of confirmed transactions.
I'm not sold on double entry here.
If a new transaction enters the system now, I could follow the advice and record it as two sources of truth. Or I could just record it once.
If I could turn a single transaction into two ledger entries today, I could do the same later if I needed to.
Keeping entries separate makes it easier to deal with data for specific accounts, the transaction will contain both of them.
the systems I’ve worked on that use your design end up in a state I call “the zoo” - because you never actually have simple transactions like you’re imagining. You end up with one object for credit card payments with stripe fees, another one for purchases with taxes, another one for ACH payouts, all meaning different things
double entry is actually the simplest projection of any possible billing system because anything that moves money can be described as a bundle of credits and debits that happen simultaneously and sum to zero. so you get accounts for taxes collected and fees charged automatically and don’t have to write custom logic for each thing
There's no redundancy. Imagine you have 2 buckets, you reach into bucket A and grab some cash and put it in bucket B. $x dollars leaving bucket A is entry part 1, $x dollars entering bucket B is entry part 2. That's all it is. I honestly don't understand what single entry is by comparison - it sounds like losing an essential attribute of the transaction.
"Two sources of truth" is an anti pattern in computing and isn't redundancy. Use replication or backups for that.
You can't imagine simpler than what you described because it's single entry.
Double-entry is twice as complicated, makes sense only to accountants and not to computing people. Your example of 1 transaction would be doubly-kept as some nonsense like
BucketA revenue:$x CR cash:$x CR
BucketB revenue:$x DB cash:$x DB
no. single entry is when you track BucketA’s balance without regard to other buckets at all
Single would be recording the two accounts, one being the source account and the other the destination, and a quantity.
single entry is like a receipt at a store, is says what you owed and what you paid, but there’s no “other side” that is getting credited or debited the same amount
> And yet, I used to work for a startup that, on every transaction, simply lost track of a couple of cents. As if they fell from our pockets every time we pulled out our wallets.
is this not the literal plot of Office Space? did you check for a Michael Bolton employee?
Im sure there is a point in the article but ive never seen a dancing cent nor can i imagine one. My numbers are all strings "5" is "5" forever. If one somehow ends up storing it as 4.99 why would the other entry be correct?
Your second sentence tells us your first sentence was a lie. You can clearly imagine one which is why you specified which data type you use for money. You know a floating point issue is an issue.
Now let's say your price is "0.00023" per unit and someone uses "213.34" units. Can you imagine it now?
Question was about double entry. Does recording it twice solve floating point issues?
> Does recording it twice
Double entry is (confusingly) not about recording it twice, it's about using a transaction model where the state of N accounts has to be changed in compensating directions within a transaction for it to be valid, N being >= 2.
So depending on how your transaction schema is defined, a double-entry transaction can be written without ever even repeating the amount, e.g.
{"debit": "cash:main", "credit": "giftcards:1234", "amount": 100}
Making it effectively impossible to represent invalid state change. Things get trickier when N > 2 as classical double-entry tend to _not_ directly relate tuples of accounts directly to one-another and instead relying on a aggregated balancing of the changes to the N accounts at the transaction level, though YMMV between different ledgering systems.
Yes -- or rather it is designed to identify the issue that floating point causes.
The main idea behind double ledger accounting is that if you add a long list of the same numbers twice (as totally independent operations), then if you have added both lists correctly, you the two independent results will be the same. If you made at least one mistake in adding either or both of the lists, then it is possible, but unlikely that the results will match.
It's easy to think that computers don't make mistakes like humans when adding numbers, however floating point addition for long sequences of numbers is non deterministic (and error prone if you don't sort the numbers first and start with the small ones).
While double ledger systems won't fix this problem, they will identify if you have a problem with your addition (particularly a non-deterministic one like you would find with floating point addition) when you go to reconcile your books and find that the numbers across the various accounts don't add up.
There was no question, that I was answering. Simple someone claiming they have no idea how it would even be possible for a $5 end up being $4.98, while literally stating they know about floating point issues.
I mean i want to understand not that the problem doesnt exist.
If the price is "0.00023" per unit and someone uses "213.34" units I feed those strings into a multiplication function that returns the correct string 100% of the time.
That much i understand. I dont get how that category of problems is addressed by the solution described.
What i also understand is that you inevtably get to deal with accountants or other finance people. Working in a format they understand is not optional. They will have you transform whatever you have into that anyway.
I learn not to wonder why but maybe i should.
> If the price is "0.00023" per unit and someone uses "213.34" units I feed those strings into a multiplication function that returns the correct string 100% of the time.
But you're not coming up with a valid monetary amount.
This is more a problem with ill-specified contracts though. It was a constant source of annoyance when I was sole-trading for a bit, because what happened was something like this:
I'd be quoted a day-rate. That was what I was actually going to get paid, one day. But then I'd be told to bill it as an hourly rate. And then actually to bill it as 7.5 hours.
But I wasn't told what the hourly was - the hourly was whatever my day rate was, divided by 7.5. So this led to the problem that it produced an irrational number as a result.
Technically this should've been fine...except no one I dealt with knew or cared about this concept - they all used Excel. So if I rounded the irrational to nearest upper cent (since that's the smallest unit which could be paid) they complained it didn't add up. If I added a "correction item" to track summing up partial cents, they complained it wasn't part of the hourly.
In the end I just send python decimal.Decimal to maximum precision, flowed through invoices with like 8 digits of precision on the hourly rate, and this seemed to make Excel happy enough. Of course it was completely useless for tracking purposes - i.e. no one would ever be able to pay or unpay 0.666666666667 cents.
Because what's not in employment contracts that really should be? Any discussion on how numbers are to be rounded in the event of uneven division. You just get to sort of guess what accounting may or may not be doing. In my case of course it didn't matter - no one was ever going to hold me to anything other the day rate, just for some reason they wanted to input tiny fractions of a cent which they actually couldn't track.
And it's not an idle problem either: i.e. in the case of rounding when it comes to wages, should it be against the employee? It's fractions of a cent in practice, but we're not going to define it at all?
> the hourly was whatever my day rate was, divided by 7.5. So this led to the problem that it produced an irrational number as a result.
The only way for this to be true is if your day rate was irrational to begin with.
Irrational is the wrong word, it was a .3 or .6 repeater or something similar. Same effect: pile in digits so excel would round it off correctly back to the original rate I was quoted.
Assuming your day rate was a multiple of 10, it can only have been a .3 or .6 repeater (or an integer), because dividing a multiple of 5 by 7.5 gives you an integer number of thirds.
The example of the price and units is actually a real-world example. If you look at how much you pay for electricity you'll see you're paying something like 0.321 per KwH and you're not billed in full units.
Your issue is just people being lazy and forcing a day rate into an hourly employment system.
Seems like your definition of units is wrong. How can I sell .34 of an HVAC unit?
I guess the lesson here is to not bank with an a16z company. I have been using Wise for years. Very very happy.
> A double-entry system is an accounting method that tracks money at both its source and destination.
Nope. Double entry bookkeeping means every transaction is recorded in (at least) two accounts.
To illustrate the difference (hopefully I get this right): A transaction needs to balance, left vs right.
For example, you receive a sales order for 1000€ of widgets, which cost you 600€. You ship them. You invoice for 1000€.
No money moved (hopefully you do get paid at some point though). However, you need to do some bookkeeping.
On the left side of the ledger (debits): Accounts receivable 1000€. Cost of goods sold 600€.
On the right side of the ledger (credits): Revenue goes up by 1000€. Inventory goes down by 600€.
These completely match. No money has moved, but the books are now up to date and balance.
Any transaction that does not balance should be rejected.
I found this paragraph
> Prerequisites: TigerBeetle makes use of certain fairly new technologies, such as io_uring or advanced CPU instructions for cryptography. As such, it requires a fairly modern kernel (≥ 5.6) and CPU. While at the moment only Linux is supported for production deployments, TigerBeetle also works on Windows and MacOS.
wha— what? why?? they must be solving some kind of scaling problem that I have never seen
Thank you for posting this.
I’m trying to get my head around how to build a fairly complex ledger system (for managing the cost schedules in large apartment buildings where everyone might pay a different proportion and groups of apartments contribute towards differing collections of costs) and you’ve just massively accelerated my thinking. And possibly given me an immediate solution.
Have you used tigerbeetle in production?
I mean, tigerbeetle looks extremely cool (I've watched the project develop since its inception), and I trust them to be rock-solid. But saying "just use this project that is very new and unproven, written in a new and unproven programming language" is just pretty unserious. At least talk about pros, cons, risks, and tradeoffs.
nit: double entry accounting only goes back to the 13th century, not thousands of years.
Fibonacci, if I remember correctly.
Losing cents is because somebody didn't use Decimal for currency. And that's just flat out malfeasance--possibly even regulatory malfeasance.
Double entry is irrelevant, here.
I love the "use crypto" to fix the problem suggestion. LOL!
It's kind of poignant since crypto has given me a fantastic club to beat executives over the head with whenever they want to do stupid handling of money. Since crypto has so many decimal places, you break dumbass floating point handling of money immediately and irretrievably for way more than just "a couple cents". Your CFO starts jumping up and down really excitedly about handling money properly when he has to worry about a rounding error forcing him to compensate a Bitcoin price jump/crash.
It could be rounding issues as well (roundings do not all cancel out), or having to divide an even amount unevenly (how do you pay $10 in three "equal" installments?) Even with infinite decimal precision there will be special handling involved.
It took me a while to realise what double-entry bookkeeping actually was, then one day it hit me: it's literally just the flow of money through a system. Understand that, and you understand the cornerstone of accountancy.
if only accountancy 'learned' from OOP and just had one mutable balance for each account :)
It's just like it was with pen and paper... Everyone trusts what is written on the financial statement but when they come to withdraw the cash, all at once, suddenly everyone finds out that the gold isn't there.
It's the same with computer systems. The charts show something, but until enough people decide to all withdraw their money or sell their stock at the same time, nobody has any idea that the money or asset simply isn't there or nobody knows just how frothy the valuation is.
Social media and search algorithms are highly optimized to ensure that people don't sell or withdraw stuff at the same time. Modern media directs massive attention towards certain topics as a way to draw attention away from other topics which could collapse the economy.
Also, imagine a bank has a serious bug which causes millions or billions of dollars to disappear every year or creates extra illegitimate dollars. Imagine they only discover this bug after a few years of operation... How likely is it that they will report it to an authority? They didn't notice it for years, why not pretend they didn't notice it for a few MORE years... The incentive to delay the reckoning is an extremely powerful one.
Precisely. Designing a transactional system can be solved. Designed a transactional system that properly entangles the bits with the assets they represent is the hard part.
The blockchain industry is basically founded on incompetence. If you go back to early Silk Road and Mtgox you can see some shocking things. Like the founders were posting questions on Stackoverflow about database basics. They then ended up building systems with race conditions (withdrawal code for both systems.)
See, if you're an engineer (from a web background) you might think that using a regular backend language with a relational DB would be fine. But this service is typically optimized for concurrency. That might sound amazing. But what you really want is a simple queue. This ensures everything is applied in order and makes it impossible to accidentally have transactions execute on stale state. Plus, a queue can be executed amazingly fast -- we're talking Nasdaq scales. If you use a standard DB you'll end up doing many horrible broken hacks to simulate what would be trivial engineering had you used the right design from the start.
You've got other things to worry about, too. Financial code needs to use integer math for everything. Floating point and 'decimals' lead to unexpected results. The biggest complexity with running large-scale blockchain services is having to protect 'hot wallets.' Whenever an exchange or payment system is hacked the target is always the funds that sit on the server. When the industry began there were still many ways to protect the security of these hot wallets. The trouble is: it required effort to implement and these protocols weren't widely known. So you would get drive-by exchanges that handled millions of dollars with private keys sitting on servers ready to be stolen...
Today there are many improvements for security. New cryptographic constructs like threshold ECDSA, hardware wallets, hardware key management (like enclaves), smart contract systems (decentralized secret sharing... and even multi-sig can go a long way), and designs that are made to be decentralized that remove the need for a centralized deposit system (still needs some level of centralization when trading to a stable coin but its better than nothing.)
I would say the era where people 'build' ledgers though is kind of over. To me it appears that we're organizing around a super-node structure where all the large 'apps' handle their own changes off-chain (or alternatively based on regular trust.) The bottom layer will still support payments but it will be used less often. With more transaction activity happening on layers above. I think its still important to scale the chain and make it as secure as possible. Bitcoin has a unique focus here on security above everything else. Making it ideal for long-term hedges. For every day stuff I can't think of any chains that have more credible R & D than Ethereum. They seem to even have been doing research on the P2P layer now which traditionally no one has cared about.
> Financial code needs to use integer math for everything. Floating point and 'decimals' lead to unexpected results.
Decimals are integers. There's no difference between integer math and decimal math, only in the meaning of the bit pattern afterwards.
> Decimals are integers
No, most decimal types are decimal floating-point. (So, sure, technically redundant, but a lot of people say "floating point" to mean IEEE 754 specifically, even if that's technically incorrect).
OK but apparently they did get to make the startup mistakes? They built the quick thing that worked well enough, got some customer traction, and then when they had bugs they were able to rework it and continue.
Frankly I'm not even convinced that double-entry is the sole right answer in this space. There are things you need to be able to represent, but the original reasons for doing double-entry (making errors when subtracting figures) no longer apply.
(I've worked at investment banks and fintech startups)
Hmm, can you elaborate on the original reasons for double entry and them not applying any.kre? I'm not in that space and double entry always seemed an extremely weird, arbitrary requirement / paradigm. Thanks!
The main point of double-entry account keeping is the notion that money never vanishes, it's always tracked as going somewhere.
I think this tends to get misrepresented by people trying to literally keep two entries like we were working with pen and paper book-keeping though.
Because if I have a simple list of transactions, I can easily use a SQL query to recreate a view of double-entry book-keeping. It's perfectly fine to just record the amount of money, and two entity names in a table.
Coz the whole point of the system is that you can explain where the money went at any given time, referenced against something which should be independently auditable: i.e. a vendor receipt which in turn says "there's a physical item of X we gave you".
The "double entry" system's actual merit is when there's multiple people keeping the books. i.e. if you're in a business and your department is spending money on goods, then when you're doing that those transactions should be getting reflected in the shipping departments books (owned by a different person) and maybe your accounting department or the like. The point is that there have to be actual independent people involved since it makes fraud and mistakes hard - if you say you bought a bunch of stuff, but no one else in the business received it or the money to pay for it, then hey, you now need to explain where the money actually is (i.e. not in your pocket).
Cool post, wish it existed 2 years ago when we started building Pave Bank, or 10 years ago when we started building Monzo.
If you're starting a bank or need a ledger these days (and aren't using a core banking provider that has one), then i usually recommend Tiger Beetle.