UniverseHacker
6 hours ago
I've become increasingly convinced that the idea of averaging is one of the biggest obstacles to understanding things... it contains the insidious trap of feeling/sounding "rigorous" and "quantitative" while making huge assumptions that are extremely inappropriate for most real world situations.
Once I started noticing this I can't stop seeing this almost everywhere- almost every news article, scientific paper, etc. will make clearly inappropriate inferences about a phenomenon based on the exact same mistake of confusing the average for a complete description of a distribution, or a more nuanced context.
Just a simple common example, is the popular myth that ancient people died of old age in their 30s, based on an "average life span of ~33 years" or such. In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- the low average was almost entirely due to infant mortality.
The above example is a case where thinking in terms of averages causes you to grossly misunderstand simple things, in a way that would be impossible even with basic common sense in a person that had never encountered the idea of math... yet it is a mistake you can reliably expect people in modern times to make.
llm_trw
5 hours ago
>In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- the low average was almost entirely due to infant mortality.
This is even wronger than what you critique.
For every period in history that we have good data for people had a half-life - a period in which you'd expect half of all people to die: https://batintheattic.blogspot.com/2011/07/medieval-populati...
For example in medieval Germany it looked something like:
| Age | Half-life |
| 0-10 | 10 |
| 10-20 | 40 |
| 20-40 | 20 |
| 40-80 | 10 |
It's called a population pyramid not a population column for a reason.The exact age varies by location, but even if we ignore everyone under 10, half of all people left would still die before they are 40.
wodenokoto
an hour ago
Parent is talking about mode, so the most common number. Let’s build a distribution that satisfy both.
Oldest age is set 80, mode is set to 75. We start by distribution one percentage point to each age. Then add an extra to the mode and all the remaining to zero.
We now have a distribution where the most common age of death, other than zero be in 70-80, and more than half the population die before they reach 40.
UniverseHacker
2 hours ago
What you said in no way conflicts with what I said. For example, if people have dangerous lives in a way that is unconnected to age they will tend to not live long, yet the modal life expectancy due to the additional mortality of actual old age can still be quite old.
What I was saying is in reference to this study I read long ago, which suggests a modal life span of about 72 years of age in the paleolithic: https://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2007...
llm_trw
2 hours ago
>What you said in no way conflicts with what I said.
Neither would a world with 88 people with the following death schedule.
| Age | Deaths/Year |
|---------+-------------|
| 0 to 5 | 2 |
| 5 to 70 | 1 |
| 70 | 3 |
I'm sure that the 3 people who make it to 70 are very happy they are
in the highest mode of the distribution. The 85 who did not may have
something to say about how meaningful using the mode of a distribution
is.UniverseHacker
an hour ago
It’s the right thing to use in this specific context not because it is explaining the full picture better than the average, but because it distorts the picture in a different way: the modal age being high is incompatible with the incorrect assumption people are making when seeing a low average life span- that people in ancient times basically never made it to an old age in their 70s or older. People then jump from that to the idea that people were dying of old age much younger, which isn’t accurate… they were mostly dying of things unrelated to aging, and few were making it to old age- but healthy active people in their 80s and older did exist even in ancient times, and are mostly more common now because our lives are safer.
Deeper than that, I think there is a modern tendency to want to believe false claims about how awful life was in the past, and how much better we have it now… so there are a large number of such myths that are nearly ubiquitous but not accurate. Not to say that many aren’t also accurate, just that the inaccurate ones go largely unquestioned.
Modern times are very different from most of human history- better in many ways and worse in many others. If we romanticize how perfect the present is, we then lose the ability to make things still better.
JumpCrisscross
4 hours ago
> if we ignore everyone under 10, half of all people left would still die before they are 40
Wouldn't it be 50 since the half-life is an interval?
llm_trw
4 hours ago
No, the half life for age 10-20 is much higher than that for 20-40 and the exponential function is non-linear.
bosma
5 hours ago
> In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- the low average was almost entirely due to infant mortality.
This isn't true: https://ourworldindata.org/its-not-just-about-child-mortalit...
usednoise4sale
2 hours ago
Our world in data pushes a clear agenda, and it isn't really to be trusted.
Consider: https://ourworldindata.org/child-mortality-in-the-past
From this article: "Researchers also collected data about hunter-gatherer societies. The 17 different societies include paleolithic and modern-day hunter-gatherers and the mortality rate was high in all of them. On average, 49% of all children died.[5]"
This is cited as coming from: https://www.sciencedirect.com/science/article/abs/pii/S10905...
Which categorically states: "Unfortunately there simply is not enough direct paleodemographic archaeological data to make definite claims about the global patterns of infant and child mortality rates of our Paleolithic hunter–gatherer ancestors."
The author of the Our World in Data piece seemingly intentionally conflates the proxy with actual archaeological evidence of the actual child mortality rates. Given the clear warning in the cited article about making definite claims, I cannot read the deception any other way.
After seeing this error, I do not know how you could possibly trust anything they have to say on the matter.
Spooky23
5 hours ago
It mostly is. The biggest gains are in childhood. Aldo consider that you’re looking at figures for England and Wales, which isn’t necessarily representative.
The largest contribution to improving life expectancy is measured by reductions in child mortality. The factors that drove those improvements (infection control, improved hygienic practices, food quality, regulation of food and drug purity, medicine) benefited everyone, but had the biggest impact on the old and young. In the 1850s, 8,000 infants died annually from adulterated milk alone.
I think of my own journey. In 1824, 200 year ago Spooky23 would have died 20 years ago, a half blind cripple. 2024 Spooky23 is healthy with no back issues and god willing a few more decades.
llm_trw
4 hours ago
From the nice graph at: https://ourworldindata.org/images/published/Life-expectancy-...
In 1850 a 0 year old would expect to live to 41.6 years. A 5 year old would expect to live to 55.2.
If we waved a magic wand and let all infants survive past childhood with nothing else changed in 1850 life expectancy would still be 27 years lower than it is today. Or put another way you'd have the same life expectancy as someone in South Sudan or Somalia.
Spooky23
4 hours ago
Sounds like we mostly agree, save some pedantry.
Nobody waved a wand. The contaminated milk that killed infants killed adults too - alcohol and milk were alternatives to unsafe water. Public health, medicine and other factors improved things.
We don’t really have great stats from before the 19th century. Was 1850 a nadir in life expectancy? I’m not sure - but I suspect it varied by region and rural/urban conditions. 1750 NYC wasn’t as gross as 1850.
llm_trw
4 hours ago
>Sounds like we mostly agree, save some pedantry.
If by pedantry you mean that I'm not ignoring the cause of 70% of the improvement in life expectancy in the last 200 years then sure.
Jweb_Guru
an hour ago
Indeed, this conversation is a good illustration of the damage that Bayesian statistics have done to the "educated" populace. Not that they're inherently bad--it's just a different statistical approach, and it's generally good not to assume a universal background, that everything is normal, etc.--but by telling people it's fine to question statistical conclusions because the distribution might be different, it liberates certain people from ever having to actually change their minds based on new information, because they can just posit a different distribution that satisfies their own biases.
JumpCrisscross
4 hours ago
> It mostly is. The biggest gains are in childhood.
The life expectancy of a 60-year old going from 74.4 to 84, or a 70-year old from 79.1 to 85.9, is significant and meaningful. Not as much as a newborn's LE going from 41.6 to 81.1. But far from "pretty stable in the 70s-80s range for most of human history."
Also, recent life-expectancy increases have come from adult morality reductions [1].
UniverseHacker
2 hours ago
What I was saying is in reference to this study, which suggests a modal life span of about 72 years of age in the paleolithic: https://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2007...
ddfs123
4 hours ago
> In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history-
I am pretty sure this is wrong. East Asian cultures celebrate 60th birthday as becoming very elderly, and if you live to the 70s it's almost as if you achieved Buddhahood.
hinkley
6 hours ago
Averages are very bad in bimodal distributions.
And that includes issues of public policy, where going left sort of works, and going right sort of works, and going in the middle sucks for absolutely everyone.
JumpCrisscross
4 hours ago
> Averages are very bad in bimodal distributions
They're bad with multimodal distributions, generally, as well as any random process governed by a distribution with no mean, e.g. Cauchy [1].
(Neuronal firing appears to be non-Gaussian, possibly lognormal [2], which does have a mean [3], but it isn't equal to the simple average.)
[1] https://en.wikipedia.org/wiki/Cauchy_distribution#Properties
JumpCrisscross
3 hours ago
Your beef appears to be with simple averages, not averaging per se.
The average for life expectancy is the mean of the Gompertz distribution [1]. Specifically, one that is "left skewed and with a flattened slope for ages under 50 years for men and 60 years for women," which proceeds to become "more right skewed and leptokurtic" as age increases [2].
So a simple average in the <55 domain would underestimate the mean while in the >55 domain it would overestimate it. Which is almost comically problematic when comparing ancient societies that had a median age below that level to modern ones above it.
> the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history
Not quite. 63 in 1900 to 83 in 2000 (in Sweden). Bigger differences when you go further back.
[1] https://en.wikipedia.org/wiki/Gompertz_distribution
[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2652977/
[3] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3000019/ Figure 1
UniverseHacker
2 hours ago
What I was saying is in reference to this study, which suggests a modal life span of about 72 years of age in the paleolithic: https://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2007...
Life spans were abnormally short in medieval Europe, even compared to ancient humans.
KK7NIL
6 hours ago
I think you'd enjoy this video on different types of means and their applications: https://youtube.com/watch?v=V1_4nNm8a6w
mcmoor
6 hours ago
It's just trying to assume normal distribution when it's not normal. Modern science rely so much on that distribution that i actually whether they have overestimated its ubiquity just because it's so damn convenient to use.
Jweb_Guru
an hour ago
While it is true that not all distributions are normal, many distributions are approximately normal (or at least, normal in some sensible space that maps onto the actual collected data). IMO the amount of ink spilled on the idea that science is fundamentally flawed because distributions aren't always normal is probably too high (especially among non-statisticians), and frankly that's not where statistical analyses usually go wrong. A much larger problem (that has nothing to do with the ultimate shape of the distribution) is stuff like postselecting from a set of plausible models until you find one that finds significant results, and claiming that was what you intended to measure all along (this is why it's important to consider stuff like hyperpriors, much moreso than lack of normality).
ants_everywhere
an hour ago
Yeah it's a common mistake, but this is like intro to stats stuff. It's not some big secret that if you summarize a distribution with a single number then you've lost information.
> I've become increasingly convinced that the idea of averaging is one of the biggest obstacles to understanding things.
I'd counter that it's easily one of the biggest assistants in understanding things. The Central Limit Theorem in particular has been enormously influential. Without averaging statistical mechanics and thermodynamics would have been impossible and with them would go the industrial revolution.
What you're noticing is one kind of mistake caused by lack of literacy in science. There are many many more similar mistakes. The solution isn't less literacy.
Mistletoe
4 hours ago
I was at the Denver museum’s mummy exhibit and disturbed to see that they said the lady died at 30, a “normal age for death in those times”. You would think a museum should know better.
https://aeon.co/ideas/think-everyone-died-young-in-ancient-s...
dalmo3
2 hours ago
I came to the same realisation about a decade ago, after being a "science" enthusiast growing up. As you said once you see it you can't unsee it. Most of science is just a scam. The exceptions are those fields backed up by real world engineering. All of social and most of biological sciences are worse than useless, they are outright dangerous.
hifromwork
2 hours ago
>Most of science is just a scam
Most of the scientists are not scammers. So if you believe that all science disciplines other than engineering are wrong[1], you should use another word that doesn't suggest researcher malice.
[1]Which is a very strong statement, because you claim to be an expert in all science disciplines at once.