Yes and no. An alternate perspective is that the output of each neuron in an artificial neural net is analogous to an F-I curve in a real neuron (spike frequency-input DC current curve). In this way, different neurons have different slopes and intercepts in their FI curves, just as a network of ANN neurons effectively have their activation functions tweaked after applying weights.

I usually only say this to other neuroscientists who have a background in electrophysiology. The analogy isn't perfect, and is unnecessary to understand what ANNs are doing, but the analogy still stands.

> the latest LLM models have almost as many parameters as we have neurons

I often see this take, but the apt comparison is between parameter and synapse count, not neuron count. You should be counting hidden units rather than weights if you want to compare to neuron counts.

In many, perhaps most, signalling pathways, amplitude doesn't matter much (it does at log-scale). Given how well we control temperature and therefore rate of the reaction, it makes sense to use timing to fight off the noise.

An elephant brain has 3 times as many neurons as a human.

They are pretty smart animals but so are dogs who have way less neurons. The point here being that the number of neurons is just one of the many factors that determines intelligence (general or not)

Human neural network build and trim connections constantly [1]. I imagine we will get much closer to AGI if we can update models dynamically, instead of just adding more neurons and more training. After all human didn't need reading billions of articles before writing an average one.

[1] https://en.wikipedia.org/wiki/Neuroplasticity

> But unless you have a way to interpret the high-frequency signal, it looks an awful lot like noise.

In other words, they're looking for their lost keys under the lamp-post because it's easier there. If there is a signal in the HF, it's not yet understood. This feels like "junk DNA" -which is I believe receiving more attention than the name suggests.

My grad school research was with an NIH neuroscience lab studying low-level sensory processing that offered a fascinating perspective on what's really going on there! At least for the first few levels above the sense receptors in simpler animal models.

To oversimplify, you can interpret gamma-frequency activity as chunking up temporal sensory inputs into windows. The specific dynamics between excitatory and inhibitory populations in a region of the brain create a gating mechanism where only a fraction of the most stimulated excitatory neurons are able to fire, and therefore pass along a signal downstream, before broadly-tuned inhibitory feedback silences the whole population and the next gamma cycle begins. Information is transmitted deeper into the brain based on the population-level patterns of excitatory activity per brief gamma window, rather than being a simple rate encoding over longer periods of time.

Again, this is an oversimplification, not entirely correct, fails to take other activity into account etc etc, but I'm sharing it as an example of an extant model of brain activity that not only doesn't average out high-frequency dynamics, but explicitly relies on them in a complex nonlinear fashion to model neural activity at the population level at high temporal frequency in a natural way. And it's not completely abstract, you can relate it to observed population firing patterns in, e.g., insect olfactory processing, now the we have the hardware to make accurate high-frequency population recordings.

It is a low-pass filter in the frequency domain with a roll-off that is not smooth. I quite like [1] as a quick reference.

https://www.analog.com/media/en/technical-documentation/dsp-...

In physics the model we choose is based on the scale - as in the macro sense all quantum effects average out over the several sextillion atoms in, say, a wood screw.

I think of summaries as the text equivalent of averaging. Some high frequency stuff you don’t want to loose in that case are things like proper names, specific dates, etc. In the face of such signal, you don’t want to average it out to a “him” and a “Monday”.

This is broadly speaking not correct. If you average together a bunch of trials with variable timing, then the result can tend to wash out higher frequency components (which you might not have realized were in the data), but trial averaging is not a low pass filter at all. There are some nice methods to recover temporal structure that changes across trials prior to averaging, like:

https://www.sciencedirect.com/science/article/pii/S089662731...

>In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- the low average was almost entirely due to infant mortality.

This is even wronger than what you critique.

For every period in history that we have good data for people had a half-life - a period in which you'd expect half of all people to die: https://batintheattic.blogspot.com/2011/07/medieval-populati...

For example in medieval Germany it looked something like:

    |   Age | Half-life |
    |  0-10 |        10 |
    | 10-20 |        40 |
    | 20-40 |        20 |
    | 40-80 |        10 |
    

The exact age varies by location, but even if we ignore everyone under 10, half of all people left would still die before they are 40.

> In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- the low average was almost entirely due to infant mortality.

This isn't true: https://ourworldindata.org/its-not-just-about-child-mortalit...

I've heard this argument a million times, but I am very skeptical: where would the reliable data on infant mortality in ancient times come from? (so that it would allow us to compute precise values of average lifespan). All we have from those times are a few bone samples and a few anecdotes preserved in fragments.

I'm not saying anything for or against the ~33 years claim, just that I doubt that it comes from a precise estimate of expected lifespan at birth.

Averages are very bad in bimodal distributions.

And that includes issues of public policy, where going left sort of works, and going right sort of works, and going in the middle sucks for absolutely everyone.

Your beef appears to be with simple averages, not averaging per se.

The average for life expectancy is the mean of the Gompertz distribution [1]. Specifically, one that is "left skewed and with a flattened slope for ages under 50 years for men and 60 years for women," which proceeds to become "more right skewed and leptokurtic" as age increases [2].

So a simple average in the <55 domain would underestimate the mean while in the >55 domain it would overestimate it. Which is almost comically problematic when comparing ancient societies that had a median age below that level to modern ones above it.

> the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history

Not quite. 63 in 1900 to 83 in 2000 (in Sweden). Bigger differences when you go further back.

[1] https://en.wikipedia.org/wiki/Gompertz_distribution

[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2652977/

[3] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3000019/ Figure 1

I think you'd enjoy this video on different types of means and their applications: https://youtube.com/watch?v=V1_4nNm8a6w

I was at the Denver museum’s mummy exhibit and disturbed to see that they said the lady died at 30, a “normal age for death in those times”. You would think a museum should know better.

https://aeon.co/ideas/think-everyone-died-young-in-ancient-s...

> In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history-

I am pretty sure this is wrong. East Asian cultures celebrate 60th birthday as becoming very elderly, and if you live to the 70s it's almost as if you achieved Buddhahood.

Averages can definitely oversimplify things, especially in neuroscience where outliers often tell the real story. Taleb touches on this in Antifragile—focusing too much on the average can make us miss what’s happening at the edges, where the most interesting things are. Instead of leaning on averages, we might get more insight by paying attention to the extremes, where the real nuances are hiding.

It's just trying to assume normal distribution when it's not normal. Modern science rely so much on that distribution that i actually whether they have overestimated its ubiquity just because it's so damn convenient to use.

> In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- the low average was almost entirely due to infant mortality.

Until the last few-ish generations, pregnancy and childbirth have been leading causes of death for women in those in-between decades of their lives.

(And obviously War and Famine, too, for both genders.)

Yeah it's a common mistake, but this is like intro to stats stuff. It's not some big secret that if you summarize a distribution with a single number then you've lost information.

> I've become increasingly convinced that the idea of averaging is one of the biggest obstacles to understanding things.

I'd counter that it's easily one of the biggest assistants in understanding things. The Central Limit Theorem in particular has been enormously influential. Without averaging statistical mechanics and thermodynamics would have been impossible and with them would go the industrial revolution.

What you're noticing is one kind of mistake caused by lack of literacy in science. There are many many more similar mistakes. The solution isn't less literacy.

@ flagged aaron695:

> The brutal reality is you don't have the IQ to understand averages or statistics like most people.

> Most of our ancestors lucky enough to made it to 45 years old in human history did not make it to 70.

> Using mode is misleading with "age", and you quickly showed you didn't understand 'the con' when you accidently tried to apply it.

> This is a political, the "past was wonderful fantasy" which is anti-science. It's used by the Woke for instance."

In this case I am using an equally "wrong" model on purpose while being well aware of its limitations, just to make a specific point. It highlights a point where people are doing exactly what you are accusing me of- romanticizing the present, and not understanding the reality of how it actually differs from the past. E.g. what are the actual reasons people had shorter life spans in the past, and what their lives were actually like. One should not forget the limitations of such a simple model, which was basically my point in the first place.

I am fascinated by "evolutionary medicine" and using such ideas as a hypothesis generator to figure out ways to treat "modern diseases of civilization." I am in no way romanticizing the past, but trying to understand the specifics, to better figure out how to develop more effective modern day medical treatments, not to return to the past. In truth I despise political thinking altogether, and like to look at mechanisms and biological details.

Your post smacks of "scientism" which is incompatible with actual practice of science. The very idea that a certain line of thinking or theorizing is "anti science" or should be taboo for political reasons is itself incompatible with creative open minded problem solving.

I can see it was a mistake to use this specific example for discussing the problem with averages- ironically because it is so accurate. Since so many people on here hold the exact misunderstanding I was criticizing, they are getting angry and insulting me instead of my intent, which was to explain a phenomenon and have this click as a simple example of it. A less charged example, where people don't have strong opinions already would have been better.

[dead]

I came to the same realisation about a decade ago, after being a "science" enthusiast growing up. As you said once you see it you can't unsee it. Most of science is just a scam. The exceptions are those fields backed up by real world engineering. All of social and most of biological sciences are worse than useless, they are outright dangerous.

They used those original average measurements to design seats for passengers instead.

Is the book titled "Brainwashed: The Seductive Appeal of Mindless Neuroscience"? The writers are apparently not neuroscientists.

> There is no handy oscillator or clock embedded in the cortex or thalamus that allows a neuron or module or us to declare “these events are synchronous and in phase”.

Brains waves drive synchronization of groups of neurons, lower frequencies broader, higher frequencies more localized.