> It's interesting seeing what comes built-in. You can see this if you watch a horse being born.

A fascinating example of this are some Labrador retrievers. Labs are descended from a Newfoundland "landrace" of dogs known as St Johns Water Dogs. They have multiple aquatic adaptations: the "otter tail", oily fur, and webbed feet. (Some of these are shared with other water-oriented breeds.) Some lines of Labradors, especially the "bench" or English dogs, normally retain this full suite of water adaptations.

But the wild thing about these particular Labradors is that they love to swim, and that most of them are born knowing how to swim very well. But they don't know that they know how to swim. So many a young Lab will spend a while standing on the shore, watching humans or other dogs in the water, and fussing because they don't dare to join the fun. Then they may (for example) eventually lean too far and fall into shallow water. Within moments, they'll typically be swimming around and having the time of their lives.

The near-instant transformation from "fascinated by water and fearing it" to "hey I can swim and this is the absolute best thing ever" is remarkable to watch, though not recommended.

I remember another Lab, who'd been afraid to go swimming, who one day impulsively bolted for the water, took an impressive leap off a rock, and (from his reaction) apparently realized in mid-air that he had no idea what he was going to do next. Once he hit the water, he was fortunately fine, to the great relief of his owner.

CAUTION: This behavior pattern is apparently NOT universal in Labs. Owners of "field" or American Labs seem to have much better thought-out protocols for introducing hunting dogs to water, and failure to follow these protocols may result in bad experiences, dogs that fear water, and actual danger to dogs. So please consult an expert.

One of the most curious things I learned about babies is that they are born with a walking instinct, long before they actually can walk. If you hold them up, they will move their legs in a perfectly correct walking fashion. But they lack the strength and agility to keep their body up. At around 3 months this walking instinct disappears, and then at around a year we 'relearn' to walk when we have the strength and agility to hold ourselves up.

But if we were on a planet with significantly lower gravity, humans would likely be walking very near immediately.

One of the most beautiful, amazing things about parenting a child is thinking about “where would this child be at this age if it were another animal.”

A three day old horse can walk.

A three year old tiger is often a MOTHER to her own cubs already.

But then by six years-old the human child can do things mentally which are orders of magnitude more advanced than anything another animal has ever done. It’s really amazing.

IIRC Andrej Karpathy in a recent talk made a point that reading a book isn't like memorizing the book, it's more like prompting the brain with the book.

So maybe this concept of being ready to go at birth isn't about the animals ability to start doing things but just a way of upbringing regardless of how ready the animal is to function. Maybe pigs just start prompting early. AFAIK human babies can swim right out of the womb. In other words, maybe the distinction between precocial and non-precocial(I don't know if there's a word for that) animals isn't that clear?

I have an anecdote that sounds like it fits this...

The house I used to live in had a ton of blue tailed skinks around it. You could always spot a baby by its size and brightness of the blue in its tail (juveniles have a brighter hue, adults are more brown). To avoid birds, the skinks would do this shimmy under the siding of the house just across from my back porch. What surprised me is that even the babies, maybe a few days old, all knew how to do the siding shimmy. Young, old, didn't matter, you could tell they just knew how (and why) to do it.

You forgot to mention the most important part: Those animals are the norm, it's us humans who are the outliers! Of course we have remnants of this knowledge and models of the world that formed our ancestors DNA for Millions of yers in us. Our ancestors newborns were as independent as these other animal children.

The only difference is that our heads are too big, so we are born prematurely. The term precocial is misleading. We aren't the norm, we are the outliers, human newborn's are not a blank plate, basing your models of intelligence and learning on the assumption we are the best fit for survival in a world is bogus.

I'm immediately fascinated by what I imagine are core questions explored by this domain. Largely the trade-offs. It's almost like choosing to ship a product with a hard-coded configuration vs. a more complex "discover and self-calibrate" phase.

Would the trade-off be that precocial animals are generally "configured" for the environments in which they've evolved? If I birth (well, not me directly) a foal on the moon, will it adapt to the different gravity in the first hour or is that something that's "built-in" to their programming?

Are these built-ins easy to override or modify? Maybe an animal being precogial doesn't negatively impact its ability to also be adaptive, which I think I'm making a big assumption on already.

> Some animals are ready to go as soon as they are born. These are called precocial animals. They are born knowing how to walk.

Early young borns that could walk, like a baby giraffe or baby rhino, often fall down or get exhausted quickly initially; tons of youtube videos show that. Humans are slow learners here, but I would not call these other animals as "born knowing how to walk" if their initial steps are so insecure. Their body structure is different though - a newborn human is basically pretty crap-built. A baby deer kind of is built differently on birth and that also makes sense if you are threatened by other predator animals like wolves or bears or lions.

I think all animals are born knowing how to walk, including monkeys and humans. However, that trait only surfaces at a later stage of their development.

> They are born knowing how to walk.

I'm not aware of any way we can know this. We do know that those species are born with the physical ability to walk within the first few hours after birth. How could we distinguish between whether they were born with the knowledge of how to walk as opposed to them learning it quickly since their body can physically do it?

> Some animals are ready to go as soon as they are born

That trade has an extreme genetic advantage when other animals see you as their succulent mains on the a la carte exotic wildlife menu.

I wonder at which point in evolution did organisms decide to embed prototypical structures to save time at birth

You can’t tell me the information and programming for all the is stored in an egg/sperm cell

If you're a prey animal being born in open terrain, you need to be able to run at full speed right away.

is it true that it's a tradeoff ? the "more precocial" the less flexibility to learn new things ? on the contrary knowing less equals less assumptions, which needs more flexibility in exchange.

Would be true that what is precocial in us is the ability mimic and abstract specific patterns into general rules ?

> They are born knowing how to walk

Most animals know how to walk. They have pre-build 'knowledge' and start with it when they get muscles. The main difference is some species develop muscles before they are borne. Others don't, some can't even see. But as soon as they develop they can walk without even seeing how others do it. The same way birds can learn, or be trained to fly without seeing examples. They start flapping both wings in sync, this is pre-build.

> precocial

I thought you misspelled presocial, but precoial is etymologically related to precocious, both originally meaning early-maturing or something along those lines.

> They are born knowing how to walk.

This is unlikely to be a good way to think about them. The norm is for animals to be born knowing how to move. Whether they actually can move shortly after birth is more of a question of muscle development than knowledge.

For example, when birds are held immobile until they're old enough to fly, they fly normally.

I think a helpful conclusion is that while the firing pattern in organoids doesn’t preclude a wetware of complex programmed instructions, it could be just the emergent properties of the underlying physics and electrochemical properties of the neurons; analogous to the phenomenon of synchronism when placing pendulums in a common place.

I'm afraid when it comes to the human brain, people just seem to talk about their favorite topics and ignore the article.

It somewhat makes sense if you think of it in terms of a really complicated 1.5GB metaprogram with a huge pile of conditionals that are triggered by the programs it itself writes (proteins). The final you is made up of an incomprehensible huge number of copies of the metaprogram, running different configurations, and spitting out programs to each other which then do more stuff. Our human brains can't really conceive of a configurable metaprogram that writes programs by interacting with itself in different configurations that it itself sets up.

Nature recently posted an interesting video [1] about what causes developing hearts to have their first beat. The gist is that eventually random electrical noise triggers a propagating wave which is then continued and repeated by the cellular automation nature of heart tissue. You don't need as much software if your system is composed of emergent properties.

[1]: https://youtu.be/SIMS2h5QsZU

> For me, it's a mystery.

For me, it's one of the last true mysteries! We've figured out damned near everything else, nothing has this level of "unknown" to it.

It's simply mind-blowing to me how such a tiny block of data can encode such high-level behaviours so indirectly!

Genes code for proteins, not synapse weights!

Those proteins influence cell division, specialisation, and growth through a complex interplay of thousands of distinct signal chemicals.

Then those cells assemble into a brain, apparently "randomly" with only crude, coarse patterns that are at best statistical in nature. Some cells are longer, some shorter, some with more interconnects, some with less, but no two perfectly alike.

Then, then, somehow... waves hands... magically this encodes that "wide hips are sexually attractive" in a way that turns up fully a decade later, well into the "pre-training" phase!!!

What... the... %#%@!

How does that work!? How does any of that work?

Y'all work in AI, ML, or adjacent to it. You know how hard it is to train a model to learn to detect anything even with thousands of examples!

PS: Human DNA contains only 750 MB (62 billion bits) of information, of which maybe 0.1% to 1% directly code for brain structure and the like. Let's be generous and say 10%. That is just 75 MB that somehow makes us scared of snakes and spiders, afraid of heights, attracted to the opposite sex, capable of speech, enjoy dancing, understand on instinct what is a "bad" or "good" smell, etc, etc...

As a person who knows next to nothing about how the brain or the genes that configure it work, I tend to think of this in terms of 80s video games like River Raid. The level data for these games, if stored naively, would fill the computer's available memory many times over. So they just store a pseudorandom number generator seed along with a few other parameters. Coupled with a few rules to make the level playable, it can generate a seemingly impossible number of levels with very little stored data.

Maybe the genes just encode a few crucial rules and the rest just emerge from that.

Oh, and I know even less about how the universe works. But I tend to think of it in the same terms: Emergent phenomena stemming from simple rules à la Game of Life.

> gets built using this information only

No they don't. There is plenty of external stimuli that also serves as input, e.g. the process of raising a child and complex thoughts that may only be transferred from grown human to grown human.

Try raising a human in a barren cell without human contact or as part of a pack of wolfs and you'll see how much a human brain is built from "DNA only".

I think you're underestimating the role epigenetic information plays. 1.5 GiB encodes every protein used to build us, sure, but which genes get switched on when and how are sensitive to factors not encoded for in DNA, including the environment of the cell and the fundamental chemistry of biology. Epigenetic information is hard to capture but can profoundly affect how an organism develops; cloned cats, for instance, may show a vastly different fur color and pattern from the original, to cite just a highly visible example.

Have you looked into the amazing things people do with procedural generation with only a tiny bit (kilobytes, often) of source code? My intuition is that this is vaguely analogous.

Here's an example from 2003, where the entire source code, from music to visuals, fits in 64 kb: https://youtu.be/HtJvSvQnep0

Here's a good gallery of such demos: https://64k-scene.github.io

The even crazier thing is that DNA does not encode any of that. Behaviour and morphology is not directly encoded in there, you'll only find recipes for proteins. The zigote will divide into billions of cells that share that same recipe book. Depending on the electric and chemical signals surrounding cells are sending, individual cells get their "personalities" or function. This cell colony forms an organism which emerges from the sum of morphology and behaviour of all cells. But you'll find no recipe for an arm in DNA, it is the result of the work of the collective intelligence that is your body.

I don't think human DNA generally codes for the behavior derectly. Rather, DNA can code for how the brain learns from incoming data streams.

If the brain naturally tunes into some sources or patterns of input rather than others, it may learn very quickly from the preferred sources. And as long as those sources carry signals that are fairly invariant over time, it may seem like those signals are instinctual.

For instance, it may appear that humans learn to build relationships with kin (both parents and children) and friends, to build revenue streams (or gather food in more primitive societies) and reproduce.

Instead, the brain may come preloaded to generate brain chemicals when detecting certain stimuli. Like oxytocin near caregivers (as children) or small fluffy things (as adults). When exposed to parents/babies, this triggers. But it can also trigger around toys, pets, adopted children, etc.

Friendship-seeking can be, in part, related to seretonin-production in certain social situations. But may be hijacked by social media.

Revenue-seeking behavior can come from dopamin-stimulus from certain goal-optimzing situations. But may also be triggered by video games.

And the best known part: Reproductive behavior may primarily come from sexual arousal, and hijacked by porn or birth control.

Each of the above may be coded by a limited number of bytes of DNA, and it's really the learning algorithm combined with the data stream of natural environments that causes specific behaviors.

I think it's a wrong way to look at it. In addition to DNA information content, one should count also the complexity of the proteins and higher-level structures in the gametes.

It’s like one computer with program (DNA) and helper programs (RNA) creates second generation of computer and programs (proteins, lipids, carbohydrates) that essentially create their own version of computer system in which they govern things like enzymes, hormones etc

But keep in mind that humans are not created in vacuum. After those two levels of computer create third level that is brain, actual programming of brain is done by other living humans.

So actual „humanity” is what persists in living population and would reset when population is culled and newborn must live and learn on their own.

Even if such newborn would live long enough to have access to things like books, computers, even sound and video records… those would be completely useless to them because they won’t even know language and skills required to use those.

> 1.5 GB information

Well, nature has a big advantage over us in that it doesn't need to "make sense" of that code :). So it can, for example, do crazy reusage optimization patterns. A "subroutine" that is used in one place could also be part of a "data piece" of another part. A "header" part can also double down as a "validator" of another part. Doesn't need to make sense, it just needs to work. The only limits are the laws of physics. I would not even call it compression at this point. It's more like heavily optimized spaghetti code.

To me, it feels similarly impossible/spooky to how image models work.

Consider a model like SDXL:

- each image is 512x512, plenty of detail

- max prompt length is 77 tokens, or a solid paragraph

- each image has a seed value between 0 and 9,999,999, with each seed giving a completely different take on the prompt

I can't begin to calculate the upper limit on the number of possible human-readable prompts that can fit in 77 tokens, but multiply even an (extremely conservative) estimate of a million possible prompts by 10 million seeds and it's clear that this model "contains", at minimum, literally tens of trillions of possible meaningful images -- all in a model file that's under 7 GB.

I suspect it works similarly to the biological side -- evolutionary pressure encoding complex patterns into hyper-efficient "programs" that aren't easily interpretable, but eerily effective despite their compact size.

This is why I’m so insistent that LLMs aren’t the best way (if they are a way at all) to getting to human level intelligence. The maximum amount of energy and input data required for training and inference is many orders of magnitude less than we are currently using.

> Human body, including brain, gets built using [DNA] information only

I think there is a good chance there are other substantial information transfers from one generation to the next. The total genome of all that gut bacteria is orders of magnitude larger for example.

Well technically yeah but consider that it takes ~9 months for the product to function without constant life support, at least a few years until majority of the basic functions work and ~15 years until it is fully functional.

Talk about compile time.

The longer I think about it, the worse it gets.

It’s not foolproof, but I can easily transmit a huge amount of information to someone by saying “Titanic prow king of the world scene.” In seven words, which could be fewer if I were really trying, the recipient has a moving image and sound in their head (as long as they’re the right age group- every example I could think of made me feel old).

"Human DNA contains 1.5 GB information"

If compressed, then there is room for more. (but afaik much is rather unused)

And for me I cannot say, that life is not a mystery to me, but this specific part I have less trouble with imagining it. As little code can create complex worlds and simulate them. (a minecraft wasm build for example is just 14 mb, but fully working)

> I just don't understand how is it possible just from a purely bit size approach.

It's ~750MB (3 billion base pairs). But anyway, that's a size of a decent Linux distribution with tons of software.

Actually the DNA is very inefficient with many areas that appear to do nothing. 1.5GB is a ton of "source code".

There is no significant evolutionary pressure to erase unnecessary parts.

You can make a brainfuck runtime in less than a kilobyte and it can run any program known to man.

Makes one curious about epigenetics!

I'll probably get downvoted for this, but recent (last ~100 years) evidence and phenomena suggest that consciousness might be fundamental to reality, and thus there could be some other information transfer we would currently consider "woo" going on here. This is hard (if not impossible) to prove, of course, but quantum mechanics has totally bewildered many aspects of the materialist ("reductionist") model of the universe. There is a large and increasing number of physicists and other reputed scientists/researchers who are adopting some variation of the consciousness-as-fundamental stance.

"Human DNA contains 1.5 GB information."

Human DNA is tightly integrated with its environment. Instead of stand-alone, think compressed, source code of a high-level language running in an interpreter and with a standard library with 10-100x more functionality.

There's also how networks have combinational effects, some things in the body use temporal encodings, and who knows what else. We can't really estimate the information content of all of this put together since we don't even understand it. It is amazing, though.

Remember the body had 9 months to "learn" a bunch of stuff already.

Recursion

[dead]

Not exactly the same, but there was the case in Fiji of 'Fiji chicken boy' - when his mom passed way, his dad abandoned him in a chicken coop, and he was basically living with the chickens. When he was later discovered, he was behaving like a chicken, making clucking sounds and pecking at his food.

And we've proven empirically that this usable "prior" can be quite small.

What exact assumptions does human brain encode and how does it use them, however, is an area of research. We are nowhere near being able to list out all of those useful inductive biases - let alone extract them and apply them to AIs.

A professor once told me that if someone asks when a concept was first discussed in philosophy, it's always best to reply "Isn't it in Plato?"

https://plato.stanford.edu/entries/innateness-history/#PlaAr...

Not quite, because the brain is an empirical object itself. Kants pure intuitions and categories are before any possible experience. Kant would say we can’t conclude anything certain from the empirical observation of the brain, only that before any empirical observation we have those a priori intuitions and categories.

If that's so, then we're back to Nietzsche's Perspectivism. The experiment is limited by what the experiment entails.

All research points to ADHD having multiple causes. Each case will have a specific causal pattern. This makes sense as it is a diagnosis defined by symptoms. Same with autism.

> This would come at a cost to social/copying skills. This idea doesn't explain "systems thinking".

The concept of envy/malice/insecurity and people lying to your face and stabbing you in the back was completely foreign to me up to the age of 36. Only in the face of overwhelming evidence and harm to myself did it all click.

Lately I'm seeing myself in this junior dev I'm mentoring, I'm strongly suspecting he's on the spectrum (that's why he was rejected initially from an internship, despite my input that he'd make a great dev, which proved 100% accurate) -- the guy is totally happy in his technical world, jabs and callous remarks from others completely go over his head.

A lot of people on the spectrum simply have a deep interest in things and systems. I could be wrong, but I think some of those spindle neurons and circuitry made to model others just get used in some people to get systems.

I often get frustrated because people seem to want to learn HOW a technical insight and it's impossible for me to tell them HOW I got to that conclusion, other than I deeply immersed myself in it and it just clicks. I get the same awe when my wife makes jokes about a behavior of mine or someone else and I can see just how deep, funny and plausible her whole internal model of others is; and sometimes how wrong it is, just like my internal model of a system sometimes is. Alas I can change my internal models of systems on a whim.

Without reading the article, the headline, taken at face value, should come with the caveat that human brain is preconfigured with instructions for understanding the world we've evolved to inhabit. Modern industrial civilization is something different. I wonder to what degree common mental disorders would count as disorders outside the highly unnatural environments and systems we've built for ourselves.

Your reality check idea is interesting - coming from a person with ADHD, anxiety and schizophrenia in their family

Nice story, but newborns can't really see well:

https://www.nationwidechildrens.org/family-resources-educati...

https://www.aao.org/eye-health/tips-prevention/baby-vision-d...

https://www.webmd.com/parenting/baby/newborn-vision

Their body and nervous system is booting up, everything is starting to adjust to being in a new environment. Masks and family doesn't matter much in that brief period, it's more important to avoid infections and have proper care if something goes wrong. That's why child mortality is down significantly.

It's A LOT more stressful than that :)

"I need to start use my own lungs to breathe and if I can't do that I'm dead in a minute". Followed by trying to get a milk from a mother who often doesn't yet produce any... Using the eyes to look around is WAY down the list :)

Babies have really poor vision until like 6 months.

So it's like, kid comes out, everything is a blurry mess, stuff happens to them and they have no fucking clue.

The book The Continuum Concept by Jean Liedloff describes exactly this

Not to mention the injections, slapping, and isolation in a room with other screaming newborns!

I feel like the retina, & sight-brain connection more generally, will turn out to be a lot more important to human cognition & consciousness than we realize

AFAIK the whole morphology is decided in distributed computation fashion via electrical potential changes, at least according to experiments by Michael Levin

Science is about finding evidence in support of such (reasonable) speculation.

every computer needs an os

IMO TFA doesn't map too cleanly onto the concept of "bootloader" (nor "microcode" for that matter). But I guess the question is, as always, can you unlock it.

Yes, its not learned "knowledge" it is evolved. Mammals are born with systems primed to fear things that look like snakes. Not cause their parents learned that snakes are dangerous but cause the parents that was born without those priming circuits died.

The word "instructions" doesn't literally imply a book of rules and practices is stuffed somewhere in their head. I'm not sure this is a meaningful distinction. It's still built-in by the time they're born.

> Sounds a bit like Chomsky's Universal Grammar.

Yes, except with things in it instead of words.

It's the other way around: neither has "instincts". They might have built-in responses/patterns that are wired into the I/O path. Coughing, etc.

I never agreed with his views on syntaxis, but the (his?) idea that large parts of our language capabilities are innate is almost beyond doubt. Are people still arguing against it?

I wouldn't say that "very upset" is a correct or fair characterization for disputes in linguistics.

Chomsky's universal grammar work was based on too few languages, too little data, and doesn't hold up when you look at all human languages and usage.

See also Jenny Saffran's empirical work on infant statistical language learning.

The broad idea that some things are innate doesn't vindicate Chomsky's specific theories.

[dead]

He should have stopped there in his career.

To make sure and confirm, not guess and assume

Because modern humans are cute.

One could ask the same question about any trait of any organism ... and the answer is always the same. Do you have a problem with birds being able to build nests specific to their species, or cuckoo chicks instinctively pushing the eggs of the host species out of the nest? The answer is one of the best understood facts of science, and the basis of all of biology. Why would anyone expect the human brain not to be "preconfigured" by the billions of years of environmental forces that produced it?

It's not "chosen". It is evolution. Your DNA has the metaprogram that sets up all the programs in your brain. Most of them are learning programs but you also have hardcoded programs on how to perform your bodily functions, how and when to cry, and how to suck on a tit.

who? The evolution. The observed instructions are also chosen by evolution

just a lot of pretraining through evolution