This essay sort of waves a hand at the sub-symbolic roots of knowledge that lie beneath text and that babies spend several years mastering before they are ever exposed to text. IMHO the proper measure of that latent knowledge is qualitative, not quantitative.

It's the tacit 'grounded knowledge' of the world that's present in humans that has the potential to fully fill in LLMs' causal blank in their text-based superficial info. This kind of knowledge is threadbare in today's LLMs, but essential to form a basis for further self-education in any intelligent agent. I know STaR and RLHF have been suggested as synthetic means to achieve that experimental end, but I'm not sure they're sufficient to connect the dots between LLMs' high-level book learning and human babies' low-level experiment-based intuition for cause and effect. But adding yet more text data is surely NOT the way to span that chasm.

It is an intelligent argument but it isn't leveraging it's own insight enough.

> So then how do humans generalize so well from so little language data? Is “pre-training” on visual data the secret to our success? No. Because… blind people? What are we doing here?

The problem there is that the author making the same mistake they identified earlier. Things like vision or touch are techniques, what matters is the amount of data that is ingested through them. Vision obviously presents vastly more data than text. But a sense of touch is actually vastly more data too, because touch is interacting with the same source of data as vision. It loses colour information and the bandwidth is lower, but there is still a lot more there than text.

If the article had estimated how much data was present in bytes it wouldn't have dismissed vision so easily. What matters is there are orders of magnitude more data than all the text on the internet available for training in the real world. Data that can't be polluted by AI blogspam, for that matter.

It's weird to me how the article in the very first paragraph mentions brute force, but as data problem. As if we hadn't seen a staggering raise in ops and memory bandwidth. Machines have been able to hold the textual history of humans in short-term memory for a while. But the ops they could perform on that have been limited. Not much point telling a 2005 person to "add data". What were they going to do? Wait 20 years to finish a round we do in a week now?

It's very clear to me that the progress we observe in machine intelligence is due to brute processing power. Of course evolution of learning algorithms is important! But the main evolution that drives progress is in the compute. Algorithms can be iterated on that much faster if your generations are that much shorter.

Why are all these AI companies falling over each other to buy the best compute per watt humans have ever produced? Because compute is king and our head was optimized by evolution to be very efficient at probabilistic computing. That's where machines are catching up.

The mark of intelligence is to not need much data at all.

Important concept for model building:

You don't need more data when the data you have characterizes a problem well. More data is simply redundant and resource wasting. In this case, talking like people about things people talk about is covered well by current data sets. Saying we can't get more data is really saying we have collected at least enough data. Probably more than we need.

Lots of room to improve models though:

Using convolution for vision learning didn't create/require more data than training fully connected matrices. And it considerably increased models efficiency and effectiveness on the same amount of data. Or less.

Likewise, transformers have a limited window of response. Better architectures with open ended windows will be able to do much more. Likely more efficiently and effectively. Without any more data. Maybe with less.

Maybe in a few decades we will reach a wall of optimal models. At the rate models are improving now that doesn't appear to be anytime close.

Finally, once we start challenging models to perform tasks we can't, they will start getting data directly from reality. What works, what doesn't. Just as we have done. The original source of our knowledge wasn't an infinite loop of other people talking back to the beginning of time.

> Current language models are trained on datasets fast approaching “all the text, ever”. What happen when it runs out?

Robots.

To reduce hallucinations our AI models need more grounding in the real world. No matter how smart an AI is it won't be able to magically come up with answer to any possible question just by sitting and thinking about it. AIs will need to do experiments and science just as we do.

To maximize the amount of data AIs can train on, we need robots to enable AIs to do their own science in the physical world. Then there is no limit to the data they can gather.

I think my favorite part of this is watching LLM-bros awkwardly rediscover basic AI concepts that have been studied in depth for 5 decades.

Language Models do not work like the human brain. Continuing to compare the two like there is an analogy at all is doing far more harm than good.

It's not DNA, it's embodiment in general. People learn an enormous amount in the process of existing and moving through space, and they hang all of their abstract knowledge on this framework.

Related: it's a belief of mine that bodily symmetry is essential for cognition; having duplicate reflected forms that can imitate, work against, and coordinate with each other, like two hands, gives us the ability to imagine ourselves against the environment we're surrounded by. Seeing, sensing and being in full control of two things that are almost exactly the same, but are different (the two halves of one's body) gives us our first basis for the concept of comparison itself, and even of boundaries and the distinguishing of one thing from another. I believe this is almost the only function of external symmetry; since internally, and mostly away from sensory nerves, we're wildly asymmetrical. Our symmetry is the ignition for our mental processes.

So I'm not in a DNA data wall camp, I'm in an embodiment data wall camp. And I believe that it will be solved by embodying things and letting them learn physical intuitions and associations from the world. Mixing those nonverbal physical metaphors with the language models will improve the language models. I don't even think it will turn out to be hard. Having eyes that you can move and focus, and ears that you can direct will probably get you a long way. With 2 caveats: 1) our DNA does give us hints on what to be attracted to; there's no reason for a model to look or listen in a particular direction, we have instincts and hungers, and 2) smell and touch are really really rich, especially smell, and they're really hard to implement.

Incidentally: the article says that we've been optimized by evolution for cognition, but what could have been optimized was child-rearing. Having an instinct to train might be more innate and extensive than any instinct to comprehend. Human babies are born larval, and can't survive on their own for years if not decades. Training is not an optional step. Maybe the algorithms are fine, and our training methods are still hare-brained? We're training them on language, and most of what is written is wrong or even silly. Being able to catch a ball is never wrong, and will never generate bad data.

People were already talking about Big Data in the 90s. If you send this idea back in time to 2005, people wouldn't be stunned by the revolutionary innovations it would unlock. They would say, "oh, someone else on the Big Data hype train."

michael levin talks about "intelligence at every scale". he has a recent study where he found some of the hallmarks of intelligence in an off-the-shelf sorting algorithm. individual cells by themselves certainly have signs of intelligence, such as memory, attention, the ability to recognize that a strategy has failed and come up with another, etc.

This article is written by an obviously intelligent author but somehow lacks a trivial observation: people learn from other people. From day one we are getting tons of feedback from already intelligent creatures. Doesn't matter if we are getting it via vision, hearing, touch, via mirror neurons or via email. It is there 24/7 and it is very compressed.

The author should really take a long pass through the multidisciplinary investigation of what embodiment means.

Multi-modal is just a start, physical agency and proception are the cores of what is missing from LLM that people mean when they say such systems don't "know" what things are.

What they don't have is not just the sorts of categorizations and propositional reasoning around it implicit in e.g. Wikidata entity maps and Prolog/Wolfram like propositional reasoning.

It's also having a body that inhabits space and time with other agents and bodies.

> So then how do humans generalize so well from so little language data? Is “pre-training” on visual data the secret to our success?

Aren't you assuming that humans are born with some kind of blank slate of a brain? What if they don't get it from their senses. What if language is an innate human ability that is genetically encoded, it isn't learned so much and uncovered, and grows with us like a heart or a lung does?

I think, we just don't have right data. What I mean, human is not pure brain, but first year of life learn physiology of himself, and this physiology is very important part of human intelligence, but it is unwritten, even some opinions named it unconscious.

Current AI learning is not even deaf, but something like learn Dadaism (or other philosophy) without understanding of human being, with some much simpler life philosophy (single cell).

Lots of good thinking in this article. A few things come to mind before we hit a data wall.

1. Sensor all things

2. Waves upon waves

3. Dynamic or Living Semiotic Graphs. Bring your own terminology.

4. General Artificial Synesthesia.

how insects and flowers found their interaction suggests biology eats impossibly hard puzzles for breakfast.

the most compact way to store data is to merely point to where it is stored.

The description of the location can combine any number of clues and still be tiny.

if there is some place where infinite semi random data is stored and biology can access it it can find places that somewhat mirror data found irl. the dataset could be the shape of our head or a crystalline structure on the other end of the universe.

if biology can also somewhat manipulate the stored data the sky is the limit, or error correction is.

there is a stupefying amount of evidence that people are able to obtain data they should have no access to.

If small children can remember a previous life in great detail then those are the facts. That you don't like it is irrelevant, it is objectively unworthy of consideration.

Thar we have people unable to overcome this cultural indoctrination is fascinating! They apparently don't have write access!

People are apparently trying to map reality onto some preexisting dataset.

I think this is exactly right, and I've been thinking that for some time before reading this article.

I'm an ML researcher and a sheep farmer. My lambs are up on their feet, nursing, thirty minutes after birth. Once they have enough blood sugar they are capable of running to keep up with mom. They come prepackaged with a fully functional quadruped locomotion scheme and associated path planning and obstacle avoidance and fully functional vision, touch, and audio processing algorithms. And this is with basically zero embodied learning.

The difference between sheep and humans is, I think, that humans actually "learn" on three timescales instead of two, unlike every other living thing. Sheep learn on evolutionary timescales, through natural selection, and on the timescale of an individual sheep lifetime. But humans learn on the timescale of society as well, in between the other two. I believe that the difference between us and animals isn't so much our dramatically increased intelligence as it is the ability to pass detailed descriptions of what we know on to others, e.g. to language. The amount of information in the world has increased exponentially since the Renaissance. Societies advance on a characteristic timescale of about a century, as far as I can tell. Much faster than evolution, but quite a bit slower than individuals.

In ML, we are doing the same thing as sheep, basically. We have two "learning loops". One is the continual development of new DNN architectures, which corresponds to what evolution does; the other is the training of these architectures on data, which corresponds to what individuals do. But the outer one still mostly proceeds at the speed of human cleverness, not computation. But we are just... relying on the fact that society, and the data it produces, is there for the consumption of ML. We do not have any ideas for speeding up the production of that data except for the hope that we already have enough of it to kickstart GAI. If we do, we'll go through the singularity. If we don't, we won't.

But we can at least solve the "evolutionary learning" problem for ML. We'll need to bring back something like genetic algorithms, or make DNN architectures somehow differentiable so they can be efficiently evolved.

I think sending back the decoder-only transformer plus the one sentence "try this on the whole internet to pass the turing test" should count as sending back less information than sending back the whole encoder-decoder transformer architecture.

This simply does not pass the smell test. If you want to analogize biological brains to ANNs then clearly evolution refines the architecture of the "natural neural network," not the data. No ANN training involves adding artificial neurons, defining new types of artificial neurons, etc, but that clearly describes the biological history of the brain.

Taking this article literally, the brain hasn't really changed much since nematodes, it's just absorbed a ton of data. That's transparently stupid. All species, including humans and worms, have evolved over billions of years to get where they are today. Why is it that only humans get to access the billions of years of data? I am guessing the author has childish views about humans being "the most evolved" species. But all species are more evolved than they were 1m years ago.

This entire article relies on a dumb bait-and-switch. It's an incoherent analogy which seems motivated by a desire to simply excuse away the shortcomings of transformers.

Interesting wildcard ideas in the article, but I don't think we can understand how the brain works using computing concepts. For a nice discussion on DNA being the "blueprint" of life I recommend Philip Ball's "How life works".

Started good by mentioning the data wall, but finished bad. It's not the DNA, we share most of it with other species. DNA can't contain human culture, and if it could, we would have been as capable 200K years ago as today.

It's the one thing we have and they don't - language. During our 200K years our species has accumulated concepts, models, methods and values. We put them in language form and transmitted them across generations.

The main problem of course is search. We search for understanding, and do it collectively. A single lifetime would not suffice, it takes humanity across many generations to travel the road from caves to AI. That is why language is key, allows for iterative improvement. It also articulates search space, goal space and action space. It is symbolic and discrete, that allows for exact replication. DNA shares the same qualities with language - isn't it interesting?

Imagine you make a discovery, but having no language you can't reproduce the steps again, because it's all fuzzy. Or you can't teach anyone else. Language, with its symbolic structure is necessary to reproduce complex sequences of steps. Without it our exploration would not yield the fruits of exploitation. We would not benefit for unreproducible discovery.

I am against all kinds of essentialism. Chomsky thinks we have innate grammar, but forgets about co-adaptation of language for learnability in children, and about learning from our five senses - they provide a better explanation than innateness.

Searle explains our special status by biology, we have biological brains that's why we genuinely understand, he rejects distributed understanding but can't explain how come no single neuron understands on its own anything.

Chalmers thinks there is an inexplainable experience (qualia), but doesn't consider relational embeddings that can model qualities in experience. Relational embeddings are created by relating experiences against other experiences, creating their own high dimensional semantic space. No, it's not panpsychism, the whole universe doesn't need to be conscious.

And this time, in the article the magic is attributed to DNA. It's not that, it is search. We search and learn, learn and transmit, cooperate and reuse. Its not even a brain thing. It's social. We need more than one brain to cover this path of cultural evolution. Progress and language are not based in individuals but in societies.

My point is that from now on we hit the data wall. Imitation is thousands or millions of times easier than real innovation and discovery. AI will have to pay the same exploration price, it will have to learn from the world. Of course, new discoveries are not written in any books. They have to be searched. And search is an activity dependent on the environment we search in. Not in brains alone, or even DNA. AI will search and create new data, but it will be a slow grind.

In Information theory terms, evolution contributes 1 bit of entropy per generation (itila book mckay). Not sure what are those claims about DNA or physics.

the real data wall is ten years of childhood's interaction with real world.

This just smells like Lamarckism with more steps.