When people say we don't understand how LLMs work, it's that we don't understand why they take the decisions they do (beyond some quite rudimentary interpretability stuff), not that we don't understand the underlying architecture. If we understood the models we could monitor them and make them behave how we wanted. The recent events pretty decisively show that we can't.

If you say that we "understand the brain" because we understand how a neuron works, you'd be using "understand" in a way that mislead everyone reading that sentence.

I feel like this is missing the for the trees a bit. I don’t think anyone who is saying “we don’t understand how llms work” means what this author claims. The underlying math of transformers isn’t the part we’re talking about.

What people actually mean is we don’t and often can’t understand an entire reasoning chain of a large llm and often can’t trace that back to the specific training data that caused that reasoning.

Anthropic itself had build all kinds of tools to try to understand llm internal “thoughts” https://www.anthropic.com/research/natural-language-autoenco...

I don’t think it’s unreasonable to say “no one understands how llms work.” What they mean is we don’t and can’t follow every one of its internal chains of thought, especially at the pace new models and techniques are being developed.

This is one of the dumbest takes I’ve read online in a while. We are nowhere close to understanding the complex emergent behavior arising from LLMs. The author comes across as so simple they don’t even know what they don’t know. Like a confident high school freshman. They would probably claim we also understand how the human mind works because we know about atoms and molecules.

[dead]