> This was openai’s entire breakthrough. Making this particular model architecture larger leads to emergent capabilities

Basically, the bitter lesson: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...

Indeed. It's pretty interesting to realize after implementing GPT-2 that the frontier models are scaled up versions of that, with various tweaks to improve performance, model-wise.

The secret sauce though is all the datasets, RL training, knowledge of what works from doing all kinds of ablation experiments, and a massive compute moat.

There is a different way to look at this: that is, actually the Transformer is a minimal complication of what the based model is: in theory the neural network could be just a huge FFN, which is anyway the part of the Transformer that does the heavy lifting. But this would be impossibile to train both numerically and computationally, so the Transformer encodes enough priors for it to work: the causal attention, and the math tricks like the residuals and so forth. But the bottom line of all this is that the Transformer works because of the incredible semantical power of simple/huge FFNs.

Same. I recall reading a paper by Stephen Wolfram after ChatGPT came out where he goes over how it works and what it does. Such a good piece and really got me going with this stuff. https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

What hopes/paths does a mere CS bachelor (not deep into stats/maths), and mid level dev (native mobile only; 10-15 years exp.), have about not only understanding it (maybe not fully) but getting possibly into this as a career? Not expecting churning out models and AI systems from the first weeks/months but entry/employment into this field?

(If I can be honest, and I am not being disparaging about anything lest it might seem so, I am looking at it from a career breakthrough/move perspective rather than an intellectual pursuit.)

I had the same reaction as you, when I learned in detail, how all this works. But then I also learned about superposition and compressed sensing, and now...I am not so sure anymore...

"Beating Nyquist with Compressed Sensing" - https://youtu.be/A8W1I3mtjp8

Could you perhaps cite the core papers for LLMs beyond „Attention is all you need“?

Yep. It's nearly identical to the neural nets we were using in the 90s. Back then even a supercomputer wasn't big enough or fast enough to do what we do today.

I have to wonder though. Is this all a human brain is? A similar thing to an LLM just scaled exponentially larger. I mean a brain is not just neurons with simple connections to each other. The neurons, axons, dendrites, <insert_unexplained_thing>, etc in a brain are all holding and processing information in different ways and doing it nearly 100% in parallel. That's a really big model.

The biological discoveries show how complex a biological brain actually is. Even the tiny brains in a bee or spider are able to solve puzzles and use tools. That's crazy.

It's not just the architecture but also the data - the decoder only approach lets you train in parallel over blocks of text (no RNN serial waiting), that allows you train on much, much more data.

I would argue that those are not emergent property of the model, but a property of how humans find insights in a plausible guess.

Since you spent a month digging into this, can you recommend any materials/projects to look into to get a decent grasp of how they work?

For anyone who is curious about the first paragraph here, this is actually a great video overview of how LLM works and the tokenization part.

Tangentially related: This part always seemed fuzzy to me, especially when dealing with data scientists and how they talk about how 'ML' looks at problems. I had this issue when working at a SIEM vendor where they kept going on about use case development having to be designed a certain way to catch things. It was all very frustrating.

aka "the bitter lesson"

Sooooo just because you are able to understand it, it's not worth anything?

It doesn't has any impact?

Ah wait it does. Mh weird.

Why are you not creating a startup and get rich?

> Over the next month I ended up drawing out a block diagram on a whiteboard I have in my office, with the math involved next to each step in the blackboard. I’d puzzle about each step along the way, and the triumph of completing the drawing was also that of this sense of deep understanding. I kept that drawing up for many months after, and would gaze at it often during meetings and idle moments in wonder.

how did you know about the steps and there was math involved. i am curious about your process and you came up with what exactly to learn to unravel the mystery.

Don't forget the stolen data from books and papers. You'll never get anything intelligent without using the stolen data they had access to.

After building some toy LLMs on my own I came to realise that architecture is not the hard part. Train is.

Architectures have evolved significantly since then. DeepSeek v4 =/= GPT-3. Even then, a great deal of complexity lies in everything surrounding the architectures e.g. how do you implement them performantly on modern accelerators, how do you distribute the model across a set of accelerators, how do you post-train, etc. And pre-training itself is a dark art. If you legitimately think that frontier labs are doing something equivalent to whatever you wrote on your whiteboard, you’re clueless.

fucking well said

Great, and won’t we all be just as surprised when human self-attentional control turns out to be just as simple or just as complex! Our minds as a strange fabric built of threads of recursions without the benefit of any explicit clock.

In terms of our brains though we can only think forward as well (if forward is time). Our brain in the future says something we did in the past was wrong (part of the sentence we wrote) and that informs our body (the agent) to go back and fix it

Could you restate this another way: I don't follow.

Comment <-> username synergy.

It's basically possible build an LLM using just routers+packets, and then hook them up to Wireshark to see it compute!

How would I set this up?

https://distill.pub/2019/activation-atlas/

I can only imagine what sort of visualizations are going on today inside of the AI labs.

You are exactly right! People do not find the writing obnoxious, they are backwards technophobes getting brought down by their superstitions.

AI assisted is a stretch. And that analogy isn't even close to being relevant

No? One affects the actual text and the other doesn’t.

This analogy makes absolutely no sense.

Rather interesting than clanker slop defenders downplay the clanker aspect and highlight the human by calling it "ai-assisted", which defeats their entire point.

I hope you do some introspection and start consciously recognizing that the human input and the clanker slop is just debasing it.

Not just that, I think a lot of people are going to waste their time losing the battle (and make no mistake, they will lose) fighting against AI writing without ever asking themselves what makes writing good in the first place.

There’s good AI writing and bad organic writing. But it’s easier to point out a few LLM-isms than to actually identify the problems with text.

Is chain of thought same as test time compute?

Predicting a word is the final objective, as in the output of the model is a probability distribution of the next token. However, choosing the right token is more complicated than just regurgitating the training data (and you won't encounter an exact example in the training data, so you need to interpolate). This makes the model learn abstract representation of things that it is able to manipulate before outputting this back into token. RL also complicates this because the "fitness" is now some arbitrary metric computed over an entire sequence of tokens.

Your casual understanding is imprecise.

At all times the LLM is, indeed, predicting the next token. Anything it does emerges from that.

It did not "figure anything out". It predicted that text describing the use of a radial gradient was likely to follow text describing your problem.

> What I mean, is the LLM is able to represent things in space . That part I don't understand.

Why do you think this is mutually exclusive to "LLM predicts the next token"?

If you tell someone from 19th century that bytes (just 0s and 1s!) can represent an opera, a song, or even a whole interactive experience, they might be really confused. But there is no reason they can't.

If you tell someone without math background that the sums of smaller and smaller sin waves can represent pretty much anything in our universe, they might be really confused. But there is no reason they can't.

There is simply no reason that a next-token predicator can't generate a nice-looking checkbox.

I understand that to be the "emergent abilities" which are spoken about. There are correlations in the dataset that are strong enough for it to seem to have an understanding which wasn't obvious it would have from simply "predicting the next word".

It's still predicting the next word. Somewhere in the gigantic dataset that the LLM was trained on, there is a phrase that says "gradient border" being in the vicinity of a CSS code that render the stuff. Therefore when you run it on an inference loop there's a good chance it output that CSS code when you tell it to render a "gradient border"

Multi-modal models that can understand visual input do exists, but no such visual reasoning process happened in the example you mentioned. Not unless you have a visual feedback loop in the coding harness.

I'm not dismissing the capability of "predicting the next word" however. The vast amount of training data enable extremely complex and useful behavior you just described.

LLMs fundamentally work by predicting the next word (token). But that should not be used to diminish their potential capabilities. It's like saying that human brains "just predict (or produce) the next electrical impulse". Fundamentally correct, but says nothing about the potential emergent capabilities of scaled-up systems that work like that.

Emergent properties of complex systems should not be diminished just because the underlying operating principle is simple.

Sorry you're being downvoted for asking a very reasonable question. I don't think any of the replies here address your question either.

If I can do my best to answer, Gemini is a multi-modal system. That means it's trained not only on text but also still images, video and also sound. The training happens in parallel and the representation of each modality is usually different, so the image recognition part is not trained on text tokens but pixels, the video part (probably) on video frames etc. There is some kind of integrated training that goes on so that text can be generated that is correlated to an image and so on, but I don't know the specifics about Gemini in particular. This kind of thing is not exactly new either, you can find systems that captioned images before the rise of LLMs simply by training on examples of images coupled to their textual descriptions.

In that sense it's not entirely correct to call Gemini an "LLM" because it's not only a "language" (or, more precisely, text) model. But LLM I guess becomes a bit of a shorthand for everything based on, or combined with, an LLM.

Anyway that's what's going on: it's not just predicting the next word. It's also predicting the next image frame or the next set of pixels etc associated with the next word.

>is the LLM is able to represent things in space

It is imitating the text written by humans who can represent things in space.

It can’t. It’s like a Redditor, it just repeats what it has seen other people say.

It has read all of stackoverflow, so it has seen your kind of problem before. Try asking it something really unusual and it will shit the bed.

I do agree bigly. Calling what is basically a superhuman brain inside a computer just a "token predictor" is peak thinkslop.

I don't want to pretend I can explain LLMs, but the same "math" can be applied for visual and non visual things. The dot product of two vectors gives you the angle between them. This is true in 2 or 3 dimensions. But it's also true in 4, 5, 6...n dimensions even though we cannot visualize a 4d space. That it's an angle is relevant for you in the space you can comprehend, but for math or a machine it works in any number of dimensions. So it does need to understand anything visually if the math checks out.

LLMs are modelled to predict the next token, and are indeed trained to do so on enormous bodies of text. But to be really good at predicting the next token (word) at the end of a long string of text, you must understand what the text means. If I give you the entire text of a long novel and at the end ask you a single "yes/ no" question about the plot, you only need to emit a single token, but emitting the correct one implies having understood the plot of the novel. This is what LLMs do. They're generating meaningful, coherent text, which implies understanding and cognition at a level that is much deeper than that of the single token they generate at each forward pass. Internally, the LLM has learned to represent the meaning of the entire prompt text, the concepts it implies and its possible continuations far beyond the horizon of simply outputting the next token.

The model is just trying to map from sequence to next token. You could say that it doesn't really care about the relationships between words/tokens - it is just being trained to learn the best attention/etc weights to make this mapping as accurate as possible.

The model could just as well learn to predict next token from gibberish text as long as there were some statistical gibberish regularities to learn. However, if you train it on real meaningful text then the statistical regularities it needs to learn (and will, thanks to gradient descent, and the capable architecture) will be those reflecting "token relationships" - grammar, semantics, etc.

So, you can say the "token relationships" (incl word meanings) are reflected in the statistical regularities of the training data, and the model architecture and training algorithm are just very capable of learning those regularities whatever they may be.

You can consider it related to Word2Vec word embeddings, which are based on the idea that the meaning of words comes from how they are used, which to a first approximation can be implemented by considering the meaning of words to be defined by the words they appear next to(!), which is what the Word2Vec embedding training algorithm does, and famous examples such as "(king - man) + woman = queen" prove that this is in fact learning the meanings of words.

At a high level, the text samples are how the relationships are derived. If we treat text samples as sequences of tokens, then the sequences of tokens describe the joint distributions they occur together which confers the relationship between them. Iirc, this is related to the idea of the distributional hypothesis in NLP: the idea the semantics of words should be similar if they occur in similar situations.

If I handed you thousands of documents which said “Jan-Michael Vincent” all over them, would you need to understand who that is in order to notice the relationship there?

How does evolution learn the form-fitness relationship?

It's the same thing here, you randomly try various token-relationship values and the ones which are slightly better will be favoured.

Microgpt is really good, if you want to understand exactly what happens. I still thought that this article was a good, higher-level complement to that article though.

It's just slop.

Hopefully someone has asked right questions and removed confusing answers/hallucinations.

Is this text generated by AI? I couldn't tell but I'd believe it if it was.

I imagine if resources were spent writing this text then one benefit of using it is not using more resources or the pollution caused from a chatbot.

There is a very high volume of them being posted every day, and they are a significant % of the total. Also, writing is hard, LLM articles can be slop whilst also being better written than average.

This is essentially an open research question. ML theory is unfortunately very weak relative to where the empirics are. I think there's a relatively optimistic paper that was posted a while back here but I would also take it with a grain of salt.

https://arxiv.org/abs/2604.21691

There's of course empirical results and relatively weak theoretical results like the UAT but I also don't think that answers your question fully, especially since it seems impossible to definitively answer questions that the industry seems to betting on like whether or not there is a lower bound to their error rate or whether hallucination as a problem can be solved. We have much stronger ideas of what linear regression is doing relative to what LLMs are doing.

considering they work with any architecture/configuration given enough compute, just more or less efficiently - then maybe it's fundamental, in the same sense as why electricity works...

See Tegmark's "why does deep cheap learning work so well" (well not so cheap anymore...)

https://www.youtube.com/watch?v=5MdSE-N0bxs is remarkably prescient given that it was written before LLMs

Universal approximation theorem, embeddings, self-attention, gradient descent. And empirically, scaling laws.

Because there are patterns everywhere!

Why does linear regression works? Why does computer works? Because it's about math and the encoding information. If we can encode words as numbers, then why can't we encode their order as a relation? It's just that neural networks are very apt at finding that relation even if it's noisy.

You're not supposed to read it, just like you're not supposed to write anything anymore. Claude can read and write more than any human. We just lean back and relax now.

I don’t think it’s absolutely embarrassing. First of all, the point of the author writing at all is to aid understanding, not produce prose. So from that standpoint, what would be embarrassing would be to include incorrect facts that suggest a fundamental misunderstanding of the topic.

From my read, it is fine. The brief history of LLMs is complicated since every single component has papers introducing enhancements. So it’s easy to ignore them or get bogged down with details.

The author appears to be a security researcher learning about LLMs for the purpose of defending against common attacks. So this piece is that person giving themselves a crash course on the topic. The fact that they cleaned up their notes with an LLM is frankly completely irrelevant.