This is an oversimplification: more data makes models smarter ceteris paribus, but mostly only because auto-regressive training (where most of the general knowledge comes from) is essentially compressing information that can be recalled later if it’s useful (or not recalled). Obviously there are differences in kind within “more data” too, you would much rather have all the books and blog posts in the world than all the fanfiction.

Yes, all three together would be even better. But it wouldn’t be if you had 100x more fanfiction, mostly synthetic, generated during RL to teach a model to be better at writing fan fiction. There are real limits to the amount of knowledge you can cram into fixed-size (downstream of hardware availability) weights. For a period scaling with data was basically “free” because we had the Internet and all the books/media that humans had already created; the data was accessible and limited (at least, the parts we think models should know about) enough and top-hardware big enough that we could basically compress the whole thing.

Post-training/RL are making this obsolete because they’re more about skill/capability acquisition rather than knowledge. They can generate much more data (most of it quotidian/useless, ie an agent made a typo in batch 382829) and clearly seem to cause a kind of mode collapse even in the most advanced frontier models.

We don’t need to make LLMs forget about SpongeBob SquarePants so they learn more about bash. But if I have a question about SpongeBob SquarePants, I don’t need to hear about load bearing seams prefaced with honest caveats after a model writes 400 lines of bash to look up SpongeBob’s family.

And there is probably a lot more SpongeBob knowledge we could put into models if we wanted to: interviews with the creative staff, a SpongeEnv/SpongeHarness modeling how the art/story team work together to create entertaining kids tv, a SpongeBench measuring entertainment value, etc. If a SpongeAgent spends 2000 years in Agent University learning how to Spongemaxx we probably don’t need or want to have it spend another 2000 years writing smoke tests

Thanks for putting this so well. The mathematical evidence for this “general intelligence underlying everything” is the “universal geometry of embeddings” paper. Fascinating read. Or as the ancient philosophers used to say, the one who knows God knows everything.

By pattern matching that SpongeBob episode to the case at hand you provided an example of what you had just explained. Nice. So meta.

This makes a ton of sense, and seems like it's coming from somebody who understands LLMs better than I do so I will definitely take your word for it - but it doesn't totally track with my experience of running small local models either. In my experience some models are better at language X and others better at language Y - and all of them are better at language X and Y than language Z-I-just-made-up. I take that to mean there is some relationship between training data and skill. Maybe not enough to make pluggable modules, tho... at least not with the architectures we're currently using.

People keep forgetting that programming is not just about knowing the target programming language, but also an enormous volume of tacit knowledge:

    - Understanding of protocols like HTTP.
    - HTML, JS, CSS, SVG, and everything "web".
    - Understanding of databases, SQL, etc.
    - Abstract code architecture patterns.
    - Understanding the users' requests in English.
    - Responding in English.
    - Command line tool usage (agents/harnesses)
    - Industry-specific knowledge that can be applied.
    - Frameworks, SDKs, applicable libraries.
    - Relevant legal requirements.
    - Etc...

An LLM works better the more disparate world knowledge it has, even if it's not immediately obvious why it would be relevant.

You just defined a liberal arts education.

>There are "experts" which do divide parts of the model that are found to activate together for specific tasks, so they can be processed in parallel to join the result at the end, but it's nowhere near the granularity of a SwiftUI expert and a python expert. The difference in those things is so trivial from an abstract point of view that it would make no sense. They would be 99% the same.

I think you're missing the point of the article though, which suggests not to mix reasoning capacities with actual knowledge. Sure working with Swift and Python is basically the same, it's programming, with the same concepts etc. Much closer than the ability to drive a car. But the point is, the methods, libs etc are all different and things are changing each time a new version of the language is released. Like you don't need to relearn how to drive a car if you go to the UK but you have to known the wheel is on the right. Knowledge shouldn't be stored in the weights.

could you share the 3blue1brown videos you're referring to?

Ah yes, so that’s why if I ask gemini for a simple low-resource trick to cast a float to its binary representation it is reminded of a work of shakespeare and suggests memcpy.

Or yesterday, when I asked it for the best low resource approach to select the matching string from a collection of strings for a shell interface it suggested an exhaustive string compare over the entire collection. Because it was thinking of how David Lynch would tackle it.

This is so right. We training Whisper Large model on 20,000 audio samples specific to a domain and it ended up reducing the ASR by 5% while improving WER of the finetuned domain by 0.5%.

Instead we ended up with no finetuning. We give audio snippet to 2 AsR models, take 3 best transcriptions and ask the LLm to pick the best based on the context. That produced significantly higher accuracy in how an agent understands the users.

> The reason is because of the “universal geometry of embeddings”, i.e all human languages have the same underlying pattern structure, so any model that is very good in any language is good in all languages.

I think this claim is acceptable in context, but taken alone, this needs to be qualified. Some aspects of language are universal, like abstract information structure and other pragmatics, but no model is going to speak rural Khmer dialects well anytime soon because not enough of it has been digitized, fundamentally speaking, hence qualifying what it means to "be good at all languages".

You 100% can finetune or adapt/build on top of models, and specialize them or extend their capabilities. That’s literally what post training is.

The problem is that “finetuning” was a 2023 AI FOTM associated with products/demos that were almost exclusively using it for LLM character role-play/output style purposes (ie not in actual systems where they served a more functional role).

This made people think you could train models without replay/real evals by yoloing it with SFT (this is partially an artifact of that era being much heavier on autoregressive training and not so much evals). You really can finetune and get results but you have to treat it like a small ML training run, with real evals, and more intentionality than just “more examples”.

You can find pretrained and -instruct models on huggingface that clearly demonstrate what specialization/staged training runs do.

I’d be very wary of conflating finetuning with specialization/extending a model’s capabilities in general.

<< all human languages have the same underlying pattern structure,

Chomsky would like to have a word. Your statement is true only at a surface leve l( they have nouns, verbs and some limits), but it breaks down the moment you start to inspect it closely:

- sign language ( I am not being petty; you put all languages ) - it is almost nothing like the underlying structure of other spoken languages primarily because it does not carry its restrictions - English vs Polish example - word can carry grammar or not; word order can carry meaning or not

Those are just two examples, but both clearly show that little about human languages is actually the same. It is kinda like the history thing. It rhymes.

I am addressing this part as other posters noted issues with other parts.

Unless you are building a factory assembly line where a model is literally doing the same thing over and over, you almost always want a general purpose model over a specialized one.

Turns out the world is made of simple, specialist processes, not generalists trying to achieve them. Adaptability may be of great benefit in evolutionary terms or for a walking anthropoid, but the majority of biology, chemistry, and mathematics rely upon specialist process for good reason. See also the old trope about robotics: that's what you call it before it works, otherwise it'd be a dishwasher.

The upshot is: use a generalist to create a simple solution once, and scale that. Don't deploy the generalist at scale, that's a waste of resources and an inefficient solution.

Sure but I bet this doesn't translate as well in this way to things like Erlang

<THAT'S A BINGO>

ie what everyone asking for this fails to immediately realize.

But the understanding of the language library ecosystem, or even better, your codebase, could let it execute faster and with less context usage.

> This is roughly what multi-agent systems are built for.

I think I disagree. For some things, maybe that works - but think of a multi-agent system where one agent understands the code, and passes it off to the reasoning agent to figure out what the bug is. This system is going to suck. Because encoding enough info to figure out what the bug is would just be dumping every single line of the code.

So say agent 1 (reasoning) asks agent 2 (swift) to explain what is happening in File.swift. Anything agent 2 passes to agent 1 short of the entire code is a lossy transfer - and then the bug gets missed.

everything is a token

> Sounds like unix philosophy.

https://linux.die.net/man/1/ls

https://linux.die.net/man/1/find

https://linux.die.net/man/1/bash

I want to pick it myself because I want to run this stuff locally on hardware I can afford today. But most people seem happy enough using the cloud, where this kind of architecture could be seamless if it existed. I.e. the query to `/chat/completions` contains an extra parameter `domains: ["c++", "swift", "navigation" "gis"]` and then those are the modules that get allocated to this query. When you start a new conversation no domains are set and it hits a generalist model, but the generalist model includes domains so the next query doesn't have to hit the generalist model. The model could even have a tool it could call to rope in new domains if the scope expanded to include other modules.

> I think the vast majority of people do want general purpose models. They want to be able to ask it any question, or ask it to perform any task, and for it to do a decent job at it.

The vast majority of people listen to the music of Ed Sheeran and think that it is good.

Would it? AGI was always about the efficiency gains of versatility, not strictly all forms of efficiency

On the otherhand, your own brain probably doesn't use your eyes to hear.

But I am not asking for a faster horse. I am asking for a draft horse instead of a race horse. I know the tasks I have on hand, I know the VRAM and compute budget I have to run them. I am not asking for AGI, I just want something to edit my little text files.

My motivation is somehow the model should be able to work without hitting an external resource every single time and somehow have all the knowledge necessary locally so we don't get rate limited but how can you stay up to date with the latest information while still remaining a good boy scout by not reaching out to tool use and scraping every single GitHub repo and issue every single time?

I would, once it is possible, specifically unplug the shitcoder expert that sometimes shows up and puts `as unknown as` and bypasses the commit hook that is supposed to enforce it. That should be somewhere near java coder expert in the whatever dimensional space. I can feel it being true.

Except assuming a fixed budget of parameters, there is clearly stuff that's better for programming than others. E.g. Qwen 27b is a better coding model than Gemma 4 31B. More params doesn't automatically win. Perhaps what makes a good swift programming model is a ton of python training, so those two things can't be separated - but that doesn't negate the idea of loading a model that's good at the specific task you want - or has specific knowledge of the libraries and tools for the language you're using at the expense of the ones you're not using.

In my hypothetical, 9B of coding and reasoning includes enough language to communicate ideas to the model.

...kennywinker is a 0.001B MoE model, with 2 active parameters.

No, that’s a common misconception. MoE despite the name doesn’t have an “expert” slice that’s an expert in any particular thing. It’s just trained models such that a subset of all memory weights is active on a single token. Often, the weights active on the next token are different. It’s used to improve memory bandwidth (throughput) and also to even out load in a distributed serving model - one GPU hosts one “expert” and the conversation is routed to it for the next token. This requires a lot of movement of the KV cache of course so it’s difficult to retain speed across multi node splits - usually it’s a rack of GPUs that you do this for.

MoE's in the abstract often get presented as if theres explicit layers of experts for any given domain of knowledge, like your coding tasks are being routed to coding experts, but it's really not that at all.

THe original MoE paper from Noam Shazeer et al. is worth a read on this bit, though the paper is admittedly pretty dense. But TL;DR is that each expert layer is learning highly abstract, localized structural and syntactic patterns in the data to minimize the loss function, and its doing this token-by-token (which in some cases may have some domain clustering, but that's just incidental).

When you start batching your queries, even if they all seem like theyre in a single domain, if you visualized the activations you'd notice that most if not all of the network is lighting up on the batched forward pass.

I mean, I get pretty damn good general reasoning out of Qwen3.8-27B quantized to 4bits, and that knows Swift, Python, Node, Ruby, Rust, etc. etc. etc.

And so far even the biggest model doesn't seem to have a working Make No Mistakes module, so maybe that's not needed

The future will be "AI operator license #123..., subclass 'Primitive Coding'", with your TouchID / FaceID enabled only. And I am not being sarcstic.

I don't think so?

Definitely not MCP, as that pulls info into the context. Unless contexts become REALLY big so that I can add 10B in swift knowledge, that's not gonna help me.

Possible RAG? I don't know enough about how that works, but I think that's not quite it either. I don't want to import facts like "the swift standard library contains a reverse array function", i more want to import knowledge - e.g. the parameters used to generate the text to reverse an array in swift.

Tool calling wouldn't do it either. You'd have to encode every single possible bit of useful info into the tool call, and the tool response would have to encode every piece as well (variable names, function scopes, types defined in other files, etc). E.g. how does it find a bug, if you have to pass understanding back and forth between the brain that understands debugging and the brain that understands THIS code?

I don’t think so because those both live in the context window and as such pollute it when they’re not performing optimally.

I think having unused or rarely used weights doesn’t influence the results as poorly as RAG injecting irrelevant facts.

It sounds to me like some sort of “dynamic MoE” where you can add/create or remove experts on the fly.

I think what you’re describing is the closest approximation we reasonably have right now though.

I'm not actually describing a specific how, I'm describing a goal. The goal is models that are made for specific work, not general work. My model doesn't need to know a single line of python, but if it's a coding model perhaps giving it some python is the best way to make it smarter. Ok. Great - give it python! But first maybe check if giving it C++ is actually better than giving it python. Because for my purposes it doesn't need to know C++ or Python, and I have a fixed parameter budget. Find me the ratio of Swift:C++:Python that gets me the best Swift coding agent in my parameter budget, and then train that.

And then ideally, make it pluggable so I can pick what I want from off the shelf components, but if that's not possible - then just train up as many variants as you can so we can all pick the best variant for our current need.

Only if you fundamentally misunderstand what I am describing

Tangent: This is often true of humans as well.

We often make a decision based on a gut feeling, and then backfill a logical reason supporting our feeling, without even realizing we're doing it -- rationalization.

This was beautifully shown by asking a model to explain how it added two numbers together (something like 45+21), and it told a plausible story, when in fact they showed it was some rotation on a helix living in some internal manifold.

Like asking a human "how did you catch that fast ball coming at you?"

> It's a statistical model with useful emergent properties. It doesn't think, it doesn't reason, it isn't aware of facts or the rules of logic.

What makes you so sure your own brain doesn't work the same way?

Senses of what? LOL. The whole Internet is AI generated by now and we all contribute to that on daily basis. get used to it or dull your senses ...

Generally I agree, with two caveats/thoughts:

Providing not just any a baseline, but a correct and useful one, is ever more important the less the model is grounded in world knowledge – misunderstandings probably compound faster if there is no general grasp of (broadly) “life on earth”, or computers, or whatever.

And secondly, I think (consumer-oriented) search becoming worse and worse is a challenge that’s mostly solvable (but far from solved!) for the big labs: (Mostly) trusted or even editorialized/reviewed sources like published work, Wikipedia, etc. is something they could index internally, it doesn’t need to come from a random blog site on the public internet. Furthermore, there’s a whole slew of companies specializing in crawling-for-LLM (i.e., bypassing bot protections) now as well.

> search keeps getting worse

For a lot of use cases, you don't need a general purpose search engine – a search over a curated knowledge base works even better.

There are plenty of freely available data sets you can use, depending on the application; plus in many cases you will want to use internal-only knowledge bases containing non-public information (e.g. documentation for a corporation's internal systems and procedures). There are also many paid subscription domain-specific knowledge services available.

search isnt that super hard to solve. the problem is that incentives are misaligned - which means companies who sell something have an incentive to share their index with the corporation that pays to be the default search provider. for a simple search engine that just delivers facts there's very little incentive to keep its index up-to-date: because that company wont get the apis to retrieve the content efficiently, so it needs more compute, bandwidth and storage to run.

the technicals of search did not get worse. the economics of the search provider did. If government wasn't stuck in the 50s mindset, search provider would have been nationalized just like a utility.

> Did we get a grip on this problem?

That's the right question to ask. For a while, it seemed that hallucinations went down as models got bigger. That may only have been because, with a big enough model, the desired data might be in the model, somewhere, which would keep the model from making up something. That's the brute-force approach to the problem.

This new article indicates that trimming down the model by pulling out seldom used info makes the problem worse again.

If LLMs had reliable "I don't know", and access to search engines, much smaller models might work.

I’m working at a company where everyone is using LLMs for everything and I’m not aware of anybody using anything truly genetic. It’s all human in the loop babysitting the tool.

Learning new things for example, general curiosities, text processing

Intelligence is still not well defined, and by more solid attempt LLMs absolutely are.

Intelligence, Knowledge and Sentience are three very separate concepts.

I know they are just next token predictors. However, Our learning also started with that only. They moved from Alphabets to sentences. Intelligence is what AGI stands for and that's still in distant future. But, we need to push them for AGI. That's the ultimate goal of humanity for now. Finger Crossed for AGI in our lifetime.

After that Space colonization will come.

As the Portal companion cube taught us, we will anthropomorphize whatever the hell we want.

At least later-era Wittgenstein would disagree with you.

As RAM prices are so nuts, I picked up a used gaming laptop early this year with an RTX 3070 (i.e. 8GB) to match the specs for my gaming tower that's run everything I'm interested in just fine. That includes recent Unreal 5.x games (Satisfactory, Fortnite, etc.) with high graphics settings on a 4k TV (60hz). There aren't a ton of games that require more than 8GB of GPU ram.