Is any RL done without unit testing? I would be surprised to hear that that wasn't the case, as it would imply a disregard for accuracy for other model makers, which would be surprising. Perhaps you can do this for small modular problems but not for a problem with a 200k token input?

CommonCrawl [1] is the biggest and most easily accessible legally acquired crawling dataset around, collecting data since 2008. Pretty much everyone uses this as their base dataset for training foundation LLMs and since it's mostly English, all models perform well in English.

[1] https://commoncrawl.org/

Haven't we reached a situation where English is the de facto language of scientific research, especially AI benchmarks ?

It's clearly impossible for me to try anything in Chinese, I'd need a translation.

One thing I thought was interesting about this paper [1] on understanding LLMs was how the models associate words/concepts in different languages with each other in what they call Multilingual Circuits.

So the example they give:

English: The opposite of "small" is " → big

French: Le contraire de "petit" est " → grand

Chinese: "小"的反义词是" → 大

Cool graphic for the above [2]

So while English is the lingua franca of the interenet and represents the largest corpus of data, the primary models being built are able to use an English dataset to build associations across languages. This might create significantly stronger AI and reasoning even for languages and regions that lack the data, tech and resources to build local models

[1] https://www.anthropic.com/research/tracing-thoughts-language...

[2] https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-...

I assume a large portion of high quality training material is in English

Chinese internet mostly consists of a few closed gardens tightly controlled by big corps. Crawlers simply don't work when each company employs an army of engineers to guard their data. Many of the most popular websites are also app only. It's impossible to get the corpus necessary to train a good LLM.

All LLMs are trained on the same basic blob of data - mostly in English, mostly pirated books and stuff.

>Or is it that if they publish Chinese-first models it won't get publicity in the West?

This is a large part of it. Kai-Fu Lee's company (https://www.01.ai/) has been publishing open source Chinese language/market focused models pretty early, but the entire conversation around Chinese tech just isn't available to you if you don't speak Chinese, in particular these days given that good English language reporting on the Chinese tech sector just seems very scarce.

They are not "English-first". Deepseek-R1, for example, reasons in Chinese when you ask it a question in Chinese.

One reason is that there is no "good" search engine in China. The most popular one, Baidu, is like garbage compared to Google search. The most useful training data in Chinese would likely be from the social media and video sharing platforms, which I guess is much more difficult to crawl and clean up.

I was under the impression that we just see the English stuff given that we're using English news channels.

I don’t see any indication that it’s English-first?

I’m going to go with: to ensure it is not disadvantaged in benchmarks

I wonder whether English text having fewer characters provides an advantage somehow.

Chinese is hard.

Why are so many American models multi-lingual, supporting hundreds of languages not commonly spoken in the United States?

Could it be that being multilingual results in a larger pool of human knowledge on the technical side compared to training on just a single language or 2. And on the business side, supporting more languages results in a larger TAM (total addressable market). Using english-language dataset for training LLMs is the default, not the other way like you insinuate.

English won. The Chinese youth struggle to write their own calligraphy characters they can read now. Typing favors English.

Nearly everyone in the urban areas of China spoke some English when I visited way back in 1995. It's a bilingual society.

The mandarin language models obviously exist, but what would you do with them if they provided access to them? And what knowledge would be in them? What is the body of knowledge encoded in Mandarin? What does that look like?

Sad reality is that not many outside of China have the facility with Mandarin to use those models. Even non-native Mandarin speakers who claim to be "fluent", are often messing up intended meaning in text. Or making literal translations that wind up making no sense.

Inside of China, llm use will be Mandarin based. Outside, it seems to me English is the natural choice.

Irony of Irony, probably the best way for a non Mandarin speaking layman to test a Mandarin based model would be to use another LLM to translate prompts to Mandarin.

It's a sad future we're looking at.

Or a brilliant one.

Time will tell.

One of the core design goals Georgi Gerganov had with GGUF was to not need other files. It's literally bullet point #1 in the specs

>Single-file deployment

>Full information: all information needed to load a model is contained in the model file, and no additional information needs to be provided by the user.

https://github.com/ggml-org/ggml/blob/master/docs/gguf.md

We literally just got rid of that multi file chaos only for ollama to add it back :/

If you ollama pull <model> the modelfile will be downloaded along with the blob. To modify the model permanently, you can copypasta the modelfile into a text editor and then create a new model from the old modelfile with the changes you require/made.

Here is my workflow when using Open WebUI:

1. ollama show qwen3:30b-a3b-q8_0 --modelfile

2. Paste the contents of the modelfile into -> admin -> models -> OpenwebUI and rename qwen3:30b-a3b-q8_0-monkversion-1

3. Change parameters like num_gpu 90 to change layers... etc.

4. Keep | Delete old file

Pay attention to the modelfile, it will show you something like this: # To build a new Modelfile based on this, replace FROM with: # FROM qwen3:30b-a3b-q8_0 and you need to make sure the paths are correct. I store my models on a large nvme drive that isn't default ollama as an example of why that matters.

EDIT TO ADD: The 'modelfile' workflow is a pain in the booty. It's a dogwater pattern and I hate it. Some of these models are 30 to 60GB and copying the entire thing to change one parameter is just dumb.

However, ollama does a lot of things right and it makes it easy to get up and running. VLLM, SGLang, Mistral.rs and even llama.cpp require a lot more work to setup.

I’ll typically use the defaults initially and then use a Modelfile if it’s something I plan on using. I think you can dump the modelfile ollama uses to have a template to work with.

Those are much much larger models, and they are proprietary. Those model providers just don't have the distilled versions identified and available.

Notice most of the models they are comparing with are 7B models. The exception is also an open weights model (Qwen-2.5-32B-RL-Zero). Even with 32B parameters the MiMo-7B outperforms it.

I believe the goal here is to compare them against similar models that are being optimized to run offline or on mobile hardware.

Qwen 3 or 2.5?

The technical report does go into a lot of depth about how they use RL, such as the modified GRPO objective they use. As far as the README, I imagine most people active in the field understand the implications of "RL" for a reasoning model.

I assume they mean "Reinforcement Learning", and it's been a decade since I studied AI in university, but isn't it perfectly valid to just say "RL"? What kind of specificity are you looking for, whether they used Q-Learning or some other algorithm?

The smaller models have been creeping upward. They don't make headlines because they aren't leapfrogging the mainline models from the big companies, but they are all very capable.

I loaded up a random 12B model on ollama the other day and couldn't believe how good it competent it seemed and how fast it was given the machine I was on. A year or so ago, that would have not been the case.

https://qwenlm.github.io/blog/qwen3/

Go look at the benchmark numbers of qwen3-4B if you think these are unrealistic.

My guess is that it is over fitted to the tests.

Every LLM is basically being trained on benchmarks so "benchmark" as applied to LLMs is a pretty meaningless term.

Today's best models will be worse models for the rest of your life.

LLM benchmarks are mostly bullshit right now. Wait a few years until the hype cycle returns to sanity.

multiple-input, multiple-output was horribly generic to begin with. Terms like multipath propagation and spatial multiplexing will do just fine.

https://huggingface.co/jedisct1/MiMo-7B-RL-GGUF

FYI making a gguf yourself isn't hard and doesn't even need a GPU.

But yeah waiting is the easier option

A lot of Xiaomi products have the prefix Mi. My initial guess is Mo is for model.

Also related reference https://en.wikipedia.org/wiki/Xiaomi#Name_etymology

(Xiao)mi mo(del) ?

Isn't that basically standard for names? What does Google mean? Even when the words in a name do have some meaning, they often don't apply. The meaning of the word brother doesn't really mean anything in the company Brother.

Rice Model?

probably μίμος (mime)

MIT - so open source

And this point everybody will open source their models or weights. The only one which will not is open AI.

Yes, not great, not terrible. I gave it my personal test (a coding task), it produced semi-decent quality code that produced a minor error, after pasting the error it failed to solve it during multiple rounds. I believe another 2-3 years and we'll have quite usable small models.

No, where can I try it? I saw a huggingface link but I wonder if they host it themselves somewhere to like how Alibaba does with Qwen chat.