I would also note that for people who want to download these models, you can find MLX versions of just about everything popular on huggingface these days. For instance go look at the "main" page for Qwen 3.6 35B-A3B and then follow the link to quantizations, and pick one of the more popular/reputable MLX variants.

https://huggingface.co/Qwen/Qwen3.6-35B-A3B

I was so excited when I saw "blaizzy" in the domain, because Prince Canuma's work around MLX has been of such uniquely high quality.

  Would be totally unsurprised if those modalities and models get integrated into this UI.

  Support for dedicated audio-only and image-generation-only models is coming soon.

Thanks for that. My first question was “What does this do that Unsloth doesn’t?”

LM Studio is trash on Windows / Linux... guess that makes sense...

Switching to mlx-vlm is basically harmful since it (like vllm and sglang) have such garbage support for modern samplers. To be clear, I am one of the authors on the min_p paper, and if min_p is the best you have (when llamacpp supports the far superior top-n-sigma), than I have no reason to switch even if you are somehow faster.

(https://arxiv.org/abs/2411.07641)

And if you do care to support modern samplers, you can start with the following:

1. https://arxiv.org/abs/2509.23234

2. https://arxiv.org/abs/2509.02510

3. https://arxiv.org/abs/2604.11012

Prince CacheAsNUMA? Nice :)

its a wrapper around mlx, so thats gonna be the portability bottleneck

I was confused too, but I believe that it refers to the Pareto Frontier: the best set of solutions to a multi-objective problem.

Look it up, it’s a bit difficult to explain concisely in words but it is intuitive visually.

If we are thinking of intelligence and price, a model will be in the Pareto Frontier if there’s no cheaper model of the same or higher intelligence. Or if there’s no more intelligent model for that price or lower.

EDIT: See this chart from Artificial Analysis: https://artificialanalysis.ai/#intelligence-comparison-tabs

So for example DeepSeek V4 Pro can be considered a frontier model because there's no cheaper model that is as intelligent.

For any solution in the Pareto Frontier, there no "no-brainer" alternative, in the sense that there's no other option that is better in some way without giving up something else. It's the best of its "weight-class".

I don't know that I agree with this specific use of frontier because it is confusable as you say.

But (off on a tangent) I do think that there are multiple frontiers generally — and I also think the open weights, small local model frontier is by far the most important and exciting one.

I keep mucking about with what Gemma 4 12B can do and every time I do I find myself thinking that all the energies in the AI world are going in entirely the wrong direction, because it is small, clever, efficient and remarkable.

If all of that research money were to be spent on improving AI models that fit inside a 16GB RAM machine with unified memory, I think really important progress could be made.

I enjoy using the Qwen 3.6 models (and BottleCap's new fine tune of the 27B) but the small Gemma 4 models are impressive in a way that I think is going quite unreported.

So while I don't think this website should use the word "frontier" here, even referring to Qwen 3.6 27B which is weirdly close, I think it could.

The frontier is a multidimensional space defined by the “best” combination of traits a model can have in multiple dimensions: parameter size [smaller is better], various task metrics, and relative token generation speed on like hardware [faster is better], and active memory requirements [smaller is better] are all possible dimensions, and a model on the frontier of the current options space is one where getting better on one of those measures cannot be done without getting worse on at least one of the others.

The frontier is a curve. https://en.wikipedia.org/wiki/Pareto_front

Had the same though. The gap between open-source and the frontier is closing in, especially with Kimi K3, but that is like >2T parameters. The Gemma 4 and other models you can actually run on an average Mac, is not in the same league.

It might be overused elsewhere but I don't think its use here is inappropriate given it's qualified: it doesn't say "frontier models", it says "frontier open models".

No, your thinking is 100% correct. It's called clickbait.

+1 came here to say this, I opened the link expecting some technical breakthrough. Misleading click bait title.

LM studio is closed source software built ON TOP OF code released by the author of Nativ.

LM Studio seems to do the same thing, except that LM Studio is not open source. So they have a point, they do something more.

"The other “local AI” apps you’ve heard of? They’re proprietary shells built on top of open-source engines they don’t own."

This is a roundabout way of addressing LM Studio.

I was wondering the same and assuming I'd overlooked something.

What spec is your macbook? I want to run Deepseek V4 Flash but its too slow for agents on my Strix Halo.

What they likely mean is, why options like LM Studio are not open source.

We've shipped some code generated by Qwen3.6 27B to production (under OpenCode). It lacks the breadth of knowledge of models like Opus, but if a change is fully inferable from the prompt and the surrounding code, it works very well. It won't be able to write something from scratch that requires niche knowledge (say, a performant inference engine tailored to Blackwell GPUs), but if it's just a PR adding a new use case to an existing project (which is usually just "load from the DB, do some invariant checks, modify the entities, store them back"), it works as well as Sonnet (provided you have the correct configuration, like recommended temperature and top-p settings, the model isn't over-quantized, you have at least 150k tokens of context available, etc.).

I don't use them as coding agents, but they can be very useful for things like text transformation, summarizing, or text extraction.

That said, if you have a subscription to a paid model already, you're not necessarily winning out on anything except perhaps privacy, which isn't nothing.

I use a gemma4 model locally to extract content from messages to a personal agent I'm building for its memory graph (to break the message up into the topic, source (assistant or owner), facts, entities, etc. in the message content (all getting thrown into a magma-esque graph using NLEmbeddings for memory search). This is for a custom personal agent that targets deepseek-v4 flash. The local model is too slow in my setup for a chat agent, but for memory extraction it works pretty well, saving API usage on every chat turn.

there is plenty of grunt work these smaller models can do. update dependencies, fix merge conflicts, write --help, markdown, or readme files for existing code. etc.

sometimes they fail but undo is just a "git restore" or if automated, rejecting a PR and having a better model take a crack at it.

They're great at helping me look up web dev stuff when I don't have internet access.

Qwen35ba3b can do a huge amount of data cleaning work on pretty modest hardware. Already have run about 100 billion tokens on it using 2x3090 gpus.

[flagged]

FWIW on my M1 Max I have not really seen any advantage at all from MLX.

I am fully prepared to believe the benefits accrue more to the M3 and up (because of changes to the Apple Neural Engine).

But with the models I've tested, unless I am missing something, the performance of GGUFs in llama.cpp has been better in some cases.

I still have not had results from Gemma 4's MTP be really worth it, to be honest; but with the Qwen 3.6 MoE it is measurable. Maybe with newer kit it is more meaningful.

(There is every chance that the above is not the experience of anyone who really deeply knows what they are doing; it feels like I am a perpetual novice at this stuff)

Same here. Tried MLX twice at different times after reading the claims here but it always does considerably worse for me than llamacpp.

I assume Nativ doesn't support SSD streaming like DwarfStar.

Agreed. Just based on this not being Ollama, so I will give it a try.

my thing is kind of an Ollama competitor (surrogate?) too. more for prose/text planning, not so much for coding, at least the harness, but i'm sure someone could set it up to do that: github.com/0gsd/enough

A lot of macOS apps are statically-linked, even interpreted programs. It's still a native app for going that route.

You can run a lot of these on e.g. M1 max 64gb

Is it? The developer page for the github repo suggests he works at Arcee. Maybe he moved?

(I initially thought the same because of the website appearance)

There is a Gemma 4 model with 12B parameters which might be worth trying. e.g. https://huggingface.co/mlx-community/gemma-4-12B-it-qat-4bit

That said, your computer will still get hot!

If you go small enough it should be no problem. For example Gemma 4 E4B in Q6 or Q4 quantization should run well on your laptop. It shouldn't be too taxing, but would still want to eat 7-9 GB of VRAM or so.

Now that model is mostly useful for writing or chatting.

You need more RAM, plus the models take up a lot of space. 32gb min but I’d recommend 48/64gb, you won’t get close to frontier but it’s still fun to play with, images are very good

heating up is normal, that cannot be avoided. it should become laggy, but you just have very little RAM (I assume 16 GB?) So most models are too big with other stuff running.

Tbh, the only reason to run a model locally is when you want to be completely safe.

From productivity point of view, it doesn’t make sense to have any notebook running a local LLM.

We have one life. We should spend it wisely.

There is also https://github.com/ml-explore/mlx-swift-lm which is what https://jan.ai uses.

That phrase itself is such an astounding performative contradiction [0].

0: https://en.wikipedia.org/wiki/Performative_contradiction

Something nobody needs is pointless hot air like "Everything you need. Nothing you don't."

You’d be surprised how hard this actually is. I spent 3 days iterating on a marketing site, where I had very explicit / “well written” copy, and it would just repeatedly rewrite it back to the most awful slop. Over and over again! Ended up adding various AGENTS rules telling it to leave the copy alone

I don’t think anyone ever looked at result. Website if full of overflow bugs. It’s pure slop.

That's what I wondered.

The webpage isn't the point. The software the webpage refers to is the point.

Bionic is an agent; this appears to be an open-source competitor to lmstudio. The initial commit is just a few hours ago though, so …

Why do ppl downvote this comment? HN makes no sense sometimes...