Qwen's ~30B-class models are genuinely good enough for use if you can find a machine with enough memory bandwidth to run them at 30-90 tokens/second. It's been extremely telling that Qwen stopped releasing 120b class models. At some point in the next 10 years (maybe 3?) someone is going to release an Opus 4.5 class 256B model you can run locally. Right now our engineers use about $800/mo worth of opus tokens; at that rate the ROI for local LLM is ~10 months

Have we reached the limits of scaling? Sadly it appears that larger model still equals better model

Very false.

I use small models exclusively. They aren't a replacement for large models. You need decent hardware to run those models efficiently, as smaller parameter models plain suck and are still slow on macbooks. And affordability of higher end hardware is very limited.

Huzzah, they’ve lost their stranglehold. Viva la revolution!

Thanks for building this! Something I grab on a regular basis, especially for doing simple education of folks about the basics of using LLMs by showing something that's not just a chatbot.

Apfel is very useful, thanks for the effort.

Yes. From the CoreAI docs:

"If your app uses model types other than neural networks, such as decision trees or tabular feature engineering, see Core ML."

This is just a bit exciting, although I wonder how the performance of this will stack up next to the stuff we already do with, e.g., a metal-optimised model which we then load into llama-cpp or whatever. (unsloth is a good example of doing this for you "batteries included").

seems they planning to replace it but overall now I'm really confused about this and mlx and coremltools. They should do better work explaining the benefits (and cons) of it and any feature parity between coreai, coreml and mlx.

Requires OS 27+, so CoreML is still useful for backwards compatibility.