That sounds awfully similar to what Opus 4.6 does on my tasks sometimes.

> Blah blah blah (second guesses its own reasoning half a dozen times then goes). Actually, it would be a simpler to just ...

Specifically on Antigravity, I've noticed it doing that trying to "save time" to stay within some artificial deadline.

It might have something to do with the system messages and the reinforcement/realignment messages that are interwoven into the context (but never displayed to end-users) to keep the agents on task.

> to decide halfway through following my detailed instructions that it would be "simpler" to just... not do what I asked

That's likely coming from the 3:1 ratio of linear to quadratic attention usage. The latest DeepSeek also suffers from it which the original R1 never exhibited.

> that it would be "simpler" to just... not do what I asked

That sounds too close to what I feel on some days xD

Turn down the temperature and you’ll see less “simpler” short cuts.

I've seen behavior like that when the model wasn't being served with sufficiently sized context window

> The main quirk I've found is that it has a tendency to decide halfway through following my detailed instructions that it would be "simpler" to just... not do what I asked,

This is my experience with the Qwen3-Next and Qwen3.5 models, too.

I can prompt with strict instructions saying "** DO NOT..." and it follows them for a few iterations. Then it has a realization that it would be simpler to just do the thing I told it not to do, which leads it to the dead end I was trying to avoid.

Claude Opus does this constantly for me, no matter how I prompt it or what is in my AGENTS.md, etc. It is the bane of my existence.

I’ve been running it via llama-server with no issues. Running the latest Bartowski 6-bit quant

Have you tried the '--jinja' flag in llama-server?

> it's has spent a fair bit of time going through an infinite seeming loop before finally unjamming itself.

I think this is part of the model’s success. It’s cheap enough that we’re all willing to let it run for extremely long times. It takes advantage of that by being tenacious. In my experience it will just keep trying things relentlessly until eventually something works.

The downside is that it’s more likely to arrive at a solution that solves the problem I asked but does it in a terribly hacky way. It reminds me of some of the junior devs I’ve worked with who trial and error their way into tests passing.

I frequently have to reset it and start it over with extra guidance. It’s not going to be touching any of my serious projects for these reasons but it’s fun to play with on the side.

Some of the early quants had issues with tool calling and looping. So you might want to check that you're running the latest version / recommended settings.

> and it's has spent a fair bit of time going through an infinite seeming loop before finally unjamming itself

I can live with this on my own hardware. Where Opus4.6 has developed this tendency to where it will happily chew through the entire 5-hour allowance on the first instruction going in endless circles. I’ve stopped using it for anything except the extreme planning now.

I don't know much about how these models are trained, but is this behavior intentional (ie, the people pulling the levers knew that this is how it would end up), or is it emergent (ie, pulling the levers to see what happens)?

Which dense model are you referring to? The dense model isn’t finetuned for code instruction according to the model card.

Around 20ish tokens a second with 6-bit quant at very long context lengths on my AMD AI Max 395+

I’m trying to use local models whenever possible. Still need to lean on the frontier models sometimes.

60 to 70 on a 5080, but only tinkering for now. The smaller models seem exceptionally good for what they are, and some can even do OCR reliably.

I'm getting ~30 tok/s on the A3B model with my 3070 Ti and 32k context.

> Do you feel you could replace the frontier models with it for everyday coding? Would/will you?

Probably not yet, but it's really good at composing shell commands. For scripting or one-liner generation, the A3B is really good. The web development skills are markedly better than Qwen's prior models in this parameter range, too.

In my experience Qwen3.5 is better even at smaller distillations. From what I understand the Qwen3-next series of models was just a test/preview of the architectural changes underpinning Qwen3.5. So Qwen3.5 is a more complete and well trained version of those models.

In my experience qwen 3 coder next is better. I ran quite a few tests yesterday and it was much better at utilizing tool calls properly and understanding complex code. For its size though 3.5 35B was very impressive. coder next is an 80b model so i think its just a size thing - also for whatever reason coder next is faster on my machine. Only model that is competitive in speed is GLM 4.7 flash

In my tests, Qwen3.5-35B-A3B is better, there is no comparison. Better tool calling and reasoning than Qwen3-Coder-Next for Html/Js coding tasks of medium size. Beware the quants and llama.cpp settings, they matter a lot and you have to try out a bunch of different quants to find one with acceptable settings, depending on your hardware.

We don't have a Qwen3.5-Coder to compare with, but there is a chart comparing Qwen3.5 to Qwen3 including Qwen3-Next[0].

[0] https://www.reddit.com/r/LocalLLaMA/comments/1rivckt/visuali...

It's the number of active parameters for a Mixture of Experts (misleading name IMO) model.

Qwen3.5-35B-A3B means that the model itself consists of 35 billion floating point numbers - very roughly 35GB of data - which are all loaded into memory at once.

But... on any given pass through the model weights only 3 billion of those parameters are "active" aka have matrix arithmetic applied against them.

This speeds up inference considerably because the computer has to do less operations for each token that is processed. It still needs the full amount of memory though as the 3B active it uses are likely different on every iteration.

I'm running this exact same setup on a Framework Desktop and I'm seeing ~30 tokens/second

https://xkcd.com/1172

Yeah, the Hyundai factory fiasco kind of dashed the idea that the enforcement would spare people working in favored industries setting up in the US.

There are 2 groups of new Chinese immigrants in the US, they are quite different:

1.Those who arrived through legal channels (most studied at U.S. universities and remained on H1B visas, with a smaller number through EB5 or other visa categories) and eventualy got green card.

2.Undocumented immigrants, which include several sub-groups/waves. In the 1990s, most came from just a couple provinces, Fujian and southern Zhejiang. After COVID, they were from different parts of China and entered through the southern border.

The contributors to AI development belong to the 1st group. They are spread across the country but a large number work in high-tech companies in Northern California.

The 2nd group was intially concentrated in New York and Southern California (Los Angeles area). Later they have expanded into nearby regions. They provide labor for Chinese-owned small businesses such as restaurants, grocery stores, and hotels.

There is an industry created largely by Chinese political dissidents helping Group 2 through asylum applications using fake materials and exploiting common Western beliefs or narratives about China like human rights concerns. For example, Alysa Liu’s father is an asylum lawyer.

ICE enforcement efforts would likely focus more on Group 2 if they are knowledgable. Ohio should not be a high-priority area. I could be wrong due to changes over time. One indicator you can observe: Are there many Chinese-owned small businesses in your area?

"Papers, please." comes to the US of A.

Yes. Yes, so true. And the phd types building these models are probably even scared in China that ICE will fly there to deport them.

Offering „You are welcome“ relocation package to Anthropic might be a good idea.

It'd be great if they went to Mistral!

Competitive models are illegal in the EU.

China city life is amazingly convenient. Trains and subways are just such an enormous quality of life boost. Add to that the relative cleanliness of having nearly zero homelessness and you’ve got something very compelling.

I will say we are winning in accessibility. China doesn’t have much of a ramp game

I got an offer out of the blue for a consulting gig in ML, offering USD 400/hr in China. Assuming this was legit (the offeror seemed legit), it looks like China is also throwing a lot of Benjamins around...

> Or In America you can get paid lots of money, and get yelled at by people online because the Government wants to use your model.

Isn't it just straight-up illegal in China to refuse the government from using your model? USA isn't perfect, but at least it has active discourse.

> China is incredibly nice to live in

I'm sure it's a very nice place to live if you're content to just stay quiet in society and never put a political sign in your yard or even just talk about the wrong thing with your friend in a WeChat.

> get yelled at by people online because the Government wants to use your model

Well duh, as recently demonstrated, an US model used by the US gov will 100% end up murdering actual children sooner than later, in this case less than a calendar year in some far flung war that many Americans do not support. Alternatively PRC model used by CCP might kill in some hypothetical future but for national reunification/rejuvenation that many Chinese support. At the end of the day, researchers and population on one side sleeps more soundly.

Chinese people are very racist towards non-Chinese. It might seem like a happy utopia, but if you aren't Chinese, then you may not really enjoy your time there. It may not be quite as bad as being black in rural US south, but being black (or anything non-Chinese) in China is still not going to be a good time.

Damn that social conscience, huh?

Where do researchers go instead of USA then? Genuinely curious

> I do think it'd be a big loss for the rest of the world though if they close whatever model their startup comes up with.

That's very likely to happen once the gap with OpenAI/Anthropic has been closed and they managed to pop the bubble.

China seems to combine the best points of capitalism (many companies taking many shots on goal, instead of the eastern bloc way of one centrally-mandated solution that either works or not) with the best points of communism (state-sponsored industries that don't have to generate a profit, for the glory and benefit of the state).

But on inference they have to compete with other inference provider that just has a homepage, a bunch of GPUs running vllm and none of the training cost. Their only real advantage are the performance optimizations that they might have implemented in their inference clusters and not made public

I'm running it on a pure cpu 2020 model ryzen server with 2x128 GB RAM with llama.cpp, it seems as intelligent as gpt4. I optimized a little bit by forcing it to run on a single RAM stick and tuning llama.cpp build parameters, going from 3-5 tok/s to a more acceptable 5-8 tok/s.

It can call tools and reason adequately enough to use them when appropriate.

That's 99% is two nines?

Anthropic is a great case study in why uptime doesn’t matter. The service is so valuable that you can have one nine uptime and add $9bil ARR in 3 months.

Anthropic also fires off the alarm bells seemingly at any sign of issue. I've personally only noticed an outage once, and the status page wasn't even showing it as down at that time. It eventually did update about 45 minutes later, then I was back up and running another 15 minutes later but the "outage" on the status page stayed up for another hour or so.

Probably good to sent alerts early, but they might be going a bit too early.

"product market fit is when people are ripping the product out of your hands and everything is breaking constantly" - seems bullish to me

Everything on that page has two nines, so not sure what you're trying to say here.

Not sure what the uptime is meant to signal. People have quite low uptime as well…

9% uptime?

+1 to this, anecdotally I’ve found in my own evaluations that if your system prompt doesn’t explicitly declare how to invoke a tool and e.g. describe what each tool does, most models I’ve tried fail to call tools or will try to call them but not necessarily use the right format. With the right prompt meanwhile, even weak models shoot up in eval accuracy.

Like 4B, 2B, 9B. Supposedly they are surprisingly smart.

There are smaller ones on HuggingFace https://huggingface.co/models?other=qwen3_5&sort=least_param... with 0.8B, 2B, 4B and 9B parameters.

The "distillation attacks" are mostly using Claude as LLM-as-a-judge. They are not training on the reasoning chains in a SFT fashion.

source?