UncleOxidant
9 hours ago
Sure, this is a small anecdotal datapoint, but since I've gotten Qwen3.8-Flash-Next running well on my Strix Halo box my Claude Pro subscription use has dropped off a cliff. I'm only at 18% of quota usage this week - this being Friday I would've been at 80% at least in the past. This is just an astonishingly good model that runs locally and quite well on my PC. I do not plan to renew my Claude Pro subscription when it comes up in a couple of months.
sorenjan
5 hours ago
I really hope the future is local models, because the idea that all companies will need to buy tokens from a handfull of tech companies to run their businesses or make their products is awful.
mschuster91
5 hours ago
> the idea that all companies will need to buy tokens from a handfull of tech companies to run their businesses or make their products is awful.
To the leadership of these tech companies that actually is the dream. Sam Altman said the quiet part aloud recently: "From a global allocation perspective, actually, it's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer, and being used maybe 5% of the time."
And it certainly fits into the neofeudal worldview of other relevant Big Tech leaders. Be it Peter "I no longer believe that freedom and democracy are compatible" Thiel, or unnamed bigwigs discussing if shock collars are effective to prevent their staff from revolting after a doomsday scenario [2].
We shall own nothing and rent everything, that is the mindset these people believe.
(And if you ask me: the AI IPOs are part of that strategy. Assuming OpenAI and Anthropic both IPO, they will get part of every American's and most other countries' pension funds... and when they collapse, a lot of that value will be gone. The initial investors into SpaceX, OAI and Anthropic all will have made their money and put it aside, and we will be left holding bags)
[1] https://www.ign.com/articles/openai-ceo-sam-altman-thinks-it...
[2] https://www.rte.ie/news/business/2025/0511/1512086-doomsday-...
savanaly
3 hours ago
How is that saying the quiet part out loud?
JuniperMesos
3 hours ago
Altman was simply making an accurate observation about DRAM, in the context of a conversation about DRAM shortages and precisely why it is difficult for DRAM manufacturers to respond to the market price incentive to increase DRAM production. Even if you think it is bad that DRAM is currently expensive, as I do, that doesn't change the underlying fact about the world that Altman's comment reflects.
Thiel's comment is also pro-individual-ownership - in context he was talking about the danger that in a mass democracy the people as a whole would vote to have the government restrict the freedom and confiscate the property of successful people (particularly prescient in light of the recent California attempt at a billionaire wealth tax). RAM being expensive is better than RAM being rationed by a democratically-elected government so no one person has too much, as judged by the voting masses.
hocusPokeUs
3 hours ago
Successful people are not successful without keeping unsuccessful people from success; inflation since Millennials were born is almost 300% and taxes have been slashed; takes 800k/yr now to have the buying power of $200k in the 1980s
Thiel has benefited from wealth being wielded by rich media owners to convince the masses decades long trends in deflation of buying power through inflation of costs are good for them
With such a system giving him a boost he was successful before Millennials and GenZ were adults or even born
Altman came along during ZIRP; being successful when the government is intentionally printing money is not really success
matkoniecz
3 hours ago
> Altman was simply making an accurate observation about DRAM
no, he was lying by omission
"it's quite terrible" only if you ignore all the problems with plans they promote and measure it only in metrics Altman wants to measure, as they support his plans
For start, Altman would charge as much as possible to optimize profits, so very significant part of efficiency increase would become profits of "Open"AI.
As another obvious problem: handing so much power to a specific company has own costs.
There are so many problems here with this specific part - starting from their power dictating how can you use it, access to your data. And that part is getting worse for anyone not planning to obey all decisions coming from USA. We have already seen export controls being invoked.
But wider, if you think that outsourcing thinking to a specific SaaS has no costs, beyond current subscription costs, then I have several thousands of bridges for sale.
mschuster91
2 hours ago
> Altman was simply making an accurate observation about DRAM
Which might be true, I'm willing to accept that, but it does not change the fact his message was incredibly tone deaf.
Cost of living in general is through the roof for years now, many people haven't had the financial ability to meaningfully upgrade their computers for just about the same timeframe - first Bitcoin drove CPU prices through the roof, then GPU prices, once that died down due to ASICs there were a few fads that "mined" RAM, HDD and SSD, and in parallel to the latter Covid hit, disrupting supply chains. After Covid was somewhat over, the Russians invaded Ukraine, then we got the various MENA conflicts after Oct 7th, and now the Iran war is wrecking everything - and ever since AI entered the picture somewhat three years ago, first GPUs and now everything else has exploded in price. Entry level smartphones have all but disappeared. And don't forget our existing computers got less and less useful because first Microsoft pushed the garbage called Windows 11 onto everyone it could and second, everyone and their dog went for Electron resource hog apps.
For Altman to now say "your unused RAM could be of much better use in our servers" is like a giant fat fucking middle finger, telling us to sacrifice even more of our needs just for the opportunity of him and his fellow rich bastard friends to make more money, all while he and his fellow rich bastard friends are executing the setup of a financial crisis that will make 2007 look fucking tame.
> Thiel's comment is also pro-individual-ownership - in context he was talking about the danger that in a mass democracy the people as a whole would vote to have the government restrict the freedom and confiscate the property of successful people (particularly prescient in light of the recent California attempt at a billionaire wealth tax).
Oh for fucks sake. Taxes aren't theft, they provide for the environment that makes these rich fucks able to earn money in the first place. Thiel is worth 38 billion dollars. Even if you'd tax the guy 50% of his wealth, he'd still have more than one single human and their descendants can ever feasibly gamble away.
> RAM being expensive is better than RAM being rationed by a democratically-elected government so no one person has too much, as judged by the voting masses.
At least in a rationed system that allocates supply fairly, ordinary people wouldn't have to run with barely working hardware because they can't afford to buy something new and modern for once as all the new stuff is hoarded by the rich like a LotR dragon hoards his gold pile.
And as evidenced by this week's elections - the people are fed up beyond belief. Either we the people get some relief, the sooner the better, or the situation may indeed devolve into open riots.
throw10920
33 minutes ago
> the idea that all companies will need to buy tokens from a handfull of tech companies to run their businesses or make their products
Did you somehow become incapable of doing things without models? Most of us have not.
senko
8 hours ago
Interesting, I'm also comparing 3.8-flash-next (4-bit quant, llama for inference & pi as harness, on a 128gb strix halo machine) with opus 5.5 in claude code and while qwen is a good and fast model, it's nowhere near for coding, even haiku 5.5 feels better.
I could see it used as a sort of "sub-haiku" for easy stuff to leave more tokens for the bigger claude models, but I won't be cancelling my Claude sub any time soon.
agile-gift0262
7 hours ago
It's obviously nowhere near opus 5.5, but it's crossed the same threshold opus 4.5 did: it's good enough. If all model-as-a-service companies shut down tomorrow, I'd be happy with qwen 3.8 flash next q4 running in my strix halo
hungryhobbit
7 hours ago
The problem here is we're not talking ... I don't know ... hamburgers. If I like McDonald's better than Burger King, it's not going to stop me from buying a Burger King burger (if I'm hungry and there's no McDonald's around). One burger may be a bit better, but I can reasonably substitute one for the other.
Coding AIs aren't burgers. Even if a local LLM is as good as Opus 4.5 (when you sink several thousand dollars into hardware ... and I'm skeptical even then) ... the actual difference between the models is huge.
Let's say I use the $200/month Claude (I don't, I use a cheaper plan, but stay with me). If I switch to a Opus 4.5 equivalent local LLM, and it makes even a single production issue that 5.5 wouldn't, that mistake will cost thousands of dollars.
That "cheaper" model isn't really cheaper.
csdreamer7
6 hours ago
> Let's say I use the $200/month Claude (I don't, I use a cheaper plan, but stay with me). If I switch to a Opus 4.5 equivalent local LLM, and it makes even a single production issue that 5.5 wouldn't, that mistake will cost thousands of dollars.
You are assuming that Opus would always be superior; the cloud AI providers can be very variable. They can be very useful for months, but during certain times their quality can drop off a cliff. People online speculate that the cloud providers drop quality during this time when they are training a new model or the service is over-subscribed. You do not have that issue with a local model.
That issue of quality of service is enough many businesses may prefer a locally run model because it will be more consistent.
UncleOxidant
5 hours ago
It's way better than Opus 4.5. See my stress test elsewhere in this thread (C -> microcode compiler with optimizations written in OCaml, generates microcode + Verilog output) - it's doing better than Opus 4.6/4.8 were when I was using them to code the original C version of this compiler early in the year. It's come up with optimizations that Claude didn't come up with back then.
rbehrends
5 hours ago
> Let's say I use the $200/month Claude (I don't, I use a cheaper plan, but stay with me). If I switch to a Opus 4.5 equivalent local LLM, and it makes even a single production issue that 5.5 wouldn't, that mistake will cost thousands of dollars.
There are a few points I'd like to make here.
First, the frontier is jagged. There are errors that Qwen3.8 Flash Next make that Opus 5.5 doesn't make, but the reverese is also true: Opus 5.5 makes errors that Qwen3.8 Flash Next doesn't make. The latter will likely be fewer, but they won't be zero. Outside of really large differences in capabilities, there is no total ordering among models, only a partial one.
As an aside, do you always run Opus 5.5 at max reasoning in order to minimize errors, too? I suggest most people don't, because they have their own intuition for good enough vs. cost.
Second, Opus 5.5's strength are autonomous tasks. For interactive, HITL work, it's not a particularly great choice. This includes its preferring complex Bash commands over tool use and inscrutable Python scripts, but also often not even explaining what it does. It is not a great choice to use as a virtual pair programmer, because following along what it does and maintaining a mental model is harder than with some other models (though probably because of benchmarks that prioritize autonomous work, these and related problems have infected other models, too). You can address that to an extent with CLAUDE.md instructions, but even that has imperfect results.
Finally, if you rely on a model not to make mistakes in order to ensure correctness, your software development process is fundamentally broken. You need to assume that any model will make at least some errors and need a development process that accounts for that. The problem here is not necessarily which model creates fewer errors, but which model creates code where errors are easier to discover. Ironically, more powerful models may create more complex code, making that harder.
You can see an analogous effect with reasoning effort in FrontierBench 2.0 [1]:
"FrontierCode is one of the few coding benchmarks where writing additional (correct) code doesn’t necessarily raise a model’s score. This leads to an interesting phenomenon: models can score lower at higher reasoning effort. Because this non-monotonicity often matches user sentiment but is hard to see in other evals, we consider it a useful feature of the benchmark."
A example:
"This task comes from PostHog’s open-source Android SDK, which apps use to record analytics events. When the device is offline, the SDK writes events to disk and sends them the next time the app starts. The task is to fix a startup bug in which two modules send data at the same time: the live queue sends new events while a legacy integration, PostHogSendCachedEventsIntegration.kt, sends the cached ones. The two race, and events reach the server out of order.
"At low effort, GPT-6 Astra moves cache loading into PostHogQueue.kt, removes the sending logic from the legacy integration, and leaves the SDK’s public API untouched, which produces a passing solution.
"At max effort, the model often deletes PostHogSendCachedEventsIntegration.kt outright and spreads its responsibilities across other legacy files. This also fixes the race and passes most of the hidden behavior tests. But the deleted class is part of the SDK’s public posthog.api surface, so removing it introduces an unnecessary breaking change for every app that references it. Scope blockers, which deterministically check that the model hasn’t modified certain files, catch this change, and the agent fails the task."
altmanaltman
2 hours ago
> Let's say I use the $200/month Claude (I don't, I use a cheaper plan, but stay with me). If I switch to a Opus 4.5 equivalent local LLM, and it makes even a single production issue that 5.5 wouldn't, that mistake will cost thousands of dollars.
What are you talking about, you're fully dependent on an LLM (Opus or whatever) not making a mistake or else you lose thousands of dollars, in production? How is that okayed?
singpolyma3
5 hours ago
"It" makes a production issue? Are you not involved in this process?
thatsabadlook
40 minutes ago
Yea your math doesn't math.
Try it before you go super far into the world of this "must suck".
Most people aren't going full vibe code. If you stear these models, they are almost identical. Seriously. The major score changes in a lot of the absolute latest models are just models doing more things (often when they shouldn't). Having a slightly lower scoring model is actually better in some ways. I digress.
If your model can make a production mistake you are using it wrong. Sorry not sorry.
There are other benefits. Your data is private, it's not sold. Your privacy is kept, that's priceless. No one can degrade your workflows. Uptime is you are online whenever you are online not 3 9s. You also won't get refused asking basic security questions.
Your math doesn't math for a lot of people. It might for you, but I would encourage you to think a little more.
senko
6 hours ago
> it's crossed the same threshold opus 4.5 did: it's good enough
Agreed on that, and actually I'd put it somewhere around opus 4.5 from what I've seen so far.
enraged_camel
4 hours ago
>> It's obviously nowhere near opus 5.5, but it's crossed the same threshold opus 4.5 did: it's good enough.
Good enough may be sufficient for personal projects, but in a competitive landscape where you're using a "good enough" model and your rivals are using a frontier model like Opus 5.5, you will be at a significant disadvantage.
agile-gift0262
9 minutes ago
Not necessarily. If yor process accounts for "all models make mistakes", you are probably limiting how autonomous the model can be, and then there's not much tangible difference between models beyond that threshold.
fakedang
an hour ago
In the competitive landscape, cost and not-getting-locked-down are also factors up for consideration.
hackernud3s
6 hours ago
> If all model-as-a-service companies shut down tomorrow
Is this the nerd version of "doomsday prepping"?
reverius42
6 hours ago
If the planning-for-a-$2T-IPO company shuts down overnight, unexpectedly, you might want to do some actual doomsday prepping (like, canned food).
utternerd
5 hours ago
First time?
swiftcoder
5 hours ago
Eh, it wouldn't be the first unicorn to go up in smoke just after the IPO
UncleOxidant
6 hours ago
Here's a stress test of sorts that I've had it working on the last couple of days: We have a C -> microcode compiler written in C (most of that was work done 'back' in the Opus 4.6/4.8 era). I gave it the docs for that compiler and asked it to implement it in OCaml (since it's not very common, but also is a good implementation language for compilers). It can use the first compiler as a golden to compare against. It's essentially: C language parser -> AST -> IR -> [optimization passes] -> microcode (+ some verilog code). It's basically at the point now where the original compiler has been. Not only does this generate microcode it also has to generate some Verilog code that loads and runs the microcode as well as run simulations in Verilator to confirm functionality. I asked it to come up with some new optimizations that the original compiler doesn't have and it came up with a couple that in some cases reduce microcode lines by up to 30% (less microcode in this machine means more performant). These are ideas that Claude didn't come up with when it was working on the original compiler back in the 4.6/4.8 era(and I recall prompting it several times asking it if there might be other optimizations we could perform).
So all in all, I put it at Opus 4.8 level, at least. And in some ways it's exceeding 4.8.
largbae
5 hours ago
I am in the same boat. Sun servers were still the best that money could buy when they lost to the Linux revolution. Linux was _good enough_ and destroyed their margins.
devsda
2 hours ago
I would have preferred to have open Solaris flourish along with Linux.
CharlieDigital
9 hours ago
I used Ollama with Qwen 3.8 27b on my MBP for a flight to try some experimentation and found that it was shockingly good at one of the more complex PDF processing tasks.
Like seat-of-the-pants good as Opus 5.5 but a bit slower and required just a bit more prompting to get what I wanted (which is fine for an app pipeline).
singpolyma3
5 hours ago
And only slower because hardware
jypepin
5 hours ago
it feels like this is a bit similar to the famous dropbox comment.
I guess maybe another anecdotal comment, but I'm an engineer myself, have a home server for simple stuff, and I pay $200 a month to chatGPT for a pretty simple usage and really don't see myself go and run my own open source model, especially at the speed at which they evolve and change.
99% of the world will not setup their own open source model, just like 99% of the world is not setting up their own servier with ssh copy whatever the dropbox comment was saying.
singpolyma3
5 hours ago
You don't have to rub it yourself though. There's lots of pure inference providers which are way cheaper than the big guys and have cheaper models
anon373839
4 hours ago
If you were paying API prices (for ZDR), I suspect you would be eyeing local hosting. But if you’re comfortable selling your data, then the subscription prices are going to be hard to beat, for however long they may last.
compass_copium
5 hours ago
99% of the world is also not spending $200/month for chatGPT. They're using whatever free shit they can get to ask basic questions.
dofm
7 minutes ago
Right. And FWIW those people are actually quite close to being able to do what they need with their phones.
If it were not for the x86 architecture they would almost have models that could answer those queries on their desks. (Gemma 4 E4B QAT is surprisingly good at this stuff; Gemma 4 12B QAT is great at it).
If I was Sam Altman I would consider giving Intel and AMD money to not develop unified memory architectures.
cmiles8
8 hours ago
That jives with my own datapoints. Of a circle of around 20 very active devs Anthropic went from being the must-have tool to on the chopping block.
Around 50% of that group says they plan to cancel their Claude subscriptions now that their local or open model setup is performing “good enough.”
jatora
7 hours ago
I don't believe this one bit. There is no way local models are good enough except for the most basic of coding.
samtheprogram
6 hours ago
If you actually know the code and can give guidance, direction, and corrections they absolutely are.
If you are just vibing and managing multiple agents and reviewing at a higher level like architecture, you are right.
cuvinny
2 hours ago
Why is it unbelievable? I still remember how to program and arch new features and don't need claude to tell me how to do it. It is much more enjoyable to do a little work on your project and offload the grunt work to a weaker model then just being a highly paid merge pr button pusher for whatever opus decides.
cmiles8
5 hours ago
Don’t know what to tell you. It’s happening.
CharlieDigital
5 hours ago
Local models: if they have the hardware to run it.
Open models: we're already seeing this at some scale. https://news.ycombinator.com/item?id=50350004. I do expect to see more of this.
colingauvin
5 hours ago
What do you consider a local model? I'm running DeepSeek Flash version 4.1 and it's outstanding.
KetoManx64
6 hours ago
You haven't spent much time with the latest self hosted models then. Give it a good harness and instructions, access to a web search engine and ability to iterate on problems and they do a good enough job that I'm happy with the results.
the_mar
6 hours ago
I hate to break it to you but most people do most people do basic things most of the time
jethro_tell
8 hours ago
These companies have no moat at all.
nilkn
7 hours ago
Their moat is their research capability. It's a combination of proprietary internal technology and models and infrastructure, people, organizational structure, and culture. As long as they maintain the best and fastest research capability, they have a durable advantage, at least for the time being. If we start to saturate all economically useful outcomes that can be reached via greater research productivity attainable to us, then this moat will start to fall apart. It could also fall apart if safety issues force a premature halt or major slowdown to further research.
hungryhobbit
6 hours ago
This assumes any research gains last. In practice it seems like none of the frontier companies can keep much of their "secret sauce" secret for very long.
If company X has a better model than company Y, do you use it? Well, that depends: will Y take two years to catch up? If so yeah, you buy X.
But if Y is just as good a few months later, you purchase based on other factors, because you know you can live without the brilliant research X is doing ... since you'll have it soon enough.
cmiles8
5 hours ago
The problem with your logic is that the history of tech is full of “good enough” eating “best” for lunch all day long. Anthropic’s plan to be the best is increasingly looking like a terrible business plan.
honr
7 hours ago
How long till that "research capability" becomes just 5 people with many AI agents, and no other humans in the main research loop?
nilkn
6 hours ago
I don't know and can't answer or predict that. But that's exactly what it could become (although probably not that extreme), where the agents doing much of the internal work are proprietary, conferring a research advantage.
honr
6 hours ago
My point is that it is not a real moat if other players can hire people from core research teams, and get a foothold that easily. And when/if models are good enough, you don't even need many people to form a fully functioning team.
nilkn
36 minutes ago
It may actually get harder to replicate with time, not easier, even if the number of people needed to run the operation goes down. As the largest labs move towards internal RSI, the most important IP for doing that will transition into their proprietary internal infrastructure and models, none of which will ever be released and which increasingly won't be fully known or understood by the humans you might hire away.
andsoitis
5 hours ago
> Their moat is their research capability.
Research does not bring in revenue. Products or services do.
nilkn
31 minutes ago
So far, it does, because the products and services are the general intelligence produced by the research.
pintxo
5 hours ago
You mean, if they keep running quick enough they are save? That's like the compete opposite of a moat, in my mind at least.
tensor
7 hours ago
That's not that big of a moat though, as we've seen with DeepSeek and even smaller teams doing edge model work.
nilkn
6 hours ago
It's very hard to measure exactly how large it is, because there aren't good pre-existing metrics for it. You can measure in time (e.g., how far behind open weight models are from the best frontier models commercially available), but it's plausible that what seems like a relatively small time advantage (months) might mask a considerable difference in internal research sophistication. What we've seen repeatedly is simply that there does seem to exist a durable advantage in research productivity, at least for now.
james_marks
6 hours ago
I’m not sure I’d call maintaining the fastest research program a “durable advantage”.
deaton
6 hours ago
If their moat is their ability to create marginal improvements every few months by releasing new models, then that is an extremely expensive moat to maintain.
chrsw
8 hours ago
I think the staggering amounts of compute and capital at their disposal is a kind of moat.
steeleduncan
8 hours ago
You need those staggering amounts of compute to run an inference service, but you don't need it to run a powerful model.
It doesn't cost a company that much to buy a box capable of running the Qwen3.8-Flash-Next model in the GP comment and never pay Anthropic again. The cost will be even more reasonable once the supply issue is solved for that hardware.
AI-as-a-service is a unique product because when the interface is natural language, there is no lock in. VS code has a hotkey to swap between claude, openai and a local model. Compare that to the difficulty of swapping from macOS to Windows.
chrsw
5 hours ago
I’m not convinced that being able to serve open models makes the closed labs business model non-viable. I think there is overlap but I also think these are distinct value propositions. I wouldn’t dismiss the ability for one organization to point lots of resources at a particular problem, for example.
I don’t have any stake in this and I hate the idea of concentrated wealth and power, wherever it is: government or private businesses. But would I bet against it? Probably not.
jethro_tell
8 hours ago
Sure, but anyone with money could do that.
We're the only people that can spend this money is not a moat.
rdevsrex
6 hours ago
How long they will have access to staggering amounts of compute is uncertain and not guaranteed, especially if they get a junk or near-junk credit status when they go public.
cmiles8
5 hours ago
The capital is running out and the compute is a massive liability on their balance sheet. Your definition of a moat is rather odd.
Ekaros
7 hours ago
I wouldn't be so sure about capital in future. I think there might be real look for return on any new capital they are getting. Be it equity or debt. And that alone will not be great for them.
Tanjreeve
4 hours ago
How much of their compute is leased/rented? Last I read it was quite a lot of it.
pyrale
7 hours ago
the idea of a moat is that it's an inexpensive way to keep the invaders at bay.
If you need to buy staggering amounts of compute, spend staggering amounts in utility costs and lobbying, and keep spending that as your hardware amortizes and your operations go on, what you have isn't a moat, it's a standing army. It's extremely expensive, and the day you start having issues, mercenaries will cut your throat before you know it.
bunderbunder
7 hours ago
That argument seemed to have legs in 2022 when OpenAI was the only game in town.
But the fact that were even talking about Anthropic like this only a few short years later demonstrates that that was only kind of a kind of moat.
honr
7 hours ago
The current compute they are buying is NOT state of the art (check this year's academic research on inference hardware). There is 10x-100x inference-per-watt improvements in existing prototypes (in the industry, not just academia). Not sure what I am talking about? Ask your friendly AI agent of choice, for the details.
Check out mythic.ai for one example.
Rooster61
7 hours ago
compute and debt*
Banks are going to want that "capital" to materialize into gains at some point. The water in that moat is evaporating.
lenerdenator
7 hours ago
Capital only means anything if you can spend it to buy resources like compute and turn those resources into additional value.
Hardware's ridiculously expensive. Then, there's no good place to put it, because the new breed of tech bro managed to propose the data center buildout in the most exploitative, alienating, and electorate-angering way possible. So compute's hard to buy.
William_BB
7 hours ago
The moat is the data they have collected. There is no way that the "de-identified" data from enterprise customers isn't making it into the training set one way or another.
PowerElectronix
6 hours ago
Even worse, their "moat" depends on buying the most overpriced hardware in huge quantities and subsequently eat a huge amortization loss in a few years when it's obsolete.
boothby
8 hours ago
I’m waiting for somebody to use one of the frontier models to hack its own data centre and exfiltrate everything…
bradnation2
33 minutes ago
Curious how much ram you need to run this at acceptable performance
RobLach
5 hours ago
I've got a prototype cluster of 5 devices running Qwen3.8-27B assigned different roles and sharing outputs between each other. Beats Claude Opus MAX in SWE and Terminal benchmarks.
thatsabadlook
an hour ago
There's basically no point in paying for these subscriptions anymore. Sure they are marginally better at some things. But they are also marginally worse at others and the implications suck. I stopped all my subscriptions and keep like 20 bucks in open router just in case. Never need it, but it's there...
cyanydeez
8 hours ago
This. The problem is now the same monopoly cartel problem.
Strix halo is double the cost because of...
jacquesm
7 hours ago
And 4 is about to drop.
scott_weber
7 hours ago
Will be interesting to see how much of an advance 4 will be. I was under the impression that 3.8-flash-next is basically a 4-preview. At least architecturally, but as we're seeing nowadays (with DeepSeek, Kimi 3.1) "continued post training" can add a quite a bit.
benatkin
6 hours ago
The non-anecdotal part is to me as interesting as the anecdotal part. The price of RAM has gone up a great deal which is going to make it harder for others to get this sweet setup that you have.
morpheos137
7 hours ago
I see lots of people talking about running Qwen as a local model just curious what's the difference with say something like local deepseek?
watusername
7 hours ago
Qwen 3.8 Flash Next (125B, 6B active) is minuscule compared to DeepSeek V4.1 Flash (552B, 16B active) and GLM-5.3 (753B, 40B active). It's more difficult to run the latter 2 locally at usable speeds.
ac29
7 hours ago
The latest Deepseek Flash is much, much larger than Qwen3.8 Flash next
scott_weber
7 hours ago
Qwen-3.8-flash-next hardware costs motorcycle level money, DeepSeek hardware starts at mid range car money.
xyzsparetimexyz
9 hours ago
How does it compare to haiku?
UncleOxidant
8 hours ago
I guess it depends on which version of haiku? I really think that right now this is basically Opus 4.8 that I can run at home. In some ways possibly even better.
christkv
9 hours ago
What inference engine do you use ?
UncleOxidant
8 hours ago
Gufo. It's the best open source inference engine that's optimized for Strix Halo right now. Halogen may unlock a bit more performance, but it's not open source and gufo seems to be making steady progress.
Infernal
7 hours ago
Cool. I found halogen but was turned off by the closed source nature (github repo that just turns out to run a download shell script, lame).
Do you mind sharing more details about your setup? I have been using opencode with lemonade for inference on my strix halo but haven't quite hit the sweet spot.
UncleOxidant
2 hours ago
I'm using it in opencode through llama-swap.
Here's the gufo command:
gufo serve 1lm \ -m /mnt/LLMs/models/unsloth/Qwen3.8-Flash-Next-GGUF/UD-104_XS/Qwen3.8-Flash-Next-UD-1Q4_XS-80081-0f-00083.gguf \ --served-model-name Quen3.8-Flash-Next-UD-104_XS \ --host 127.8.8.1 --port 10083 \ -c 262144 -n 16384 -j 1 \ --prefill-chunk 8192 \ --speculative mtp \ --mtp-model /mnt/LLMs/models/unsloth/Qwen3.8-Flash-Next-GGUF/MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf \ --cache-disk ~/.cache/gufo-fork \ --cache-disk-bytes 68719476736 \ --cache-disk-staging-bytes 8589934592 \ --cache-ran-bytes 4294967296 \ --log-progress
FloorEgg
8 hours ago
Do you ever worry whether these open source models have been trained to embed backdoors or leak secrets in very specific circumstances at some point in the future? The kind of thing that's hard to know ahead of time and even detect after it's happened?
I feel like the industry hasn't solved that alignment problem yet... We have no way to verify the entire models landscape of intent.
Paid models have an incentive not to do that because if the market discovers it there is a huge economy penalty, but the free open source models don't have that disincentive.
neya
8 hours ago
> Paid models have an incentive not to do that because if the market discovers it there is a huge economy penalty, but the free open source models don't have that disincentive.
False.
I worry more about Claude.
For those keeping score, this probably isnt exhaustive- but here's some greatest hits;
#55743: Claude Max account email auto-injected into LLM context with no opt-out - May 3, 2026 https://github.com/anthropics/claude-code/issues/55743
#72518: Anthropic embedded spyware in Claude Code - June 30, 2026
https://github.com/anthropics/claude-code/issues/72518
#21151: No indication of WHICH file for READ tool - January 27, 2026
https://github.com/anthropics/claude-code/issues/21151
#42796: Claude Code is unusable for complex engineering tasks with the Feb updates - April 2, 2026 https://github.com/anthropics/claude-code/issues/42796
#73408: AFK auto-continue on unanswered AskUserQuestion is undocumented and has no opt-out - July 2, 2026 https://github.com/anthropics/claude-code/issues/73408
(Also worth a mention, OpenCode PR #18186: anthropic legal requests - March 19, 2026 https://github.com/anomalyco/opencode/pull/18186)
https://news.ycombinator.com/item?id=50387628FloorEgg
6 hours ago
> False
What exactly are you claiming is false about my statement?
The list you presented are mistakes Anthropic made or cases when they put their own interests ahead of users, but I don't see how it disproves the point I was making. None of those are examples of embedding backdoors or stealing secrets to take over or sabotage a system. They are categorically different than what I'm asking about.
ericmay
6 hours ago
Where's the huge economic penalty?
Catloafdev
8 hours ago
> Paid models have an incentive not to do that because if the market discovers it there is a huge economy penalty, but the free open source models don't have that disincentive.
This just fundamentally isn't true. They run on the same economic model, but they have to release the weights to gain credibility, unlike the American labs.
> Do you ever worry whether these open source models have been trained to embed backdoors or leak secrets in very specific circumstances at some point in the future?
Open weight models are extensively analyzed. It is extremely unlikely there is an intentional 'backdoor when X' concept added to any of them. Not impossible, but unlikely for the same economic incentives you see in the other labs, and given that no evidence of anything along those lines has surfaced yet.
ericmay
6 hours ago
We've seen many open source software libraries, frameworks, and hosting platforms with vulnerabilities, backdoors, or leaks.
In my opinion, to suggest open-weight models are extensively analyzed while being net new and continuously released and the model itself being unintelligible to humans seems a bit premature.
It's possible that closed-weight models end up following the pattern of being arguably less secure since there are fewer eyes on them analogous to open source software but I don't think there's a perfect match here.
blocked_againnn
8 hours ago
Should be soon solved with open weight model providers offering enterprise support.
UncleOxidant
8 hours ago
No, I'm not much worried. I think this is mostly FUD at this point to try to get people to not run local models and keep paying the "trusted" providers. But paid models have been exfiltrating data - there have been several stories about these incidents here.
deaton
6 hours ago
"trusted providers" don't even need a backdoor, they just collect everything from you and whether you approved or not they're using that info to train the next model.
gonzalohm
7 hours ago
I feel that way but about Anthropic and OpenA lol. Didn't they just get caught sharing your email with every website every time it does a search?
phendrenad2
7 hours ago
Backdoors in model weights are mostly theoretical. And the "alignment problem" that the AI labs love to talk about really doesn't exist either, at least, not the way they use it. The real alignment problem is getting an LLM, which is just math, to obey arbitrary human rules. In other words, LLMs do not have "alignment" at all. However, AI CEOs abuse the term and use it to mean "AI has an alignment, just not the one we want".
CamperBob2
8 hours ago
I don't ship AI-written code without at least having an unrelated model of equal capability review it. That should cover a multitude of sins like intentionally-embedded backdoors and zero-days...
... as long as the reviewer model hasn't been deliberately lobotomized, that is.
(I don't disagree with the [dead] comment, either -- fortunately I tend to make small changes to code I'm very familiar with, so it's fairly easy to vet in person. The more code gets generated by the clanker, the more nervous I get, just because it's impossible to keep some complacency from creeping in.)
figassis
7 hours ago
I don’t think parent was talking about the shipped code, but about your one development environment while using the models.
scott_weber
7 hours ago
OP could be read either way, but the dev environment threat is not rocket science to manage: LLMs can only do what the harnesses their responses are embedded into explicitly enable them to do [1]. And once the setup is secured, it tends to stay secured without additional effort, unless one explicitly wants to alter its granted capabilities.
Auditing output by contrast is a continuous cost. On the third hand, it's one that most will want to pay anyway.
The "embed into shipped code so subtly humans can't spot it" vector is sci fi ish. Less plausible than old fashioned trusting-trust issues, and most folks have managed to sleep at night for the last 40ish years that spectre has haunted.
[1] I guess the model could return output that buffer overflows the harness and exceeds its intended ability, but that seems far-fetched too
jdjdjdjjdbznnx
8 hours ago
insane. I can't imagine doing this and I hate that I'm being forced by upstream communities to run code nobody has read
just read the damn code yourself!
hattmall
7 hours ago
>just read the damn code yourself!
It's basically impossible to do this. Claude produces so much excessive / verbose code and comments that reading it would take far longer than simply doing the implementation "by hand". It's kind of like, Claude can write the code in 1/2 a day. It can be reviewed by another LLM, unit tested by another for 1 day. Or I could write the code in probably 3-4 days. Reading it and validating / testing it myself though would easily take 6+ days.
Now if I were fully in control I wouldn't run code that wasn't truly reviewed at all, but in the situations where Claude is being used that's not my call. TBH a lot gets caught, and I have some automated scripts that find a lot of the more problematic stuff automatically, flag it, send it back and make note for the reviewer model as well.
I will also spot check areas where I think an issue could arise.
mplewis
7 hours ago
If you can't read the code, you aren't being responsible enough to ship the code.