They're not competing against each other, they're competing against free/local/open weight models that they're also lobbying very hard to regulate/ban.

The brutal free market for job seekers like me but regulations and safety nets for thee

Price per task has been dropping exponentially for years. This isn’t a sudden new phenomenon. They’ve been competing with each other continuously. There is even more competition now than ever before.

How could they do anything but implode financially? Genuinely curious. The numbers are not right.

That's why Anthropic are rushing the IPO. You can explain a single quarter slowdown, you can not explain a multi quarter one.

> or they will implode financially

They are too big to implode.

> If this is mostly competition enduced, I have to assume either their usage has plummeted or they will implode financially in short order.

If we assume it's due to competition, then their usage has precisely not plummeted because they cut prices by 80%.

How?

At this point I'd rather try to invest effort and usage in a harness that is FOSS and is actually steered by a non-profit. Goose comes to mind and to be honest I haven't tried it beyond installing and giving it a prompt or two.

Comiong to harnesses, even Pi seems like a weird choice. I mean Pi, until it's "your own customised Pi", is pretty useless and pointless. Besides they are also starting to have their own models/infra business (or is it mostly proxy like OpenCode?).

I have also given up on Herdr and came back to Tmux (and doing a bit of customisation slowly, as I need it). I wanted to try dsh but somehow couldn't brave it. Maybe once I have a good container/vm setup is ready. I know I am biased/discriminatory here, but there are things about which I'd be better safe than sorry.

As for OpenCode: I used V1 a lot and I am struggling with V2 currently. Yes, it just doesn't feel right. It's neither here, nor there. I am not a harness reviewer, so I won't be able to put that into industry accepted words and tests but it doesn't feel useful and right to me. At least Pi is completely stripped (it might be changing though) and then you can make it as you want it. Even Claude Code is better than OpenCode and that's saying something. I find Codex to be quite effective but it doesn't feel familiar. I am still in the middle of some work so I am using OpenCode v2 a lot (and then there's OpenCode Go and I won't want to play around a lot with it) but I will move away. Where? I am not sure yet.

What I feel will work for me? I guess a harness of my own! Forked something smaller/similar? I don't know. No, really! Just like I have never found the "my" notes app and journal app yet, I don't think I should use any of these harnesses if it doesn't work for me, especially now that I can kinda make one (hail LLM/vibe coding :P). It doesn't even have to be a TS/Node monsters most of these harnesses are. More I am reading about, digging deeper (haven't been quite far yet), how models and the "local setup" interact, the more I end up scratching my head – why all this drama and why all these 5 billion GitHub star "harnesses" with trillions of lines of code! Why? It doesn't make sense!

I have a harness for my harnesses now. My Hermes manages Codex, CC or Cursor now and interact with me via Slack and PRs on GitHub.

It's nice because I can keep going from anywhere, other team members can see and interject in my Slack threads, and Hermes auto updates its skills with any adjustments that we make so we don't have to ask twice.

I found Claude Code to work best. You can use it with any compatible API.

Personally I've found Zed's agent to be much superior to OpenCode. IDK why but it gets better results from the same model on the same task. Using with GLM 5.3 and DeepSeek 4.1.

Pi.dev at home.

And I’m slowly writing my own. But it’s garbage at the moment.

For me, I used Zed Agent with Codex as it has native integration with Zed's ACP so it's pretty convenient.

However, Anthropic is pickier about the harnesses you can use with Claude Code and I noticed discrepancies when using Zed's External Agent feature with Claude Code, so I just use the Claude app. It's fine.

Crush is great and easy to hack on: https://github.com/charmbracelet/crush

I've been pretty satisfied with OpenCode 2

Massive increases in efficiency and https://en.wikipedia.org/wiki/Jevons_paradox

If the beef producers don't believe beef is a commodity and they want to get you addicted to their specific beef then it makes sense. Right now in these companies life cycles profit doesn't actually matter. The only thing that matters is how much of their beef is being consumed. They want to give people so much free beef they can't even imagine the idea of having a meal that doesn't contain beef.

Missed the memo. Whats the proof of positive free cash flow?

OpenAI and Anthropic have more compute and the best frontier models to throw at the optimization problems. They have published articles on these efforts. For example, Dots, and Anthropic's upcoming "persistent agent" is because of these optimizations. Both companies often publish articles detailing a lot of their technical efforts. They just get buried among the flood of other news.

Yes, I would not be shocked either.

Sonnet 5.5 is 16x more expensive at $10 per million output tokens. Kimi K3.1 is also $10 per million output tokens, and its number of active parameters is only 6.5x larger than V4.1 Flash (104B compared to 16B). These things are not directly comparable of course, but just to give an idea.

Even if inference of those models needs 16x, 32x, or 48x the number of GPUs compared to V4.1 Flash, still seems like potentially great revenue per GPU.

They published all/most of their work and it would be unbelievable if Anthropic had not already used that research in their own architecture.

What is the TPS of DeepSeek? All of the LLMs I have tried are slow as molasses. \

> its per GPU-cluster, not a single gpu.

No, it isn't. It's revenue per hour per GPU.

Based on translating their Chinese text on Firefox, it says "per card":

> The current service throughput of a single cluster is 155 trillion tokens per day, and the overall revenue is $38 per hour (at the lowest price) per card.

With Kagi translate (best), it says GPU

> A single cluster currently serves 155 trillion tokens per day, generating overall revenue of $38 per GPU per hour (at off-peak electricity rates).

https://zhuanlan.zhihu.com/p/2095560295930508953

Free markets don't exist to help individual companies, they are explicitly meant to hinder them.

They are meant to help the consumer.

Unfortunately neither of the two parties in the US even pay lip service to free markets anymore.

Any sources on training costs in free fall? My understanding was training costs are proportional to model size. Is it just that compute costs are falling?

I would think that access to power is more critical than compute

I too would like security issues to stay unpatched because only three letter agencies have access to uncrippled tools for finding the bugs.

I think that the frontier labs knew what they were doing when they invested in cybersecurity starting in the spring (or maybe earlier). They knew it would create hype. They knew it fit with their narrative that they are building a dangerous thing (something they believe because it’s fun to believe in doom, because it’s a great story for investors, and because it’s not inconsistent with a moat - in fact it makes a moat via regulatory capture).

“Black October” sounds a lot like Red Scare.

Well, OpenAI is at the root of Black October, so maybe they are the problem

You remind me of those politicians who try to scapegoat people who point out security holes. https://www.theregister.com/security/2022/02/15/missouri-end...

Regulate the bad actors, not the neutral tools. Lockpicks aren't illegal, but breaking and entering is. Imagine if MasterLock used a lockpick to break into a government building and steal stuff, then tried to lobby to make lockpicks illegal. That's what OpenAI is doing, and you're falling for it.

In the hands of 4th party system Progressive Republicans and 5th party system New Deal Democrats, government was an extremely powerful force for trust-busting.

In the hands of 6th party system Republicans, namely Ronald Reagan and Robert Bork, the government worked hard to usher in a a new era of Robber Barons. We live in the culmination of this project.

History says the result depends on the hands, not the tool. 6th party system Democrats have largely failed to undo the Reagan/Bork policies, but that's an act of omission not comission and even then Lina Khan was a big step in the right direction. This issue was on the ballot in 2024 and will be again in 2028. Hopefully we make better choices in 2028.

If you take away all regulation you’d probably end up with a monopoly anyway, as companies would no longer be prevented from merging with their competitors or using underhand tactics to gain market share.

This is the goal of some, but very few overall, government regulations. Not even close to "all".

That's wrong, but why let reality get in the way of a pithy ideology?

Government regulations exist to regulate activity, to prevent, punish, and/or remediate harms.

Environmental regulations exist because businesses wouldn't clean up their own messes. Financial regulations exist almost exclusively because of harms and abuses committed by members of the financial industry. Building codes are updated to reflect lessons learned about people died, to prevent more people from dying the same way.

If AI regulations are passed, it will be because the AI industry decided not to self-regulate itself and stop illegally hacking computer systems, generating CSAM, or various other illegal activities that should easily be preventable if their systems were actually intelligent.

“All” is doing a lot of work there.

Also there are real trade offs. Food would be cheaper if, for instance, it were legal to cut it with any old cheap substance regardless of health effects.

But I totally agree that some regulations are dumb or are regulatory capture monopoly schemes.

Unfortunately like many other things there is no absolute simple rule to save us from thinking. Each case must be evaluated in depth hopefully with domain expertise.

DeepSeek hasn't proven anything except what they say. You're inherently trusting a company's statements over others. When an independent third party corroborates the numbers then we know for sure. Until then, every company is just selling a story for investors and users.

Does DeepSeek offer a heavily subsidized subscription that provides 20x+ more usage than the equivalent value in API tokens?

API usage being profitable is completely plausible, but almost nobody pays API prices for Claude and GPT, nor would they be able to afford them if the subscription suddenly stopped existing.

Not DeepSeek themselves, but their open models are run by third-party companies in a profitable way. The unsustainable interference theory has been debunked.

The conditions for DeepSeek's profitability are very very narrow. A key piece being that they have cheap energy, a second key is that they have west/east clients and thus extremely high uptime.

Except Scam Altman has stolen from literally everyone to train his models.

Agree with you, I do

Language modeling is fundamentally compression: https://arxiv.org/abs/2309.10668

The entire endeavor is ultimately fundamentally coupled to reducing compute

Eventually we will compress models further by removing outdated knowledge like Python and Cobol cause who needs 1960s like languages to then layout data in the screen when the model has knowledge of geometry and can mutate the screen geometry without needing a Python interpreter or C compiler

Yes - exactly.

Because all these newly trained models using the compute are doing really well and growing!

A historical episode that may arguably be related -- regarding stock market euphoria & overbuilding capacity -- was the dot-com era Telecom bubble & crash [1].

Even if something is a reasonable infrastructure project with a sound long term business plan, investors can still lose money if they pay too much for it. For the Telco bubble & crash, there were a few of these Telco companies engaging in financial shenanigans / securities fraud to juice their quarterly financials by leasing capacity to artificially inflate their operating revenue etc. E.g. we booked $100m of revenue by leasing X units of our capacity to $competitor! Sure, we're still extremely cashflow negative, but look at our operating revenue grow! Fine print: we also incurred a symmetric $100m expense to lease X units of capacity back from the same $competitor, but for some reason we didn't classify that as an operating expense, we've thrown it into a different accounting bucket.

That said, overbuilding fiber optic capacity by laying undersea cables is likely different to overbuilding GPU datacenter capacity in that most of the cost of laying a cable is the physical cost to lay the cable -- given you're already laying an undersea cable, the incremental cost to add a bit more capacity to it (more optical fibers) is negligible -- at least before you've sunk the cable! Whereas for a data center full of GPUs, maybe 50%-60% of the total cost is the GPUs, so it isn't cheap to massively overbuild capacity.

[1] https://en.wikipedia.org/wiki/Telecoms_crash

Are those statements audited or fact checked? Inference profitability is great but how much of that profit gets eaten up paying back training costs, R&D, and operating expenses.

At best, this tells you v4.1 flash is profitable, but can they really get $38 per gpu hour? Sounds like a lot.

V4.1-Flash is probably quite smaller and much more resource effective than the frontier models. Add to that much cheaper energy in China.

Clearly potentially profitable. Not clearly profitable.

How much did it cost to train Deepseek v4.1 flash?

SPCX had a pretty wild ride so far with a jump to 200 a few days after IPO and a drop down 108 a few months ago so I'm not sure the market really knows how to properly price SPCX right now.

The same evidence could just as possibly point to supply being too great for demand. The OP article references this as a distinct possibility: if models really have gotten more compute efficient, we may wind up in a situation where we have more inference capacity than we actually need.

To give a simplistic example: if a city as a ton of available AirBnBs, you might say, "Lots of people are willing to rent out their apartments, it must be profitable for them to do so." But the fact that all of those rooms are not already booked could also mean that way too many people have attempted to rent out their apartments.

And for what it's worth, it's possible for supply to outstrip demand while still leaving inference profitable. The question is whether it's (a) profitable enough to service massive debts and (b) sustainably profitable over the long run.

Edit: to add one note, it's possible for the LABS to be profitable on inference without inference itself being profitable. We don't know what the contracts look like, but it's entirely possible that labs received sweetheart deals from neoclouds so that the latter could flaunt big utilization numbers.

Solar panels (which continue get cheaper every year) on my house makes charging my low-TCO EV free in 5 years

If you make it easy for lots of people to make the fuel, yes. EVs and smaller scale power generation anyone?

That would likely require innovation in a renewable liquid fuel that can work at scale.

Electrification will drive the demand for oil down, making the overhead more expensive per gallon, and eventually we will largely move off of fossil fuels, though I don't expect completely. It will be many decades due to the turnover times for industrial/commercial transportation systems.

Yes if China somehow gets the fuel and subsidize it to sell it for cheaper in the global world, even the gulf state price will go down.

Tokens per second? That only really matters for things like chat and sessions where you are monitoring a session as it runs which is going to be an increasingly small percentage of usage. That's bottlenecked by human attention. Agents in the background, long running tasks, unattended tasks, all that stuff is what is even capable of driving significant usage because you can just increase consumption massively by parallelizing without being the bottleneck. I wouldn't be shocked if already the vast majority of token usage was from unattended sessions. TPS is not that important there

DeepSeek is serving more TPS than anyone, about as much as OpenRouter does in a week in a day, at 38$/H800/Ascend so about 900k/~90% profit over 3 years (or ~80s% profit including training expenses).

Meanwhile, Oracle is handing Tencent 100.000 GPUs at 1.60$ hour, dangerously close to breakeven...

What does this tell you?

Wait what, OpenAI is trustworthy now?

It won’t be a ban, it will be some “certified safe” badge from a federal agency. Companies will be scared into not using anything that is not “certified safe” and business insurance will even require it. Whoever (openAI or Anthropic) gets to the politicians first will be the only certified safe model provider.

Maybe the FDA is a good analogy, you wouldn’t want to buy meat at the store not FDA certified would you? OpenAI and Anthropic want you to feel the same about AI.

Not just models, won't be long before hardware is regulated too.

Sounds like an eventual Supreme Court case.

I do not expect an outright ban. (1) the bots are too useful, open is the way, re: HF handling the incident (2) you can train without certain capabilities, see Kimi K3.1 (3) reminiscent of the encryption is dangerous debate of the '90s

There will likely be requirements like registering with the government (China already does this, every business already does everywhere), certain usages require more or have deeper regulations (this is already done with most technologies, depending on application)

I can't believe the actual answer is so far down the thread. All praise to our overlord DeepSeek for ironically helping American labs make cached tokens much cheaper.

"Unlimited" for a consumer will probably take a form like limited parallel agents at max intelligence for X amount of tokens, followed by either single threaded agents or nerfed intelligence or both.

This is the same mechanism used in unlimited mobile internet for example.

> if you get unlimited tokens, with scheduled/automated jobs and subagents you will burn a LOT of money

Maybe, maybe not. There could be a saturation point of compute demand from ordinary consumers. (See: hot-water analogy.)

> You can only spend so much time speaking on the phone.

Tell that to my dial up modem as I played Active Worlds in 1996.

Mobile video data and tethered devices would tell a different story. Usage growth has been significant.

I expect they will be "unlimited" in the same sense the mobile / ISPs do that 99% of people never reach the limits.

We could have "unlimited" electricity too, if we choose, by printing solar panels (like the Chinese) and hopefully fusion before long.

> The cost of serving a model grows linearly with use (energy)

The cost of processing tokens however does not. Throughput and efficiency are improving around 10x per year. Hardware is not the only factor.

Power and memory are apt to become the biggest billing constraints.

Something close to how we rent other cloud resources today makes sense. We don't worry about electricity costs, it's built into the hourly prices, use as much of the hardware as you can. Electricity rates vary over the course of the day as well.

People don't like complex pricing. This would be for businesses running larger workloads. For individuals, the unlimited plan seems the most probable to me. This stuff is going to get real cheap in the coming years.

The energy needed per token is in dropping rapidly, even though the energy per-GPU is going up. We are seeing something like 10x improvements per year.

correct.

Commercial airlines are of great value to our lives. Who's questioning or denying that?

Horrible business to be in though.

People get obsessed with wanting companies to profit ASAP.

That's not how you grab market share. Early on, any profit you aren't reinvesting into the business is basically hurting your growth rate.