Those data centers are specifically for AI workloads. Let’s say everything crashes and we now have all the data centers, what do you do with them? GPU are pretty specialized hardware, without AI a data center full of outdated graphics cards isn’t really too valuable.

It’s really not obvious the infrastructure we are building for AI stuff is something that will benefit humanity over time.

Without talking about the fact that bubbles are extremely destructive. Bezos is obviously someone who came out ok from the dotcom bubble but we are talking about something that destroys a lot of value globally. That has real, direct consequences, not just investors losing some money. The US economy is currently only growing because of the AI bet

> Stocks going down didn't un-research drugs

Drugs cost pennies to manufacture after they are researched and make their way through the approval pipeline. There are many generic drug manufacturers who can work off the existing formulas.

The more apt comparison is that LLMs won't be un-trained. Opus 4.8 now exists. Even if Anthropic somehow went bankrupt, that particular asset could, at the very least, be sold for proverbial pennies on the dollar to a "generic" inference provider.

Datacentres aren't the same as infrastructure or research though. All the hardware in them has a finite, useful lifespan. In 10 years time it'll be totally useless

Hardware fails, and also scales out in terms of efficacy to run it as more power efficient, modern hardware turns up. It requires constant investment to keep it useful, and cost efficient

When AI pops, we'll temporarily have some extra compute capacity that will be horrendously uneconomical to run due to the high grid load and low consumer demand, before they get shutdown. There's simply no real use for them at this scale

In order to not un-build the data centers, they at least have to make more than it costs to operate them, and also not have some attractive liquidation value (the land, maybe).

I could imagine something like “inference is done at home or in China, that’s the price to beat” and it’s not worth keeping all those GPUs cool out in Nevada.

I'm surprised people think LLMs, a thing which mainly excels at advertising, spam and writing code is going to generate that much economic activity.

A few things, I think you’re missing the point here

- most tasks do not require the latest frontier models, even if they are a magnitude more intelligent (we don’t actually know if that will be the case). Current Gemini flash is cheap, fast, and pretty capable with good guidance for most tasks

- now that companies pay API costs instead of a subscription they will be setting restrictions on token use to not have their budget explode (like Uber in this submission), that’s a strong incentive to NOT use expensive models, and limit their thinking budget

- there is competitive pressure from China and others who can offer very decent performances at a fraction of the token price

- the price of tokens for the frontier models is likely to go up, but the price to access older models is what depreciates! The overall price per token is going down now that we are in a new world where companies understand that token maxing is one of the stupidest concept ever created by humankind.

The $1T number seems more promises than reality, which is closer to the $300B to $500B level. Still a big number, but between a third and a half of the value used in the popular media.

These are similar numbers to the dotcom bubble. With GDP growth and the percentage of productivity AI contributes staying the same in this scenario this requires regular gains in revenue or growth. If things just stumble, like with most datacenters going unbuilt the bubble will pop.

For ~equivalent tasks, or because we’re expecting more from tasks?

The real measure should be cost per ~equivalent task, not cost per token nor tokens per task.

Local privacy respecting inference can be worth it. I use a local model to log everything I do all week to automate my timesheet. I also have it do a bunch of other data tasks. I won't say that larger SOTA models wouldn't do these tasks better than a local model but PII is a concern and my employer wouldn't approve of me just setting tokens on fire everyday to do what I could do myself.

> more work for the user

Model routers allow this to happen automatically without any more work by the user.

> a shittier model

A ton of tasks don't require the most expensive frontier models, etc.

> I’m not sure why anyone does it

1. Faster solutions from the LLM - also reduces employee costs of having the employee waiting on the LLM

2. Avoiding things like the half-billion dollar per month bill for a single company’s LLM use recently reported in Axios

What you call a shittier model is what was considered frontier and fantastic one generation ago…

I sometimes let Claude Opus create plans, DeepSeek v4 pro implements and writes tests. Claude reviews and corrects.

Saves like $2-3 per session. Same quality code.

Chips age and fail with age. You can check hot-carrier injection, bias-temperature instability and electromigration as they are the main aging mechanisms. All if these are a linear function of time but exponentieal of temperature. 90-100C these chips are running at are really tough, so they are likely to fail at couple of percent to 10% range in 2-3 years depending on the margins they have in the design.

The solder joints are notorious to fail at a high rate too.

There are data centers that use and rent out 10 year old server GPUs.

They can't run larger modern models. They can't run smaller models as fast as newer servers. So their remaining market is applications where customers are okay with older, smaller models and slower performance.

They have to price the service lower than competitors due to the lower performance. The older GPUs are less efficient so it costs them more to keep them running. They're paid off, but they're taking up valuable power, space, and cooling in a data center.

Eventually there is a tipping point where it's better to replace that space and power budget with something new that has more demand.

The parts are sold off on the open market. There's an equilibrium demand for the parts from other data centers keeping older servers running and from hobby people who are okay with a jet engine sounding toaster of a GPU running in their home.

In addition to the physical depreciations other comments mentioned I'd also mention that old chips will settle into a low price and then actually go up on a per unit basis if you're trying to buy a significant amount of them. With a limitation on fabrication facilities continuing to pump out older cards is an opportunity cost to the manufacturers that would prefer to be producing newer cards. If you were in a place where you suddenly wanted to buy 10,000 3080s, as an example, I'm not certain if the market could actually fulfill that demand and no one with the ability to increase the available supply to meet that demand actually wants to do so.

Chips do wear out and need to be replaced (entropy do be like that and durability is not a primary concern for chip design) so you'll need to refresh your stock and, even if you don't need cutting edge models, the price of all chips at scale will go up over time. It may feel unintuitive since, when the PS3 was released PS1s were extremely cheap - but if you're struggling to understand this effect from your experiences in the consumer market you're actually looking at the price factor that starts making antiques increase in value since at a certain point they become scarce goods. The market price for an NES is higher today than it was in 2003 because the price had already bottomed out from demand from the general consumer market but the demand remaining (speedrunners and the like) is now fixed or growing while the supply is inevitably shrinking.

Gradually, and especially when hot. Modern chips are pretty close to the physical limits of how small they can be made, and that means atomic/chemical effects like electromigration are accounted for and determine the lifetime. Every extra 10 degrees Celsius of temperature doubles the speed of chemical reactions.

When they stray too close to the line ... you get Intel's 13/14th gen chips that wear out after 1-2 years instead of 10-20 years. Intel calls it "Vmin drift" because that doesn't sound scary, but the actual point is that various wear-out mechanisms push the chip outside of its design envelope - increasing the voltage or lowering the clock speed may get it to run for a while longer, but you're living on borrowed time as the various circuits just stop working right and you get unpredictable instruction mis-execution: https://fgiesen.wordpress.com/2025/05/21/oodle-2-9-14-and-in...

I used to work in datacenters, during spinning disk era we had technicians from vendors basically every couple of days to replace some broken part. When the massive switch to ssd happened instead of having them every couple of days it was 3 or 4 times per month.

Despite no moving parts things broke anyway and, even if it doesn't break, the vendor can make you change the technology just by playing with maintenance cost of the older one, limiting or removing spare parts from the market.

They do degrade physically, but the bigger thing is they stop being competitive quickly. Each year or so we see doubling of GPU speeds for the same amount of power.

If you build a 100MW data center with GPU compute and three years laster a new data center opens with the same cost for GPUs and same electricity cost you do, but can do twice as much compute, you quickly lose business unless the market is just so constrained customers can't afford to be picky. But the moment there's slack in the market you'll see major migrations off of providers that have the same cost but half, or quarter of the same performance.

So when you see someone talking about GPUs fully deprecating in value in 1-3 years this is what they're talking about. Right now it's not a big deal because there's no slack in the market. But once there is, the bottom will drop out.

Nothing is stopping them, it's just not worth it: Have a look at e.g. vast.ai's pricing (https://vast.ai/pricing).

The V100 (2017 -> 9 years old) can be rented from $0.02 to $0.37/h (right now I can find a V100 with a Xeon Gold 6140 and 48GB RAM for $0.165/h). Let's assume the guy you rent it to pins it at its 250W TDP and let's ignore the running costs of CPU/RAM/etc... Then you draw 1/4 kwh for that compute hour. The industrial electricity prices in the US vary between 7.5 and 25 ct per kwh (depending on state, time of day, etc...), so at 100% efficiency, assuming nothing ever breaks, and the CPU consumes 0W you earn about 14ct/h.

And remember: V100s hours are sometimes sold at 1/10th the price.

If I pick average conditions you need to start thinking of whether it is worth it to rent them out: Usually it isn't unless you have them anyways and just sell idle capacity.

It's barely worth it to run them in a pure "is it profitable" sense, if we also account for the opportunity cost of taking up a slot in your datacenter it seizes to be worth it really quickly.

I assumed the issue was similar to crypto mining, where given finite amounts of space and power it makes sense to always be running the latest and most powerful GPUs instead of keeping older hardware running. There's definitely a secondary market for these GPUs as well.

Today's data center GPUs are essentially overclocked, and so at limit of how much the chip materials can physically handle, and therefore degrade over time. For example, GH200s operate at 1W/superchip but the actual safe power is somewhere around 650W which will allow them to function for a decade or more. But that leads to around 15% slowdown and that is unacceptable in today's competition. So current GPUs are destined to be depreciating assets.

In future, we might have fixed cost GPUs but not today.

Chips do deteriorate and fail naturally at datacenter scale or in timescales of decades, though not exactly like on financial reports. Leak current increases or electro-migrations occur at junctions or whatever those words mean.

And yeah, it does feel like GPUs will start losing values slower going forward with Moore's Law being dead for a while. It used to be that 3-5 years old GPUs were more useful as space heaters than GPUs, but that's much less of the case today.

> There are no moving parts, I dont think memory chips or GPU chips deteriorate naturally

I believe they do, but I too would love to know more details because there are several ways this can happen. Electromigration, package failures, VRAM failures, dielectric breakdown... Hopefully there will be studies soon similar to that old Google paper on HDD failures!

GPU do depreciate indeed, but here the depreciating commodity is the token, not the hardware. You sell cheaper token with the same hardware

the hardware itself is still useful, but random failures happen every so often, so if you're trying to run a fixed sized fleet then your fleet shrinks when you can't get spares any more

Your laptop doesn't have a 100% duty cycle. If you ran it like a data center it would indeed wear out much faster.

Transistors do wear out. Not going to elaborate as it is easy to ask GPT

When it was profitable to mine crypto with GPUs people used to sell these miner GPUs on the used market after about two years.

These were about half of the cost of an used GPU just used for gaming. By that pricr, I'd say a GPU kept busy has twice as high a chance of failure after two years of use.

Not great, not terrible.

Yes, even if the hardware is untouched. As technology advances, the power cost per compute cycle goes down. A gpu using old tech costs progressively more to operate compared to the newer models. So its value goes down over time = depreciation.

As for duty cycles, the chips are perfectly happy at 100% operation. Cooling and power componants fail, not the chips. But it costs manpower to repair such things and manpower is inconveniant these days. A gpu with any sort of fault just gets dumped.

The “you wouldn’t download a car” meme applies here

They'll ban them because (unless run locally or self-hosted) they are just data capture tools for the China.

So, have you ever been to China and could hadely found anything familay?

- Oh, they must have been blocked from entering the Chinese market!

But none of that is true. You could see global brands everywhere here — Tesla, Unilever, KFC, Apple, and so on.

---

Or have you ever actually done cross-border trade? Or any international business collaboration? If you had, you’d definitely realize that what’s really stopping you is U.S. legislation. At least, that was the case with our former U.S. partner

Google also needs fabs to build their TPUs.

From what I understand it’s mostly TSMC and the memory providers being out of capacity over the next few years.

So it’s not even about datacenters.

Here’s a Reuters article about TSMC: https://www.reuters.com/world/asia-pacific/broadcom-flags-su...

So this is actual committed contracts with all kinds of companies such as Apple, NVidia, AMD.

Also, the whole reason they can’t build data centers faster is precisely because of this.

> they can't build datacenters at anywhere near the rate they want to

That was because the supplies the datacentre needed were constrained - supply-constrained, not end-user demand constrained, so would be in agreement with the GP comment (and the article I read didn't imply anything about lying).

Today's frontier models will be tomorrows low-end option. I think whatever model you are using today will be less expensive to use a year or two from now.

They aren't going down, but in the meantime they'll cover their ass by bribing their way into the S&P 500 and then use your 60 year old mother's 401k and teacher's pension to fund their risky capital expenditure.

A key point here is open in terms of being able to download and use it, not open as knowing what data and instructions were fed into it when training.

A paranoid part of me thinks that these models are all inherently biased and instructed to be pro CCP, with specific gaps in their training data related to undesirable historic events and political ideas.

Here's a free startup idea: operate an open-weight model service, and offer "Verified AI Integrity," which signs the input tokens, the seed for the randomness in selecting outputs, and the model ID, proving that the result of the call to AI was completely "organic" and was not interfered with.

Your main audience would be snake oil salesmen trying to prove their AI products are unbiased and not under the thumb of any outside influence. This doesn't address the biases of the model itself, but that's not your business. Your business is selling tokens and security certificates. If you can get the right angel investor, you could maybe have your new standard required for some government applications.

Yes, you can. There are multiple inference providers out there. The problem is, it’s hard to beat the Chinese providers in cost. And you also have to compete with frontier model providers’ subsidized offerings.

There are plenty of US-based inference providers available, including AWS, that serve Chinese models at competitive prices (vs frontier US models). They also have lots of usage. Not necessarily for coding, but for other enterprise tasks.

Have you heard of openrouter? There's 1000 of these companies already. Do something else.

It's called AWS. Bedrock is right there. Price or data policy is never the issue. The models themselves are the problem -- most large US companies are not going to touch them.

Source: directly involved in these discussions. You can downvote as much as you'd like but you can't ignore the facts.

> Deepseek has some models in Bedrock.

Just looked into it, seems like at most they have just 3.2, not 4: https://aws.amazon.com/bedrock/pricing/

Looking around their catalogue more, most of their models seem quite outdated, aside from the OpenAI and Anthropic ones (but those get more expensive). I wouldn't willingly pick Bedrock and would instead throw money at OpenRouter, that has both a bunch of providers, as well as almost any model for you to try.

Makes me think of how my Claude.md files specifies to use the built in framework code-generators (rails). Those generators are deterministically right every time.

I wonder how often the Agent actually follows the guidance. I do see them follow it when I look. But it doesn't seem so every time.

A lot of the time if you're copying code from one place to another what you actually want to do is abstract it so you can reuse it in both places.

The LLM can easily do this type of stuff, just tell it and it'll happily do it. This is exactly what I mean when I tell people they need to work closer with the AI, tell it how to do things. Don't just tell it what to do and get frustrated when it does it differently than you would.

A good way to achieve this without writing huge prompts is tell it to plan the change first. Just give it some vague low-effort directions. It'll usually get most things right, you tell it what you want different and once you're happy you tell it to go ahead.

AI may get so commoditized for certain use cases that you will not even be able sell inference at a profit. AI might be bundled in with other services, just like cursor bundles in their own AI model for auto complete with their editor. I.e. cameras might have AI for image recognition bundled in etc.

I agree with all of this.

So my question remains the same: How are the players investing 100s of billions in buildout going to hope to make this back? Market capture looks bleak, inference looks like a race to the bottom. End users look like they could be beneficiaries. Where do the big boys go?

> Speculation here but I think openAI/antrhopic api inference is insanely profitable, it just needs more volume to amortize the training costs.

Well, they just rent their hardware, so I'm not so sure. But they'll both be public soon and we should get that breakout in their cost structures, somewhat.

The majority of Deepseek providers on OpenRouter for v4 pro are in the US. Especially interesting is that they are in the same ballpark for pricing.

The costs to hire management and "support staff" like TPMs that scale with SWEs that help them meet goals is proportional to SWEs - often that is taken for the higher end fully loaded costs, depending on how you define it. Office space in downtown SF, Mountain View, or Palo Alto costs more than office space for back office workers in Nashville or Utah. Firms that hire SWEs often have fringe benefits like free food etc. and while they may apply to all workers, it tends to go along with hiring lots of SWEs.

But yeah, double is insane. When I saw prices for COBRA from Facebook, it was $3300 a month, and that was god-tier insurance - the insurance benefits were so good they had a custom list of what was covered that was probably way better than anything available on the market (e.g. you want brand name drugs? no problem. You don't want to try both ambien and trazadone before taking a sleep medication doctors actually recommend? No problem - etc.) - but for my needs it was barely better than COBRA costing way less than half. $3300/mo, or even $1200/mo for an entry level ops worker is a lot of their salary, and probably where the double comes from. At SWE compensation most of it ceases to scale.

The fully loaded costs including proportional management costs isn't relevant to the true marginal engineer, but estimates I've gotten from higher-ups definitely factor into engineering decisions about "should we spend engineering time to save money/make more money - how much will doing this thing cost the company" (opportunity costs are also relevant, but usually less grounded, since most projects don't have concrete benefits like "we will save $x/yr in infra costs")

While the fully burdened cost of an engineer being double his salary sounds suspicious, this is indeed broadly the case. It has been (sometimes significantly) more than double in the case in every US employer where I worked and where I saw both numbers. In one case it was a hair under 3x.

My experience was not with pure software houses; we had some labs, measurement and RF equipment, but even without the hardware component the offices, insurance, admin expenses, HR, janitors, conference travel and so on would easily bump the total employee cost to double the salary. My 2c.

There are plenty of expenses in this order of magnitude that are not tied to direct increases in productivity. I think it may become a serious hiring impediment for companies to be really skimpy on these budgets for example.

Nearly no one is doing anything that is “only possible with AI”. This doesn’t seem like a relevant calculation. People spend on AI as an investment in their current productivity.

There's a lot of opportunity cost to waiting 14 months to build something.

It also presupposes that open models will bridge that gap towards opus4.5, which was really when I drank the AI coding koolaid

Yes, they are all already doing this

He was making a joke.

Let's be real. Most of the time you ask an LLM "Why did you do it like this?", it responds with something along the lines of "Oops. My bad. You're right to point this out."

You even have a fair chance of getting a response like that when there isn't anything wrong and the question wasn't rhetorical - which perfectly illustrates the level of the genuine understanding LLMs operate at.

I believe the “them” the OP was talking about was referring to the people opening the PRs, not the LLMs.

And you can certainly tell it the flow you want (and any other constraints) in the prompt.

Sorry, I meant interviewing the PR author for certain choices.

What happens if you just keep vibe coding is? Does it whack-a-mole fix one area and break another?

Google Jam Board (and other digital whiteboards) had high upfront capex and lowish opex. Probably close to the price for how often they were used before being killed off.

Same with the MS surface(?) tables (not tablets). I saw load of companies buy into the hype and then discard.

Companies love to waste money on that kind of service, before this website became everything about AI, every week someone would post how they saved a gazillion dollars by leaving vercel or AWS to self hosting as an example.

There are so many. Can I start with Oracle databases?

> Can you name a service that charged companies thousands/seat/month that turned out to be almost or completely useless?

The Concorde turned out to be fad (not "useless" - which was your reframing.) Touted as the future of travel, each seat cost about $20,000 of today's dollars, but it turned out even at those high prices people and companies were willing to pay per-passenger, supersonic trans-Atlantic air travel is not economically viable, and was discontinued.

Every consultant ever, but to be fair that's not per seat.

Not a service, but do you remember Scrum Masters? We had them as full time employees not so long ago. Pure fad.

Oracle and some company wide Microsoft licenses.

Between AI and the stock market (which of course relates directly to AI), I’ve lost count of the number of times I’ve heard lately another variation of “this time is different.” Sometimes so close to those words that I wonder why the person speaking them doesn’t feel a bit tingly. Great big warning signs all around.

The examples I gave, and the arguments that usually support them don’t really translate into “building complicated systems”. I was talking about the arguments in support of variable naming flamewars, etc.

I’m not proponent of AI generating everything without any supervision as of now. But willing to change my mind when it gets better.

Most software engineering jobs are not cutting-edge tech, or research, or solving unsolved problems. Integrations, APIs, figma-to-react pipelines, devops and etc. is what people get hired for. All those can be done much faster in the same-or-better quality by an experienced person with the supplement of AI. It’s hard to imagine any company would go against the grain and slow things down on purpose.

Just to go on record, as of today, I’m a big believer that a person that knows all that stuff is much more productive with AI-coding than a person who doesn’t.

I have no idea how we can get people motivated to learn these through trial-and-error when AI coding exists though. I remember the days of spending hours on stupid bugs that AI can resolve within a minute. But I recall learning heavily from those experiences. Oh well…

When I started I learnt something about coding from VBA macros to automate excel.

Often that started with the macro recorder. Then you worked out what that "recorded" code/sludge did, removed the crud you didn't need or want, improved the logic and so on. I bought books to understand it better. Now you can ask a (different) LLM "what is this? why is it used? How would I?" etc which is probably a faster learning curve than books, newsgroups and old school personal home pages with good info.

I would have been quite surprised when I first used a VBA macro in anger just how far I would go down the rabbit hole. C, asm, verilog, Linux were no part of what I originally signed up for!

Some people will specialise in the equivalent of recording macros and go no further. And this will be fine for code that gets it done but doesn't matter too much in the other dimensions (security, reliability, usefulness without the authors' support, etc.) Much like VBA utilities inside companies that were useful way back when. Other people will want what they produce to be better, even good, and they will learn about floating point [1] and all the rest, much as I did. Probably learn pretty fast too. [2]

[1] https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h...

[2] Working out how to write an excel vba webserver and using it to collect and and collate summary data from various divisions into reports was seedy as hell, solved the actual business problem (given ridiculous but intractable constraints) and isn't something you can record. We all have stories from a misspent youth that we're simultaneously ashamed and yet somehow proud of.

And, you don't have to vibe code. A competent developer can make great use of AI. I think a developer that can develop the system themselves is the most accelerated user.

> You don't actually need to know the answer to those questions in order to vibe code

No, but you do need to know the answer to respond to that 3AM page about prod being down.

heh yeah, i'm also selling trading advice :p

All of the articles and CFO’s saying so, and companies like Uber cutting back on AI spend.

They can be reinforced with best practices and context windows etc will increase.

If anything the snake is eating it’s own tail because now it’s training on vast amounts of its new slop…dragging down the average bar of quality.

Any kind of rug-pull is a serious concern. Companies are re-orienting their entire development processes around these tools. Sure they can go back, but it will require a much larger and more expensive effort than to transition in the first place.

All companies who make this transition will be more or less at the mercy of model providers.

Every employee doesn't need $1k in token spend per month, either. That kind of spend makes sense for technical workers in r+d.

Most other workers are served fine by $20-30 worth of tokens on a budget model. You don't need Opus to help support write emails.

The question you always have to ask is what problems does it directly solve. I personally think most of the current problems in software development and really the world at large are not time-bound problems but alignment issues, and all an LLM can really do there is be some 3rd party oracle that gives you an answer without needing other humans to agree with you.

There's some software that can cost $1k or more per seat/month, but it's pretty rare. Big tier ERPs usually fall in the ~$600/seat/moth range, specialty engineering stuff can hit over $1k, Bloomberg terminal, etc. I wonder if what Uber's building with that $1.5k/month/employee is actually delivering the same value that something like an ERP would to the entire org...

I mean, there's an "enormous incentive" for people to run their own data centers rather than using AWS. And yet, cloud is growing and on-premise is shrinking.

While I hope local AI continues to exist, I'm skeptical that it will take over, for the same reason running your own servers hasn't taken over. It's just hard, and involves spending huge sums of money up front.

It's also not really clear how much tokens are being subsidized. The discussion reminds me of Uber. For years people on HN claimed that Uber was going to collapse once they ran out of VC money. Then... that never happened, and everyone just moved on to discussing other things.

It's a mix. If the current wave of LLM businesses crater, demand for LLM specific hardware (and related hardware) will crater. GPUs were propped up by crypto currencies and now by LLMs. They're still great at doing fundamental math operations, but for their value to stay up another massive business opportunity involving matrix multiplication and the like would need to rise as soon as the current business cycle winds down.

Not impossible, not unlikely, probably 50-50.

Running locally is even worse for this, because if you're running 4 jobs at once they just run at 1/4 speed. Not literally, you can make up some of the difference with batching, but you have limited resources instead of spreading your requests out on an API provider's nodes.

I would expect the overwhelming majority of output tokens would not be the actual code but used for analysis, reasoning, testing and iteration. If you only use the agent for autocomplete then yes, the calculation is probably different.

> I don’t think companies will do that. Why don’t they just buy local on-premise infrastructure even though it’s cheaper than AWS?

For customer facing, production software, its worth paying a cloud tax to get the reliability guarantee. For tools that are used by engineers for code development, there is no need for such bulletproof guarantees.

The massive misalignment in large companies is no secret. But neither is the fact that when someone comes to cut, they also have no idea of who is doing load bearing work that matters, and who doesn't. I look at recent cuts around my large corp, and it's clear they are made at levels that have no visibility of the ground, and are uninterested in said visibility. Obvious mistakes that are worse than what claude would have told you (yes, I asked Claude to pretend to make the budget cuts in our org y looking at the same data an exec could probably get. They were better than what happened)

I think it's a general problem, but in my rare conversations with execs nowadays, they seem rather uninterested in improving their decision making there. The actual performance of the organization does not appear to be all that relevant to them.

But most Platform Engineering teams in smaller companies (and especially non-US) add a layer on top of existing technologies. A layer that usually maps to the specific culture and idiosyncrasies of that company; a bit like the deployment flow which is usually very specifically shaped on how a company is.

But in Uber's case, they tend to reinvent lower level pieces of platform/infra.

That's going to stop eventually, and I think at that point we're going to see business models more like the major CAD providers.

I don't think they'll have a choice, open weights models are not far behind. At some point it's essentially a commodity game

It's an extra 18k a year for developer tools when they're paying how much a year per developer? Having software developers at all isn't cheap.

Also, I don't believe you need to spend $1500 a month on a coding agent if you optimize usage at all.

There's models for every price point. What was SOTA and stupid expensive to run a year ago is a cheap flash model today.

Why are smaller non-IT companies "screwed" because they can't pay out the nose for their developers' AI usage? They're non-IT companies, developers are presumably not on their critical path, or not their bottleneck. Developers can keep on writing code the old way, or doing it with a more reasonable AI spend. I don't see how this "screws" any company.

"Windows XP+Dell" should have been in quotes. It's similar to the way enterprise productivity software was developed, packaged co-designed with hardware, and sold on an 18mo upgrade cycle assumption. It's not literally windows xp.

3/4 of Microsoft's BUILD conference the past two days were about local AI, foundry local and Windows ML along with a big section in the keynote about running local workloads on their new hardware with Nvidia. Say what you want about Microsoft's reputation, but they are a "big player" and seem to be moving in the direction of local AI first.

About the same ~40 FTE team. We're doing the same thing. Smattering of internal tools, but no net gain in external revenue. Who knows which of those tools will have any value or ppl are just doing it because it's cool now to make fancy dashboards.

OK. I guess that's good, too.

Yeah this seems to be a pretty widespread story, from what I've heard as well. The thing about those janky dashboards and spreadsheets though is that somebody understood them and built them with intent to solve a particular problem. Despite the rickety appearance, they're trustworthy tools. A polished single page app might look nicer but it's harder to debug than an excel sheet, and much less transparent in its internal workings--especially if nobody actually wrote it...

    > We are shipping more features

If you increase your spend -> ship more features -> no correlated increase in revenue, that's just burning money.

If a team of 10 spends 1 extra headcount ($180k/year) and ships features with no corresponding growth in revenue, what does that mean?

There was probably a reason it was on the backlog (because it didn't really have value).

> doing a days work in an hour then fucking off in a variety of ways

Until companies start hiring 5x less engineers than they did before and well.. we are clearly moving towards that direction

But that's an inefficient use of dev salary. Y'all are gonna get ground to smooth well-compensated paste.

Yeah I think this is probably most accurate.

I agree the most interesting use cases I've heard of are about increasing the rigor of software development practices, but there's definitely a lack of coherence in methodology.. I believe that some users and companies are successful in this effort, but the odd (and interesting!) thing is that so far we don't seem to know how to communicate how to do it successfully.

You're a content creator; you define your revenue stream.

Uber engineers do not define their revenue stream; the product leadership team does.

$1500/mo of AI spend by engineers does not equate to revenue. They need to figure out revenue first before zeroing in on AI spend.

Can you share some examples that you would say justify that price? Not a gotcha, I’m genuinely curious where you’re seeing a return at that level.

It is not only for devs

https://openai.com/index/codex-for-every-role-tool-workflow/

> Office 365

An entreprise license for 0365 is something like $75 per person per month. Totally different order of magnitude.

And regarding Bloomberg terminals, Bloomberg only has 1 million users (semi random guess).

The reality will be that some places just won't pay for any licenses or will try to set up their own, local LLMs.

What makes it look like one? All their dead comments read pretty normal to me.

Nope I'm a couple levels too far removed from the code at this point for that. Closest I get is during meta-management (modularizing, complexity reduction, etc) with agents

Studies show that LLM-generated context files have a negative impact on LLM performance: https://arxiv.org/abs/2602.11988

that's an interesting approach and something i also considered (using git to avoid conflicts). one thing i needed was a "database" (basically a folder of markdowns) with a fixed schema so i can let the agents record their decisions in (for example when the code conflicts with product design spec). this combined with search has been a real lifesaver.

this is how it works: https://help.markbase.cloud/humans/collections/overview

Your unwavering praise of LLMs' performance which does not match anyone's reality?

OpenAI or Anthropic would be paying you, like they pay bot farms and other influencers, and they would expect marketing in return, which you provide in boatloads.

Your job is to be an influencer, I'm not sure why anyone would be surprised that this is a possibility.

The issue is he’s not actually balanced at all. I’ve never seen him say anything negative about an AI product.

Yes, a paid shill. You can find a clear point in time where he shifted from sceptic to 1000% fully onboard non-stop praise, with no reason.

Theyd get a bad review for leaving performance on the table. When has finishing your work ever resulted in anything other than more work?

No, we know from the financials of these companies that API prices are close to being at cost and the individual developer plans are heavily subsidized (because they are roughly 10% of API cost per token[1]).

If plans were at cost and API pricing was marked up that would mean there’s a 90%+ profit margin on tokens and instead of raising money and talking about revenue, Anthropic and OpenAI would be talking about their obscene profits.

[1] the caveat is that the average plan user probably doesn’t use all of their quota, I guess maybe 30% is the average across all users.

yes, and theres no evidence that they arent (or can't) use profitable inference to subsidise those other expenses. Some companies will keep spending massively to train better models, and some other companies will not, and offer good api prices. Which will end up being used? That depends on whether the spending turns into better value models

The fact that Anthropic models are offered at the same API pricing by not just themselves but AWS, Azure and Vertex despite Anthropic taking a major slice on licensing along with the cost an open weight 1T parameter model like K2.6 costs to run on any third-party provider, make it unlikely that API inference cost are subsidized by the labs.

Openrouter? i.e. Even excluding Deep Seek inference for very large open models is way cheaper. Maybe these providers are not very profitable but its highly unlikely that they are losing $4 for every $1 they make since selling inference is their only product...

Yes; they ban various uses of their subscriptions but say you can do whatever if you’re paying for the API without limits

Assuming this were accurate, then presumably the AI companies would be betting that inference costs come down before the bill is due - I don't see enterprises being willing to absorb another ~10x price increase for tokens (as they've just done going from subscription prices to per-token pricing)

Yeah, that's not going to work if you can get e.g. 80% of value by using 10-20x or more cheaper open models. At some point it would just make sense for large companies to rent compute and deploy their version of DeepSeek or whatever (if they don't trust Chinese providers)

None of what you said is true

goated comment