Unfortunately not everything will cleanly show up in PnL, which is a function of the entire company’s processes.
If developers are shipping more code (and most of the enablement is in software right now) does that show up in top line? Maybe… if sales and marketing are also selling more, and product people are building the right thing, and…
It’s not a straight line from AI -> $$$
btw, shipping more code isn't necessary a good thing. If quality isn't controlled properly, it eventually shows up as negative in the top line.
With AI, keeping good quality gets even harder because it normally only produces tricky defects.
It’s not just code quality, it’s all the other business fundamentals like the product itself having value.
I see this in a very non-AI way working at a company that’s a hodgepodge of acquisitions. Some of our products just don’t make a lot of money, don’t resonate, or are behind the competition in various ways.
The amount of code or general engineering complexity of the application seems completely detached from profitability a lot of times.
Now the bar is going to be higher for software to actually be worth buying when it’s so easy to make your own for yourself, so that also means that our ability to pump out code loses value.
> It’s not just code quality, it’s all xxx
> I see this in a very non-AI way
That's very non-AI of you
I did not use AI to write any of my comment. Please don’t make that accusation.
I don’t use AI to write prose at all in any form, not even drafts.
> The amount of code or general engineering complexity of the application seems completely detached from profitability a lot of times.
Of course, they're almost entirely unrelated. It's just programmers believe solving difficult problems with specifically code is valuable in of itself. The goal is to get as much value as possible from as little code & complexity as possible.
Maybe?
But I also think:
- there are a lot of companies that don’t need to be 5 9s[1]
- I have a feeling it’s possible to ship fast without drastic quality drops
[1] Claude code is 99.85% available and is one of the world’s most important products
> world’s most important products
Really? If we delete CC today, will humanity stop functioning? Nope.
It's so far from one of the world's most important products.
About 5 9s, reliability is just one of the axes which suffers from fast generation.
There are a lot more: architecture, accuracy, etc...
> I have a feeling it’s possible to ship fast without drastic quality drops
at what scale and for how long?
> btw, shipping more code isn't necessary a good thing.
This is FUD. It isn't necessarily a good or bad thing.
At the very least, using AI to audit your code is responsible use. The testing is the same as it ever was. If there was a chronic problem with shipped AI assisted code, it would be a popular talking point.
> If quality isn't controlled properly, it eventually shows up as negative in the top line.
This an idealistic pillar of capitalism that has been falsified across time. Depending on what someone means by "controlled properly", I don't think this wasn't been true before AI and I don't think it's demonstrated post-AI. Momentum (notably capture) keeps the software markets resistant to quality defects.
> It’s not a straight line from AI -> $$$
That seems like it might be a bit of a problem for those frontier model CEOs who have been making exactly the opposite claims.
Is it?
Most of the claims I’ve seen are around productivity (bottom line).
And while it’s hard to say “we would’ve had to hire more devs to be this productive without Claude” it kind of feels like the case, doesn’t it?
I don't think 2 trillion USD was invested into AI for "more productivity", without any financial gains.
Or maybe it was! And investors were actually great humanitarians!
It should be especially when new features are used to hook new customers (or renew contracts) and lower the amount of time wasted on bugs…
It can also be the case that it has zero impact on PnL, but is now necessary to operate a competitive business and keep up with competition.
Usually, things that aren’t clearly correlated with profit tend to get ignored and canned by businesses.
Think security, optimization work, refactoring, etc. Today everybody is drunk with AI, but what happens when the hype is over and you can’t justify to the finance dept. why the company is spending X on tokens?
An agent found a SQL optimization that at least a month of eng effort hadn't found.
Agents accelerated rollout of a data model change by at least 2 months of eng effort.
Those unblocked a $100k/month infra downsizing. That 2 months pays for my team's AI spend for the year.
This post is rage bait.
Your experience is the modal outcome
This comment is rage bait. You have absolutely no way of knowing this
I know a very specific use case in management consulting where hours spent by a consultant for a repetitive task (mainly report writing and source data gathering) went from on average of 40 hours to 12 hours per project. It costs the firm roughly ~$150 in token spend to save those 28 hours. They used to be roughly $200/hr (so $5k~$6k).
That's about as hard of data as you can get.
You haven’t shared enough data to tell whether there was a financial payoff for the firm or not. The consultant is a fixed cost to the firm, so saving consultant time doesn’t necessarily translate directly to higher margins nor revenue per worker.
Are the consultants delivering more billable hours per person now?
All sorts of reasons why the answer to that may be: no.
The firm charges a fixed fee to the client. The client doesn't see anything different from what they got before, the firm increased their gross margin overnight.
Yes, assuming the consultants are salaried like most, you can increase gross margin while operating margin stays flat. The question still stands — did revenue per consultant increase for this firm?
The consultants are contractors. Even if they were FTEs, it would mean the consultants could do ~3x the work.
You seem hellbent on disproving me. Why?
> You seem hellbent on disproving me. Why?
Uhm, no. The OP makes specific claims wrt to AI and financial payoff; you offered up a case study, with incomplete data, as possible evidence that AI is generating these financial payoffs. I'm asking questions to uncover the data necessary to reconcile your case study against the actual claims from OP.
> Even if they were FTEs, it would mean the consultants could do ~3x the work.
Could. But again, are they? That's the whole point of Apollo's claims: people keep saying AI can or could increase capital efficiency, but it's not showing up in any actual financial results.
You're extrapolating a 3x increase in throughput from the optimization of a single process that exists in a whole business system. Given your background, I don't think I have to explain to you why that doesn't make sense.
Also, do we think the S&P 493 is so incompetent that they'd miss these optimizations? Or is it more likely that this firm you're referencing is a small inefficient player, and the S&P 493 already realized these efficiencies decades prior without AI?
> it's not showing up in any actual financial results.
Really? Not any results?
This assumes the client is oblivious to the fact that they are paying the same for 12 hours of work as they used to pay for 40 hours of work.
For non-technical clients that may well be the case for a while, but I suspect even the most non-technical clients will eventually realize they can claw back a good portion of the efficiency as discounts.
I'm negotiating some contracts where I could do the work in-house for about 1/20th of the previously-reasonable vendor cost. I'd rather not do it, so I'm happy to pay 1/20th plus a reasonable percentage above that, but that's the limit. The vendor can accept that, or goodbye.
Interesting times on vendor contract negotiations these days.
> I suspect even the most non-technical clients will eventually realize they can claw back a good portion of the efficiency as discounts.
If the outcome is the same then the fees shouldn't matter to the client in the long run. Said differently, if the client who pays the management consulting fee gets the same return out of the service, why do they care?
Now, what may end up happening is that some other management consultant vendor in the market comes in an says "ya know what we're willing to do the same work for less" (which means their margin goes down). This could indeed drag their pricing down.
> Interesting times on vendor contract negotiations these days.
For sure.
I actually feel like it’s really premature for a relatively new technology to be showing profit improvements. We are talking on the order of a few years of investment and “figuring out” how to use it productively in business. Most new technologies take a decade or longer to really yield tangible value.
This is the first cycle in my (relatively) long career where the demand for return felt instant. The tools and techniques are almost comically immature and the average person interfacing with the tools is largely bewildered.
For word processing for instance people spent years upon years coming up to speed on wordstar, word perfect, word. Decades later they’re a common part of life and the tangible benefit over secretarial pools is obvious - there is no such thing as a secretarial pool.
AI it feels like almost a blink of an eye and the demand is “well, it’s been a few years, where’a the post scarcity society I was promised?”
The difference is that other new technologies didn't have a literal trillion dollars invested into them within the first few years, nor were the industry leaders warning that the tech could destroy humanity. Given those conditions I think "Has it made any money?" is a reasonable question to ask.
[deleted]
I think part of the issue is-if AI makes you twice as productive at X, that doesn’t necessarily result in any increased revenue, because now your customer expects twice as much for the same price, and if you don’t deliver that one of your competitors will.
A new technology can deliver significant increases in real economic value in terms of utility per $ expended without significantly increasing revenue for the average firm.
I agree. Being twice as fast as X might not help depending on how X impacts Y and Z.
As an example, my X is that boilerplate and general speed/effort of modifying my code is lower, but I’m noticing that this doesn’t necessarily make my team any more effective overall.
It seems like it’s been a code review burden and it’s led to some team members being too ambitious with picking up more projects that don’t make sense.
I have seen some managers in particular toss prototype work that they shouldn’t be doing at underlings, and one of those projects in particular really ended up taking me longer than if I had done it from scratch without AI. The fastest result would probably been if I did it from scratch with AI, but because I took it over from someone who didn’t understand the fundamentals and farmed out decisions to AI I had to spend a lot of time correcting mistakes.
Obviously, good management beats bad management all day long, but I bet you most companies don’t have stellar management.
Can’t share more than that publicly. Just that the time saved on this one thing alone is saving us way more money than the subscription per employee that is needed (~100$). I don’t doubt there are companies where this is not the case. Would like to share more details, but the rough math of doing something in a day vs several days + engineering salary have to do here.
> subscription
Well yes when your tokens are nearly free, then getting literally any business value is a good outcome.
Speaking annecdotally because im an engineer, not in sales:
Our company has secured projects and executed them to a standard which we would not have otherwise were it not for AI or hiring more people. Now, these are at least 3 year projects, so we are only seeing ~3 being AI accelerated.
For us, yes, it's a multiplier, directly translated to more $$.
With the advent of the internet and email, many people surely made similar observations. I wonder what the fate is of the companies who refused to accept these productivity enhancements.
The fate of companies who rushed into it before those technologies were ready was "a lot of wasted money".
Unfortunately we're at the mercy of Goodhart's law and in an interesting twist, AI is really good with things that have an immediate feedback mechanism, eg writing code that doesn't compile or pass tests, but you're trying to grade AI o something that doesn't have an immediate feedback mechanism. AI can slop out a PowerPoint deck in a fraction of the time it used to take a human by hand. The competent human then goes in and unslopifies it, but overall, still time is saved. Unfortunately, that employee is salaried and not hourly, so they can't click out early and even if they were, what's their incentive to clock out early if it means less money in their pocket? Or they can use the hours now free on something else. My gut, as an Internet random who's used AI is that it helps. I don't know how to measure it, just like how do you measure how much better it is to live in a clean house vs a cluttered one? How would you put that in a metric? If you can't put that into a metric, why bother cleaning your house? We clean it anyway, because of a certain je ne sais quoi.
> If you can't put that into a metric, why bother cleaning your house? We clean it anyway, because of a certain je ne sais quoi.
That "je ne sais quoi" is called hygiene and it's hard to measure but when you stop cleaning, generally bad things happen.
> And what revenue and/or profit increases can you cite publicly as a result of the use of AI in this work?
Nobody even tries to measure this. Their CEOs and board members have AI shares, and potential investors need to see your AI story to justify your valuation. It's a prison of lies.
It's probably just another Red Queen's Race. The benefits of LLMs won't show up in higher profits for most companies because their competitors are also using the same tools to move faster. You can't opt out of the race or else you'll be left behind.
Sounds like involution
Next step is 9to9/7days for the American worker to compete, or else you'll be left behind
Internally the hard data is there. On a budget of $2000 of API-priced tokens (Claude models + GPT models) we were able to migrate ~6 teams of infrastructure in 1 month, done by 1-2 people. Without AI it would've taken 10x longer wall clock time to do, and involved far more folks, and probably would have been much bumpier of a rollout. The big lift was mostly using AI for massive information gathering, digging through huge volumes of discussions, documents, commits, PRs, alerts, dashboards, etc pretty exhaustively for the past 5 years of history, to discover all the "chestertons fences" far more thoroughly than even the folks on the team who built it were able to. Then the 1-2 folks could go to each team with a super comprehensive "here is every need, every interaction, every discussion, everything you've ever done w/r/t this infeastructure, and here's how we're going to handle each of those with the migration we're going to do. We will do all the work for you, is that acceptable, yes or no?"
The teams throughly reviewed and brought up 2 overlooked cases, but mostly we all reminisced about things forgotten that AI discovery had turned up which would have been totally overlooked otherwise. Forgotten discussions, designs, outages, odd decisions, weird edge cases prepared for but never encountered, etc. Then they were migrated, it went smoothly, all to save a low seven-figures per year dollar amount.
The fundamental approach of exhaustively researching to that degree would never have been possible without AI. It's just not feasible to analyze 5 years of even one of those discussions sources (e.g. slack) to figure that out, let alone ALL of them. Now a team of ~1.5 engineers has tackled all that by leveraging AI.
The hard data is piling up in companies, but it'll take a while to get out there.
(Numbers and details lightly greeked in the above account)
It can take a long time for the economic gain of new technology to be visible in GDP because people have to learn how to use it effectively and processes have to change to integrate it. It took about 40 years for factory floors to be fully reconfigured around distributed electric motors rather than central steam shafts.