Assuming the employees are calling themselves "engineers" here, who else is supposed to be doing that work? If we were going through a renaissance in material science, don't you think it would be the mechanical engineers who try to figure out how to figure out what sorts of materials might be suitable for their use case? Surely you don't want the sales team or executives figuring that out.

Shouldn't software engineers have informed opinions on databases, programming languages, frameworks, etc.?

> Most employees can't tell you what database to use, what software programming framework to use, what document management framework to use, but they are expected to know which of the 25 models available to use for a task, budget appropriately, monitor efficacy, update models to the most relevant for a task, continue to manage architecture patterns??? for LLM agents, this list goes on.

All while suffering from the increased the mental load/lack of focus from even more workload. "You've got AI, you should be able to get it done today, right?"

There's way more anxiety here than is necessary. If you don't want to optimize, then it's pretty simple right now, just use Opus 5.5 or GPT 6.1 Sol. If you want to quickly test out the alternatives and see if the decreased ability in long horizon tasks is offset by the decrease in costs, toss a few bucks into your favorite neocloud (Fireworks was fine last time I tried them) and give GLM 5.3 and Deepseek v4.1 a spin, and see if they're good enough. OhMyPi makes it trivial to swap models without swapping your harness.

Everyone has a great researcher piped to their desk now. You can have it follow the Twitter zeitgeist for you, you don't have to do it yourself.

Was it ever not stupid?

A company I know well complained people weren't using enough and now that they mad everyone use it, it complains it is too expensive.

I am one of those employees who can make architecture decisions, and make them well, and have for decades.

The level of appropriate LLM usage in my opinion? Ask Gemini some questions when you need to search the web then read the sources it gives you.

LLM written code still has the problem IBM identified. A computer cannot be heald responsible, and so it cannot be allowed to make decisions. That applies to executive management AND software engineering.

Of course that requires working in an industry, like aerospace, where engineers are (usually, Boeing not counting) heald accountable.

I use "auto" in vscode which likely selects for cheapest. Good enough!

[dead]

Most employees can't tell you what database to use, what software programming framework to use, what document management framework to use, but they are expected to know which of the 25 models available to use for a task, budget appropriately, monitor efficacy, update models to the most relevant for a task, continue to manage architecture patterns???

Right, the problem is frontier models can do ~all of that pretty well, a lot better than they could 9 months ago. Luna 6 can do most of it practically free and instantaneously. So, your quote is likely what's increasingly being said in c-suite meetings as they decide to start mass layoffs.

It's not just being accustomed to AI. It's that the adoption of "agentic" workflows can make teams dependent on AI.

This summer my team basically shut down when we hit a usage cap because we had recently pushed a decent portion of our devops workflow into agent skills. It had been done in such a way that it was difficult for humans to navigate. Relevant scripts turned out to be buggy and poorly documented, and nobody realized because these harnesses that are tuned to be absurdly tenacious about searching for workarounds had been quietly burning heaps of tokens on muddling through instead of raising any alerts about the horribly broken state of the system.

I do agree that home (or at least independently) grown harnesses might be the next logical step. Harness vendors who charge by the token have an inescapable conflict of interest here. Moderate, well-governed LLM usage isn't good for their revenue.

I have some bias as a software dev/manager but at least in our org I noticed you can get a lot better value out of your tokens with some proper configuration of context and skills. Providing the latest models with good context can result in extreme savings.

After we started excessively documenting and extracting skills everyone has stopped complaining about running out of tokens, because agents stopped having to reconstruct the full context each time from scratch. Harnesses like claude code also push the model to aggressively keep this documentation in sync so there's little concern about drift.

The worst thing to do from a token usage pov is to give a model a vague open ended prompt because they are so scared to be wrong that they'll waste a ton of tokens "thinking" through the issue and verifying everything. Whereas they almost trust skills blindly and skip all this unnecessary work.

No one would tell truck drivers "We hit our gas budget for this week, deliver the rest of the packages on foot."

> I believe we'll see a shift to more organizations building their own harnesses with model routing logic to get central control over who can use what AI and for what.

I can also see middlemen agencies coming into the picture to determine token use, best model, etc. Effectively a middleman to the LLMs that will sell token use on the popular models bundled together.

My company setup tiers (as of this month - I can't tell you how it works since this is only day 5). I automatically get so many tokens, if I use them up I'm auto approved to upgrade to the next tier. They told everybody wait until you run out to request a tier increase - the real goal is a few people had a runaway job burning tokens that they didn't stop, the tier is a force stop point in case that happens to you.

Well at a big bank I work, if you finish in 1 week you do manual coding the rest of the month until next month. Management doesn’t give a shit. And they have daily talks about using more AI.

This has happened before with cloud computing. There was lots of people talking about how every employee would spin up their own servers etc and huge productivity gains. In the end all of that leverage mostly got absorbed by the engineering teams and gatekept and the rest of the business gets to use it through a GUI.

> I believe we'll see a shift to more organizations building their own harnesses with model routing logic to get central control over who can use what AI and for what.

This is what happened with cloud computing. People with expertise wrote software around the software creating machine because just giving naked compute to people en-masse didn’t actually achieve anything valuable. Likewise all of the talk of enterprise agents etc, they’re all custom software that wraps around a reasoning model.

My rough sense based on personal experience is that token consumption heavily depends on what you're doing with it. I fit pretty comfortably into $20/mo on my personal projects. At work my individual requests were averaging more than $1 apiece and I was doing well over 20 of those per day.

At work, my most expensive calls were asking questions about the (criminally underdocumented after 6 months of vibecoding) codebase, Plan mode, and asking it to diagnose and fix bugs for me.

It seems like actually generating code is one of the cheaper things you can do with it. But that cost grows with the size of the codebase you're working in, and how much stuff you have in your agents/ directory and AGENTS.md files.

Both of which can be pretty outrageously bloated if you're working at a company that's all in on AI coding. At work I recently did some git hacking to hide the team's shared skills directory and replace it with my own personal, heavily stripped-down version. It did wonders for both my context utilization and the quality of results I've been getting out of the agent.

There are multiple camps of people, and everyone thinks the other camps are doing it wrong.

- full autonomous camp

- developer augmentation camp

- no-ai camp

The trouble I see with all of it is that the future seems unpredictable at the moment. The costs related to AI are low enough at the moment that full autonomous seems to be possible, but we have reasons to believe that costs will rise significantly, which may change that calculus. The no-AI camp is in ostrich mode, and is betting on this all going away once the bubble pops. The developer augmentation camp treats it like just another tool, which is somewhere in the middle.

The trouble is that even if there is a clear advantage today, the ground truth of costs built in is probably not stable.

That's cool.

I have agents going pretty much nonstop for hobby projects. This is great value for me because I get to see a lot of things I'm interested in come to life. It's very token-inefficient though and if usage limits were reduced much this wouldn't be worth doing.

For my actual job though, I don't let agents go for hours on end and I care about the code. I spend between $1000 and $2000 per month there. Of course we pay API pricing. I wonder how much you're getting for that $20 in API pricing, it could be 10x or even 50x. At least on the max plans it can be that high if you are hitting usage limits, I don't know about the 20.

Depends on what you're doing. For example, I'm having one chat extend/optimize a (mildly-novel) CUDA program. Opus 5.5 (high/xhigh) is very efficient, but a single thread would still cost me the Pro plan usage in ~2 days (less if I were just running it unattended). This is one mostly-bounded program, and it's not even that large. In fact, before 5.5, I would have just used Fable instead.

The trick is having enough speculative explorations going on at once so that you can see if the ones going off for hours on end strike gold while doing so or not, then being prepared to either cut them off or get them back to the point, and repeat.

If you don't have at least some workloads doing that you're missing out on the biggest wins from the current phase of the technology.

Many businesses are on Enterprise plans where usage is all billed per token, rather than having a flat rate seat with rate limits. You may find it interesting to look at your token usage and calculate what you'd be paying monthly if you paid API rates instead.

they're not doing it right _or_ they're doing it correctly.

We're sorta in the age of alchemy. Lots of cranks out there, but there are real recipes.

[flagged]

That is what we have been doing, you get a default monthly limit, you can request more, managers chat with you about why, so there is flexibility but you feel some pressure to reduce your spend, so more of us fine tune our effort levels and model choices depending on the task, rather than just OPUS HIGH all day long.

corporate has no idea what stuff is "useless stuff" . Previously it used to be invisible, now token spends are putting a number on it.

If i had to guess upwards of 80 percent in corporate is "useless stuff".

>You don't need to max out tokens

Alternate take: it takes a lot of up front effort to ensure when you "max out tokens" the work is valuable.

The same thing goes for token use operating business processes (vs. building software that runs business processes): Let's imagine buying $250k tokens per year to operating some business processes - how much human effort is needed to ensure that level of spend is valuable? I'm using a number that could be "we could hire a skilled human at that level" as that changes the feel of the question.

> Companies should handhold employees, establish clear SOPs, and train them on responsible and effective AI assisted coding.

The whole point of AI is for companies to invest less resources into employees. There's no way they start spending money on training people now, when that was already something that made executives roll their eyes before AI.

Maybe I'm in the bubble, but I don't see a lot of "software factories bros" here, there are plenty of "coding is solved"-bros (they're also annoying btw)

edit: grammar

That "intuition" is not intuitive. How do I choose an appropriate reasoning effort?

I use these models all day long, experiment, and have no clue how to choose that.

It just showed up one day in the interface with no explanation or guidance. Its use is mysterious, its effects unclear except through intensive experimentation, and to this day it mostly seems "how many bad decisions will Claude go forward with when it finally dumps out screenfuls of text instead of getting better guidance early on" though it's certainly not a guarantee on anything.

These models are being released at breakneck speed even before their creators know how to use them. It's a big project of collective discovery to figure out what they are doing and how to use them.

Now get the 1000 engineers in your company to be as well-behaved and mindful as you are.

(Couldn't even get this to happen in a 30 person team...)