What is your relationship to this team? Their manager? A senior colleague? Something about this sounds like a failure of communication or leadership. Why are you stand offish from them? Why are you treating them like children? If they don't know what they're doing why did you hire them? Now that you did why are you treating them like a burden and not an opportunity to mentor, teach, and uplift?

> 30 minutes of patient, old school troubleshooting

This is also the best way to understand a codebase, and it's quite enjoyable.

I think the models are getting too egotistical. They're so confident of their fixes they won't bother suggesting basic techniques like isolation of the problem by disabling code paths, for example. They'll keep shotgunning less-and-less likely fixes with undiminishing confidence until the code is full of fixes that do nothing. Then they'll double down on why these should be kept.

It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself.

EDIT: don’t get me wrong. I still think AI is incredibly useful for a lot of tasks! But when implementing an architecturally hairy thing, I find it less stressful and equally quick to jump down to the editor level and use AI just for code completion.

I'm seeing this happen in the security space right now. Someone on my team I was helping train and bring along is all of sudden regressing in their understanding of the issues we're working on, and instead focusing on AI tool outputs to do their job for them.

I gave the team an exact solution on a silver platter and they still failed to identify how to go about it

I think what's happening is ... poorly aligned and led engineers [in] this ego inflation feedback loop where they are completely detached from reality

A story about a team of humans with some very human problems.

> The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patient, old school troubleshooting.

This critique (if it can be termed such) admits that the code itself has little value especially compared to the work of “30 minutes of patient, old school troubleshooting”.

In my past experience, developers who critique more-junior colleagues in this way work better in isolation than as part of a team because working as part of a team would require mentoring colleagues through the often difficult process of troubleshooting.

Giving “the team an exact solution on a silver platter” may not be helpful if implementation depends on knowledge withheld because one does not have the resources, ability, or motivation to mentor.

Something about giving a fish rather than teaching to fish comes to mind.

I've felt that AI can figure out and fix 90% issues, but it rarely does minimal, non invasive fixes. That still requires manual effort. But going from a broad to minimal fix is still a different skillset from actual debugging, so in the the end it does lead to skill atrophy.

When I hear this I do wonder to myself how they're using AI.

For me, as long as I'm properly RPI looping it and not blindly pressing "yes" then it will nearly always reach the solution, usually a fair bit quicker, because it effectively becomes an ideation machine that can keep more thoughts and knowledge in it than my brain can.

However, I'm using it through giving it the context, it has access to the repos, access to the sytems, I'm telling it where the logs are, I have docs to show it what each part of the system is for.

Along with that, it does depend on my own instincts/knowledge for me to read its response and for me to say "no, you've over-thought/engineered this and this is actually the better solution", but its very are nowadays for me that it can't find the 1 liner, as long as I've fed in the right context, mostly pre-done because I've spent a bit of time building out the context tree for the repos/systems in a way that allows it to know what it needs to know.

I'm not doubting that people spin their wheels and couldn't find the one liner, but if its as simple as you say, that does seem like people who aren't great at LLMs along with a lack of instinct/experience.

I guess in essence, I think you can use LLMs in an old school troubleshooting way, and I find it still speeds that up the majority of the time. Its basically how I use it most of the time. And like old-school troubleshooting, if you build out the LLMs context over time, it also grows in capability, as long as its being used as a tool and not blindly trusted.

Should I not assume that most senior developers who are serious about LLMs do this?

I it usually doesn't get me in this weird state of mind, but I once spent 6 months (all-in) building a thing that I, once finished, just left alone completely (on disk gathering dust). Weird experience. So I'd say AI physchosis is real.

Sincerely , I think you're blaming the AI incorrectly there. You just got incompetents on your payroll.

Are your silver platters producing something faster, better, or cheaper?

Do your silver platters give you some competitive edge? If not, then is the ego problem yours, or your coworkers?

Having said all that, I'm aware of the intoxicating effects of feeling empowered from knowledge. There's an old saying: a little learning can be dangerous....

That happens all the damn time... Some otherwise competent developer decides to use AI, and falls into the pattern where AI will pretend to solve their problem for a week, while if they would have to think about it for an entire long hour to discover how to solve it by themselves.

As a bonus, that person will spend half of the week asking their peers for crazy delusional stuff.

Are your silver platters producing something faster, better, or cheaper?

Do your silver platters give you some competitive edge? If not, then is the ego problem yours, or your coworkers?

> “A natural evolution of engineers losing touch with the customers and users”

I’m a neophyte to software teams. I work at an accounting consultancy and deal with lots of SaaS platforms. I’m pissing in the wind about this or that product quality issue through the support queue on a weekly basis. There’s prolly an XKCD comic about it.

It’s a rare company that responds to the issues raised through support. Anything that could make engineering more responsive without degrading the product quality would be welcome to us.

I think LLMs have some of the same risks and benefits of stimulant drugs. They can make you more productive if used effectively as a tool, but they can also delude you into thinking you are better than you are and create a dependence such that you aren't just less productive without the LLM/drug, you fail to be productive at all because you don't know how to function without it.

I think I've heard late-career engineers from the past century say similar things, although I can't remember exactly quotes.

I wonder if every generation of engineers ends up with such hand-wringing about the next.

I'm also genuinely curious what fraction of the time they were right to worry.

I don't think it's engineers, it's the rest of the org insulating the tech workers from every side of the business

If the solution is so simple, why claude did not found it? At this point we can assume, it is better than 90% of engineers (including me).

After three decades of outsourcing to lowest bidder, I do not buy that humans are somehow better!

> patient, old school troubleshooting

I usually see similar arguments around systems with major red flags (no docs, poor CI, decade ago no CVS...). And engineers with private stash of workarounds for job security!

Claude does not do anything special.

Or perhaps claude was misconfigured, it had no access to relevant part of system, and it tryied to work within its limitation. Often it means decompiling binaries in desperate loop...

Hmm that’s strange, in my experience it is the other way around - Claude is super diligent with infra and will _insist_ on double checking and trying everything for real before committing.

When I was doing this myself I would read the docs and just implement them - claud is going about doing real software archeology to figure if what is said is actually the truth or it’s stale/inaccurate/buggy.

I’ve become 10 times more diligent because it is a lot easier to do. It’s no longer Urgh it’s good enough let’s ship it, now it’s “sure put a leg on it to figure it out and double check it”.

Backups are _tested regularly_ now because LLMs make it cheap to do so.

The only problem is when new engineers who haven’t learned these things Pre-ai now don’t really get why it is needed in the first place and will often lead the agent astray.

I think to address this we need to change or improve our training routines in general for humans. I think a lot of companies nowadays just skip that and deploy a company wide skill/policy for the agents, but don’t transfer the underlying skills to the devs themselves.

>Why not? Because even pre-AI, very few companies spend time practicing restoring their backups, or disaster recovery, or picking infrequently-used runbooks to practice, or seeing whether they can easily rotate secrets without downtime, or trying to redploy the system onto another vendor's cloud/platform, or, or, or...

Exact. Let insurance cover it, say sorry to your customers twice and shwoop never happened.

No worries, at some point your hit the wall with it and the reality will force you to look at the code. It won't be nice, but until that point delulu land is sustainable enough to fall forward

It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself.

I find that it helps a little to maintain an iron grip on the design. Take the time for really detailed change planning. Be pedantic and push back against every little thing that seems wrong or even a little odd in any plan document the LLM spews out. Likewise, watch for the for the moment when the LLM eventually stumbles and starts to make excuses for deviations from the plan. Interrupt immediately and force it back onto the plan (or your vision), potentially with pointers to resolve immediate problems. Completely unattended agents just don't work for important code.

>The more I use it, the more I have to rely on it to make changes/ fix things in the same system. In the end, I come out feeling empty; no intuitive knowledge of the system "I" built or fixed.

Most of us build a system to deliver a product to make money. We don't feel empty for not knowing how the system works, since we don't really care: it's just a means to an end.

I think this has as much to do with how hard software became to understand as with the new shortcut to refusing to understand it and the shortsightedness in willing to take it.

We lost a lot of traction in the name of ease of staffing and speed. Using LLMs to generate more code that is harder to understand it catalyzes it but the root of the problem, in my opinion, was letting go of great design and deep understanding for short term profit.

I've been thinking about this lately - is it like using 3rd party libs to achieve stuff faster? As much as I would lovr to hand craft the datetime logic in my app, I might as well use luxon and invest this time somewhere else. Only now with llms, you get virtually infinite 3rd party libs you can use, you create them on the fly. So if you have strong engineering values, I would say simply it boils down to "contracts over programs", you can still be in touch with the logic that glues it all together and treat some logic as a blackbox the same way we do with 3rd party libs?

Pilots aren't expected to be design engineers and SREs aren't expected to be software developers.

At least in my experience, operators and on-call engineers aren't necessarily responsible for remediation and validation.

Before you take me too literally, there is, of course, an overlap of skill and trade, but I believe the pilot analogy is more apt than you argue. I do agree that the difference in consequence and stakes is meaningfully important, however.

In aviation all code and doc are available to inspectors when a catastrophic failure or near miss happens.

It's not the case for other industries, and in particular car industry is able to hide about everything software related from inspectors.

I mean I think this is just how AI already operates. I’ve seen multiple models go digging in the git blame or past PRs, and even unmerged PRs. When agents work on a project that has well-defined in-repo docs, agents normally update them without prompting. So I doubt this will be a big issue for AI, but agreed we’re losing a skill

Will AI remember ? Or rather how will we make AI remember ?

It’s tough. The models can at this point very quickly identify issues in a Kubernetes cluster, for example. This because these systems give you a TON of observability, and it can easily see all the different moving parts.

That doesn’t mean the proposed solution is always right, but it is absolutely landing on the root issue faster than most humans would be able to, even pre-AI. Just because it can remember and run through a bunch of commands more quickly that I can.

There are lots of incidents where the symptom doesn’t always clearly point to the issue, so having something that can fairly exhaustively check a lot of different things very quickly is pretty useful!

But I at least partly agree, I think the more complicated and obtuse a system is, the harder it is for AI to do this. But we’ve invested time in making systems much more straightforward to understand and operate from one place (Kubernetes in general), and AI thrives on that.

And yeah, it means your debugging skills wane a bit because, yeah, if the bot can diagnose the issue in 5 minutes, it’d be irresponsible to not use it.

And I’m not really happy about it, and personally I’ve always been able to figure out a tricky bug given enough time. I don’t want to loose that skill. But everyone is under a lot of time pressure these days.

Like the fact that software "engineering" is mostly nothing like real engineering (and it’s further regressing now due to LLM coding!), the general lack of drilling is again one of the things that make software-related stuff look really naive and amateurish from the perspective of those dealing with the real world. Imagine if the military, police, fire service, and so on did not drill and rehearse incident response?

> If capability increase continues as it has, then an incident that cannot be resolved by AI will stump humans no matter the practice.

I disagree with this. Whatever the AI produces must be embodied in some kind of information system. The moment the output is on disk, it's fish in a barrel for any competent operator.

I've worked in environments that are beyond the pale with regard to complexity. It will take AI another 10 years to product something as complicated and coherent as a semiconductor manufacturing operating system, which is clearly feasible for humans to manage today.

If you are in a situation where you dont know what happened when something goes wrong, the business incentives will not accept “its too complex” as an answer.

Firms aren’t just selling products, they are selling reliability and taking on liability.

Nah, LLM models are already the new compilers. A commodity only engineers know how to use (in the context of software engineering in production environments)

For more in this vein, look up "Automation should be like Iron Man, not like Ultron". Sad to see so many people let go of their agency.

> when wielded competently.

And what happens when the competency drains away?

Who is going to teach programmers to wield AI competently?

> Robotic process automation is a type of business process automation that automates tasks within business and IT processes using scripts that mimic human interaction with application user interfaces.

For anyone else wondering what RPA means. Never heard that abbreviation before.

This isn't abstraction though. Outsourcing is a better term. If things continue moving up that latter, you will see that your agent/agency will pass the buck too. But there should always be some last turtle. Maybe that turtle will be the human that thought he was climbing the latter, who knows.

>introducing new high level abstractions...

Coding via LLM is not similar to using an abstraction. Imagine a car. The controls like steering wheel, the pedals, the gear levers. Those are abstractions.

But using LLMs are like driving using a remote control that has probabilistic behavior. You just loss what it feels to be in a car and you fail to improve as a driver because of the erratic remote control.

That’s the big question for sure

You would need to give it AWS credentials and MCPs for that. I’ve been sending agents into prod env for over a year now.

> We’re still roughly on year one of this transformation.

OpenAI Codex was released in 2021. Artificially compressing what we see now to one year of growth is an example of why it's very hard to trust anything AI proponents say.

> But certainly not in the 70s and 80s.

Some already developed even earlier and most ideas were already there in the 80s. The problem was just that the field was growing so fast that it was filled with people without formal training or fresh out of university. That way the knowledge did not spread.

For instance the 1986 paper „no silver bullet“ already described the need for iterative approaches later described as agile.

After 2 minutes of reading that article, I inspected and deleted the node. Very annoying.

If you have something like ublock origin use the block element functionality to target the blinking cursor. It's gone on my end :)

Not just you. Must be nice being fully neurotypical and „not seeing“ all of this kind of stuff.

https://en.wikipedia.org/wiki/Ironies_of_Automation

https://www.sciencedirect.com/science/article/abs/pii/000510...

The article is based on evidence, and states it clearly in the body.

I think the biggest loss is now not knowing if what the agent is claiming was actually done:) but yes, right tools are necessary, adding a boundary will change the position, being out of touch will be recoverable. Otherwise one bad call might be unrecoverable and no amount of familiarity will save you

Nope, because as I’ve seen first hand most people are too incompetent to use an LLM effectively, as easy as it is.

We already know this happens though, Bainbridge’s paper came out in 1983.

It's just a picture of a drill! Thanks, Magritte.

I'm not sure if I agree with the plane analogy either. Aren't planes supposed to have exceptionally stable hardware and software platform? Does that not significantly reduce failure modes? Most production software by comparison is constantly evolving. Unless you design very simple or basic training examples, most of it's going to be out of date soon.

The former CTO of a large MSP software company once told me on a call the reason their product had so many features removed with price increases was "you can't maintain a plane while it's in the air"

The immediate response was "We don't, your updates bring the on prem RMM down for hours at a time, the plane is grounded for maintenence regularly"