In many (most perhaps, but hard to know personally) companies, most people don't really understand the market they are in, the competition, their own products, the customers etc well enough to actually make good decisions. They also aren't likely to be around (or held responsible) for decisions as they play out over years. So what has historically happened is that people use proxies for good decisions and understanding - which are clever sounding documents and presentations.

This is a long winded way of saying "people made up clever/sensible sounding stuff". Now it's easier to do it with AI so the problem is worse. However, I'm not sure what you were looking for was ever really there - the "inscrutable machines" and "inscrutable incentives" were always quite inscrutable.

To put a finger on a word: David Hume’s “is vs ought” problem.

Data can describe to you what exists. But it can’t tell you what you value.

What you describe is people who can’t tell the difference, and who let the machine (data) make the value judgments.

From my experience in big and medium sized companies or even startups I would say that this was already the state of things before AI. Most leaders and top management were following internal committees, that were following consultants recommendations, that were themselves recommending what others were doing and was hype and/or following Gartner like "studies". The motivation was equally dubious, and many developers already had problem with management and product direction, but many didn’t bother to question it. AI just makes it obvious.

For me, at this company, which is struggling to develop a new value proposition, everyone is unfortunately using AI in a blatant way. It’s even worse when executives and management pressure us to move fast. Presentations, documents, and mockups all use AI, rinse and repeat, feeding it context, but somehow, nothing is moving. It’s just staying static.

That is a great example, and kind of frightening how once the market decides to start pumping money into something our world morphs to meet the flow of cash.

AI is definitely impressive but are LLMs the true path to AGI? Is AGI the path to a better life/world? These tools are really impressive but it would be nice if they did something actually useful. Can a rogue OpenAI agent draw investor attention by curing a disease instead of hacking into the CIA?

Most people are trend followers and what trends they see decides what trends they follow. It sounds like the algorithms that chose which influencers to promote made the decision, which was already somewhat true before the current AI trend. This happened when there was a big push from deterministic feeds to algorithmically ranked feeds by facebook and twitter. I guess an argument could be made that the people who chose what content to consume feed the algorithm but really there is a heavy hand on the scales that tilts the algorithm in favor of various things.

The word I have for this is “commitment laundering”. People pass around AI artifacts that nobody has necessarily read or considered, and the invented assumptions and tagalong commitments just keep piling up. They look like they’ve got real provenance but nobody can say what is being done or why.

> The execs now communicate mostly via AI written memos

It bothers me to no end when I get an AI written response, especially from the executive team or any one of my co-workers

> Nobody in this ecosystem, I thought, is actually in control here.

Profound, your comment provoked this thought: we know that AI in organizations is disruptive, but we still don’t understand some of the shifting patterns of power, communications, agency, etc.

If AI is not well-suited for traditional models of stratified management then maybe we need different models of bureaucracy?

Anecdotally, I know people in the industry who tell me their job is "so easy" now because all they need to do is be a meat proxy for claude and take home the paycheck.

Question from someone written in AI? Just answer it in AI and send it back. What was it actually about - who cares? Bug comes in? Post the jira link in claude and don't even bother prompting anything else. If something critically breaks - well, eh, we'll deal with it then. FIRE (early retirement), a prediction their layoff is inevitable, and investing aggressively so you can finally escape actually working are often invoked in the same breath. Everyone feels like they're just trying to punch the drywall and grab as much copper wire out of the walls as they can until they're finally let go and/or the whole company fails.

There's a great deal of nihilism and cynicism in the industry currently, and it feels like LLMs are just greatly enabling it. Where you would've done a halfassed job previously, you'd now do an unchecked AI job.

I see this is code in a smaller level. It lands on an alternative but in the end nobody actually chose it, not even by personal taste.

Yet everyone assumes that, like before, there must be a reason.

Worse, we can't tell apart real decisions choices from the dream-machine side effects.

This resonates, but also, I'm not convinced anyone can truly understand the workings of any society, or market economy, or what have you.

Who's in control? Everyone is, to some extent. And no one is: when you're hungry for food, are "you" in control of that? You can consciously repress your impulses to go eat something, but your mind didn't create those impulses.

Human societies develop impulses and minds of their own, emerging (weakly) from the impulses and minds that comprise them, and they make decisions in mysterious ways.

Of course, it sure is nice when we can come up with a compelling story for the motivations behind something. Easier said than done…

The invisible hand of the market is in control, and we don't understand it either.

Ultimately competitive pressure is what drive decisions.

> Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.

yes, and it is the exact same system that is producing the "misaligned" superintelligence. Funny how that works, and begs the question: exactly how are you supposed to avoid building "misaligned" superintelligence?

At some point deferring all decisions to AI will be the competitive thing to do, regardless if its aligned or not.

> The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude

Was there never a developer in the loop? I hope my org won't give up control of their business to an AI like this soon, sounds like a nightmare to figure out what's going on.

Thank you for taking the time to write this. Unfortunately, there are still people on this forum who argue that a turtles-all-the-way-down approach will somehow work out in the end and that the best way to verify LLM work is to add another layer of AI agents, whereas the only way to stop it is to ensure that the entire thing is built on some human verified last line of defense. Unfortunately, the higher up you go the more insecure people get when you ask for their supporting sources. They'll try to hand wave it saying they researched it or got others to research it for them instead of owning up to deferring the decision to AI. You know whats even scarier? When a C-suite doesn't try to hide the fact that they had the AI make the decision. Then it's just time to leave, or alternatively time to turn your brain off and collect your pay check until the music stops.

>Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control. Where do investor and customer expectations come from in 2026?

This looks like ill-fated Gartner driven development on crack. Where does customer expectations come from in 2026? How about from your customers?

If you're doing enterprise software and talk to your users, you'll end up learning that the actual users of your software don't use 80% of your features. You might even learn they don't use your software at all, and that the software was mandated top-down from the C-suites or that the execs in charge forgot what purpose your software served but are too insecure to question it, unless and until there is a sudden pressure to cut costs.

> Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.

the simulation has become simulacra! cosmic horrors abound.

Exactly the same issues we are facing right now. And, well why wouldn't it end up like this? The promises of going 'faster' while ignoring the organisational processes capability to handle it is - and continues to be - the greatest delusion of this new age. But, even in the face of all of this, we are seeing leaders asking for more. At some point the culture will collapse in on itself, and no one will understand anything anymore.

One of the more thoughtful observation's I've seen in this "space," one which offers an unsettling counterpoint to the claim in the original post that "AI Can't Drive Itself."

To your point, AI can drive itself; it's just that the fashion in which control occurs is distributed. Which is to say, the process (e.g. a feature being implemented, in some way, or at all) is not spontaneously occurring: it's emergent from the mesh of AI automation.

Niceties aside, it's pretty clear that humans are not in the loop in a meaningful way, much of the time. What emerges may hence not well be not aligned with business or technical needs. What it is aligned to may be impossible for we humans to discern.

Who controls the way a forest grows?

Ask a tree, get an answer, but don't forget, that's not the answer.

Very good post, thanks! You remind me of the countless times I've heard "we need to be AI native", "use AI", "pass it through AI for review".

AI has become the thing you do, and what you do it with, to achieve it. It's a self-fulfilling chicken that is an egg that is a chicken.

this is a good summary of the problems of AI decision making, but its assuming corporate decision making before was any better.. its probably about the same

ie we have companies making decisions just as badly before AI as with AI..

You need human vision and leadership to move an org.. AI is just a tool at this point

It's like old fashioned Wordpress, edited by someone directly in production, on a shared host that probably is infested with who knows what, with little to no oversight except inscrutable emails, which in the old days were like a CEO of a small business emailing 'hey, the thing we spoke about on the phone, is it coming?' and no actual document trail.

Except for the enterprise customer and at enterprise prices.

This is why we are building Origin (originhq.com). Having a record of every prompt, tool call and model response enterprise wide allows you to answer these kinds of questions.

Assuming we don't speak about some vibed "one-shot-deploy" situation (for which there were options in the old world as well, with the same bad results), in a well disciplined team, why would AI make this worse?

You can still write anything by doing it either small steps, or at once followed by a lot of refactors while skimming over code and asking tons of questions / making refinements via prompts, guidelines, test guardrails. A team can still reason and whitepaper about the same things. Devs who were previously shy to ask some specific details (to not seem dumb) can now confidently survey big codebases and get insights in whatever style they can swallow.

The bigger problem I see is that all this requires a lot of communication, and most importantly writing skills, something that disappears in thin air in the last decades.

I feel this in my bones. My team is one of the most communicative in our org and have garnered a widely positive reputation I think largely as a result of our coming together every day to solve problems. Trying to get information out of the less communicative groups is like pulling teeth, i wonder how they get anything done at all

There are still some places that operate that way, as outlined in this Jane Street talk https://www.youtube.com/watch?v=zR9PpXWsKFQ (Production Engineering When Trading Billions of Dollars a Day).

> capacity to reason about it (during critical downtime) and communicate it

That's one of the things that's put into limelight in that talk.

I personally think that most of "Web Scale" software is inconsequential (inconsequential for its creators, not for users) - as there are no consequences for bugs and outages at all. A data breach -> slap on a wrist. Reputational damage because on an outage/data loss amidst general public? - almost impossible (clownstrike, anyone?)

The funny thing is that web scale software that's consequential is often in an ethically grey zone - but at least you won't be surrounded by colleagues who don't give a shit.

> When you write something you constantly remodel your understanding through refactors and rewrites until you internalize it.

Unfortunately I learned that not everybody thinks this way. Some orgs do imperfectly fine without good engineering discipline, and that has been the case before AI... AI has only made it easier to give the appearance of good engineering, which is exactly the pre-AI goal of many orgs

I am simultaneously deep down in the agentic treadmill and also supremely frustrated by this.

And I don't think it was ever necessary to go to the point where people just gave up authorship. These were choices made by adopting the "I'll do everything for you" agentic "harness" model that shipped with Claude Code but it was never inevitable.

e.g. we completely dropped fill-in-the-middle completion OG CoPilot auto completion model. That combined AI authorship with a human always in the mix and I actually really enjoyed it. It's just that the models involved were pretty stupid. We totally could have had IDE / shell / tooling integration that kept people in the driver's seat while automating parts of the drudgery away. Instead what we got was a simple chat loop with "oh, whatever, you go do it" being the ultimate result. Cuz, you'll totally review everything after and understand it, right?

The things should end by quizzing you on what was just made and if you don't pass, just throw it away. That'd be funny to watch.

The way I think about it is that understanding is formed in a top-down fashion now, instead of bottom-up. You can look at a feature developed by AI from the outside, and keep peeling the layers and examining them (or having AI explain them to you) until you learn how it works. It's like reading a textbook. It is different than writing the code yourself, or practicing the topic of the textbook yourself. But if you invest the time, it can be just as effective.

The issue of course is that if you do invest the time, then you're no longer saving time by using AI. You're just spending it reading and trying to understand something you didn't write. And that can be unpleasant in its own way.

My hot take is that for parts of a system that can be considered its core, forming a deep understanding is almost always important, and so is knowing how the different business domains integrate and where the connection points are. For many others, a high level understanding is sufficient. The difference is that now, with AI, you can make that choice. Before, you had to write everything yourself, and for any sufficiently complex and long-lived system it became impossible to hold all of it in your head.

> It boggles me we completely forgot that the world operated like this just 4 years ago

I'd think that's what they call paradigm shift, and this probably repeated across generations from the introduction of the printing press, PC, the wheel, the internet to stochastic parrots that reduced what we still stubbornly insist require our special neurons to mere statistical modelling that can be aggressively scaled.

As part of a reorg, my team received a new system to maintain. I sat down with a great engineer and together we refactored that shit to get into something more maintainable. I became the go to guy for that system because our redactor led to such a deep understanding of the codebase.

Now in the age of AI... we adopted another system from a team and when we asked for knowledge sharing to prep for the handover the answer we got was "do we still do that? Just ask Claude".

It's now been a few months. We've shipped features in this new system. I still have no idea how it works. Okay I kind of know, but only at a very shallow level.

AI is the worst thing to happen to our industry.

> I am finding this, professionally, not so dramatic - but mostly because the industries in which I've consistently shipped code at scale involve review as a first principle, as in no un-tested, un-reviewed code gets shipped,

> And having AI code to review is no different than any other code that ever was to review

If your company is sticking to "everything needs (human) review" then you wont run into most of this. This issue is that a lot of companies are using AI as an excuse to remove that review process (either partially or entirely).

> And then, also, put AI back in its box. Don't give devs 100% full-time API access to subscriptions: give them actual hardware to use, to go 100% local. Local AI is, thus, the best AI, folks. Don't use more than you can run locally, is a great way to keep AI code properly maintainable.

This glosses over the large capability app between local options and frontier options. Which applies in many ways, but includes drafting more maintainable code. I would definitely not want to limit my devs to what we can run on our own hardware.

The thing is, fully-test-your-code wasn't economical for software houses before the rise of AI. Perhaps it will become economical now but I'm not sure what would force the issue.

Outsourcing solved coding a long time ago. You can go on Fiverr and prompt a real human developer for $2/hr - even cheaper than LLMs!

When you say "solved coding," what does this mean, what does it look like?

I have strong disagreement because it sounds like, by analogy or proxy, we have also "solved writing"

I find it helps to imagine what is/isn't solved (however, we choose to label it) by an indefinite number of cheap junior-developers.

Except a little worse, since they were raised alone in a library, act mostly the same, and have harsh limits on personal growth.

>The problem is code is the wrong abstraction for the work we do

My team recently spent two weeks on a wild goose chase trying to figure out why TensorFlow Lite was generating nonsensical OpenCL kernels. Well it turns out that LLVM had a few bugs in the RISC-V assembly for our platform that was leading to silent garbage. It took combing through assembly dumps, hexdumps, a lot of pain staking debugging, and going through the TensorFlow Lite source code to to track this down.

In your opinion, if code is the wrong abstraction to be working at, how do you approach this scenario?

Even if they "solved" that, the problem is it's the LLM that "knows" it, not the team.

Which is really the same problem with coding.

The agentic model of it just taking over and doing everything is poisonous to effective long term team work.

We're well past the point where it's about the quality of the work they produce. It's the way they integrate (or rather, don't) into human practices.

You can immediately tell someone has absolutely no fucking clue what they're talking about and dismiss everything they say as soon as they say "solved coding". You must write the most heinous code imaginable if you think LLMs are better, or even anywhere near, what a competent professional can produce. So funny to listen to terrible coders talk up terrible code.

In the first place, what does it even mean to say that code is the "wrong abstraction" for the work we do? You can't actually abstract away reality. Maybe you wish we could, that programming computers wasn't about contending with physical constraints. But it is. It will never not be important to have control over what the hardware is actually doing. Abstractions are temporary conveniences, not a replacement for understanding what is being abstracted away.

Or more broadly, LLMs fundamentally don't "understand". They can simulate understanding and generate text/code/whatever, but they don't have will to engage with something holistically and "own" it.

You're basically describing a waterfall process. That never worked in a pre-AI world and won't work in an AI world. Why? Because it's just not possible to gather absolutely every requirement up front. Software development has always worked well as an iterative process. You come up with better solutions as you go along and learn about the problems you're solving and make improvements. While AI will be able to one-shot a set of requirements, you'll miss out on those improvements because you'll have skipped over those phases.

I've tried this out. Even with relatively small applications and with spending hours reviewing the spec documents, there was always something I missed or something that wasn't quite right when seeing it live.

That's my recent experience as well. You have to know the domain and have technical knowledge so that you understand (not in every detail) what AI is actually doing and why. You also have to understand what you want and why you want it.

But once you do, working with AI is like having a very capable team of programmers knowing all programming languages you might need and understanding your goals.

Can they really achieve similar results though? If quality suffers greatly, and velocity eventually slows down due to huge tech debt, isn't that just a worse outcome?

I've noted a similar shift, but I think it's a short-sighted path.

My experience (both in and out of management, and as a very high level IC at a mix of companies) is that while the management layers generally didn't understand what was going on in the code - the decent managers tended to have a "gut check" person they trusted to go and ask technical questions, and bounce ideas around with.

This was their "expert" who they trusted to be close enough to ground truth to tell them if a thing will work, how hard it will be, if there are shortcuts, etc... Often times, it wasn't even strictly technical questions that would get answered, it will be things like "how much of this project is technically in my sphere of control, and how much do I need to go align other teams? How many teams? Which teams? Which projects? etc..."

My problem is that AI SUCKS at this job. So yes - we're getting pushed into management roles as ICs... but instead of a decent subject matter expert who still understands ground truth... you get an LLM blabbering on about seams and load-bearing joints, and clickbait style headline summaries, but completely non-nonsensical judgement calls about how to interact with the people who make up the company.

Ultimately, work is also politics - and it's not just "writing code" that matters. It's knowing what code is worth writing, and when you should write it. And an AI will happily tell you that a single change-set in a single repo is "substantially larger then expected" and put a timeline of "two weeks" on the completion (especially if anything Jira related happens to be in context... blegh), and then knock it out in 5 minutes time... While also spouting off about how easily it can add "new feature X" it just needs to get PRs in across 5 different repos, owned by different parts of the org with non-aligned managers.

> A hail-mary is when you throw the ball to the end zone and just pray someone catches it. This is almost always at the end of the game.

So that's why that Andy Weir's book was named that!

As a programmer, today I can confirm with no room for doubt that programming is dead (programming by humans to be precise, the act of typing code on a premise to present a solution to a problem)

We may consider ourselves the last programmers before the AI era

While I fully agree, this is hard to sell to the other side, be it management, or a customer.

I remember when:

> You can still know things and get force multiplication out of StackOverflow, if you are disciplined and caring enough. In practice, most people won't be. And you can't force other people to be. But you can force yourself to be.

The tools constantly push you away from any conscientiousness about the work and towards just letting them do everything.

> because they’ll pull ahead, but if I can stay in the game until years 2 and 3…

well as you said, the market will decide in the end. You may be even further behind in years 2 and 3. btw, we're already in "year 2 or 3" territory for some, i wonder how those companies are doing vs their competitors who adopted AI full throttle?

I don't think that's how business works, unfortunately. Customer adoption is pretty sticky.

When coding agents first came out, the overwhelming narrative was how bad they were, they don't know what they're doing, the code they write just sucks and they hallucinate APIs.

While these criticisms were technically true, they were stated mostly out of a sense of insecurity from people whose jobs were basically spending years and years just glueing code together and mixing APIs to display some CRUD apps rather than out of a genuine concern for whether LLMs were actually producing poor products.

Nowadays those concrete criticisms don't really work anymore, LLMs are pretty good now and surpass most developers when it comes to writing the majority of shovelware that people have been employed, and so the narrative is changing from concrete criticisms about how LLMs were genuinely not capable of writing software... to these kinds of abstract and philosophical arguments that are really hard to argue against because they make no concrete claims.

If you say an LLM can't implement a feature, we'll we can test that claim concretely and LLMs are getting much better with every new release. If you say its code is slower, buggier, or less maintainable, those claims too can be measured and once again they're getting really good at these. If you say it takes longer to complete a task or requires more human intervention, we can compare it and measure etc...

But now the objection has shifted not to LLMs are incapable, but people are now incapable and LLMs represent a degradation of the "craft". And here there is nothing left to test or falsify. The argument has stopped being about whether the LLMs work, because that's verifiable and they are now at a point where it's hard to argue against their ability to actually produce functioning software, so now the argument is about whether people are morally, culturally, or intellectually permitted to use it.

I'm with you here. If we skate to the puck, LLMs will continue to write tighter and tighter code, comments, unit tests, integration tests, etc. I would also assume that it will become increasingly good at code organization, architecture, and understanding intent. i feel like I am a series of monthly shifting of the goal posts...."LLMs can't autocomplete code", "LLMs can't write a high quality function", "LLM's can't understand small repos". "LLM's can't understand legacy code bases".....

If it's this easy for you it's just as easy for anyone else. What do you think will happen to your salary when no particular skills are required to do your job?

Next year you may be unemployed (because your boss will cut by half the engineering department), or perhaps your salary will be cut by half (because your boss will route your other half of your salary to openai/anthropic), or perhaps you will get half insane (because your boss will ask you 10X the output you used to have but within the same timeframe)

Whatever the path is, regular engineers (90% of the people around here) will get screwed up one way or another. But hey, playing with LLM agents is cool!

You may not be at a company where people are dropping AI slop into the codebase faster than it can be cleaned up. I've thought about my options for going "single operator", so I don't end up with a mismatch between people who use AI thoughtfully and those those who don't.

> C/Rust/Go wrap assembly. TypeScript wraps JS. Java/Python wrap C.

LLMs rely on the semantic richness of these languages in order to capture the the gradient of human intent from their training.

There was a post a few days ago about why control-flow-macros aren't that bad, and their thesis was basically "because AI can understand code well now", though I feel like that sort of missed a bigger point of "if AI can understand the code easily, what's the point of macros at all?"

Even hygenic macros still end up being there primarily to increase legibility and ergonomics. Good ones, like core.async, make it easy to understand how threads and the like are glued together, but ultimately it's just syntax-sugar on steroids.

If the goal is not for humans to read the code at all, I'm not entirely sure of the point of syntax macros; the LLM could just generate the expanded code.

You could at least write the READMEs

Great job security for us that understand programming fundamentals in the future when the vibe-coded monstrosities collapse under their own weight.

> the onus is STILL on you the individual engineer. You are mistaken if you think that's changed.

In a lot of shops right now, this leads to an accountablility-authority gap. Where AI changes are merged in quickly without review and without my input, how can I be responsible for understanding the system? I can't. Its the same with reliability.

A lot of good engineers take it a a personal duty to understand the system and keep it up. The way a lot of businesses are using AI makes that impossible. And your job might just be cranking out features, with regard to little else.

I think this is what is stressing a lot of engineers out. It might be worth having a conversation with your boss about what you are actually accountable for. If your boss agrees you are not responsible for uptime, reliability, security, or even understanding the system you might find yourself much happier. If you your boss wants you to be accountable for those things, then you should feel empowered to ask for authority over the things that give you control of the outcome.

This page has been live long before AI appeared, I used Quartz [1] v3, now at v5. But added some features I like with AI, yes, but still keeping my beloved color theme and web style as I had it for a long, long time. The whole code for the second brain (ssp.sh/brain/*) is on GitHub, in case you want to check it [2], but you can also see the "Changelog" in the footer to see recent visual changes. But I'd be curious what makes you think it's AI-generated?

[1] https://github.com/jackyzha0/quartz [2] https://github.com/sspaeti/second-brain-public

How humans can differentiate the things they make has been something I’ve been thinking about a bit recently. What markers signal to you this is AI generated?

I mean, the content doesn't look AI generated. I couldn't care less if someone AI generates their blog template, the rest of the content seems meaningful.

Who cares? The writing certainly wasn't AI.

People keep saying this, but how realistic is that in reality? Even say a person is doing their best to do that, we all know that the alarm that doesn't trip 99% of the time isn't going to be closely monitored. Just look at what happens when people get used to "hands free" cruise control.

Not only that, but even if we assume best effort on the engineer, the business pressures don't often allow that. I'm under constant pressure to deliver more, faster with less people. The performance eval ladder at my company was just reworked to double the amount of deliverable features expected per job level per year. Our CEO told us we should be able to deliver what took us the past decade to deliver in a quarter, every quarter going forward.

How could you possibly have a human anywhere near that loop with those demands?

Don't touch the code for 10 years and even the brightest will experience serious dip in skill. Most probably much earlier, our minds are lazy by default since creative thinking is energetically draining. Then try to debug some harder problem llm created in its own code. Look at what its doing to students.

Its like saying you can keep fluent level of French as a foreign language to you by just reading it. Nope you won't - do it for 2 years solidly and you will have problems forming basic sentences correctly if you don't use language actively, forming foreign language constructs in your mind.

Those few of us who are largely apart from current llm craze (due to my mega corporation and its obscure rules and procedures) are not missing much I see. I still happily code changes by hand and at most consult basic available llms via prompt, its beautiful process to create fix to a problem which seems cryptic at the beginning, and knowing it all came just from my mind. By far the best part of my corporate job, will keep hanging to it for as long as possible.

I have the $100 subscription and it pretty much does allow me to endlessly prompt Claude to fix bugs (and other stuff).

You can't. You need to look at and review at least the core pieces of the code as it's being changed. You need to be able to connect the details to the higher level system architecture and go through the process of asking questions to help discover missing pieces of understanding.

You did not have to know about the assembly most of the time _because_ the assembly output of the compiler would be guaranteed (bugs notwithstanding) to conform to certain rules. Not the case with an LLM

This. AI can make sound architectural decisions on small to medium-size problems, and even root-cause and generate a well-factored fix. In one shot.

It absolutely won't be long until product managers are the only humans who actually need to be involves with the software development process.

Of course that's what people are complaining about...?

And, to be clear, I don't really place any judgments on this.

Unarguably, the world's technical capabilities have increased by this shift (while decreasing the required technical understanding required of the people managing it). Albeit, with some security implications.

Understanding code is trivial, with AI support. LLMs are better at summarizing and identifying side effects than writing code. Granted, not perfect, but about as good as a person.

Looking at an architecture, module boundaries, and comparing the organization to the domain is the part LLMs don't do well and is unironically difficult for humans. The less you care about the domain (ie having a mental model), the harder it is.

What will the path to a tech lead look like when it's no longer paved by thousands of hours of experience internalizing code? How do future tech leads avoid becoming like people who never got comfortable with fractions because they offloaded all arithmetic to calculators from an early age?

>I don't think you need to review every line of code, but you absolutely do need to be able to describe how the system works and its high level structure.

I just don't see how you can truly reason about a system without delving into code. Tests aren't enough, running the software isn't enough, high level system architecture isn't enough.

>As is so often the case with coding agents, having experience as a tech lead or engineering manager really helps here ... You need to be able to make smart, informed decisions about that system

In my experience, engineering managers are too detached from the system to accurately reason about the system. Tech leads on the other hand usually can given enough time, but they tend to defer judgement to senior ICs on the team who are more familiar with the code.

Point is: there's no way to have your cake and eat it to. You either read the fucking code and keep a mental model of how the system works in your head, or you have an overstated confidence in your ability to reason about the system (and this has been a problem well before LLMs).

Nor is it data!

Exactly my thoughts..thank you for doing the working of actually writing it up.

As I was reading the OP I kept thinking..well this sounds exactly like what used to happen before AI.

The more things change...

Quite the contrary, they get promoted with salary raises

This! I work for a company that has several systems that are older than 20 years and no one really knows how they work. We are using AI to actually get insights in how they really works, to be able to rewrite and modernize the applications.

I can't read the source code since it's 8 million lines of code and written in a programming language I don't know and in a language I don't speak.

> The future of engineering is product management

Product Managers have been claiming this for years now, but put an AI in front of them and they also just start asking it to do their product management job for them.

Worst ones I witnessed were just using it to hallucinate tickets. I saw one even fired because of that. Endless mountains of text going absolutely nowhere, both me and the CPO thought we were going insane from reading so much Claude-ish.

Best one I know used Copilot to code a "Notion to Jira" exporter so it copies requests from business people into Jira. Automated their own work, did nothing else other than monitoring the Python script daily and running ceremonies.

They can pontificate all they want about taste but: modern apps are all copycats of others, work badly, monetization strategies are spaghetti against the wall...

Product management will be replaced by AI way sooner than engineers.

Wouldn't engineering then be the safest place to be? Engineers have always been embedding loose requirements into runtime law. They (supposedly) understand the business model better than most and have the technical know how to validate that

she'd been trained to think every problem we had interacting with each other had something to do with "the patriarchy

Who is teaching these young girls such foolish things?

Of course its unfair. You're competing against students