I would reframe this as: Quality was always a significant bottleneck and is even more so with agentic coding.

Code review is where many of the design & code quality problems are observed/highlighted.

We shouldn't give up on line-by-line review for serious applications.

My company's leadership is oscillating every few weeks on AI goals. Their most recent story is "we're expecting developer velocity to approximately double over the new 2-3 years as people get better at using AI". But of course that's completely non sequitur. People on my team spend about 10% of their time writing code and 90% doing design and analysis. So if AI doubles our code writing velocity, then it will make us 5% faster overall.

I agree. The latest models are getting quite good at validation, as long as they are clear on what the requirements are. Recently I got frustrated that while we have a huge test suite I noticed the LLM was hardly ever breaking a test with its changes (PRs only added tests, never changed old ones) which seemed suspicious.

I had it do a full audit. There were about 100k lines of tests, of which it determined about 60k were near useless. Many were overly mocked, some mocked to the point that they really weren't testing anything at all. It deleted those and wrote more expansive scenario tests to replace them, caught a lot of bugs on the way and fixed those too. Total time was about 2 hours.

I think humans need to review requirements, specifications, and test scenarios. And for some industries and uses they're still going to have to review every detail. But for a typical CRUD app, I think we're at a point where we can already set the process and quality bar higher than ever, in addition to moving faster.

> I think senior engineers find it very difficult to give up on line-by-line review, no matter how much better the agents have gotten.

I actually dislike traditional code review. In part this is because most of our code review tools are very cumbersome to use (GitHub's and GitLab's web interfaces in particular).

But also because humans aren't all that great at code review when it comes to correctness and security. Part of the problem is that you don't have access to the author's mental model for non-trivial changes and have to reconstruct it, otherwise your understanding of the program logic will be inadequate.

The problem is that purely agentic code review has orthogonal problems, and typically remains subpar when it comes to judging architecture and design considerations, compared to human reviewers. Both false positives and false negatives also remain a concern.

My best results so far have been with agent-assisted code review in a proper IDE, helping you to quickly construct a mental model and to analyze the underlying logic (in addition to having agents do a first pass for correctness issues).

IME, the biggest source of rework cost that happens at (or after) code review is when someone notices that the code, correctly or not, implements the wrong thing. For example, the feature, as implemented, doesn’t sufficiently solve the original business problem. Or it solves it in a bad way, like because of overengineering or taking on unnecessary technical debt. I haven’t really seen much evidence of AI code review catching either of these problems.

I’ve heard people complain that human code review doesn’t reliably catch them, either. And I have seen that with my own eyes. Just not universally. Teams that have a strong culture of shared ownership and collaborating on designs regularly catch these kinds of problems in code review. If they’re really good at it, they catch them even earlier. e.g., I know planning poker isn’t fashionable anymore, but when I’ve been on teams that take it seriously they tend to be pretty good at finding minimally expensive, correct designs pretty early in the process. Which makes both coding snd code review go faster because simpler designs are easier to manage. And design flaws that do make it to code review tend to be smaller and less expensive to fix. So those teams didn't tend to bottleneck on code review in the first place.

I think that this is the kind of thing that led to folks like DORA observing that AI coding is something of a multiplier for the effects of both good and bad software development practice. If you’ve got a cowboy coding culture, throwing AI into the mix will probably just lead to agentic cowboy coding.

As a senior engineer that is struggling to give up line-by-line reviews, I agree with your take.

I've spent literal decades where one of the most valuable practices has been code reviews, for many dimensions. It is one of the best ways to ensure quality production releases. It is one of the best ways to transfer skill to junior engineers. It increases tribal knowledge throughout the engineering org by exposing others to systems they may not be working in. The number of benefits to a traditional human-centric engineering team is hard to overstate.

But I'm starting to feel that instead of starting at one extreme (heavy handed reviews) and moving towards less review, it probably makes more sense to start at the other extreme (no review at all) and move towards a brand new practice tailored for the new agentic reality.

It is looking increasingly likely that we don't need to modify code review for the agentic era, we need some completely new practice that hasn't yet been invented.

You don't give up on line-by-line review just because the human is getting better either, and senior engineers want other people to do line by line review on their own code. It's not a matter of how good or bad the person or program generating the code is.

> I think senior engineers find it very difficult to give up on line-by-line review, no matter how much better the agents have gotten.

Our industry has spent the better part of the last two decades interviewing with brain teasers and leet code questions. Reading code is a skill that we just weren't assessing for, and lots of people aren't really good at it.

I would argue that code review is mostly theater, politics, bike shedding and pedantry.

There is a lot of work to get away from "coding" and "programing" and get down to the actual engineering that really needs to happen in the industry.

All of these model-based approaches are fundamentally nondeterministic systems, and there's no escaping the math of their fundamentals, which is textual representation of natural language.

The "nondeterministic" part is enough to flag these tools in some industries. We can't risk having a different answer to the same question in some applications. Even if the differences are small, differences compound over time, as anyone who's worked in inertial guidance can tell you.

Aside from the "nondeterministic" part, that "Fundamental Math of LLMs" is a very different beast from math that runs much of the rest of the cosmos. It's just human language. Just because you can talk about something does not mean you understand it.

Existing "LLM-type" technology performs beyond all prior preconception because there's pretty obviously some kind of basal mathematical relationship between different kinds of meaning. But that relationship, no one's really grappled with it formally in its entirety. It's probably adjacent to the big new science/mathematics of Emergence. It's also, from a pure science aspect, probably the most interesting aspect of these systems. All the crap we've been trying to wrench - for DECADES - with taxonomies, semantics, a swirling universe of computational linguistics in all its variety - that crap could conceivably have a single " $ E = mc^ 2" moment. But we don't fundamentally understand it, and it's a real mistake to hand critical systems to models that aren't understood. Down that path alchemy lies.

It's not code review that's slowing things down. It takes time to get the thing you want, and then you have to be reasonably sure that it's right anyway, or you risk shipping a piece of crap to production and tanking your own reputation. Even on teams that rubber stamp PRs I see people pushing drafts and waiting to merge until they feel comfortable with what they've vibed out.

I think non-senior engineers don’t realize just how much of a burden is barely reviewed ai generated code.

I’ve seen a few projects that heavily rely on ai with not very good (though senior) devs driving it - and its a mess. Not the code. The resulting product. Code is actually sound as ai is pretty good now.

All because ai does code reviews and dump decisions go into prod

We havent done manual code review for months actually. It is not a problem if you have adverserial review loops that automate your QA and mechanically enforce the orchestration of them. That is basically what the zeroshot framework we are building is for.

> I think senior engineers find it very difficult to give up on line-by-line review, no matter how much better the agents have gotten.

Interesting framing that casts senior engineers as the impediment. If the models are now good enough now, perhaps engineering leadership could[1] do away with code review entirely - or maximally automate code reviews. I'm sure seniors would love to get their individual productivity unleashed for performance metrics/bonus reasons.

1. I'm stating this as a hypothetical, but this "vibe-coding your core product" experiment has been repeated many times and I'm yet to hear a single success story

Imagine no code review: poof. You still aren't shipping at lightspeed because it still comes down to knowing where to shoot. Oftentimes this is like pulling teeth from various involved stakeholders.

You say small startups would be faster as they won't be doing code review. There's probably always been startups running without bothering with code review for as long as this has been a field. Bad practice? Sure, but all that matters to the business case a lot of the time is just shipping something that works alright, not the greatest piece of efficient code in the world. Moore's law has bought us a lot of slack in that.

AI is great at reviewing code. It is not great at reviewing intention. If the prompt was bad, not understanding or thinking about how to implement the feature properly, the review usually doesn't point out exactly what's wrong. This can lead to a sort of "polishing a turd" approach to code reviews where you patch some of the negative side effects for an implementation that is completely wrong.

This is a result of AI still being bad at telling the prompter "no". Not only because it will blindly reproduce what you tell it to while writing code without pushing back on assumptions without asking, but because it will often blindly accept an implementation approach while reviewing.

I suspect this is a hard problem, because if AI stops listening at the wrong time it can be a massive inconvenience.

I have noticed that AI is slowly getting better at pushing back on different types of things each time the frontier models get better though. I've also noticed on reviews that it will tend to hallucinate or nitpick around the problem. I assume it sees something wrong there, starts generating output, and points out something different there instead because there is pressure to be both agreeable and helpful, which are not always compatible.

I think it is getting better, but it can't always see the forest for the trees.

I think senior engineers find it very difficult to give up on line-by-line review

Thanks, hearing that renewed some of my hope for the future!

I'm definitely in that camp. The tools need to be as reliable as my compiler at faithfully expressing my intent.

Two inherent headwinds:

Their non-deterministic nature makes this difficult for me to trust.

And the imprecision of the human language.

Elaborating on the second... Source code can be a handy shorthand. It was designed for the purpose. If my level of abstraction (or "programming language") is the English language, it's going to be a lot more "code" to navigate.

That demands highly structured organization, reliance on diagrams, methodical vigilance for edge cases which may not be as readily elicited, etc. Admittedly, design hygiene that should already be the case if you're diligent, but I've seen how those areas can be a weak point in practice - particularly in some corporate environments where the developers are stretched thin, and as a result the culture sacrifices discipline.

In other words: Exactly the place where everyone says AI is supposed to be a revolutionary force multiplier.

I agree with you organizations who leverage the tool most effectively will be the ones who build or reinvent themselves across more than just the engineering teams, to avoid some of the habits human nature tends.

In the meantime I'm very keen to see how long term maintainability pans out for the codebases being churned out today (and fully expect as commodity code becomes cheaper to produce, we'll see it become much more disposable - just throw the whole thing out and start from scratch).

Of course I'd love to hear thoughts on this from people who've spent more time behind the wheel of these tools than I have!

We're going to need to start putting the month in brackets after the article title, not just the year!

> shipping significantly faster

If you're right (and you probably are), we're in for a wild ride further down the quality death spiral.

The era of user telemetry and conversion tracking already decoupled product design from customer experience. No need to wait for customer's to articulate their experience or desires or for discerning analysts to amalgamate and ponder their opinions. The numbers tell you everything you need to know! Just throw 3000 pounds of spaghetti at the wall every month and wait to see what sticks.

But now you really were bottlenecked on engineering.

Only 3000 pounds of spaghetti per month? And throwing more bodies at it barely even makes a difference? Oof.

Of course the solution is to just flush pureed spaghetti-slop through the hose as fast as we can. Brilliant!

What could go wrong? The users are sure to love it. We'll know because they keep clicking. That's how they tell us the truth, even as every forum fills with louder and louder complaints!

> I think if there is a major change in code review culture, it's not going to happen first in established engineering teams. It'll happen in startups that adopt a no-human-review policy from jump and then maintain it as they mature.

I've been hearing this for more than a year. But if these startups are all moving at superhuman speed, a month instead of a year from idea to a shipping product, where are the products? Every time I ask that, people talk about the amazing velocity gains they experience for the thing they are about to ship. It never does, unless we count throwaway hobby projects at GitHub - by that metric, AI is adding a lot of software. And I guess never-downloaded slop-games on mobile app stores.

But if you actually go shopping for consumer or enterprise software that's camera-ready, the landscape is basically the same as in 2022.

I lean towards a hybrid review process where humans accept their limitations, focus what they're good on(higher level concepts and architecture of the proposed change), and let the LLMs do the rest.

Doing a line by line review as a human is silly now.

> I think senior engineers find it very difficult to give up on line-by-line review, no matter how much better the agents have gotten.

Why do you feel like that? Agents have got better but better how? Looking at the frontier model output, I’d say the same myopic approach that causes long term issues without frequent course correction is still there.

The study was updated in August 2026.

Whenever I've tried skipping code review I've later found janky crap or weird decisions made.

Agents allow me to build cool stuff but atm they're far from good enough to not review the code.

You have to classify each code review based on what it changes and decide how to handle it, agent review and/or human review. You can have an agent do that.

For me reading ai code feels a lot like that time I tried reading Joyce. I don’t understand but I’m sure it’s beautiful.

My experience with prototyping games is that you need very strong human driven guidance and review in the beginning, then you can increasingly take your hands off the wheel.

If you take it off too early you end up with shit and will spend weeks fixing all these baked bad assumptions your project is built on top of. I spent weeks fixing some of this shit then the LLM revived a not insignificant portion of it in a later change because it got "confused" (its words) because some of the comments it previously made were left around.

What could possibly go wrong?

[flagged]

[flagged]

[dead]

[flagged]

> I think senior engineers find it very difficult to give up on line-by-line review

i am pretty senior. I hate code reviews so much. They are so boring, easily the most tedious part of my job. Its such a chore. Who is clinging to line-by-line review. Ppl dont really give that much about their employers.

Both Claud and Codex would run circles around 80% of the developers I’ve worked with through my 20 year long career. This whole code review thing is a hold over that probably won’t last six months.

I agree, I do like them for creating believable interaction prototypes though.

The study was updated twice with the last time on August 2026.

They've updated it for the agentic code era, and have found basically the same thing. Code can be created faster but that has moved the bottleneck to review.

By that logic, it is not ever possible to do research on LLM-assisted work.

> The models involved in the main agent results were primarily Claude 3.5 through Sonnet 4.5, GPT-4.1 through GPT-5, and Gemini 2.5.

What we have today is not meaningfully better than what we had a year ago, so that isn't relevant.

"The overall performance improvement gained by optimizing a single part of a system is limited by the fraction of time that the improved part is actually used" - Amdahl's Law

IMO it's a lot more like a new more powerful engine is now available. Sure, faster drivers/teams will run faster lap times, no doubt, and it'd be a bad idea to not use it. But, less mature teams/drivers aren't going to catch those faster teams by just slapping the new engine in and going. They are still largely held back by the myriad of other things that go into being fast on track consistently IE (driver + datalog review, and just good ol seat time), vehicle [other than engine] setup, aero tuning, pitting practice (esp fueling for endurance stuff), etc etc. All those same things that were making them slower than the fastest teams before. And even still, if you give only the slower teams the new engine, they probably still won't be competitive with the faster teams and, quite honestly, the not great drivers will probably just make the same mistakes they were making before that slowed them down, except faster.

But, even if the new engine is the golden egg, now what? You're still competing against every other team that has the golden egg... And it's right back to the driver mod, vehicle setup, etc.

That's not to say that the slower teams have any choice but to run the engine too to not be even less competitive. It's just to say that having the engine isn't gonna fix the underlying problem, no matter how much of the team you lay off to fuel the new, more power, engine.

The hardest part always has been and always will be - a simplified vision and strategy.

Everything else is a case of project decomposition, planning and scaling up - aka this is when having code written faster is great.

The SV approach of the past decade has been about experimentation, A/B testing and data science - it has its limits.

Or we could just recognize these llms crank out poorly architected bug infested slop. The coding work of these llms is just as garbage as all the other bland, flat, wtf writing they crank out with prose. Even worse because there's no logical mind behind the garbage. With bad human written code there was at least one human there thinking about it.

Except the problem with this is an engine designer doesnt want to pivot to designing a stronger driveshaft, or strengthening the gears of a gearbox, or designing more efficient brakes. They're an engine builder.

Same goes for software. 20 years ago I got into software development for the love and passion of solving problems and writing code. Now I've been forced to pivot to a glorified BA/Products Manager. It sucks.

Maybe I'm missing something, but it seems like the obvious solution is to have human accountability on the API design and tests, not on line-by-line source code. If the human writes an appropriately-scoped API and the code passes human-reviewed tests, then the implementation details don't matter so much (depending, of course, on project-specific requirements around performance and security).

Has someone presented an alternative to code Review , i mean, instead of just not doing it? Formal verification sounds expensive, asking an llm to Review another llm otput does not seem bulletproof. Is there another approach out there ? Can it exist? I've been thinking on this latelly but i can't find a solution

I think it highly depends on what field you're in. Personal projects that I find useful in my everyday life? Churning them out like crazy. Enterprise software? It has slowed our productivity down to a halt--part of this is due to people not knowing how to use these tools, and the other part of it is the collective AI psychosis of our leadership (I work at a Fortune 500 software company).

I’ve caught bugs in code reviews before, but I would agree that it’s probably not the most crucial thing you should strive to do with code reviews.

That said, I think I’ve done a tremendous amount of PREVENTING bugs from being introduced via code review. Caught all kinds of weird code patterns, duplication, wrong levels of abstraction, etc. Essentially, the kinds of places that bugs like to hide and grow in.

I still think that’s valuable, and it’s the kind of thing that models generally aren’t great at, to this day. It generally relies on having a large amount of context about the project and about what’s being solved, so if you wanted to do this via automated review, you maybe COULD, but it would probably get pretty expensive, especially on larger projects.

I review code line by line and build a mental model of the software as I review it. I find plenty of important bugs that way. PRs are an opportunity to learn (both committer and reviewer), communicate, and improve the quality of code a team ships. Sadly, many people decline that opportunity for the sake of short-term velocity.

No. You are on to something here. The utopia I see programmers lamenting the loss of, the one where human code review was perfect and never let any bugs slip through, never existed. I see people on this board and others complaining all the time about LLM generated spaghetti code and bug-ridden codebases. Well, I've been working those code bases for 20 years. We didn't agents to make them. LLMs are a new tool, but the problems that come with them are nothing new.

It was more important for giant C or fortran code bases. You will legit catch pointer errors or memory mgmt stuff, going line by line. Once Sun busted in with Java is when I really saw the shift to skim reviews. So much stuff is caught by compilers now, or SAST, that using dev time to find code smells is mostly a waste of their time. And it's not like the old way caught everything, it's just that the stuff we take for granted being caught by compilers or Sonarqube now, just used to get shipped to prod and cause abends and production support calls at 3am.

No. That's just the socially accepted norm and so people fight for it because they don't know any better (and a lot of these guys have egos the size of a small nation that leads them to "exert dominance" due to their insecurity problems—they're also frequently wrong).

Turns out just using the damn thing and testing what you've built on your own can solve a ton of bugs.

I think a code review should be just that. A review of the optics of code itself not the solution or anything like that. If that is important it should be done earlier in the process before coding. Even more important when coding with AI. The reviewer is not a tester. I've heard horrible stories from friends who mainly works with fixing AI generated code that junior devs pushes at a rapid pace.

Yes I have often caught stuff like "not using DB transactions in the right way" or "this is not going to be race safe" for instance from juniors just by glancing on a PR.

I find recent LLMs don't do such mistakes though, in my experience. And also LLMs will find such mistakes if asked to review, if not perfect then more predictable than humans.

The best code review process I've done are more like oral exams ie: the person walks through the changes and we listen as they describe it.

It's a good way to catch thinking errors, which are the insidious errors.

Code review is like a dam holding back a huge body of water. Nobody does it perfectly and almost all software turns to shit given enough time.

[dead]

Yes, I am literally inundated by cool new tools I've built that I never would have before.

I'm literally building a black-box client + server emulator for one of my favorite old MMOs, remotely, from a coffe shop in SF as I type this, on my laptop at home via Claude remote-control.

Insane.

I wonder how much of this is "cool new tool I want to try and play with" vs genuine technical capabilities? I'm guessing you know how to write software already. I think part of what might be happening is the inertia of starting with a project seems to be lifted. It is almost like instant gratification. I'd rather have the cup of coffee in front of me vs having to up and make it sort of thing.

But I wonder how much of that putting off was actually a real assessment of how much time and effort a project would take, vs a sort of feeling or assumption that it would be tough, and perhaps avoidant behavior as a result where the task is constantly backburnered?

This is how it is with my personal projects. Hard to decide to sit down and started but once the ball is rolling, I pretty much get the entire thing done in one sitting, learn something and feel satisfied and accomplished. Never takes too much time as I assume. But boy to get started is the real tough part sometimes.

> Projects I never thought I’d even have the time for are now possible.

Put another way: "Projects I would previously have considered a waste of time now seem worthwhile."

Organizations that use Jira have not reaped efficiency gains from AI, news at 11

Based on a sample size of one or…?

> true, but what if you just don't

A question being experimented on by what seems to be an alarming share of professionals who use social media. We'll see. I suspect it will depend on the context. If you weren't going to review a change anyway, and wanted the outcome, the team committing it won't perceive a negative outcome, whether it has been had or not. But hey, they'll be "using AI," and so will be compliant.

Before it was all coding on its own, I asked a model for bugs in a new project, saved me the 15 minutes of new web project I waste every time trying to remember obscure first time setup things you dont touch for years in under a minute.

I suspect theres a lot of hours saved that are otherwise wasted by devs when some simple thing throws off their implementation completely. Basically we cant measure the saved developer hours by LLMs because it could be literally anything.

This stuff is still 90% garbage hype. See also:

https://www.youtube.com/watch?v=NnV_cWeoo5Q&list=LL&index=3

[dead]

You read the paper, saw that the stated collection dates ranged from January 2021 to March 2026, and are claiming the authors are lying?

Listen man. Let me give you a tip.

Don't take it seriously.

Efficiency is not your job, until you can 1) solve it, 2) present the solved and implemented problem to senior leadership in one fell swoop.

Corporate America is a giant slush fund and this is how it keeps going.

Be grateful!

What kind of professional would spend 4 hours of their time per day playing video games?

The true professionals spend those 4 hours making snarky Hacker News comments.

I would be more concerned with playing video games half the day than whether the PMs are slowing you down by figuring out the requirements

You are free to not comment in the future!

Ars Technica in general has also been catering to their knee-jerk anti-AI audience. I used to find their comments section somewhat interesting but now it's the same old tired, outdated dunks on AI.

> the processes we love to make rarely help us

I'm going to make a potentially bold claim here. This depends _heavily_ on what the threshold of "rarely" is, and whether we actually _love_ to make these processes.

In the case of people before process, as stated in the Agile Manifesto, the process is begat _from_ the people in many cases. That doesn't mean it shouldn't be questioned, but when the chimney sweeps that built the process are saying "I don't want to go back to things that are similar to the way that gave me mesothelioma", I don't think it's fair to write them off as the minority.

Its the content not the process that matters - Steve Jobs.

I actually find it amazing all this money has been spent and all the same lessons will come back to haunt firms.

Whats wrong with humanity? Same mistakes made over and over again.

I get that the generating higher profits is alluring but damn, have some discipline!

[flagged]

>This is software that would not existed without coding agents: https://fluximp.org

> FluxImp is an AI agent harness

That is a tautology

A rocketship pointed in the wrong direction is not going to get to mars. Steering is probably the next evolution of code review

I confirm this diagnosis. The UIs are always clunky, disorganized, and kinda ugly. But they work well enough. I guess UIs are opinionated enough that we probably all customize them anyway. Maybe the agents know this so don't bother putting too much effort in? (Deep thought of the day: acquired)

[flagged]

We got rid of end to end QA as head of engineering decided to "shift left" on automation.

Is he not allowed to use Claude to do the reviews? I've heard from friends that the engineers on the team are basically now just routers for Claude/ChatGPT.

The paper was updated in August this year.

When people have gotten excited about having so many tests, I've asked them what their plan is when 500 of their 1000 test suddenly stop passing. I've always favored 50 tests I know are meaningful, where any failure is something to respond to, over 500 tests with a bunch of overlap and edge cases I'm not sure matter. But now people aren't even thinking about that, they're just letting the agents make all those decisions.

I think this approach will continue to work for some people and orgs, but others are going to hit walls where their agents can't manage the mountains anymore. It's going to be interesting to see how people and orgs respond when that happens.

Classic test theatre behavior. You can catch this with an adverserial reviewer though. This type of antipattern is why we build zeroshot

Every senior I've ever worked with at some point embraced the idea of minimalism. Produce minimal changes with a minimal surface. Worked well at startups and especially well on large spaghetti legacy projects. Seeing all of a sudden now how suddenly number of kLocs, PRs, token maxxing, number of tasks in progress is again a metric with which non-engineers brag about on engineering conferences feels like the industry has reverted to some braindead pre-infant state. I guess all mistakes need to be relearned at every hype cycle.

> Look at the shipping speed of the frontier labs themselves. It’s all the evidence you need to know that this isn’t true.

I certainly look at how their software sucks, and wonder why their fans think the rest of us should adopt such a bad approach to making software.

That's pretty sick.

[flagged]

The link is a piece of crap wrapped in candy packaging, able to fool anyone who couldn't be bothered to unwrap the candy. DOA slop. It has nothing to do with either art or craft. This is actually a perfect example why agents' code cannot be trusted.