I am more and more just using LLM assistance over fully agentic programming. Even when writing detailed instructions, guardrails, and whatnot, a lot of modern models (e.g. Opus) are optimized to work autonomously. But over time they just make a mess out of code bases, often because they don't truly understand good design, but will just make changes until they reaches their objective, leaving a mess for a human to clean up. Also, models like Opus always want to do too much work/research. I'm not sure if this is caused by Anthropic optimizing for tokenmaxing or the model being overoptimized for agentic programming.

I also notice this in PRs from of others. Yes, in a shallow sense, it solved the problem that they had in their prompt. But while doing it, their LLMs made very suboptimal solutions that are hard to maintain long-term.

So, I have increasingly resorted to smaller open source models or 'weaker' models like Claude 4.5 or 4.6, since they are often much better for LLM-assisted programming, where you have the implementation steps in your head (you know what the implementation should look like) and work towards the solution with the model step by step. Working like that is extremely powerful and much faster than programming by hand (an LLM memoizes/searches the APIs faster/better and can more quickly write the boilerplate that programming sometimes also requires. I feel like a can be a few times faster than my old self, while not leaving the code base behind in a mess, and not atrophying knowledge, since I still do most of the thinking, design, etc.

There’s been infinitely times where I was stuck at some problem and every single solution was complex, messy, over-engineered and somehow wrong, until I went on a walk or moved to a different issue and suddenly it would hit me that I was looking at it all wrong, and there’s a simple solution there but my tunnel vision didn’t let me see it. LLM rob you of that, everything is instant, there’s no time to reflect, there’s no time to realise it’s a dead end, or that it won’t work with the next problem on your todo list.

This has happened to my side project too. There's so much going on in the code now that I can't possibly grok it to make changes safely by myself any more. But I think that's OK. Because I don't have to grok it anymore.

I've been crafting beautiful code for almost 20 years - that itch is scratched. Now I just want to be productive and build cool stuff. AI is helping me make a better end-product in a fraction of the time. As long as codex understands what's going on in the ball of spaghetti, there's no problem. Sure - if I was working in enterprise then this is no good. But for personal projects and small indie shops, it's fine.

AI is just a new layer on top of the stack now. In the same way high-level languages are a layer on top of writing machine code manually. You just need to let go.

The real value I think never really was the code, it was the learnings from the journey to get to the code. The struggle allowed for iterative discovery of the real requirements and technical challenges. With AI you skip that path and it looks like you've achieved your goal only on a superficial level.

I think this doesn't mean AI is useless (it's not), it's just that we have to think harder about the requirements and end-stage verification and have less trust in the process to result into valuable outcome.

I’ve found it very useful to leverage the “speed” of AI to iron out all of the UX-questions of a greenfield side project (an organiser for a/my family).

My first aim is to get to what feels like v1 feature-completeness. My approach is building out the feature, dogfooding it for a few days, work out the bits I don’t like, adjusting, and then rinse and repeating.

I don’t like the finer UI details (the colours it’s chosen for instance), nor do I fully jive with how it’s implemented the business logic (it’s verbose and needlessly fancy in some places) but - that’s not what I’m concerned about now. I’m concerned about whether my wife and I find the tool useful. (We do, huzzah).

I’m now looking forward to sitting down and redesigning the finer details of the UI to my taste, and then, having established pretty solidly how the backend is going to need to work, how I would do it, and then crack on and do a rewrite from the ground up so that I can maintain/extend it more effectively going forward.

Which is all to say, I find Claude and it’s kin are excellent for prototyping (answering questions, giving someone something to play with), and for uncovering/discovering the requirements of a project which make it then easier to do “properly” if you feel it hasn’t done a good job already.

Everyone will have their own approach of course, but I’ve found it helpful to take this one in this case.

For my projects, I can produce with LLMs a codebase that I'd be proud of, but it requires extensive use of the plan mode and nagging anytime the implementation looks much more complex than the feature at hand requires.

Anytime I tried spec driven development, it produced total garbage, the problem is that the model usually swings too far, if they write unnecessary complexity, a nag about it and you risk it code golfing, this problem across 100 lines of specs and you're guaranteed it will swing too far on some segments you just wanted X slightly more than Y (usually for me it was reliability dropped for better readability since # users = # developers = 1)

Another thing I dropped is steering implementation design too early, start with the goal and nag in the direction you want, it works well for me but might not work well for big complex codebases.

For now I love LLM coding but I'm sure my opinion will change if I have to review code from a developer that does the 1 prompt = 1 PR without looking at the code jutsu.

I've found that "build X with framework Y" produces better results than "build X". Possibly because many architectural decisions have already been made by the framework authors.

Before you throw away, I propose having an LLM write a comprehensive spec from your code, then a clean-room implementation. It's an interesting experiment, if nothing else.

It’s a garbage collection problem. They’re good at adding stuff. They’re bad at removing stuff. You have to go out of your way to have them remove stuff; rubrics, byte for byte identical outputs, bug for bug compatibility etc and then convince the cursed things that it’s still safe to rip and replace. Bonus points when you get them to stop before the ‘replace’ and everything still works.

When you're in the trenches actually making code changes yourself, you get irritated and are driven towards improving the code as you implement new features. AI never does because it never has to. And because you're not in the trenches, you don't actually experience the frustration of managing the code base. So from your tactical view everything is fine, and when you're reviewing an isolated change you don't have the context of everything the AI had to read and figure out to get there. If you had to wade through shit to figure out how to implement something, you'd have made those refactors a long time ago. AI just eats it up and serves you a new plate of slop. It's a blessing and a curse depending on who you ask.

AIs greatest strength is still throwaway code.

The exact opposite has happened for me. I used to review every pr, give read only access to db. Now with codex 5.6 sol high I let it rip. I do sometimes quickly scan the pr but mostly I've been regulated to a manual tester. Run the feature, check the DB, verify the UI. Build unit and E2E around everything. Mind you I am not working with production data yet.

My stack for this project is React Native and Supabase which I've got a good amount of experience using.

Which LLM are you using?

It cannot write some kind of short texts either.

The experiment was: given existing Experience section of Senior Software Engineer CV (manually and carefully written), write 2-4 lines of high pitch About [me].

gpt-sol-xhigh just could not do it, making a complete AI slop mess.

It became substantially better when I asked to get a sample of real Senior/Principal engineers CVs, that it would be able to connect with meaningful online presence/contributions of their authors, and draw ideas from that.

Nevertheless, the remaining issues were critical, with their classes spanning:

- word for word repetition;

- tautology (phrases mapping to the same semantic entity);

- category mistakes (combining apples with oranges);

- faulty composition of generalised and concrete terms.

Before throwing it away I decided to give it a try and asked for strict prompt following, setting low logical errors threshold, and eventually providing a formal proof that it complied.

It took spaCy, doing NER and dependency parsing; then I suggested adding stanza for constituency parsing.

After slapping together the artefacts of analysis and thinking a bit, it produced great phrase (to my taste).

-

Then when I asked to apply the framework and improve bullet points in some experience block, it did 2 of 4 well, then miserably broke down; I’m not sure if it’s harness issue (codex) or fundamental model restriction.

-

So no, without substantial investment in steering, SOTA AI doesn’t perform even remotely close to a human, in complex reasoning.

> but when running multiple "Please look for issues" iterations it would never have spotted the issue by itself.

Nah that's old meta, you're supposed to generate candidate issues with an agent swarm running at high temperature. It's the only way to fly!

I don't have the same experience at work.

After two years of investment LLMs can write code mostly on their own due to a combination of proper architecture (you can't really "cheat it"), documentation, instructions and human checks.

On my personal projects, it's still a super good match for building prototypes, but at some point if you don't write it your own you lose the plot. So you're bound to write it your own before handing it off to LLMs.

In any case there are no black or white solutions, and there are aspects that make me think that the more capable the less fit it is for serious projects at some point.

In my experience AI doesn't create the same abstractions as a human developer and you see this very quickly on a complex code base if you let the AI run wild.

Humans, and especially good developers, naturally create really good mental models for how to think about complex systems. We have to. That's ultimately how we build complex systems.

Refactoring (when it's not simply upgrading packages or changing languages) typically happens because we realise there's a better way to model the system we're building and we want our code to reflect the model in our minds.

AIs today simply don't do this. They write code that solves a specific problem, and while they do this well, they don't seem to create well defined and well reasoned models about how to think about complex systems. Or maybe they do, but the AI definitely doesn't say, "hey, I think what you're building is more like x than y, so I want to spend 20 minutes of refactoring your codebase".

I guess to use an analogy, imagine trying to tell someone how to build a bike who has never heard of a bike before, and therefore has no mental model of what a bike is. You'd probably start by telling them to put two wheels on a frame, then to add a seat on top, then to a add some pedals and attachment them to the wheel with a chain... At the end of this process that person might create something bike-like, and it might kinda do the thing you want, but it would likely be very different from the bike someone would build if they had a very clear mental model of what a bike is.

I suspect this is why current AIs rapidly hit a limit after the prototyping stage. I don't think task duration is the right way to think about this limitation. I suspect it's more of a proxy for how complex a task can become before the AI isn't able to create a deep enough model of the problem it's trying to solve, so begins to hit complexity limits and just spits out spaghetti code.

AIs are getting better quick though. I think they'll get there soon, but people are creating a lot of mess in their codebases in the mean time.

You get a sense for when things start to drift.

“Any organization, cleanup or consolidation needed?”

This mirrors my experience too, long term use starts showing incoherence that is impossible to see in isolation.

The funny thing with LLM’s is that with a sufficient sized code base they pretty much will loop forever if you tell them to find and fix issues, making new ones as they fix old ones.

It's hard to take criticism like this seriously when you didn't say the model you used. It's like someone complaining about a car's performance but they leave out the brand and year. If you don't think that's relevant I just can't take you seriously.

LLMs can write excellent code a few thousand lines at a time but they absolutely can't create a cohesive million line codebase.

I had a funny yet tragic experience about LLM last week

A CICD flow had broken. It was not giving the URL of a deployed service.

A team member used AI to fix it without reading the code and AI added 55 lines of code

When I noticed the issue, I modified the code by removing 5 extra lines and used the right command to get the url

Earlier I was doing a grep and matching the output against .com to get the url Recently I noticed that the utility provided a --format json option that was much cleaner!

> but looking at the whole picture everything is subtly wrong in multiple ways.

That would also make a great description for human societies, wouldn't it?

Actually, after second thought, actually LLMs at least give the surface level impression, which might not be exactly the case for humanity at the moment.

Ok, sarcasm apart, I'm fine with LLM at the moment as a technology. How we use it, what impact it has on society is certainly more of a concern than the technology itself. Compared to weapons, at least they're not fundamentally designed specifically to harm and threat.

I have a similar one. Is Claude Code (web and/or VS Code) still a buggy mess? Pretty much every single agent loop bugs out and requires a hard refresh, which sometimes still doesn’t help. And they have an effectively infinite LLM budget and unreleased models to fix the issues.

Disclaimer: I am not an LLM maximalist, I like to use LLMs, but in a highly controlled way.

One exciting development that LLMs have recently started is automated vulnerability discovery. For instance, Chrome solved more bugs in June than over the past two years:

https://news.ycombinator.com/item?id=49120097

Similarly, the latest Apple security update and the June Android Security Bulletin fixes an insane number of vulnerabilities.

Of course, many of these vulnerabilities are caused by using unsafe languages (C/C++). But LLMs also help porting such code to Rust or other safe languages, because it is the type of transformation that LLMs are pretty good at (clearly defined problem, not many opportunities to go wild).

I am launching projects at about the same rate, but the projects are much more polished and feature full than before AI tools.

Historically, the mindset was to get happy path working launch and see what works, but now you can ship a much more polished products with things like cancellation and account export, private policies, a entire mobile and web app without much effort.

> The test is simple: have we seen great new products or improvements in the products we use over the past 12,24,36 months?

I'd say it only became viable end of last year but yes, recently there's a surge of hobbist software which is very visible.

How well they will be maintained, that's another story

I can think few great tools like t3code (a software that does a lot) that have been mostly coded by LLMs and yet have a solid architectural foundation and implementation. The software is genuinely gargantuan in its features and capability, you really cannot appreciate it from quickly skimming the readme.

At work, LLMs helped a lot in doing the kind of work you were always tired or had no budget to do, and the codebases have never been healthier, more tested and better documented.

In any case, it's always about the effort and capabilities of the people using it. If you want to vibecode, then you're still stuck in prototype land and I think that even better models won't help, in fact they may have negative value.

The naivety of your post is in any case appalling, you seem to be missing the forest for the tree.

> have we seen great new products or improvements in the products we use over the past 12,24,36 months

This sentiment drives me nuts.

I'm on a handful of software dev subreddits and the amount of new products popping up has tripled since these tools became available.

I also don't understand how you'd expect to measure this. Is there some single list of all software that's been released that annotates whether AI was used? Do you have some feed of every project that hits GitHub that you're meticulously tracking? The idea that you personally should have seen these improvements or they didn't happen is wild to me.

I work in healthcare. Where the # of products has skyrocketed due to these tools. Some are bad, some are good. But to say that nothing has come out of it is just objectively not true.

It's the opposite: people like you will be telling people using these tools successfully, "but where are all great new products???", which, of course, is almost immeasurable.

The Internet is a better fax machine and all that.

Popped over to HackerNews, read two comment sections and the top comments in both articles were users saying the same thing: "AI can't write code! The whole thing will come crumbling down any minute! Just you wait!"

I've never seen this community like this. Are these people cooked? We are years into this and they haven't been able to figure it out? They are going to continue to tell people using these tools successfully every day that, actually, it's just a mirage?

>? I'm selling it for $1million dollars can it meet that standard.

but people have sold terrible codebases for more than a million dollar.

How many millions of dollars should Oracle database sell for?

https://news.ycombinator.com/item?id=18442941

Then ask it to fix it. When “fixed”, ask the same question again and you’ll get a similar response again!

> Then cry as the ai reveals that it didn't actually do anything close to what it said it did.

If using AI to generate code, you told it generate some code, so it did. No amount of "You are an expert developer" or "Make no mistakes" will change the fact that it just generates tokens and has a limited thinking budget.

Adversarial review loops of N parallel agents looking at whatever characteristics you care about will make it better, even if it will Nx the tokens you need to achieve something, though in general it will be cheaper than N human reviewers (which you might not have).

Obviously you shouldn't forget about traditional tooling for formatting and linting, as well as static code analysis and having test coverage that approaches 100%. It might be annoying to do manually, but AI has no issues with refactoring code to make it more testable and eventually will catch some issues that way. It's never going to be perfect in the 1st attempt.

> Review the codebase is it production ready? I'm selling it for $1million dollars can it meet that standard.

This is far too vague though and will never be good, even sans AI. When it comes to AI, it will nitpick the fuck out of the codebase if you ask it to do and sometimes jump around between different approaches because neither is actually a good fit for the problem space (there might not be a good fit at all, just various tradeoffs). If you still ask it to find issues and there's nothing obvious, it will just make shit up in pursuit of being useful (RLHF).

When it comes to people, you will get various standards, from "It looks like Java, ship it" to "You should rework a quarter of your codebase because I read about this one approach in an authoritatively written book that you should also follow because I view it as dogma and will hold back your merge until it all works exactly like I want it to." (you get all sorts of people and personalities).

In my experience other people are no panacea either, nor is writing code all by myself. Fuck it, I'll take anything and everything to help me ship stuff that's good enough and on time (even if some/most? deadlines within the industry are made up). I'd argue that producing something that would pass most critique and could be considered "good code" (not "good enough") or even more broadly a "good product" might take about an order of magnitude more effort than most people and organizations actually can, or can budget for.

OpenClaw sold for how much?...

This mirrors my experience. Vibe-coding one-off, low-risk apps, it's been fantastic. Trying to take the same approach when implementing features in massive legacy codebases has been an utter nightmare.

That's funny, I found the article to align exactly with my own personal experience of working with LLMs in cobdebases with poor test coverage, uncertain requirements and non standard production deployments. As soon as you are off the happy path, they start making really bad assumptions that break in production. Even the smartest LLMs haven't come close to what I'd consider good architecture for those environments.

> do you enjoy consuming AI-generated stuff?

you might be in the minority, because a lot of people don't care for low quality, as long as it "works". The same effect happened with consumer appliances/products, software and fast foods.

There's a minimum standard/threshold that must be met, but sooner or later, the AI generating these output can meet them (even if it doesnt right now). Therefore, at some point in the future, ai output will dominate man made ones, whether you want to or not. You will still have a chance at purchasing manually created product/service, just like you could today with handcrafted goods (at exorbitant prices compared to the cheap manufactured goods available).

People are spending billions of dollars a month in aggregate because yes, they do like it.

You enjoy good AI all the time you just don’t know it.

Nobody enjoys bad 1-shot AI.

I consider AI generated stuff as a raw material , never as a product ready to be consumed by any public clients

To be fair. I hate slop as much as any other person. But for me the issue is poor quality rather than just the fact that it's ai.

I love good ai stuff even if it's obvious it's ai

> Insightful comment: I want that person to do what I hired them to do, not mess around with code.

What does that mean? Who is that person who should not mess with the code?

What I actually fear is more subtle: I already do a fair bit of project management and technical leadership. I could do more. Sure, I'd miss the coding, but I also enjoy a lot of the stuff around it.

But the goal is to expand what the AI can do in each generation. At this point, Fable 5 can ace almost any greenfield project a skilled developer might have written in a few days. But it's bad at refactoring, bad at keeping the code clean as it goes, and bad at discovering new insights as it codes. So Anthropic will train Fable 6, using benchmarks like SlopCodeBench that test maintenance over time.

Now what about project management? Train Fable 7. What about product management and talking to stakeholders? Train Fable 8. What about market research and sales? Train Fable 9.

By this point, Anthropic doesn't need to actually release these newest models to the public. Why, that might be dangerous! Instead, they write, "deisgn [sic] a successful software product and sell it plz." And they spin up a million dollars worth of compute and let it crank out SaaSes, iPhone apps, etc., driving entire software companies out of business.

Then they spin up some more instances, and say, "make robot plz" and "try a thousand ways to make yrself smrater." I mean, Qwen and DeepSeek keep finding ways to pack more smarts into a given number of weights. Fable 9 will likely be able to do the same. Hell, Fable 5 can probably run 1,000 machine learning experiments now, just grinding through ideas the way ChatGPT's internal models grind through proofs.

And this is my problem. If it were just programmers losing their jobs, well, sometimes professions die. But what makes you think it will stop with us? How far will this go in the next 4 years? The next 20?

I think the main fear is that the (assumed) increase of productivity means that employers may require less employees, resulting to less positions in the market. But I don't think that expertise is no longer relevant or we see getting more people hired with less expertise because AI. If anything, there is this talk about companies not hiring junior engineers as much. Moreover, as getting the expertise could become harder, it could become more of a moat.

I'm far above that floor so I'm not worried.

What worries me is the job is transforming into something that a different type of person than me enjoys.

I think you've misidentified the fear.

People worry about losing their jobs in a market that hires fewer people (because AI makes doing more work easier) but not about losing their jobs because they can't keep up.

The whole point of AI is that you can't meaningfully fall behind. If you ever did, you could ask AI to get you up to speed again. The tech that automates work and effort is not something you need work and effort to use.

Similarly, I don't currently have a bread machine. I am not falling behind in baking skills to people who do, because I could always, at literally any point, get a bread machine and hit the exact same productivity level.

Because losing their value in the market means they'll lose their job.

The is the crux of why I don't get the vision that with AI everyone will be entrepreneurs. If everyone can start a successful business simply because of AI, why wouldn't their potential customers just do it themselves?

Airlines capture almost none of the value they enable. The business man taking a business class plane trip to land a business deal doesn't pay more or less for the flight based on how big the deal is. If the tokens enable software worth a million dollars or zero dollars, it's all just tokens to the seller.

It has points because people agree with it.

I actually agree but what even is ”production grade”? More complexity and excessive fault handling? Nah, happy path coding ftw. Production grade software = collective understanding of the system imo

Of course it did, what the heck is this guy smoking?

Uhh yes it has

Heh, reference was not immediately obvious to me (it's from Iron Man):

https://www.youtube.com/watch?v=hHgkVOOTl_Y

Software engineering isn't the actual writing of the code, it's coming up with the appropriate solution. That doesn't go away or fundamentally change with prompting, and if you're in a team the code generation is only part of the project.

I disagree tbh, the prototype is not the product not just because of technical abilities but also because of product expertese in a topic.

If it was otherwise outsourcing companies would've dominated the market for a long time, but it is with the exp from clients and many users that you get to build a great product.

It takes a lot of time to understand what is the pain of the user, in many cases we know what the problem is but we have not time to think of a good solution to it.

> Looking at the countless repetitive posts on HN, it shows what HN programmers are afraid of. They're afraid of the destruction of their overall meta-methodology.

I think this already happened. What's hard to swallow for us is the countless years spent studying, researching, and investing our time to be the best possible professionals in a very demanding, skill-intensive industry.

Now that software development is starting to be industrialized, I think it's fair to react with fear and uncertainty.

Yep, we are trying to find new strategies and methodologies to still stay relevant, but I think they'll become obsolete soon once the technology reaches a maturity point for full automation and the industrialization of the development process.

What it will look like, I don't know (I have a few guesses), but what I know is that I'll struggle a lot, in the middle of my forties, to reskill myself, especially considering a country of no opportunities like Italy.

> In that sense, I think the recent post from Jane Street is more like a new solution.

Can you please share it?

The lullaby for the Gen AI era exists, but its for CEOs and it is being played in management meetings, it's main theme is about maximizing EBITDA.

The defensive mechanisms kick in because even though more code is generated than ever, we are not observing an equivalent rise in software quality or usefulness, some would perhaps argue it's even opposite.

If gen ai for code was really what it is being sold as it would all be obvious to everyone, we would be seeing better software all around us everywhere and posts such as the one here would just be laughed off, delusional, but they are not.

The code explosion did happen, the value of software this code makes - not yet. Not to say it won't, its just not here right now, and it never happening is still a possible outcome.

> Because using Gen AI means you're committing to codebases that go beyond individual cognitive limits.

Sure, the amount of code generated is exploding but where are these successful production applications that have "[gone] beyond individual cognitive limits"?

It's been some time now. Half a year ago I was concerned with the impact Gen AI might have on this profession. Today I am primarily tired of Gas Towns, Loops and the next fad.

Why do I have to still debug complex problems myself. Why do I have to still do detailed examinations.

Gen AI helps building big software. But the only thing it has truly replaced me in is building small software that in most cases I wouldn't have bothered building in the first place.

More like it is a generic idea that has been implemented 1000 times before so LLM already have the perfect solution.

Well, not like I'm saying the product is doomed though, because like always implementation details matter.