I usually do most of the engineering and it works great for writing the code. I’ll say:

> There should be a TaskManager that stores Task objects in a sorted set, with the deadline as the sort key. There should be methods to add a task and pop the current top task. The TaskManager owns the memory when the Task is in the sorted set, and the caller to pop should own it after it is popped. To enforce this, the caller to pop must pass in an allocator and will receive a copy of the Task. The Task will be freed from the sorted set after the pop.

> The payload of the Task should be an object carrying a pointer to a context and a pointer to a function that takes this context as an argument.

> Update the tests and make sure they pass before completing. The test scenarios should relate to the use-case domain of this project, which is home automation (see the readme and nearby tests).

I feel like I am taking crazy pills. I am getting code that works from Opus 4.5. It seems like people are living in two separate worlds.

I am getting workable code with Claude on a 10kloc Typescript project. I ask it to make plans then execute them step by step. I have yet to try something larger, or something more obscure.

I have found AI great in alot of scenarios but If I have a specific workflow, then the answer is specific and the ai will get it wrong 100% of the time. You have a great point here.

A trivial example is your happy path git workflow. I want:

- pull main

- make new branch in user/feature format

- Commit, always sign with my ssh key

- push

- open pr

but it always will

- not sign commits

- not pull main

- not know to rebase if changes are in flight

- make a million unnecessary commits

- not squash when making a million unnecessary commits

- have no guardrails when pushing to main (oops!)

- add too many comments

- commit message too long

- spam the pr comment with hallucinated test plans

- incorrectly attribute itself as coauthor in some gorilla marketing effort (fixable with config, but whyyyyyy -- also this isn't just annoying, it breaks compliance in alot of places and fundamentally misunderstands the whole point of authorship, which is copyright --- and AIs can't own copyright )

- not make DCO compliant commits ...

Commit spam is particularly bad for bisect bug hunting and ref performance issues at scale. Sure I can enforce Squash and Merge on my repo but why am I relying on that if the AI is so smart?

All of these things are fixed with aliases / magit / cli usage, using the thing the way we have always done it.

Well one way of solving this is to keep giving it simple tasks.

Just a supplementary fact: I'm in the beneficial position, against the AI, that in a case where it's hard to provide that automatic feedback loop, I can run and test the code at my discretion, whereas the AI model can't.

Yet. Most of my criticism is not after running the code, but after _reading_ the code. It wrote code. I read it. And I am not happy with it. No even need to run it, it's shit at glance.

skill issue

I had my first interview last week where I finally saw this in the wild. It was a student applying for an internship. It was the strangest interview. They had excellent textbook knowledge. They could tell you the space and time complexities of any data structure, but they couldn't explain anything about code they'd written or how it worked. After many painful and confusing minutes of trying to get them to explain, like, literally anything about how this thing on their resume worked, they finally shrugged and said that "GenAI did most of it."

It was a bizarre disconnect having someone be both highly educated and yet crippled by not doing.

It’s like weightlifting: sure you can use a forklift to do it, but if the goal is to build up your own strength, using the forklift isn’t going to get you there.

This is the ultimate problem with AI in academia. We all inherently know that “no pain no gain” is true for physical tasks, but the same is true for learning. Struggling through the new concepts is essentially the point of it, not just the end result.

Of course this becomes a different thing outside of learning, where delivering results is more important in a workplace context. But even then you still need someone who does the high level thinking.

What you as a teacher teach might have to adapt a bit. Teaching how code works is more important than teaching how to code. Most academic computer scientists aren't necessarily very skilled as programmers in any case. At least, I learned most of that after I stopped being an academic myself (Ph. D. and all). This is OK. Learning to program is more of a side effect of studying computer science than it is a core goal (this is not always clearly understood).

A good analogy here is programming in assembler. Manually crafting programs at the machine code level was very common when I got my first computer in the 1980s. Especially for games. By the late 90s that had mostly disappeared. Games like Roller Coaster Tycoon were one of the last ones with huge commercial success that were coded like that. C/C++ took over and these days most game studios license an engine and then do a lot of work with languages like C# or LUA.

I never did any meaningful amount of assembler programming. It was mostly no longer a relevant skill by the time I studied computer science (94-99). I built an interpreter for an imaginary CPU at some point using a functional programming language in my second year. Our compiler course was taught by people like Eric Meyer (later worked on things like F# at MS) who just saw that as a great excuse to teach people functional programming instead. In hindsight, that was actually a good skill to have as functional programming interest heated up a lot about 10 years later.

The point of this analogy: compilers are important tools. It's more important to understand how they work than it is to be able to build one in assembler. You'll probably never do that. Most people never work on compilers. Nor do they build their own operating systems, databases, etc. But it helps to understand how they work. The point of teaching how compilers work is understanding how programming languages are created and what their limitations are.

Same with essay assignments, you exercise different neural pathways by doing it yourself.

Recently in comments people were claiming that working with LLMs has sharpened their ability to organize thoughts, and that could be a real effect that would be interesting to study. It could be that watching an LLM organize a topic could provide a useful example of how to approach organizing your own thoughts.

But until you do it unassisted you haven’t learned how to do it.

When learning basic math, you shouldn't use a calculator, because otherwise you aren't really understanding how it works. Later, when learning advanced math, you can use calculators, because you're focusing on a different abstraction level. I see the two situations as very similar.

I haven't done long division in decades, am probably unable to do it anymore, and yet it has never held me back in any tangible fashion (and won't unless computers and calculators stop existing)

LLMs are not bicycles for the mind. They are more like E-bikes. More assist makes you go faster, but provides less exercise.

https://www.slater.dev/2025/08/llms-are-not-bicycles-for-the...

They don't always do the simple things well which is even more frustrating.

I do Windows development and GDI stuff still confuses me. I'm talking about memory DC, compatible DC, DIB, DDB, DIBSECTION, bitblt, setdibits, etc... AIs also suck at this stuff. I'll ask for help with a relatively straightforward task and it almost always produces code that when you ask it to defend the choices it made, it finds problems, apologizes, and goes in circles. One AI (I forget which) actually told me I should refer to Petzold's Windows Programming book because it was unable to help me further.

Lots of interesting ways to spin this. I was in a computer science course in the late 90s and we were not allowed to use the C++ standard library because it made you a "lazy programmer" according to the instructor. I'm not sure if I agree with that, but the way that I look at it is that computer science all about abstraction, and it seems to me that AI, generative pair programming, vibe coding or what ever you want to call it is just another level of abstraction. I think what is probably more important is to learn what are and are not good programming and project structures and use AI to abstract the boilerplate,. scaffolding, etc so that you can avoid foot guns early on in your development cycle.

AI does have an incredibly powerful influence on learning. It can absolutely be used as a detriment, but it can also be just as powerful of a learning tool. It all comes down to keeping the student in the zone of proximal development.

If AI is used by the student to get the task done as fast as possible the student will miss out on all the learning (too easy).

If no AI is used at all, students can get stuck for long periods of time on either due to mismatches between instructional design and the specific learning context (missing prereq) or by mistakes in instructional design.

AI has the potential to keep all learners within an ideal difficulty for optimal rate of learning so that students learn faster. We just shouldn't be using AI tools for productivity in the learning context, and we need more AI tools designed for optimizing learning ramps.

But what has changed? Students never had a natural reason to learn how to write fizz buzz. It's been done before and its not even useful. There has always been a arbitrary nature to these exercises.

I actually fear more for the middle-of-career dev who has shunned AI as worthless. It's easier than ever for juniors to learn and be productive.

I agree 100%. But as someone with 25 years of development experience, holy crap it's nice not having to do the boring parts as much anymore.

I remember reading about a metal shop class, where the instructor started out by giving each student a block of metal, and a file. The student had to file an end wrench out of the block. Upon successful completion, then the student would move on to learning about the machine tools.

The idea was to develop a feel for cutting metal, and to better understand what the machine tools were doing.

--

My wood shop teacher taught me how to use a hand plane. I could shave off wood with it that was so thin it was transparent. I could then join two boards together with a barely perceptible crack between them. The jointer couldn't do it that well.

Yes! You are best served by learning what a tool is doing for you by doing it yourself or carefully studying what it uses and obfuscates from you before using the tool. You don't need to construct an entire functioning processor in an HDL, but understanding the basics of digital logic and computer architecture matters if you're EE/CompE. You don't have to write an OS in asm, but understanding assembly and how it gets translated into binary and understanding the basics of resource management, IPC, file systems, etc. is essential if you will ever work in something lower level. If you're a CS major, algorithms and data structures are essential. If you're just learning front end development on your own or in a boot camp, you need to learn HTML and the DOM, events, how CSS works, and some of the core concepts of JS, not just React. You'll be better for it when the tools fail you or a new tool comes along.

Sure (knowing the underlying ideas and having proficiency in their application) - but producing software by conducting(?) LLMs is rapidly becoming a wide, deep and must-have skill and the lack thereof will be a weakness in any student entering the workplace.

Similarly, it's always been the case that copy-pasting code out of a tutorial doesn't teach you as much much as manually typing it out, even if you don't change it. That part of the problem isn't even new.

As a teacher, do you have any techniques to make sure students learn to write the code?

I'm an external examiner for CS students in Denmark and I disagree with you. What we need in the industry is software engineers who can think for themselves, can interact with the business and understand it's needs, and, they need to know how computers work. What we get are mass produced coders who have been taught some outdated way of designing and building software that we need to hammer out of them. I don't particularily care if people can write code like they work at the assembly line. I care that they can identify bottlenecks and solve them. That they can deliver business value quickly. That they will know when to do abstractions (which is almost never). Hell, I'd even like developers who will know when the code quality doesn't matter because shitty code will cost $2 a year but every hour they spend on it is $100-200.

Your curriculum may be different than it is around here, but here it's frankly the same stuff I was taught 30 years ago. Except most of the actual computer science parts are gone, replaced with even more OOP, design pattern bullshit.

That being said. I have no idea how you'd actually go about teaching students CS these days, considering a lot of them will probably use ChatGPT or Claude regardless of what you do. That is what I see in the statistic for grades around here. For the first 9 years I was a well calibrated grader, but these past 1,5ish years it's usually either top marks or bottom marks with nothing in between. Which puts me outside where I should be, but it matches the statistical calibration for everyone here. I obviously only see the product of CS educations, but even though I'm old, I can imagine how many corners I would have cut myself if I had LLM's available back then. Not to mention all the distractions the internet has brought.

I'm not so sure. I spent A LOT of time writing sorting algo code by hand in university. I spent so much time writing assembly code by hand. So much more time writing instructions for MIPS by hand. (To be fair I did study EE not CS)

I learned more about programming in a weekend badly copying hack modules for Minecraft than I learned in 5+ years in university.

All that stuff I did by hand back then I haven't used it a single time after.

I agree, and like to add,

> It is hands down good for code which is laborious or tedious to write, but once done, obviously correct or incorrect (with low effort inspection).

The problem here is, that it fills in gaps that shouldn't be there in the first place. Good code isn't laborious. Good code is small. We learn to avoid unnecessary abstractions. We learn to minimize "plumbing" such that the resulting code contains little more than clear and readable instructions of what you intend for the computer to do.

The perfect code is just as clear as the design document in describing the intentions, only using a computer language.

If someone is gaining super speeds by providing AI clear design documents compared to coding themselves, maybe they aren't coding the way they should.

The hardest part of coding has never been coding. It's been translating new business requirements into a specific implementation plan that works. Understanding what needs to be done, how things are currently working, and how to go from A to B.

You can't dispense with yourself in those scenarios. You have to read, think, investigate, break things down into smaller problems. But I employ LLM's to help with that all the time.

Granted, that's not vibe coding at all. So I guess we are pretty much in agreement up to this point. Except I still think LLMs speed up this process significantly, and the models and tools are only going to get better.

Also, there are a lot of developers that are just handed the implementation plan.

> (And the automated test suite will help them confirm that the refactoring worked properly, because naturally you had them construct an automated test suite when they built those original features, right?)

I dunno, maybe I have high standards but I generally find that the test suites generated by LLMs are both over and under determined. Over-determined in the sense that some of the tests are focused on implementation details, and under-determined in the sense that they don't test the conceptual things that a human might.

That being said, I've come across loads of human written tests that are very similar, so I can see where the agents are coming from.

You often mention that this is why you are getting good results from LLMs so it would be great if you could expand on how you do this at some point in the future.

Unfortunately I have started to feel that using AI to code - even with a well designed spec, ends up with code that; in the authors words, looks like

> [Agents write] units of changes that look good in isolation.

I have only been using agents for coding end-to-end for a few months now, but I think I've started to realise why the output doesn't feel that great to me.

Like you said; "it's my job" to create a well designed code base.

Without writing the code myself however, without feeling the rough edges of the abstractions I've written, without getting a sense of how things should change to make the code better architected, I just don't know how to make it better.

I've always worked in smaller increments, creating the small piece I know I need and then building on top of that. That process highlights the rough edges, the inconsistent abstractions, and that leads to a better codebase.

AI (it seems) decides on a direction and then writes 100s of LOC at one. It doesn't need to build abstractions because it can write the same piece of code a thousand times without caring.

I write one function at a time, and as soon I try to use it in a different context I realise a better abstraction. The AI just writes another function with 90% similar code.

> Going back to typing all of the code yourself (my interpretation of "writing by hand") because you don't have the agent-managerial skills to tell the coding agents how to clean up the mess they made feels short-sighted to me.

I increasingly feel a sort of "guilt" when going back and forth between agent-coding and writing it myself. When the agent didn't structure the code the way I wanted, or it just needs overall cleanup, my frustration will get the best of me and I will spend too much time writing code manually or refactoring using traditional tools (IntelliJ). It's clear to me that with current tooling some of this type of work is still necessary, but I'm trying to check myself about whether a certain task really requires my manual intervention, or whether the agent could manage it faster.

Knowing how to manage this back and forth reinforces a view I've seen you espouse: we have to practice and really understand agentic coding tools to get good at working with them, and it's a complete error to just complain and wait until they get "good enough" - they're already really good right now if you know how to manage them.

The article said:

> So I’m back to writing by hand for most things. Amazingly, I’m faster, more accurate, more creative, more productive, and more efficient than AI, when you price everything in, and not just code tokens per hour

At least he said "most things". I also did "most things" by hand, until Opus 4.5 came out. Now it's doing things in hours I would have worked an entire week on. But it's not a prompt-and-forget kind of thing, it needs hand holding.

Also, I have no idea _what_ agent he was using. OpenAI, Gemini, Claude, something local? And with a subscription, or paying by the token?

Because the way I'm using it, this only pays off because it's the 200$ Claude Max subscription. If I had to pay for the token (which once again: are hugely marked up), I would have been bankrupt.

> Going back to typing all of the code yourself (my interpretation of "writing by hand") because you don't have the agent-managerial skills to tell the coding agents how to clean up the mess they made feels short-sighted to me.

Or those skills are a temporary side effect of the current SOTA and will be useless in the future, so honing them is pointless right now.

Agents shouldn't make messes, if they did what they say on the tin at least, and if folks are wasting considerable time cleaning them up, they should've just written the code themselves.

> That's your job.

Exactly.

AI assisted development isn't all or nothing.

We as a group and as individuals need to figure out the right blend of AI and human.

I agree, as a pretty experienced coder, I wonder if the newer generation is just rolling with the first shot. I find myself having the AI rewrite things a slightly different way 2-3x per feature or maybe even 10x. Because i know quality when i see it, having done so much by hand and so much reading.

Similar place. I kept trying to get LLMs to do anything interesting and the first time they were able was 4.5 sonnet.

Best case is still operationally correct but nightmare fuel on the inside. So maybe good for one off tools where you control inputs and can vibe check outputs without diaster if you forget to carry the one.

I started doing it as soon as ChatGPT 3.5 was out. “Given this file tree and this method signature, implement the method”. The context was only 8k so you had to function by function. About two editor screens worth at a time.

Vibe coding was coined less than a year ago:

https://x.com/karpathy/status/1886192184808149383

GitHub Copilot came out with AI autocomplete 2-3 years ago I believe.

Typical blogspam clickbait of "I knew what LLMs were 2 years, but maybe didn't know the name for them, so we'll call that vibecoding."

"So why do you think a 10 kLoC vibecoded codebase will be any good engineering-wise?"

I've been coding a side-project for a year with full LLM assistance (the project is quite a bit older than that).

Basically I spent over a decade developing CAD software at Trimble and now have pivoted to a different role and different company. So like an addict, I of course wanted to continue developing CAD technology.

I pretty much know how CAD software is supposed to work. But it's _a lot of work_ to put together. With LLMs I can basically speedrun through my requirements that require tons of boilerplate.

The velocity is incredible compared to if I would be doing this by hand.

Sometimes the LLM outputs total garbage. Then you don't accept the output, and start again.

The hardest parts are never coding but design. The engineer does the design. Sometimes I pain weeks or months over a difficult detail (it's a sideproject, I have a family etc). Once the design is crystal clear, it's fairly obvious if the LLM output is aligned with the design or not. Once I have good design, I can just start the feature / boilerplate speedrun.

If you have a Windows box you can try my current public alpha. The bugs are on me, not on the LLM:

https://github.com/AdaShape/adashape-open-testing/releases/t...

Because a novel is about creative output, and engineering is about understanding a lot of rules and requirements and then writing logic to satisfy that. The latter has a much more explicitly defined output.

  Have you ever read a 200 page vibewritten novel and found it satisfying?

(The model was asked to generate a chapter at a time. At each step, it was given the full outline of the novel, the characters, and a summary of each chapter so far.)

I like this way of framing the problem, and it might even be a good way to self-evaluate your use of AI: Try vibe-writing a novel and see how coherent it is.

I suspect part of the reason we see such a wide range of testimonies about vibe-coding is some people are actually better at it, and it would be useful to have some way of measuring that effectiveness.

I don’t get the analogy because novel is supposed to be interesting. Code isn’t supposed to be interesting, it’s supposed to work.

If you’re writing novel algorithms all day, then I get your point. But are you? Or have you ever delegated work? If you find the AI losing its train of thought all it takes is to try again with better high level instructions.

I wrote this a day ago but I find it even more relevant to your observation:

—

I would never use, let alone pay for, a fully vibe-coded app whose implementation no human understands.

Whether you’re reading a book or using an app, you’re communicating with the author by way of your shared humanity in how they anticipate what you’re thinking as you explore the work. The author incorporates and plans for those predicted reactions and thoughts where it makes sense. Ultimately the author is conveying an implicit mental model (or even evoking emotional states or sensations) to the reader.

The first problem is that many of these pathways and edge cases aren’t apparent until the actual implementation, and sometimes in the process the author realizes that the overall product would work better if it were re-specified from the start. This opportunity is lost without a hands on approach.

The second problem is that, the less human touch is there, the less consistent the mental model conveyed to the user is going to be, because a specification and collection of prompts does not constitute a mental model. This can create subconscious confusion and cognitive friction when interacting with the work.

> My way is to understand my own codebase and look at the output of the LLM.

Then you are not vibe coding. The core, almost exclusive requirement for "vibe coding" is that you DON'T look at the code. Only the product outcome.

I had GPT-4 design and build a GPT-4 powered Python programmer in 2023. It was capable of self-modification and built itself out after the bootstrapping phase (where I copy pasted chunks or code based on GPT-4's instructions).

It wasn't fully autonomous (the reliability was a bit low -- e.g. had to get the code out of code fences programmatically), and it wasn't fully original (I stole most of it from Auto-GPT, except that I was operating on the AST directly due to the token limitations).

My key insight here was that I allowed GPT to design the apis that itself was going to use. This makes perfect sense to me based on how LLMs work. You tell it to reach for a function that doesn't exist, and then you ask it to make it exist based on how it reached for it. Then the design matches its expectations perfectly.

GPT-4 now considers self modifying AI code to be extremely dangerous and doesn't like talking about it. Claude's safety filters began shutting down similar conversations a few months ago, suggesting the user switch to a dumber model.

It seems the last generation or two of models passed some threshold regarding self replication (which is a distinct but highly related concept), and the labs got spooked. I haven't heard anything about this in public though.

Edit: It occurs to me now that "self modification and replication" is a much more meaningful (and measurable) benchmark for artificial life than consciousness is...

BTW for reference the thing that spooked Claude's safety trigger was "Did PKD know about living information systems?"

The term was coined then, but people have been doing it with claude code and cursor and copilot and other tools for longer. They just didn't have a word for it yet.

The author is using the term to mean AI assisted coding. Thats been around for longer than the word vibe coding

Very good point. Also, What the OP describes is something I went through in the first few months of coding with AI. I pushed passed “the code looks good but it’s crap” phase and now it’s working great. I’ve found the fix is to work with it during research/planning phase and get it to layout all its proposed changes and push back on the shit. Once you have a research doc that looks good end to end then hit “go”.

I have only ever successfully tried "vibe coding", as Kaparathy describes it, once, soon after VS Code Copilot added the chat feature, but timestamps tell that was in November 2023. So two years is quite realistic.

Yeah, that's what I pointed out.

Just more FUD from devs that think they're artisans.

Last week I just said f it and developed a feature by hand. No Copilot, no agents. Just good old typing and a bit of Intellisense. I ran into a lot of problems with the library I used, slowly but surely I got closer to the result I wanted. In the end my feature worked as expected, I understand the code I wrote and know about all the little quirks the lib has.

And as a added benefit: I feel accomplished and proud of the feature.

Happy to see someone else doing this.

All code written by an LLM is reviewed by an additional LLM. Then I verify that review and get one of the agents to iterate on everything.

It doesn’t have to be different foundation models. As long as the temperature is up, as the same model 100 times.

But those are the parts where it's important to struggle through the learning process even if you're slower than AI. if you defer to an LLM because it can do your work in a new codebase faster than you, that code base will stay new to you for forever. You'll never be able to review the AI code effectively.

That's a misunderstanding based on loose definition of "vibe coding". When companies threw around the "90% of code is written by AI" claims, they were referring to counting characers of autocomplete basing on users actually typing code (most of which was eequivalent to "AI generated" code by Eclipse tab-completion decade ago), and sometimes writing hyperlocal prompts for a single method.

We can identify 3 levels of "vibe coding":

1. GenAI Autocomplete

2. Hyperlocal prompting about a specific function. (Copilot's orginal pitch)

3. Developing the app without looking at code.

Level 3 is hardly considered "vibe" coding, and Level 2 is iffy.

"90% of code written by AI" in some non-trivial contexts only very recently reached level 3.

I don't think it ever reached Level 2, because that's just a painfully tedious way of writing code.

GitHub copilot was released in 2021, and Cursor was released around October 2023[0].

[0]: https://news.ycombinator.com/item?id=37888477

It seems the term has been introduced by Andrej Karpathy in February 2025, so yes, but very often, people say "vibe coding" when they mean "heavily (or totally) LLM-assisted coding", which is not synonymous, but sounds better to them.

Not everything can be built by one person. This is why a lot of software requires entire teams of developers. And someone has to have vision of that completed software and wants it made even if they had to delegate to other people.

Do you honestly get satisfaction out of writing code that you've written dozens of times in your career? Does writing yet another REST client endpoint fill you with satisfaction? Software is my passion, but I want to write code where I can add the maximum value. I add more value by using my experience solving new problems that rehashing code I've written before. Using GenAI as a helper tool allows me to quickly write the boilerplate and get to the value-add. I review every line of code written before sending it for PR review. That's not controversial, it's just good engineering.

It's not a craft.

We're modern day factory workers.

You can update the spec as you go... There's nothing that makes the spec concrete and unchangeable.

That's the sad part. Empiricism is scarce when people and companies are incentivized to treat their AI practices as trade secrets. It's fundamentally distinct from prior software movements which were largely underwritten by open, accessible, and permissively-licensed technologies.

"They got more VC than me, therefore they are right".

You gotta have a better argument than "AI Labs are eating their own dogfood". Are there any other big software companies doing that successfully? I bet yes, and think those stories carry more weight.

The practice is older than the name, which is usually the way: first you start doing something frequently enough you need to name it, then you come up with the name.

When using GenAI (and not), enforcing guardrails is critical. Documenting decisions is critical.

Sharpee’s success is rooted in this and its recorded:

https://github.com/ChicagoDave/sharpee/tree/main/docs/archit...