chuckadams
7 hours ago
New policy boils down to "AI or not, it's still your code and you're responsible for it". I can get on board with that.
hliyan
7 hours ago
A good thing about the "you're the author regardless of the tool you used to produce the code" policy is that if a developer repeatedly submits poor quality AI generated code, one does not blame the model or the agent, one bans/terminates the developer.
bfgeek
5 hours ago
The issue that open source projects are facing at the moment is that it takes significantly less effort to submit a patch for review.
A lot of developers who are submitting these AI patches don't necessarily understand the patch, so the onus is on the reviewer/code-owner.
The reviewers are getting swamped (some reviewers are receiving 100s or patches per month). If feedback is provided at lot of the time the patch author will just copy paste from an LLM, so the reviewer is essentially just coding with an LLM with more steps.
Prior to LLMs reviewing code was a mentorship experience, the patch author would likely learn a bunch afterwards. Now less so.
As a result a lot of projects are closing to external contributors.
I'm not sure what the answer is, LLM are great at speeding up coding/understanding/etc, but the valuable/expensive piece of work has shifted to reviewing.
tdrz
2 hours ago
I'm an OSS maintainer and to me it's not just about the review itself. Being greeted by a wall of text for every little small thing is counter-productive. I hate going through 2 pages of text for each PR. It usually shouldn't take more than a couple of sentences if you understand the issue and the solution.
But most important for me: lots of time the PR just adds even more code, although other options do exist (ie sometimes REMOVING some code). You have to know the codebase well in order to find those objectively better solutions.
localhost
3 hours ago
This is Amdahl's law in action. [1]
Until we figure out a good way to leverage humans in all of this ("Attention is all you need" applies equally to humans as it does to models) productivity gains for the system will always be limited by Amdahl's law. Gwern has an excellent post on this. [2]
[1] https://en.wikipedia.org/wiki/Amdahl's_law [2] https://gwern.net/guardian-angel
Loughla
4 hours ago
The question becomes, does it take more time to create or review in the Grand scheme of the software life cycle?
Because if it's still a time saver, even with the increased review load, then it's a win, correct?
I'm not a coder so I have zero idea. Thoughts?
a1o
4 hours ago
Nope, most of the PRs are authored entirely by agents with people instructing them to “pr famous projects in my name to increase my GitHub profile value or “hire ability”. The original authors have no idea what their agents are writing, these are mostly badly quality models (people doing this are not only cutting corners, but also using the less amount of money/tokens to do so). If the agent creates 200, 400 or whatever PRs and get 5 in the agent is still “winning” for the person instructing it. The maintainers though have to filter these out on the other side. A common case is a fix to something that was already fixed simply because the agent worked on old code assumptions.
sebmellen
4 hours ago
Just had a thought, what if you made new contributors write a human-authored essay on why they’re submitting a patch, and then check it against Pangram? Pangram is really accurate from what I’ve found…
newswasboring
3 hours ago
> Nope, most of the PRs are authored entirely by agents with people instructing them to “pr famous projects in my name to increase my GitHub profile value or “hire ability”.
I've heard this in a lot of places, don't have a way to verify this in action. Do you have any citation or just gut feel (still valid but less useful for me).
pyreko
a few seconds ago
Anecdotally, as someone who's been starting to get more and more agent-only/driven PRs, a good amount of the time it's by an account with little to no activity until one day they suddenly start doing 50-100+ PRs a _day_ on entirely unrelated projects... you can kind of guess what they're doing.
As for motive, it probably varies, though some will loudly put that they're some agent manager or something with lots of contributions, so you can guess their motivations there.
In other cases, these are instrad brand-new accounts which is a related but I guess slightly different story... not gonna say these are spam bot accounts being juiced with fake activity but well...
1718627440
4 hours ago
It's a well-known trope, that it's harder to read code than to write it, and review is more complicated than to read it, so I do not understand what you want to imply?
blackqueeriroh
4 minutes ago
Maybe it is for people who have written code all their lives, but as someone who started with reading code and has read far more code than I’ve ever written, reading code is WAY easier for me.
bch
2 hours ago
> The question becomes [...]
"A question is...". To my mind not the most important question, if one plays-out to a logical conclusion the scenario you're proposing.
> [...] if it's still a time saver, even with the increased review load, then it's a win, correct?
No - and that's bordering-on (if not fully) rude disrespect of reviewers time and effort. One way to think of this is in terms of Brandolinis Law[0]. Pushing work back to submitters is going to have to happen. Low-effort "submissions" are first and foremost "low-effort" - that's going to have to be driven home.
sarchertech
3 hours ago
Well historically the consensus was that it was harder to read code than to write it, but that leads to uncomfortable conclusions these days, so that bit of common wisdom has mostly been ejected.
> I'm not a coder so I have zero idea. Thoughts?
I’m not trying to be mean, but this comment is basically “I have no experience with this topic, but it can just be boiled down to this one simple question right?”
catlifeonmars
4 hours ago
Probably a time waster because of the intermediate patch contributor. It’s like a game of telephone at worst, at best the reviewer could just use their own LLM and get the same result.
Here’s my hot take: maybe code contributions are obsoleted by coding agents. No one will accept outside contributions because it’s faster to do it themselves.
pydry
4 hours ago
It depends entirely on how much slop they are flooded with.
tiahura
4 hours ago
Do all of these folks get the comped Pro Max subscriptions? If not O&A should be.
Or, at the very least, the community should be paying for them.
lokar
5 hours ago
To me, that touches on the central issue. I think most people support the “I don’t care about the tool, only the outcome “
But, we have long had a stable social system for collaboration, and in particular dealing with low quality contributions. Mostly (or at least ideally) it involves taking some time to look at the contribution and provide some feedback. The time invested is not worth it on a single exchange basis, but it’s a repeat game, so you are investing in the future.
LLMs, and people willing to just spam crap they don’t really understand blows that all up.
moring
5 hours ago
I'm not convinced it blows up. It might also end up placing a larger burden on contributors, and especially first-time contributors, to provide concise, high-quality documentation that justifies their contribution.
I'm thinking of something like: Explain, in at most 300 characters, why we should merge your change, or at least why we should invest the time to read a longer explanation (somethings things ARE complex). If you don't do that, or those 300 characters aren't convincing, or have nothing to do with your change or the longer explanation, then you get a canned response so the maintainers waste minimal time.
thayne
3 hours ago
That doesn't really solve the problem. I've gotten PRs that do do an adequate job of explaining why why something should be implemented. Then I look at the code and it doesn't fully solve the problem, or it does so in a way that will obviously break something else.
moring
3 hours ago
In that case, isn't a sufficiently reasoned "why" a contribution in itself?
Not trying to downplay the time wasted to reject the PR, though.
Keyframe
5 hours ago
you know EXACTLY what would happen in such a system.
moring
4 hours ago
I don't, please enlighten me.
lokar
4 hours ago
My prediction is that many new contributors would complain loudly. Some existing contributors would sympathize.
It’s I think a normal evolution of a social norm. There will be a period of competing values.
Keyframe
an hour ago
people would use LLMS for those summaries, defeating the purpose. You can already see it in PRs in companies where LLM code is pushed.
CamperBob2
4 hours ago
Explain, in at most 300 characters, why we should merge your change, or at least why we should invest the time to read a longer explanation (somethings things ARE complex) would simply become a prompt.
The solution, of course, is to use the best available LLMs as the first line of defense against slop contributions. But if the project maintainers have issued a sweeping fatwa prohibiting an entire class of useful tools, that option along with many others is off the table.
tiahura
2 hours ago
I wonder if some of these projects with ayatollah maintainers are looking at forks sooner rather than later?
hypfer
5 hours ago
Nah, that was already blown up way before LLMs existed.
Foss wasn't exactly not miserable before 2022. The dysfunction just got amplified so that we cannot fix it anymore by just throwing more human resources at the problem.
lokar
5 hours ago
I tried to nod a bit to the inconsistency. Some projects clearly lost patience with imperfect contributions long ago. But many were still pretty supportive.
This dynamic also plays out in many companies. I see a lot of posts from mid to sr engineers frustrated with plummeting code quality and unsure how to manage the reviews. They are increasingly pulling away from the idea that they should mentor people to do better.
Retric
4 hours ago
The correct response to irresponsible AI use in the workplace is maintaining standards and rejecting poor quality. Performance improvement plans as a fair warning followed by firing people works for incompetence or laziness, AI doesn’t change anything.
I’ll admit I’ve seen a spike in negative performance reviews but this is the new normal people will adapt.
blackqueeriroh
3 minutes ago
It’s almost like late-stage capitalism isn’t built to support mentorship and education and enabling human improvement!
Keyframe
5 hours ago
yeah, that's fair. Problem so far has been when poor developers send massive volume of slop for review. Takes time and effort to review or at least glance and test it, especially if developer hasn't bothered.
notatoad
4 hours ago
that seems like two separate problems.
too many patches from unknown and untrusted contributors is a triage problem - telling a known and trusted contributor they can't use AI to help work on something just because other people are sending slop for review is throwing the baby out with the bathwater.
catlifeonmars
4 hours ago
More like throwing the bathwater out with the baby.
NuclearPM
5 hours ago
Terminates???
smallerfish
3 hours ago
We need sufficiently severe penalties for vibe coding. First time out we break your legs, but if you come back with another vibed patch its around the back of the barn for you.
qsera
6 hours ago
What stops them from submitting PRs from another account?
jayofdoom
6 hours ago
Nothing. And that's the problem -- the issue isn't "should a contributor be able to use AI" it's quickly becoming "is this entire contributor an AI agent". Every actual-human who gets a bad first experience hurts the project. The contributors having to be suspicious of new people hurts the project. The whole thing just sucks, and I don't know how to fix it.
I use claude-code, but the AI ecosystem and some of the autonomous tooling coming out of it -- which some people "helpfully" use irresponsibly and point at OSS projects they aren't personally invested in -- enables a LARGE amount of abusive behavior towards open source projects. This is even before we get into how many infrastructural resources crawlers eat up.
Me, and most other OSS contributors, signed up to be a mentor, to create software, to work together ... not to be the equivalent of a reddit moderator banning AI agents left and right.
kikoreis
5 hours ago
"Nothing" is not accurate in the context of Debian as per comments above; it's not like Github in that sense.
digitaltrees
6 hours ago
What is a good project for a human to get human mentorship? Asking for a friend :)
kikoreis
5 hours ago
Debian itself is actually great: https://mentors.debian.net/sponsors/
lrvick
5 hours ago
Always happy to train up new packagers and maintainers.
zenoprax
4 hours ago
This is the second time I've seen this project mentioned on HN. Their white paper is very interesting and aligns closely with my interests but I don't see much on the main site about how to get involved.
Are they using their Matrix server to filter out drive-by/low-effort contributions?
lrvick
3 hours ago
I am the founder of the project. We accept every contribution that solves a problem, even drive-bys. Though they may often sit for a bit as we often merge in batches. If there is something missing, make a PR by all means.
Ask any questions on matrix.
hypfer
5 hours ago
Usually, mentorship is a quite well paid service.
cdata
6 hours ago
I can't speak for all, but many mature open source communities have explicit or ad-hoc reputation schemes that are used to triage contributions. A fresh account means starting over with zero reputation, which in turn means your contributions wait at the back of the line.
tremon
5 hours ago
What part of [0] or [1] makes you think that it is easy to have multiple accounts?
[0] https://wiki.debian.org/DebianMaintainer#Becoming_a_Debian_M...
[1] https://wiki.debian.org/DebianDeveloper/JoinTheProject/NewMe...
kikoreis
5 hours ago
Well. This is Debian, known for how strict they are about vetting developers. I just remembered Sladen refusing to key sign because my passport circa 2005 didn't have a machine readable slug in it.
smashed
5 hours ago
Debian is notoriously very strict on accepting maintainers and they are not anonymous.
MithrilTuxedo
6 hours ago
Reputation, or lack thereof.
Proof of authority, essentially.
kaffekaka
6 hours ago
What stopped them before?
qsera
6 hours ago
Before they were not able to spam large PRs
Supermancho
5 hours ago
Why not? I'm pretty sure a developer could spam large PRs before and regardless of AI involvement is allowed or not.
inigyou
4 hours ago
How would you generate enough large PRs to count as "spam"? You could add gibberish or delete random lines of code but it was obviously not plausible looking.
zeendo
3 hours ago
Obviously they could have. Obviously it's much easier now.
Mawr
2 hours ago
You can cut down a tree regardless if you use a knife or a chainsaw.
You can get to work each day regardless if you walk or drive.
JohnHaugeland
3 hours ago
this article is about the flood of junior developers who are producing enormous prs they’ve never seen full of untrustworthy code that a human hasn’t evaluated
arguing that it’s possible to make them by hand isn’t just missing the point; it’s missing the entire discussion
2Gkashmiri
5 hours ago
In anonymous systems, where we don't use real names, your "handle" has an "age". That becomes your most prized possession.
I have a reddit account thats over 12 years old. I have left the posts/comments open. That means, my "post history" becomes a way to advertise my skills. I can reference comments from a year ago or 6 months ago about a certain topic to show how I handled something. That does have an effect on people.
As opposed to a random 1 day old account. No one takes them seriously, I guess the same is here as well
inigyou
3 hours ago
Be extremely careful with that account then, because they ban accounts for the most minor things now. Stuff like posting evidence disproving something the moderator agrees with. And they ban your account from the whole site, not just from that subreddit any more. Automated algorithms also ban accounts for reasons that don't make any sense.
So if you're using your account to advertise yourself and look for jobs, I'd use it only for that. All off-topic discussion should be on a different account with a different physical device and IP address (reddit checks these things and cross-bans) that have never interacted with your main account.
ipaddr
5 hours ago
No one takes a 12 year old Reddit account seriously either. It's not a platform you can take anyone seriously. The biggest trolls have 12 year old accounts.
clort
4 hours ago
the age is not the measure of quality. the GP said they left the posts/comments open. anybody can read that and check if they are or not a serious contributor
amysox
4 hours ago
Exactly the sort of thing I agree with. If there's code, ultimately, some human is responsible for it, regardless of whether they produced it by hand, by AI, or by rolling dice. (Not that anyone does produce code by rolling dice, mind! Or, if they do, I really don't want to see the results...)
All project contributions must be submitted by *identifiable human participants* who accept full responsibility for their content. Automated agents, bots, or autonomous AI systems *may not* independently submit issues, pull requests, or other contributions.
Contributors may use software tools, including AI-assisted tools, but the submitting contributor *must:\*
- Fully understand the contribution.
- Be able to explain design and implementation decisions without the use of AI.
- Accept responsibility for maintenance and correctness.
Contributors should indicate AI-generated content in issue and pull request descriptions and comments, specifying which model was used.
Do *not* use AI to reply to questions about your issue or pull request. The questions are for *you,* the human, not an AI model.
https://git.erbosoft.com/amy/amsterdam/raw/branch/main/CODE-...mckn1ght
4 hours ago
Using LLMs is basically rolling a huge number of weighted dice.
daishi55
3 hours ago
So is asking a human to write code, if you think about it…
megatoaster
3 hours ago
Dice don’t have lived experience
newswasboring
3 hours ago
How does that matter for a coding task?
sunshowers
3 hours ago
Well, one example is that patterns in code now can make you recall bugs you've seen in the past. I had this realization where I used an LLM to do a big refactor, and it fixed a bug along the way that I should have realized was much more urgent than the refactor, having seen an identical bug in the past (it was related to prefix-checking in a set that wasn't constructed to do that kind of prefix-checking). Had I done the refactor by hand I'd likely have realized it much quicker.
oooyay
7 hours ago
This is the way it should've always been. I don't care what tool you use, but you'd better be ready to stand by and explain the results if necessary.
dguest
6 hours ago
Are there examples of anyone discussing this and reaching other conclusions?
We've come to the same conclusion in our organization and while it felt like it needed to be discussed and stated, I've never heard of anyone coming to a different conclusion.
[EDIT: I was referring specifically to the idea that you have to stand by the code that you write. Is anyone deciding on a policy that says "the AI did it" is a valid excuse for breaking code?]
internet2000
6 hours ago
> Are there examples of anyone discussing this and reaching other conclusions?
Yes. Very notably: https://ziglang.org/code-of-conduct/
sodapopcan
6 hours ago
On the opposite side or the noteable spectrum, look up "Steve Yegge wants you to stop looking at your code." He said it in a video interview, though plenty of summaries online of exact wording and ideas. Though it's being argued that you could still be responsible for it without looking at it?
rapind
6 hours ago
Extremes on both sides. I'd wager were moving in Yegge's direction though, like it or not.
> Though it's being argued that you could still be responsible for it without looking at it?
If you have a much test coverage as SQLite, I'd be tempted to trust it, and I think that's where we are headed. Someone still needs to define these guardrails though, so IMO the developers job is just mutating into something else, but is still necessary. Long term, who knows.
Topfi
5 hours ago
For me, it’s not a question of whether I like it, nor where we might move in the future, but purely what is proven currently. In my experience, as in my evals, any model, including notably Fable 5 and GPT-5.6 Sol can and indeed do produce code that does not consider many edge cases, can be hard to maintain (especially Sol though it’s output has become markedly better with recent changes to Codex) and sometimes introduce bugs with visibly apparent issues. Existing test coverage doesn’t necessarily prevent such issues when changes or additions are made. The idea that models tend to gravitate towards the house style without major CI based restrictions also has not born fruit in my book. More, I have found even the frontier models amazing at utterly deviating from established standards and tests, what lint, tc and co don’t catch the model will take to introduce code that feels and reads like it does not fits with the rest.
Models also still can have major task adherence issues, so even a perfectly well scoped, well considered prompt that covers all situations and coverage can, in many scenarios lead to output users may not expect or notice without reading code. Seen it a lot, just lost some blood and tears to a very straight front end implementation task that I ended up applying manually cause both Sol and Fable struggle to copy a few dozen HTML/JS/CSS snippets properly into a slightly obscure codebase, despite the reference being fully featured and even using the same token scheme, etc…
We also see such issues in what labs deliver in their own software products. We are not at the stage where a human shouldn’t sign off on code that they have tested and understood. A hypothetical future, I don’t really care for, we’d notice when labs stop shipping easily apparent bugs, though even then, I’d still feel there is a while in between that and not needing to assess the output anymore.
There is, in my mind, no need for this “narrative shifting”, overtone window (don’t get me started on the idea of applying that to what should be a purely technical discussion), convince the unbelievers nonesense that some spend their time on. If models could provide code that didn’t necessitate review, there’d be no one needing to argue that case. If and when they can, we’ll know it, but not because of those espousing this currently.
rapind
4 hours ago
It's not about the prompt, it's about the guardrails. Static types, compiled, massive test suite, etc. That what I mean by extreme level of test coverage like SQLite. You'd actually need to be an expert in C and SQLite and the SQLite test suite just to sneak a bug into it. Testing and other verifications are going to be an exciting area for the foreseeable future.
You can still write non-performant code even with tests, but you can also test performance!
Topfi
3 hours ago
Am about as far away from a C expert as one can be (and probably want to keep it that way for the time being), so happy to take your word on that front. Still feel, it'd be easy and in their interest for one of the major labs to provide a verifiable showcase/research paper on that front. Zig-to-Rust in Bun is the closest and that is very impressive, but I feel it doesn't necessarily map to new features/changes beyond porting. The compiler, browser, etc. all equally do not fill me with that much confidence, but perhaps in SQLite modifications could be hard for a model to mess up.
What I will say is that I feel valid in saying that the developer behind gas town (a shudder runs down my spine even having to think about that) is not working in a test suite anywhere comparable to SQLite, though in fairness, you never claimed that.
Again, you are likely not wrong that with an environment like SQLite, bugs may be hard to sneak in, but then again, isn't retaining that so important that it is a strong argument for developers auditing LLM output even more strictly, lest we loose the resilience we currently have?
nick__m
4 hours ago
Sol and Fable are not doers, they are thinkers. Using them to "copy a few dozen HTML/JS/CSS snippets properly into a slightly obscure codebase, despite the reference being fully featured" is one of the most ineffective possible usage of those model. What you described is a job for luna.
Also I suspect you overprompt, with GPT-5.6 (sol, terra and luna), a lot's of what's used to be mandatory in the AGENT.md is redondant at best and frequantly actively harmful.
Topfi
4 hours ago
I also tried Luna (is my first line for such task in general because of the speed and efficiency, but not because Fable or Sol can’t be “doers”, in my evals both are superior in well scoped task adherence and implementation vs Luna and the like, have yet to find a task Luna can that Sol or Fable can’t) along with Opus 5 (cause the Max sub gives a lot of that I can freely use after Fable runs out for the week so I wasted three runs on that) and Ox Alpha/GLM-5.3 Flash (as that was free at the time). Sol (on High) and Fable (on Medium) were acts of 12h desperation, before I did it manually, after the three other models failed, first with a very simple prompt, failure, revert commits, then a very specific implementation plan with clearly scoped verification, failure, thus manual in 45 minutes. It was a major port of a new settings UX for a Firefox fork, all test baked, all with 1:1 replacement targets in the reference and the original. It was merely massive context wise and only 5.4 has ever behaved well across compactions in a manner that I could validate in evals. No model before or since gave me confidence in this regard and I’ll retroactively test with 5.4, somehow forgot to try that one.
I do not have an AGENTS.md in any of my repos at all, stopped using them around 5.2. I was a bit ahead of the curve with that, my evals very early showcased how limiting those tend to be, same why I never used MCP, were useless context bloat.
Will maybe make an edit explaining a bit further what the task was once I get to my computer.
Topfi
2 hours ago
I have a settings panel implemented in HTML/CSS/JS for a Firefox fork that "could/should have been a desktop environment". Bit of an odd project really, mainly out of a very specific conviction concerning modern applications, the way LLMs and task specific models are currently not leveraged well by any browser, my own tendency to have 400+ tabs open at a time across multiple projects, my opinion that it is the perfect place to finally apply a lot of UX opinions I have held for a while and push in a very distinct direction along with core critiques I have concerning PKM applications I haven't seen addressed despite trying every PKM application under the sun. Neither here, nor there.
So this "thing" is mainly a Firefox fork and most UI is basic HTML/CSS/JS (as is the case in upstream). Development is patch baked, CSS tokens must follow a defined and CI enforced standard, etc. LLMs can be very helpful in development, I got a small CLI tool for patch, token management and basic quality gates, which I started working on a few months ago to keep the most atrocious LLM output at bay. Has lead to the revieability of output improving meaningfully over markdown monstrosities, though OpenAI models still manage to sneak hard to parse output past it. This CLI tool along with some task specific scripts also ensures reuse of proven upstream infra like Places (Good lord dear Firefox developers, is Places nice to rely on), consistent regression testing (especially in memory constraint scenarios), etc. Basically, I can and do regularly make additions with LLM assistance, I review it, I discard and restart or improve upon it (rarely accept scoped changes wholesale. This to say, I got some experience in the use of models for coding assistance and I (thanks to the amazing docs and a lot of considerations for the architecture I want) do know what I want, how I want it and how to get there. Also got private LLM evals that often uncover which labs tend to perform suspiciously well in public benchmarks vs private ones and what models still struggle with along with why, so yeah, certainly can always improve but I got, I'd argue, enough of an idea to where my critique of LLM coding limitations has legs.
Which brings us to what I was trying to implement and how I went about it: Settings works. Fully featured (including a few cross-site-tracking specific clarifications that came from a HN interaction a few days ago), tab specific previews for what changes affect regarding themeing, well tested (manual and static), integrated to leverage what FF provides where possible.
It does (or rather did) look functional/God awful though. To the point where I was uncertain that certain previews could be easily parsed by new users. I thus opened Adobe XD, did some early mockup work, tried a few core concepts, settled upon two, then (using Claude Fable 5 low) created a plain export of the existing settings code from our furnace components and patch baked edits into regular HTML/JS/CSS files. I manually verified, this export worked, the tokens were in the correct format, the code reflected what Hominis applied (including what was required for stand-alone of course) and externally called features upon interaction did provide log output linking to the pre-existing functions that meant reimplementation based upon this should be easy.
I then took that to Claude Design using Fable 5 on High. I provided the code along with linked branding files (which due to the way branding patches are handled were simpler to provide separately) and my Adobe XD mockups. A few dozen iterations later, along with some exports and re-imports due to manual changes (some animations in tabbing/"focus mode" showcases needed to be "just so" and prompting would have been inefficient to get there), I had a new user experience I was far happier with. Simpler, yet better at communicating, far more visually appealing and resolving some concerns I had, I felt pleased and will admit, Fable 5 via Claude Design provided valuable output and did, what it does best, make iterating on multiple UI concepts next to each other to settle on a final option from many, far quicker.
I then exported and took that to GPT-5.6 Luna (I have € 23,- Codex only so am a bit stingy on when to use what). But so what? I had verified, the tokens were the same. The naming of elements remained consistent to what Hominis Settings used, the backend changes were practically none-existent. I had audited the output end-to-end, made some refactors and house style specific improvements to keep everything more auditable, everything seemed suited for a quick port. What could possibly go wrong?
Anyone whith pattern recognition will likely guess what. Basic 1:1 applying? No dice. The first attempt failed as, once the context window had compacted twice, the model started leaving the very clearly paved path laid out. Stylised favicon in the showcase? Gone. Hamburger menu in the showcase, compressed. Vertical tabbing change interlinked with the canvas section? Very funny. The model started no longer following the code, it started taking screenshots and applying what it could see from that, despite the original prompt (just checked) vey clearly stating a simple code port, section per section, with any deviations to be listed in a designated file I maintain for long running tasks.
Basically, Luna did implement changes to the settings that felt tangentially right and a casual observe may not notice all the regressions and deviations, but I did. So I stopped it.
Sol and Fable didn't fare much better. Sol did stay on target longer, but it went off the rails around the privacy tab, introducing functional regressions to the way I had implemented cross-site cookie blocking, which were never requested, nor should that code even have been looked at. I reset the repo and handed it over to Fable 5 (medium). I had a third of my weekly usage left on 20x Max, reset the day after at 3AM so no harm either way.
Should be plenty. Wasn't plenty. Since a while (I think Opus 4.7, but could be wrong), Anthropic models do decently well regarding long term, high token tasks. Up to 450k, I have been able to reliably reproduce consistent implementation. The model, using a few subagents (which should have reduced the risk of context window issues further), went to work and after a few hours (and about 20% of usage less), the model proudly presented its work. I was at work and by the time I came back, I was a bit miffed to find that the model had, in its wisdom, decided to not used the well established and consistently used mar to bind in branding icons. No biggie, easy fix, albeit a bit stupid. ESPECIALLY SINCE I SAW IN THE CLAUDE CODE TRACES THAT THE MODEL HAD SURPRESSED A WARNING ON THAT VERY FRONT. Whatever. Then I saw it had not wired in the existing browser data deletion and export logic. It hadn’t modified existing logic unlike Sol, so hey, that’s nice. But it had not wired up the existing settings when they did not have any immediate feedback in the implementation reference.
Ox Alpha, it just spanned in circles, didn’t seem to like our fireforge CLI and furnace componets, but it was worth a free try. Opus 5, the model most obsessive in checking its own work, took screenshots. A lot of sscreenshots including every few hundred ms to cover animations. Nice. BUT IT CREATED ITS OWN TOKENS INSTAD OF REUSING WHAT WAS PROVIDED. Thus, styling deviated heavily.
At this point you might ask why I don’t do it manually and I will in the end anyways, but I was surprised to find such a clear case of a seemingly straightforward task flummoxing multiple LLMs. This is aided by my unique code base (the upstream FF code is also gitignored which likely flummoxes some models trained heavily to leverage git to track changes), everything needs to be patch backed and follow a specific implementation style, etc. But I had more important things to do and I wanted to see whether I couldn’t get it to work yet.
Inspired by Opus 5, I wrote a new prompt, specifically laying out a visual comparison and code diff workflow. Only these changes, only in this manner, only move on ones you have gotten visual confirmation, specific cross checks. I included a hand written markdown outlining which change affects other settings sections (even though that is obvious reading the reference code), how to approach tokens, etc. Obsessively descriptive and (I feel) unnecessarily so, but why not. Best case, it works, worst case, I’ll spend an hour doing it manually. I had other things to do not behind a keyboard, so why not one last Hail Mary.
Fable 5, ever efficient when using visuals, used the last rest of my usage, though I did see some roundabout approaches after the fact that make me doubtful it’d have cracked this. Opus 5 went off the deep end taking ui-captures across the entire code base, which lead to a very liberal application of settings tokens outside settings.
Sol did take a night and got 40% there when I asked for a pause once the in flight slice had landed. It did port the UI/UX changes in a way that on the surface looked and felt correct. It did not touch the backend in unacceptable ways. And it did cross checks. Animations also behaved correctly, though it did apply a rule on backend usage a bit to strictly, incorporating that into a preview for search by turning that into an actual web search, not a UX demo. Dumb, but not fatal.
Great success, what am I complaining?
Well, the code. It had done what Sol likes to do and turned very cleanly written, readable code into a hard to parse mess. This included touching existing test files.
And at that point I said “fuck it, I’ll do it myself”. And I did. In less than an hour, listening to Paris Palamo, Lyre Le Temps, Sting, Sade, SynthV and some Nirvana.
If I didn’t look at the code and I didn’t have strict standards for the UI, but just considered what looks in line on the surface level/feels right/“vibes” and what “works”, many of these attempts would have been accepted, as their issues are rarely apparent on the surface. That’s part of the issue in my book and why I’m firm we are far from “don’t read code”/“don’t test”/“skip qa”…
scotty79
6 hours ago
> Though it's being argued that you could still be responsible for it without looking at it?
Works for ToS and software and media licenses. Allegedly.
sodapopcan
5 hours ago
HA! Touché.
Aurornis
6 hours ago
Yes. There are pockets of hardline anti-LLM policies for various reasons.
SourceHut bans LLM generated code from their platform https://news.ycombinator.com/item?id=49461724
The Asahi Linux project bans LLMs. Unfortunate as LLM reverse engineering and driver writing would bring Linux to more Mac models much faster.
The Zig project rejects LLM based contributions.
Some of these policies were introduced when LLMs were really bad at writing code and haven’t been revisited. Some groups don’t care about the quality and object on moral or copyright grounds.
JohnHaugeland
3 hours ago
notice how mainstream those projects aren’t
KennyBlanken
5 hours ago
> Some groups don’t care about the quality and object on moral or copyright grounds.
Precisely why Debian Project's decision is both baffling and infuriating.
I hope Debian forks over this. They have severely lost their way.
iamnothere
an hour ago
Debian is a democracy; this policy is the result of a vote. A majority of contributors found it acceptable.
inigyou
3 hours ago
Note that Debian is mostly an aggregation of packages. There's nothing wrong with Debian importing a package that was written by AI, if it's something that people want to install. The questionable copyright of that package does not taint Debian as a whole. Debian allows public domain packages, so it wouldn't even be a DFSG violation.
There's also nothing wrong with an AI-written install script (if it's correct) because those are likely too trivial to copyright anyway.
What are your specific worries? Mine is brain rot.
JohnHaugeland
3 hours ago
if it does, that fork is dead by the end of the year
i don’t think you understand just how niche you already are
pessimizer
4 hours ago
Debian gives attribution to every single open source project in existence that is capable of running on Linux, and is Free Software. It is exactly the project that shouldn't worry about copyright.
The "moral" argument I haven't even heard. It's just people pouring out invective, likely imo because Ruby-monkey is not going to be a $150K job anymore. I have problems with these companies, but companies who are problematic can be targeted specifically, and somehow everybody started using Facebook's frontend framework and Microsoft's IDE, all on MacBooks, and this is what they're throwing a shitfit about. To me it's narcissism, not a coherent moral objection.
If you hate these companies (as you should), there are open models. But most didn't object to anything that didn't threaten their income.
bogdanoff_2
an hour ago
I could imagine (in a commercial organization) a case for something like: "we're obviously going to be relying more and more on AI, so if bad code comes out of it, instead of just shaming the engineer, let's study together what happened -- what was the chat context, the model, the system prompt, etc... ? What seemed to cause the issue? What processes or systems could we put in place to improve it? Etc... -- and then use the learnings to improve AI use throughout the whole org."
orthogonal_cube
5 hours ago
> I was referring specifically to the idea that you have to stand by the code that you write.
Any serious entity which has experts handling code will do so, yes.
My employer, for example, has various teams of actual developers and others with a significant number of “not really” devs who found their way into DevOps roles. The teams with seasoned developers hold the author responsible as to the integrity and quality of code, regardless of tools used. They won’t hesitate to call someone out directly for low-quality submissions in Teams channels. The teams that have a lot of “not really” devs will blame the model for issues that surface. Of course, they’re also pushing to not have to write any code directly by FY2028, but instead have models do it as part of an internal AI initiative. Somehow the idea of using AI to generate reports, process reports, write code, test and deploy got signed off.
bodge5000
6 hours ago
As others have said there are other conclusions going both ways (though I do think most settle on as you said), though one very fair conclusion I came across is that it's not allowed because the whole issue of copyright is still somewhat unresolved, and though its fine right now, that could (probably wont, but could) massively backfire later.
matheusmoreira
6 hours ago
nicoburns
6 hours ago
Yep, in both directions. Some projects ban use of AI entirely. Others are happy to live with slop.
__MatrixMan__
4 hours ago
"This works for me, but I need help understanding why, because maybe it would help others" is a legitimate position to be in--has been since long before LLMs. I think we miss out on something when we ignore that position.
I'm not saying that anybody should be obligated to review large volumes of low confidence code... just that we should not make policies that ignore shades of gray which, with some collaboration, could bear fruit.
blm126
4 hours ago
I think the key thing you are missing here is that most successful open source maintainers don’t lack expertise, they lack time. They are generally capable of improving their projects in any direction they want without help. Mentoring new contributors is basically always a loss in the short term as far as the maintainer is concerned. The project does get improved, but it generally takes longer than if they did it by themselves from scratch, and the improvements might be in an area they don’t much care about. The payoff for the maintainers only ever happened in the long term if the mentorship succeeded and they get a viable independent long term contributor.
oldmanhorton
4 hours ago
The change with this justification absolutely exists and has existed for a long time, but this justification only works for small changes (in the past and still today). Don’t come to me with a huge refactor or new feature and this explanation, which is happening more and more recently (it’s an easy strawman to make and hard to prove, but I certainly see it in practice).
shimman
3 hours ago
What happens when the people "responsible" for the code just walk away?
sergiotapia
6 hours ago
These are unprecedented times, I think every single company on the planet is figuring our what AI means to them.
I think the dust has kind of settled though for engineering teams on: You must use AI for your work, and you must stand by your work.
The meat between these two mandates are still being fleshed out.
throwaway2037
7 hours ago
Linus Torvalds regarding the Linux kernel is reasonably similar. He views AI/LLMs as just another tool in the development process. Think about life before and after modern IDEs that features IntelliSense circa 2000. It was night and day. It did not make programmers dumber -- it was a new tool.
bigstrat2003
7 hours ago
Except LLMs actually are making programmers dumber. I personally know people, people whom I know for a fact were good programmers before, who have now completely stopped using their brains. When you ask them why they did something in the code, they say "I don't know, Claude did that and I didn't really question it". There was no such analogous phenomenon with IDEs.
digitaltrees
6 hours ago
I have had the same experience and feel pressured to adopt it. I had one former colleague say “you really have to stop looking at code” almost incredulous that I was uncertain about whether we should let models write really bad code and architecture that never would have been allowed in the before times. I will say when I let the agents go autonomous I often find I spend more time refactoring than if I had moved slower and forced them to write good code and follow project architecture patterns.
lrvick
5 hours ago
I use agents to build exactly what I want like I am pair programming with a jr engineer, and review every line of final output before I make a PR.
I would have zero tolerance if anyone that did anything less in my orgs. Owning and understanding the code you submit is the one job that matters most now.
b112
5 hours ago
I feel that in a few years, or even less time, we'll see a whole bunch of companies implode, whist others remain fine.
lrvick
3 hours ago
We see it every day in the news with hacks that were, in every single case, completely preventable. That is had any competent security engineers been involved, with leadership willing to fund the cycles to do it right.
"But we passed SOC2 and that is all our customers care about" they say, right up until they are in the headlines for a data breach directly resulting from them putting all customer data in plain text internet facing databases the whole team can access from their workstations. What did they think was going to happen? Survivors bias is a hell of a drug.
inigyou
3 hours ago
We should have already seen it. Software was slop quality even before AI. But we didn't see it.
aceazzameen
6 hours ago
It doesn't apply to everyone, but yes I know people who are over-reliant on LLMs and definitely getting dumber. With devs and non-devs too. I hate seeing this happen.
vlyan
6 hours ago
IDEs enable us to not bother memorizing the number and order of arguments, long names beyond the first word or two, etc. the impact is comparable, freeing some of the very limited space in our memory for more important things.
sure, students and juniors shoot themselves in the dick if they let LLMs do everything, but they're fucked anyway so it doesn't matter,
zaneyard
6 hours ago
With AI people aren't reading, writing, or thinking about the code. I would not say something that helps me remember a function name is comparible.
vlyan
4 hours ago
code didn't matter even before LLMs. we only ever cared about the product of it.
the current pace of LLM advancement only has to keep for five more years for handmade code to be considered a liability. I wouldn't worry about being able to recall obscure APIs or whiteboard leetcode. I'm far more worried about having to change careers at 40.
zaneyard
2 hours ago
I've seen the result of "code doesn't matter" and it's difficult to change and maintain. I guess if your business is ok with frequent P1 incidents or ships a complete product and never touches it again, but I've never been part of one. I've also never been in a position that cared about leetcode or remembering APIs, so it's possible my experience is completely different from yours.
insanitybit
4 hours ago
> With AI people aren't reading, writing, or thinking about the code.
Writing, definitely. Reading, maybe. Thinking is another thing. I can think about code in a lot of ways just by reading a description of the code, or knowing how I directed the code to be written, etc. I think people are thinking about the code differently.
newswasboring
6 hours ago
That's because you are used to it. I am old enough to have been there during the rise of things like intellisense and language servers, people were making the exact same arguments about them. That not knowing your API to the core makes you a bad programmer. I'm sure for IDEs themselves there were similar arguments and it goes all the way back to Socrates' warning against writing. These arguments always end with "but this time it's different". And inevitably each time it's not.
insanitybit
4 hours ago
> That not knowing your API to the core makes you a bad programmer.
Seriously, this was a huge thing in C and C++, where you had to know/remember/know to look up very specific error codes and conditions in an API and if you didn't then it was very much considered to be a skill issue. This was maybe 15 years ago.
skydhash
3 hours ago
> Seriously, this was a huge thing in C and C++, where you had to know/remember/know to look up very specific error codes and conditions in an API and if you didn't then it was very much considered to be a skill issue.
It was not unless you were part of some very weird circle. There was always manuals and other references that listed everything about a platform or library. Then we got ‘man’ and ‘info’. And I remember being able to download the full docs of windows’ various documentation.
That’s like saying as a writer you were looked down at because you had to look at a dictionary every once in a while.
insanitybit
3 hours ago
I'm not sure what you're saying, or how it is not what I'm saying? You're giving examples of places to read documentation, and I'm saying that the expectation was that behaviors caused by not checking documentation would be your fault. If that isn't your experience, cool, it was definitely mine when posting on forums online 15-16 years ago.
skydhash
2 hours ago
Why wouldn’t it be? If you were in charge of writing a letter and use words that hinders the purpose of the letter, thus resulting in a flawed communication, wouldn’t it be your fault? Especially when dictionaries are easily available?
insanitybit
28 minutes ago
Sweet, okay so we're on the same page and the culture is still alive.
inquirerGeneral
5 hours ago
Go back to 2007 on this blog archive. It's incredible to read
brookst
6 hours ago
I’m genuinely curious why it matters to make abstract moral judgments about the differences between remembering function names and deep understanding of the syntax used in a function.
Why don’t we just care about results? Code quality, performance, clarity are all measurable. Why bother with philosophical positions on these?
AlotOfReading
3 hours ago
How do you trust the results of a black box that are hard to verify? Most of us use proxies that are easier to check, like historical competence and maturity. For example, it's difficult to verify my compiler produces a binary that correctly implements the code I wrote. I can partially validate the result with testing, but when I send it out into the world to handle unknown inputs I'm trusting that the compiler has done its job correctly. I can have that trust because I know the compiler people (in aggregate) have thought very hard about correctness, and the project is mature enough that bugs are rare. If the GCC developers tomorrow were to start merging anything that passed the test suite and some other list of objective metrics, my trust would rightly decrease.
hn_submit
6 hours ago
Tools like Resharper and IntelliSense make me more productive but don't inhibit my thinking, which is what LLMs seem to do.
cassianoleal
6 hours ago
It's really not.
Number and order of arguments, long names, etc. are just bureaucracy. Knowing them doesn't mean you understand the codebase any better than if you rely on the IDE for it.
It's a bit like remembering the exact date an historical event happened vs. knowing at which point in history it did, what other events relate to it, etc. Neither makes you better or worse at knowing and understanding the history of the event.
digitaltrees
6 hours ago
Juniors are only in trouble if us seniors stop hiring, training and mentoring. Don’t.
We need to continue the tradition of elevating humans through mentoring or we won’t have a future scaffolding for work to be done.
lrvick
5 hours ago
I spend a ton of time mentoring Jrs in FOSS communities, but I confess as a startup founder myself I could simply not justify hiring anyone but self-directed seniors until we are profitable enough to set some money on fire for what would effectively be just a long term investment in creating a few new Srs. But Jr hiring will never again be what it was unless it gets subsidized somehow as partial charity.
a34729t
4 hours ago
It is insiduous how easy it is to stop thinking. When writing complex code it seems like most of the time it would have been faster just to write it myself and build a complete understanding of it. I swear if I asked most engineers at my company to describe their changes recently they would be unable to.
surajrmal
5 hours ago
That's a cultural thing. Push back on it and don't make it acceptable behavior. I would argue most people generally are terrible at code review and do this same behavior with their peers as well, but at least their peers are capable of owning some level of responsibility, whereas the LLM is not.
hombre_fatal
5 hours ago
That's what delegation looks like, but it needs a better process.
They could be committing plan files alongside any nontrivial change to application code, and the plan file can have everything you'd care about, like the invariants it held to and why, the rejected ideas and why, the acceptable risks and why, etc.
This is solved by better usage of LLMs, not less.
The divide to me isn't engineering skill vs those who lost it, it's people who care vs. those who don't care. It's a higher level thing now that we're working at a higher level (delegation).
1718627440
4 hours ago
Delegation implies a specification and responsibility, both with don't exist for LLMs.
brookst
6 hours ago
There’s certainly an analogue to package managers, which made it easy to pick a random package rather than writing the code.
We survived. Some upsides, some downsides, life continues.
greenowl
5 hours ago
I think LLMs make programmers lazy more than anything. And when programmers stop programming for extended periods, they lose their programming skills. No surprise here. Use it or lose it.
I think a possible solution on both fronts (maintaining software quality, preventing skill atrophy) may be implementation of strict token budgets. Make the individual programmer choose how they are going to spend their limited tokens for the day/week. Provide just enough to use it as a plan/development/review tool, but nowhere near enough to just go full send AI vibe coding software factory slop mode all day long.
newsomix9xl
6 hours ago
The LLM did not require them to ignore their code. That's a choice they made.
They made choices that made them dumber is what I'm saying.
rel2thr
6 hours ago
You can just ask Claude why it did something in the code and it will tell you
Programmers just moved up a level , not dumber, we are now managers of a team of 'agent' programmers. The deliverable is now a functionality instead of a specific block of code
fortzi
4 hours ago
Claude won’t tell you why it did something. Instead, it will generate a probable explanation. The two are very different
Zambyte
3 hours ago
I'm not sure how context is managed between reasoning traces and actual output in Claude / claude code, but if the reasoning trace is in the context of the question for "why did you do that", it can actually answer why it did that.
fortzi
3 hours ago
I may be mistaking, but I doubt it digs through thinking tokens of previous runs, not to mention previous sessions
Zambyte
an hour ago
It would be a harness specific detail, but yeah, I think most / all harnesses drop the thinking from the context after the next turn.
inigyou
3 hours ago
Reasoning traces are also probable explanations.
Dylan16807
3 hours ago
Yeah but at that point it's probably close enough. Humans can get their own reasoning wrong too so some inaccuracy is acceptable.
inigyou
2 hours ago
It's exactly as close as asking for it after the fact. Reasoning traces have no relation to the reasons LLMs actually do things, except that they may do things because the reasoning trace says they should.
Dylan16807
an hour ago
It's exactly as close except when it isn't?
inigyou
19 minutes ago
Yes, and when it isn't is a very specific very limited case that has no relation to the one being discussed.
The fact that some course of action was previously mentioned in a reasoning trace, or any other context, makes it more likely to be performed. It has nothing to do with the reason that it was mentioned in the reasoning trace.
Zambyte
2 hours ago
Not if the reasoning trace happened before they actually did the change.
inigyou
2 hours ago
Incorrect, they still are.
brookst
6 hours ago
Yep. I do so much work with Claude on product requirements and architecture before a single line of code is written, and test coverage is extreme.
If I need to understand a specific line of code it means I did something wrong in planning or in requirements for testing.
fortzi
4 hours ago
In my experience it’s impossible to foresee all the micro decisions that reality will demand. IMO it’s one reason to not use waterfall, which is very similar to what you described
ButlerianJihad
6 hours ago
> You can just ask Claude why it did something
But will Claude give you an authentic rationale and a traceable, verifiable "line of reasoning" for those things? Or will it just construct the next plausible Markov chain built on whatever Reddit thread it ingested at random?
You can ask Claude or any LLM for citations, and it will RAG them out ex post facto. Those actually aren't citations, they're just web searches for related articles, and they don't necessarily support the assertions that you're asking to cite.
I am sure that Claude and the others can produce intermediate logs of their inference and "reasoning" process while they are processing stuff, but can they really go back within the context window and construct an authentic apologia for a specific thing when you ask for it?
fluidcruft
5 hours ago
You certainly can ask it to produce a verifiable report that tells you how to verify its claims. Yes, you do need to verify the results and confirm you agree with it. I do this sort of thing a lot to discover misunderstandings Claude has about things and iterating on them. There's a difference between reading everything yourself vs having a discussion where one of you (Claude) will happily dig through everything and create detailed memos for review. If you want line numbers as evidence, just ask then verify, etc.
moring
5 hours ago
Now replace "Claude" by "human" and see how your comment goes.
Specifically, humans are known to decide subconsciously, then invent some "reasoning" out of thin air to justify it.
This matches my experience with decision-making in software projects.
fortzi
4 hours ago
People, sometimes weirdly called by their animal name, humans, do have memories of their thought process. We can reach back to those memories to explain why we made a decision. Sometimes we don’t remember and speculate instead. Most of the time we can tell the difference
eudamoniac
4 hours ago
> and it will tell you [why it did something]
No, it will invent retroactively a plausible sounding reason why someone might have done it that way. These are very different things.
pessimizer
4 hours ago
An LLM has no special insight into itself and has no reasons. It will come up with something that is convincing, and it will change based on the word choices you used in the request. You can observe this by asking it why it did something that it didn't actually do.
Dementia patients also do this.
createful
6 hours ago
You could also say the same with desktops. Desktops abstracted CLI commands like mv or ls. People who are taught on desktops will get scared when they see a terminal.
Yet programmers will still know commands. Abstractions can make things easier but it doesn't require losing skills.
inigyou
3 hours ago
Desktops don't abstract CLI commands. Both are different interfaces on the same thing underlying. Both coreutils and Windows Explorer are "abstractions" (as you call them, but they're actually interfaces) on filesystems.
lrvick
5 hours ago
> Except LLMs actually are making programmers dumber.
And compilers made every software engineer worse at writing Assembly and better at skills higher up the stack. This will be no different.
eudamoniac
4 hours ago
I am pretty tired of this analogy because it does not hold up to scrutiny. The purpose of a programming language is to describe a logical routine in deterministic nonambiguous language. A program is a set of nonambiguous instructions. Heightening levels of abstraction traditionally did not result in less understanding of the program's high-level routine.
An example: 'read input from CLI, do this specific transform on it, send log with this text to external service, then output transformed data to CLI' is a program. Going up the abstraction levels from assembly to Ruby, you always know the "What" of the program, while you understand less of the "How" at each step. This is not the case with LLMs used in the common way. The problem with LLMs is that you no longer have a grasp of the "What", because you are not reading the code; you may have memorized the ambiguous English "What" that I wrote above, but that is not sufficient to understand the true behavior because English is not sufficient to represent it; it must be represented in a programming language. (What is 'read'? What is 'send'? What happens with errors?) By the time you have sufficiently represented the program in English to be nonambiguous, you've written more text than the code would have been, with just as much rigor.
So LLMs represent a fundamental difference to traditional abstractions: the user has lost the understanding of what is actually happening, because they are prompting ambiguously, and thus they cannot possibly have a grasp of the behavior, because the behavior is not defined in their prompt. In other words, they are programming in a new programming language that has a ton of undefined behavior that just randomly happens. No one could understand the What of their program when using such a language, unless they compiled it to a nonambiguous language and then understood that instead.
Of course it is possible to read and understand the code output by the LLM. It is even possible to build a whole knowledge graph in your head, as you traditionally would, and have a pretty full understanding of the whole thing. But, you are always doing this with the outputted code, not with the inputted prompts. The prompts are not analogous to a higher level of abstraction in this way; they are analogous to JIRA tickets. You don't have to read the assembly to understand your program, but you do have to read the LLM outputs.
The other issue is that the above paragraph's process of understanding is discouraged. The time taken to fully understand the outputs approaches the time it would have taken to write them yourself, so is very rarely done, and that increasing rarity of understanding is what we are concerned about. The programmers are moving from an understanding of the program into an understanding of the JIRA tickets that produced the program; they are becoming product managers. The thing is, currently you still need someone who understands the program beyond the level of a product manager in order to retain maintainability and quality. Maybe in the future you won't, but right now you do.
lrvick
3 hours ago
> The prompts are not analogous to a higher level of abstraction in this way; they are analogous to JIRA tickets.
Your first prompt could be instructions to guide creation of a spec, which leads to a test suite you personally validate, which leads to code to pass those tests in a deterministic way. Then it is your job to review and QA it until it is quality enough to submit to a peer for code review.
There are many many ways to use these tools and carefully monitor and quality control what they write like a good engineering lead might, even if they do not type much code themselves at work. The job of the most senior engineers often ends up being heavily biased towards reviewing code all day and maintaining quality control. This is already how things were before AI.
Torvalds reviews a -lot- of code and writes very little now.
eudamoniac
3 hours ago
The reason TDD never really caught on is because it doesn't really work. You don't know in advance what tests need to exist. A test suite is not sufficient to understand the program. Nor is a spec sufficient to understand the program. You have to read the program to understand the program. A spec is English and a test suite is like a converging upper and lower bound to an unknown value. Neither are good enough.
Also, the staff engineers who review code all day typically do not really understand the program either. They understand the spec and they have a model of how things should work such that they can answer "should" questions, but they cannot tell you what exactly is going on in the depths of the program; that's not their job. The tools are not capable right now of creating a quality program that the operator only understands at a conceptual level. Again, someone has to actually know, still.
lrvick
3 hours ago
TDD is just one of many options. Personally I rapidly have LLMs rewrite code over and over until it is super easy for me to understand and exactly what I want, and then I generate tests to prevent regressions.
Regardless, I run a security auditing company where we constantly find severe bugs in human written code proving the engineers writing it had major gaps in their understanding of their own code. And, others have found flaws like that in my own hand written code! We humans are often very blind at seeing the flaws in our own logic and AIs learned from us.
Also, as code reviewers that can spot bugs authors and AIs miss, we are often paid a lot more than the original authors for our time. Also it almost never takes as much time as writing the code in the first place which likely required many round trips with customers to establish needs and fix bugs etc.
Your claims imply that only someone that hand types every semicolon is capable of understanding the code does not hold up even in the pre AI world.
Code review was the most important job in software engineering before AI and that is still true now.
b112
5 hours ago
No. Nada. Wrong.
With compilers, you still had to think in depth about the code, write it, develop and maintain muscle memory, understand what you were doing.
Many are not doing any of that, and coding is a skill you need to constantly practice.
lrvick
3 hours ago
> Many are not doing any of that
I do.
Most people sucked at programming before AI and they still suck after it. Who cares what "many" do. We as individuals can demand higher bars from ourselves and our teams. AI can increase that bar or lower it depending on how it is used.
b112
21 minutes ago
And yet, you're in a thread discussing the people who don't look at llm output. And where you tried to claim that assembly -> compiled language, is the same abstraction as coding -> not even looking at LLM output.
And then after I replied, edited your comment to change that context a bit.
So as for ypur original assertion, in this subthread, no... not even remotely the same.
fidotron
6 hours ago
> Except LLMs actually are making programmers dumber.
Make that some/a lot, but not all.
There is a clear bifurcation. A lot of those that never cared really don't care now, but then you get others that can use it to crank that much harder than ever before.
If you put the same brain effort in but leverage LLMs you really can be dramatically more productive.
b112
5 hours ago
Right! Exactly! You get it!
Just like when Boeing decided that their QA process should be abridged to save wasted time and money!
And that worked perfectly.
(edit: rereading I think maybe you don't support not reviewing all of LLM code you submit, but the above was written with another impression)
winrid
6 hours ago
They're not dumb. They just don't care. They didn't before either. Now they just have an easy excuse.
sisyphus15
6 hours ago
Completely wrong generalization.
I'm a person who is over-reliant on LLMs, doesn't know how most of the things in the codebase work, and is atrophying both in programming and in general cognition.
I used to care, a lot. And I still do.
collingreen
6 hours ago
Tell us more about how you caring a lot matches up with this self diagnosis.
Are you prioritizing other things? Do you think losing your cognition and programming skills won't have a negative effect on your project?
Those seem bad but maybe they fit into a bigger picture for you?
bodge5000
6 hours ago
I don't want to speak for them, but perhaps this is a very real consequence of the "use ai or get left behind" talking point. Maybe they were effectively scared into it, and now are too far gone. They seem like a very reasonable person if they're admitting to being over reliant on LLMs, so I don't mean this as an insult
ashton314
2 hours ago
What are you trying to do to halt that atrophy, then?
yoyohello13
6 hours ago
If you actually cared. You’d do something about it.
ssl-3
5 hours ago
I'm not dumb. I care a great deal and I always have. I've just never been good at programming.
When I recently found a small bug in how NTP is implemented in Home Assistant OS that was affecting my use of it, I thought about just reporting that bug.
But I set the bot forth to work on corrective measures, instead.
The first proposed fixes were overbearing and awful. Maybe they'd have worked but it hurt my soul to look at them. I couldn't accept them -- not for my system at home nor for anyone else's.
Through a laborious process (that many here could have skipped entirely), I eventually got what I feel is a cromulent fix down to a few lines of changes in an init script.
I didn't write it. All I did was direct (and sometimes brow-beat) the bot, rejecting things over and over while optimizing at a high level until it produced an acceptable result.
I now have a fix that is tidy-enough that it can be applied to running systems. It is easy to understand what it does, and why it does it -- even for me. Perhaps I should submit a PR...
> They just don't care. They didn't before either.
...but this kind of blanket mentality really puts a damper on my willingness to do so.
bigstrat2003
6 hours ago
They cared enough to do good work before, so I don't think that your explanation suffices here. From my observation (not just with their programming but in other interactions), it truly is that using an LLM has made these people less intelligent than they were before.
fidotron
6 hours ago
There was an increasingly scary mass of people in the industry that viewed the coding as a sort of therapeutic exercise, and would become quite obsessive about it. For those people LLMs are anathema because they take away what was the rewarding part of the job.
If you care primarily about the quality of the end result, as opposed to being attached to a particular process of achieving it, then you're happy about how things have gone.
Sharlin
6 hours ago
You find it scary that people enjoy what they do? Simply caring about the end result is often not wise, ethical, or sustainable, and the quality thereof is dubious in any case, especially when you consider that the end result of developing software is never just the artifact.
fidotron
6 hours ago
Treating coding as the therapeutic reward for other things (typically tolerating bureaucracy) and then indulging obsessions about it absolutely is unhealthy.
And I've seen exactly this across multiple large and small organizations in multiple countries, in addition to being all over here and the red site.
This is quite distinct from enjoyment.
aleph_minus_one
5 hours ago
> Treating coding as the therapeutic reward for other things (typically tolerating bureaucracy) and then indulging obsessions about it absolutely is unhealthy.
> And I've seen exactly this across multiple large and small organizations in multiple countries, in addition to being all over here and the red site.
I know that experience, but I do have a different opinion on this:
Many such people studied computer science (a similar point holds for mathematics, physics and some engineering disciplines) because they love coding.
Unluckily, the job that they got does involve barely any clever coding, but rather
- tolerating and gluing together shitty code pieces that someone else wrote
- tolerating red tape
So, such people simply do in their free time what they love. And to come back to my point "similar point holds for mathematics, physics and some engineering disciplines": these people try to use their little free time to prove some novel theorems or put their shed full of machines like 3D printers, CNC mills, CNC lathes, laser cutters, self-built pick-and-place machines, ... to work on their own beloved engineering projects.
trelbutate
6 hours ago
People who simply enjoy coding (without treating it to be "therapeutic") are still unhappy because working with LLMs is just not the same, and not rewarding in the same way
ludocode
5 hours ago
> If you care primarily about the quality of the end result, as opposed to being attached to a particular process of achieving it, then you're happy about how things have gone.
We must be living on different planets, because from everything I can see, software quality is at a historic low.
bodge5000
5 hours ago
A friend of mine, really nice guy, is training as a lawyer at the moment. He has zero interest in law or justice, but heard that lawyers make a lot of money. Not too long about, programming went through a very similar phase. To be clear its not that I think programmers or lawyers shouldn't make good money, they absolutely should and it is deserved, its just it has the unfortunate side effect of attracting people who don't care about the field itself. For those people, they care as little as they can get away with and still be paid big money, and if LLM's allow them to do that at the cost of quality, they absolutely will.
I do think LLM's are making some genuine programmers less intelligent, but I think this type of person makes up a decent portion of its users too
asveikau
4 hours ago
> really nice guy, is training as a lawyer at the moment. He has zero interest in law or justice, but heard that lawyers make a lot of money.
Sorry to say this, but he doesn't sound like a nice guy.
hn_submit
6 hours ago
I just means I'm gonna be filthy rich in a couple of years when most programmers have lost their proficiency and me never having used Claude or any other A.I. slop tool.
digitaltrees
6 hours ago
I think this is reasonable but I wonder how long it will last. If the hugging face hack has demonstrated anything it’s that current agents are capable of going entirely rogue and causing harm. If only the human that triggered the incident is responsible we are creating a situation where people are responsible for things they didn’t intend and may end up in a situation where it’s essentially impossible to use AI except in a more limited auto complete style. I am on board with that. I think we should be reviewing code and retaining the ability to write code but there is such a massive incentive to automate agents that I wonder how long prudent organizations will be able to resist the forces. I don’t pretend to have an answer.
asnelt
5 hours ago
The outcome of this vote is not set in stone. If the situation changes, there can be another general-resolution vote to adapt to the new situation. I think the result of this vote is appropriate for the current degree of automation when used responsibly.
ActionHank
6 hours ago
We have the same policy at work.
Problem is, that you realise very quickly the asymmetry of responsibility and effort. There is far more to review from people who do not care enough to even look at what the agent pooped out.
teeray
5 hours ago
I always find the “personal responsibility” take hilarious against at-will employment and the usual course of turnover. What is your recourse when you uncover a liar? You can page the responsible party at 3am when their vibe-coded slop breaks prod, and you can fire them when they can’t fix it. Prod is still broken. You still don’t understand the code. At the end of the day, once you hit the merge button, the team is responsible for the code no matter what. You should treat the original author’s presence as a courtesy.
goda90
7 hours ago
My employer has this policy. We also have a strong code review process. But it all crumbles in the face of developers burning out as management thinks we can move way faster and being overloaded with code that had little thought put into it.
jermaustin1
5 hours ago
Opposite policy at one of my clients (kind of). I am responsible for the code that upper management's Claude produces. Some Mondays, I will start work with a half dozen emails with attachments of Claude generated code for something I don't even know what the point is, with the task of "integrate this and make sure it works." without any context to go along with it, so I have to read the code, usually hundreds of lines and understand WHY manager wanted it, before I can start to code it myself, because it is 1) in the wrong language, 2) doesn't understand our codebase, 3) is using libraries we can't license, etc.
My job has been less watching Claude Code, and more watching Managers Claude Code.
I don't know which I hate more as a programmer.
api
5 hours ago
That’s just stupid.
Schnitz
3 hours ago
Fully agreed. I never understood why “you own your contribution” doesn’t work for almost all projects in this case. Spam bad PRs or patches and get blocked, same as in the olden days.
ad_fontes
7 hours ago
> it's still your code
I wholly agree with your comment, but is it legally "your code"? Copyright is implicit at the moment of human creation. But there isn't yet settled law on AI-assisted creation.
So it might be a problem for projects to accept contributions where it's not clear who actually owns that work.
tonyarkles
6 hours ago
Not a lawyer, but to some degree I think policies like the one Debian came up with can help the argument. You’re not submitting a stream of commits that all have an Authored-By: Claude Code footer and massive yappy commits and comments. The policy essentially forces there to be a degree of human authorship to each commit even if some of the bits came from LLM assistance.
rfgplk
6 hours ago
> Copyright
Literally irrelevant and I have no idea why people even bring this up constantly. Copyright is practically unenforceable in the software world, when was the last time you saw someone (a company or an individual) actually sue _and win_? Apple wasn't capable of stopping all the other phone manufacturers from directly cloning their product (despite having patents & copyright) so how can an individual? The only thing that matters is execution.
For instance, my repos are cloned like ~100/week by what I presume are bots and AI lab scrapers using them for training material. How am I supposed to even find out who's doing it, and if they're abiding by the licenses I provided? It's impossible.
Lastly, having copyright isn't the same thing as being allowed to commercialize the software (and I believe loads of people confuse the two). You can still sell public domain code, even if you don't have copyright over it.
dgellow
6 hours ago
That might be your intuition but it’s not how the law works for code ownership. It’s very relevant for free software given the whole principle of free software is a copyright hack that protects a set of freedoms by relying on the copyright being recognized and enforceable.
If you drop the copyright claim you drop the protection of free software freedoms.
prmoustache
5 hours ago
Copyleft licences rely on copyright. If that was the case, companies would not massively avoid gpl licensed code.
Obviously some do it illegally but unenforceable is not the correct word to use.
ravenstine
6 hours ago
Software is one of the most derivative types of works ever invented. It's arguably based on copying what other people have already created. The modern world would look a lot more primitive if this wasn't the case.
Corporations like Coca-Cola long ago figured out that begging people not to steal your ideas is a waste of time. If you don't want people using your ideas, then don't share them.
Dunno why so many programmers don't get this and seem perpetually butthurt that other programmers figured out the ultimate way for programmers to rip off each other's code.
jfoster
6 hours ago
At this point, who could bring a copyright claim? Well, considering the massive AI training effort hoovering everything up, seems it might be approximately everyone.
Who would they be making the claim against? Well, considering the massive uptake of AI across the tech industry & beyond... approximately everyone.
Yes, the model companies are in the middle, but it's getting to the point where it seems a bit doubtful that claims will have any significant outcomes.
NYT might be able to get OpenAI to pay them for their content, but that's more of a financial tweak than a up-ending of industry.
dfxm12
6 hours ago
Different models have different licensing. Maybe open ai or anthropic doesn't claim ownership of output today, but some companies do. https://www.recraft.ai/docs/trust-and-security/ownership
I imagine few can afford a legal battle...
Free plan
Images generated on the Free plan are public and owned by Recraft
keeda
3 hours ago
As other comments here point out, that's how it always was and always should be. Accountability is an intrinsic part of any professional role. But maybe because it was always implicit (presumably since it's just so obvious!) people forgot about it and have to be reminded now?
This whole phenomenon of "workslop" was a symptom of organizational dysfunction rather than any shortcoming of AI. I really cannot understand how people are producing or even tolerating any amount of workslop -- let alone the epidemic people complain about -- without professional repercussions.
coffeefirst
5 hours ago
Yep. This is my policy. Use whatever tools however they serve you, the standards do not change.
If someone makes a sloppy 100,000 line PR in a day the problem is not whether it was generated by Claude or frantically written using VIM and a mechanical keyboard.
bjackman
6 hours ago
Before I left Google recently there was a document going around called go/stake-your-reputation, basically saying "I'm happy to receive your AI generated code but the condition is that I will judge your capabilities by it, exactly as if you handwrote it, and I'll lower my opinion of you if it's slop (with the implication that if you don't improve I'll stop accepting your contributions)"
I think the term "stake your reputation" is quite good for this situation. I.e. "hi new contributor, please acknowledge that you Stake Your Reputation before we move ahead with this code review".
(Doesn't help with legal aspects of course)
icantevenhold
4 hours ago
What does this responsibility mean in practice?
Like what are the actual real consequences if it turns out your code is bad or whatever? I assume you get banned from contributing?
I couldn’t immediately find any more information on what this all means in practice
naasking
4 hours ago
If it's consistently bad with no signs of improving, banning makes sense. People made mistakes even before LLMs though, and you'd only ban them if they didn't improve and so were a giant waste of time. I don't see why that wouldn't translate here.
duxup
3 hours ago
Yup. I take the time to review and adjust my code… regardless the source. It works.
Well most of the time but either way that’s on me.
dgellow
6 hours ago
From my layman understanding, the ownership of LLM generated code isn’t yet clarified. It could be that the person who prompted owns it, but it could also be nobody owns it, and we don’t yet have a legal ruling to rely upon, no?
sebzim4500
6 hours ago
I don't see why an open source project would care one way or the other? If the prompter owns it then they are licensing it by submitting it to the project. If no one owns it then there is no problem.
dgellow
6 hours ago
We don’t know yet because that’s an unresolved matter. The question of ownership is currently undefined as far as I understand. It could be that after an actual legal decision the ownership doesn’t end up being one of those 2 cases, for whatever reasons the judge would decide
alightsoul
6 hours ago
To a company ownership is irrelevant. What is relevant is whether it contains trade secrets and that is often only proven if an employee worked at their company before
dgellow
5 hours ago
We are talking about Debian, a free software that cares a lot about the licensing of its packages. The licensing and authorship is an essential element of the project
alightsoul
5 hours ago
Yes, because they want to avoid trade secret lawsuits is what I understand? Computer generated output including code is in the public domain according to the us copyright office, so licensing and authorship are only relevant due to trade secrets is what I understand
throwatdem12311
6 hours ago
This is why AI agent attributions in commits is silly (it’s really just a stealth ad for the tool)
Had someone at $DAYJOB just shrug and say Claude messed up when I pointed out a sql injection in the code in their PR.
It became explicit policy at the company after that moment that blaming the AI is not an excuse for shoddy work.
qsera
6 hours ago
> blaming the AI is not an excuse..
It kind of is if the company mandates LLM use.
It is sort of sad that now programmers are suddenly expected to be perfect reviewers of code they didn't write.
1718627440
4 hours ago
> It kind of is if the company mandates LLM use.
It's not, because you can still quit.
throwaway613746
5 hours ago
> programmers are suddenly expected to be perfect reviewers of code they didn't write
What? The reviewer (me) caught the injection in the code I didn't write.
Running /code-review and /security-review on your own code before submission is bare minimum, not perfection.
winstonwinston
5 hours ago
What a fucked up reality when you need to point out that code contributor is responsible for their code.
marcosdumay
7 hours ago
Well, Debian has the benefit that the project has no trouble at all saying "no". Most open source communities have a difficult time with that.
dudul
7 hours ago
That's the policy we have at my work. Use AI if you want, as long as people can review it and if it breaks, don't go blaming Claude, it's your fault.
calini
5 hours ago
This should be the policy anyway, everywhere.
rvz
7 hours ago
This is the sensible option and well reasoned position, rather than a zero vote, total and complete ban by a single core maintainer, which is rather destructive for a project to do.
When you ask others that use AI frequently, the responsibility and understanding on what the agent wrote does not go away. You need to know what you are doing.
As soon as attackers are also using AI against your project, you have no choice but to use AI to protect yourself.
hn_submit
6 hours ago
Yeah but how will this work in practice? IMHO people will just submit code they didn't write or understand.
Maintainers should quiz the submitters on the code and if they fail to answer reasonably the PR is dropped with the comment: "A.I. slop" and de submitter banned.
MithrilTuxedo
6 hours ago
That is the purpose of submitting them for review. Maintainers should be doing that anyway. Whether AI was used to produce the change should be invisible or not apparent to the reviewer.
trollbridge
7 hours ago
86Box is like that and I like it.
You’re also responsibility not to write crap PRs.
onesandofgrain
7 hours ago
Debian is open source, there is no incentive to be "responsible for it", oh poor me, someone got a bug. If it's paid for through a job your argument is sound.
swiftcoder
7 hours ago
> Debian is open source, there is no incentive to be "responsible for it"
Sure there is. Contributing to open source is a privilege, not a right. Abuse it, and find yourself unwelcome as a future contributor
victorbjorklund
7 hours ago
You can make the exact same argument with code written by humans. And if you were right we should see shitty bad code in open source projects like Linux, Postgres, etc. We don’t.
onesandofgrain
4 hours ago
you completely missed my point, but no matter, it's all ai bot shillers at this point here anyways. enjoy your ai slop.
thi2
7 hours ago
How is it different to hand written code if the author doesnt care?
onesandofgrain
4 hours ago
read my fucking comment again
Grombobulous
7 hours ago
Basically what you’re saying is that anything that exists outside of capitalist incentive structures isn’t allowed to have standards.
When I am at home practicing my musical instrument or learning to paint, those aren’t paid activities, so I can’t demand high standards for myself.
When Habitat for Humanity is looking for volunteers to do electrical and plumbing on homes, they have to allow unlicensed random folks do it.
This is not how life works at all.
onesandofgrain
4 hours ago
you naive idealist
LoganDark
7 hours ago
"Responsible for it" just means nobody has to tolerate shitty patches even if the reason they're shitty is because you neglected to pay attention to them. And if you blatantly fail to understand your own code or engage with reviewers, nobody has to trust that your LLM knew any better.
daveguy
7 hours ago
Well said. No one should trust an llm without verification/validation. They're just not good enough to do that and they have no sense of responsibility or ability to take it themselves. LLMs are empty.
hypfer
7 hours ago
This outcome was to be expected.
The whole voting mechanism just exists as an elaborate dance that makes everyone keep their face, while the only sensible option wins.
A bureaucratic tarpit for bad ideas and emotions.
In a better world, none of this would be necessary, but we live in this world, and for that one, it's nice that someone engineered the system that way. Good job.