I've caught Claude Code generating some pretty egregious security vulnerabilities. I'm using it to build an AI RPG site and the goal is to use web assembly as a bridge between author submitted code and LLMs in order to help shore up state management at the game level.

The language that I picked for the game runtime is Python. Claude really thought that the best way to validate user submitted Python was to bypass the WASM sandbox and execute it within the application container using shell exec - essentially opening up an RCE vulnerability.

I also find that the quality of Claude Code degrades substantially. Claude really wants to implement every feature in as bespoke way as possible. This is fine when you first generate the project but over time you'll find that every web modal is implemented differently. Every button is different. Business logic is disconnected. It's why agentically produced codebases are MUCH larger than they should be; every feature is developed in a vacuum.

Then I'm trying to shove stuff in my AGENTS.md or CLAUDE.md files like "ALWAYS look for existing patterns within the codebase to keep it consistent." But the harness doesn't always work and it'll generate useless, verbose code anyways.

In some cases it's useful - like if I am shaky on the DSA knowledge needed for a specific operation or optimization then Claude can replace Stackoverflow. But, man, I'm so frustrated with it.

I think there are some factors beyond just skill too - the kinds of tasks you're giving the AI, and how involved you are in ensuring the output is good (via either extensive planning guidance, extensive review/testing, or a combination).

I used LLM to teach me how to code and get through obstacles that would have me spending a lot of time doing ???. Typically, I just write code that I know a lot of time is absolutely wrong but the LLM helpfully point out mistakes.

I am slowly doing more of my own code and cutting out the LLM out of the loop in the unfamiliar territory I am working in.

My main concern is not so much productivity but understanding the code I have written and feeling agency over it.

The LLM is a very good teacher.

Well, I have bad news for camp 1..

It's bad if they work in a part of the industry where code quality or efficiency matters. That's maybe 10% of the total though.

I disagree this is the source of the polarization. Maybe it's part of it.

I have been coding since about 1983 or so. I shipped high quality products that have been used by millions of people. From embedded software to desktop applications to distributed systems.

I don't think I'm in the "don't understand what code should look like camp" (I mean you never know but the evidence seems to show that I do know what I'm doing). I use AI as a tool and it helps me be more productive. I don't "one shot things that would take me days to do". I use it to help me automate things that I could do manually where it is faster and more effective. I review every step and if I don't like something I adjust. There are some specific situations where it basically does as good a job as I would do in running some experiments, doing some analysis or writing some small amount of code. I still know what the changes need to look like broadly, where to make them, and what patterns to follow. It just automates the work and sometimes does have some additional insight that can complement my views. Unlike me it is all knowing about everything in terms of access to "knowledge". It knows all the details of how a certain runtime manages memory, Linux internals and various open source software. I could go look it up myself (which I'd do before AI) but I don't hold it all in my head like AI basically does. It is also "all knowing" in the code base I work in (more so than me, it's a huge code base, I have an outline and a high level picture in my head but not every single code line) where again I can dive into the code but it helps me extract the relevant information faster.

I think the polarization is more on the how you use the tool, what situations you use the tool for, which domain are you operating in (languages, applications etc.). You can also one-shot simple tools and helpers that are not the production software which is another way to accelerate your workflow.

Yup, pretty much.

The hard part too is it's not like you can just learn the basics and be able to tell good code apart from bad -- the more you learn to code, the more intricate your understanding of good code is. It's like becoming a good writer; just knowing grammar and spelling doesn't make your writing interesting. Not to mention that there's just a lot of bad advice out there that you can't recognize as bad advice if you're not a regular practitioner. Like, "Clean Code" is IMO a terrible book, but a ton of people follow it because it has the sheen of respectability.. until, hopefully, they learn some new patterns and realize those old ones aren't very good. But you pick these things up with experience and doing the work! Otherwise if you're just reading other peoples opinions, you'll see a bunch of people say "Clean Code is great" and a bunch of other people say it's rubbish, and you'll have no way to know who you should listen to. (If you disagree with me on Clean Code the book that's fine -- I'm just using it to make a point -- sub in a different book/ideology if it suits you)

I think looking at an LLM code and thinking you're now a coder is like watching a someone play guitar and think you can just pick up a guitar and play a song. The truth is, if you want to be good, you have to do the work.

One of the things I hate about AI is that we're going to have a generation of "programmers" that are absolutely shit at programming, create problems for everyone else, and will have absolutely no idea how bad they are. And they'll probably never get better, because you can't get better by just asking claude to do shit for you. And then the LLMs themselves will probably start to degrade because they'll be trained on the slop since it'll heavily outnumber handwritten code..

Is this really a split that exists?

In my case I see Claude produce code much worse than I would, but it's certainly much quicker and, even after reworking, it makes me finish tasks in less time.

It really depends. If you're cranking out prototypes or testing ideas, it's genuinely great. But if you're familiar with the code it's very easy to spot its (many) mistakes. It's Gell-Mann amnesia.

Then again, I just caught Claude writing setTransparent(!opaque == false), opaque being a bool, on a purely vibecoded project. Which was pretty impressive. ("• You're right, that's nonsense.")

The difference between prose, art and code is that we can define "good code" with deterministic tools. Not perfectly, but to a large degree.

I'm in camp 3, where sometimes I don't really care how good or bad the code is. For internal tools for example, you can let the LLM crunch out code really fast, you can validate output but don't even have to look at the code. These kind of "weekend projects" can get finished in an hour or two, and so are really 10x.

For bigger production ready code, you indeed have to guard the architecture. But for the code, in some corners you can get away with sloppy code, as long as it kind of works.

What I'm saying is, code doesn't always has to be great. You will just have to judge the places where it needs to be high quality, and other places where you can get away with sloppy code.

There's a third camp between these extremes who is like "goddamn it just type this shit out for me so I don't have to do it myself".

It isnt though.

The industry largely has selected for camp 1 long ago.

If you don't get immediate negative feedback camp 1 can go quite a ways before problems surface.

LLMs can generate code, but the quality of the code at scale is just not there currently by all important metrics such as security, maintainability, separation of concerns, etc.

Today, it's a kind of chaos magic wherein you summon the beast and try your best to contain him, knowing that someone will probably die in the process. Sometimes literally. It's still a force multiplier in the right hands and domain, and agentic coding is a paradigm that won't retract, at least until something better supplants it.

The problem is that few engineers actually have the discipline available to constrain these models appropriately and instead rely on a hodgepodge network of "skills" aka prompt fragments which are passed around and glued together.

I consider myself as having such discipline, being strongly architecturally-minded, user-first, etc. in both design and implementation. And I still struggle to contain the beast many days. I just got through screaming at Claude for intentionally taking a shortcut that I'd forbidden, leading to a ton of wasted time and tokens.

Sometimes I feel like I saved weeks of R&D with a single ten-minute task handed off to an agent, other times I feel like I'd get better returns playing slots in Vegas at the alarming rate Claude burns through money.

I, for one, welcome our new AI overlords. They provide me with only the finest Gell-Mann amnesia, straight from the tap.

I think this is a straw man.

> the polarized differences in opinions on HN about the quality of LLM-produced code

Are there strong differences of opinion about the quality? I've seen very few people claim that LLMs write better code than they do.

> one shots things it would take me days to do, and has made me 10X more productive

This is an entirely different claim from the former, and you're conflating them.

The boost from LLM-assisted code isn't _expertise_, it's the power of having an always-on team of reasonable junior developers from every discipline you can possibly imagine willing to do your whim.

Take for example Jesse Vincent / obra[0], who is an exceptional developer, with great taste, and a stack of well-received open-source software to his name. He posts a lot on how he's being made more productive by AI-assisted development. Do you have bad news for him about the quality of his work...?

0: https://en.wikipedia.org/wiki/Jesse_Vincent

Eric S. Raymond has basically stopped writing code by hand altogether. He consistently delivers high quality code without intervening to fix the LLM's output himself, much faster than he would have been able to alone. This is very bad news for camp 2 because it means one of three things:

1) he is extraordinarily lucky

2) he is extraordinary brilliant at manipulating LLMs

3) you really are "holding it wrong" and you are hobbling yourself with your failure to properly learn the tools

The first two seem rather unlikely.

I’ll take “what is Dunning–Kruger” for $200 Alex.

On factual things, potentially. But if I want to read your writing, wouldn't I be trying to pick your brain? Otherwise why don't I read wikipedia or usage documentation?

I haven't used these things for writing recreationally for a while (since the Claude 3.X days), so my opinions might be outdated - but they definitely weren't bad - after all they had a huge library of witticisms to pull from, and like Stable Diffusion that pulls from master artists, so do LLMs from skilled writers. Pro writers did come up with an absolute dearth of interesting ideas, and there are mountains of skillfully written prose out there - and its all in the training data, and AI is quite good at pulling from it.

The advantage of the writing vs images, is that it takes longer to absorb the whole with text, so its less apparent that the whole thing doesn't quite come together.

My problems was with Claude's prose and ideas is that it kept recycling the tropes and phrases after a while - something that has been observed that these models have very strong statistical biases - when asking for a random number for example, LLMs are far more predictable than even humans, this shows up in unguided writing exercises.

But as for actually crafting text that is both terse and to the point - such as oneliner explanations, or writing summaries - these models are quite bad. The best I have seen is they could turn a given length of prose into an even longer version - with generally some loss in the tonal accuracy or the points made in there.

As such they are a terrible tool for professional communication, but unfortunately, lots of people have started using them for exactly that.

This is just the last domain people can desperately cling to because style is totally subjective.

Depends on the type of writing. Blogs and the like? LLM generates prose that, to me anyway. is unbearable.

However, in fiction I’ve found it a useful collaborator. There have been more than a few occasions when, given some notes of how I want a character’s arc to develop in a particular scene, that the LLM gives some excellent pointers and ‘new’ ideas I hadn’t considered.

As far as editing my prose, I use it as a ‘thesaurus of phrases.’ When lazy, I can give it a rough sketch of the paragraph, giving it the gist of what I want, and have it generate a dozen or so versions. I usually can find nuggets of good phrases therein that are useable… much as I would refer to Roget’s to find a more precise word.

That said, one has to resist tbe temptation of using a chunk of generated text verbatim; no matter how good it sounds in isolation, the repetitive grammatical structure and other LLM-smells add up quick and become nauseatingly obvious if used frequently.

In any case, I think LLM’s get a bad wrap for writing… when used correctly, it is incredibly useful. And, it’s tiresome to hear pretentious snobs assume that an author who uses LLM simply lacks the taste to appreciate how bad the prose sounds. Not true in all cases.

> I can't fathom thinking that LLM writing is even remotely passable. People that think this should honestly read more.

This makes me think you're only exposing yourself to high quality writing online and from an intelligent circle of friends and coworkers. The average person's reading and writing abilities are _atrocious_ and only getting worse. We're almost at the point where kids are communicating through abbreviations and emojis exclusively. LLM prose is significantly better than what the average person can produce.

Really hard to take your comment serious when the only post on dvt.name is a hello world page, because at least OP is trying to publish and you are lacking moral high ground to judge him thinking LLM writing is good.

IMO LLM writing hasn't significantly improved since maybe GPT4. It still does the exact same "It's not x, it's actually y" tropes and many of the other common LLM smells. Most LLM generated text is immediately discernible as such.

Well, no. The LLM is just better than you in a narrow band that appears wider to someone below than above.

From above, or from below with adequate exposure, it feels facile and hollow. It is good at weaving grammatical structures. It is not good at thinking in words in a way that invites a fellow human along for the journey. Because it doesn't think.

Well, practice to get better at writing (and, therefore, judging writing) yourself. It seems obvious. Your skills are not frozen in time and set in stone.

Well, the third choice is to develop as a person and become better at writing. Which you do by doing some crappy writing and learning from it.

>If I am not a very good writer, that means an LLM IS actually better than me

If you're not a very good writer, I'll at least skim your work to see if it contains any good ideas. If it's slop, I'll just close the tab. You already told me it's not worth caring about, so I'll agree with your decision.

> My two choices are to keep producing my own crappy writing, or use an LLM to create better (but not great) writing

Are you also an LLM or do you have the capacity to learn and grow?

I mean, you have the ability to learn to do stuff better to a certain extent, so it's not like your only choices are "suffer through the writing I'm capable of producing today for the rest of my life" or "give up on ever writing anything myself". Writing stuff yourself is pretty much a requirement of getting better at it, and arguably even if you do intend to use LLMs to supplement it, having a better baseline will be valuable for additional iteration with the LLM.

It's like a gross looking slice of pizza, it's mindbending because at first it looks good, after all it's pizza, but something in it makes it really disgusting

> Smoke test though, I can't say I've ever see irl usage.

We use it all the time at my employer, and have for decades. They're basic tests to tell you if the app is up or not.

Like: you go to this page, and it shows a big green banner if the app can connect to the database and its disk isn't full. If one of those basic things are wrong you get big red banner or you can't load the page at all.

Heard smoke test IRL & was confused to see it used indeed in place of “sanity check”. Weird.

I frequently use “belt-and-suspenders” — probably not in wri… crap I just used an em dash.

Huh, we have a process which has several exit criteria - which are pretty expensive to calculate with multiple rest calls to get through each...

I've always called them exit hatches, entirely unrelated to llms...

Now I wonder if I need to reword the docs... But realistically speaking, llms are the only readers of them nowadays, so I guess it doesn't matter.

Which seems logical - if they were somehow consistent between all LLMs i'd be even more curious how LLMs are... crab-evolving to 'isms, but hopefully that's not the case hah. (i say crab-evolving in jest)

Considering that LLMs output continuously becomes more human-sounding (by design), you’d either have to continuously run what you write through various detectors and keep changing it or you must resign to inevitably be called an LLM at some point.

Simultaneously, because humans subconsciously mimic what we see, we also converge to sound more LLM-ish.

The harsh reality is that no matter what you write and how much research you put into it—especially if you try to be legible to others and not make grammatical mistakes—someone could discount all that and claim you just prompted an LLM. If they want, they can always find some magic “AI checker” that will return a high enough probability. We all know that with a good enough prompt and with round-trip validation against a checker (there are definitely products with this all built in) it will avoid the common tells, it’s just the matter of a few extra tokens.

It’s somewhat demotivating.

> They must

Nope. LLMs are RLHF'ed to the brim.

I'm going to play devil's advocate, who cares if it all looks generally the same? I grew up before everyone had dial-up, and I remember back then, when Web surfing on Netscape Navigator, all the websites also had a similar structure. Homogeneity in this context, to me, signals a design system that works best in the moment.

This has been a bugbear of mine (and many other people) since long before the advent of LLMs. It is not an intrinsically virtuous thing for a web design to be "effortful". Some of the most well-regarded UX systems in the industry are defined by rigorous sameness. If the baseline is good, departing from it is as likely to reduce legibility as increase it.

Before LLM you could sum up the web as the hamburger menu, bootstrap and materialize. Even Apple threw everything in a hamburger at some point.

Ironically, your entire post can be read as such, almost perfectly!

Labeling each sentence (J)ab and (T)hrust, and using colon ":" to indicate arguments, one gets:

```

J: J. J. T.

J: "J. J. T".

T: T.

T: J. J. T.

```

> I kind of wish we'd stop talking openly about what the tells are.

Least this way it’s out in the open perhaps, since enough users have training enabled labs will naturally learn what annoys us.

Had the same thought though

I call it "I'm not like other girls" writing.

Imagine a world where everyone talks like an Apple product page.

My friend also made a discord bot for this. It got too real so it was shut down.

I often think it’s the opposite in fact, that the LLM smells come from LinkedIn text.

https://github.com/JetBrains/JetBrainsMono/blob/master/OFL.t...

Which is fine too me; I never read those emails anyways...

That's a funny one. I don't use LLMs at all but "load bearing" is such a common/over-used internet joke for DIY building projects and stuff like "load bearing caulk". Have never heard it in a software sense really so am slightly perplexed

Hah, ChatGPT constantly says "that's real" or "less about X, more about Y."

You could stop using it first I suppose

I meant interviewer. Sorry!

People are being forced to express in suboptimal ways, just to avoid being detected as slop. Some others are instructing AI to inject some typos or write grammatically incorrect sentences. The whole "hey I detected that you have used AI" is so childish. It is same as "hey I knew that you used a car to get here, instead of walking". Ok, I will drive my car at walking speed and maybe replace it wheels with legs.

an LF is 1 character, just like a space. Its just looks bigger to our puny human brains.

wat