I think this is true if you're the kind of "non-technical" person who reads HN which is still the minority
The smallest amount of framing and architectural forethought pays massive dividends but I imagine the person who says "build me an accounting app" while being apathetic to what language and stack it uses like apps such as Loveable imagine will still get bad results
Yesterday, Codex helped me resurrect a project[1] from 2013, where the code was published, but trapped inside an exotic format that can only be read by a single application. That application had ~14 breaking releases (rewrite everything!) over the past 13 years. Needless to say, no current version of the app can read the 2013 file, and no current system can run version 2.0 (it's now at version 14) of the app in 2026. Codex helped me create a containerized environment that pulled dependencies from that era, connected the container to my host GUI session, and successfully exported the code to normal-ish text files. It took half an hour (of Codex time, not mine) to check changelogs and try different version combinations. I was quite happy with the results.
Today, I wanted to capture screenshots every 5 seconds on a Windows machine. Codex created a dotNet project that compiled to a (self-contained!) bundle. It worked, of course, and I even ended up using it. The problem was that the bundle was 167 MB in size, held together by lots of XML config, and implemented the "RBGA-bitmap to PNG file" dumping from scratch; the whole thing was close to 1.5k lines of code. 30 LOC of PowerShell would be a perfectly acceptable solution here. I was... well, not impressed, and I would never show this solution to anyone, but it at least works and I didn't need to brush off my PS skills as a result, which is kind of good enough in this specific case.
As a programmer, I can immediately recognize when the LLM output is something nobody should ever see. It doesn't matter if nobody's going to see it anyway, I agree - yes, "one-user applications" fall in that category. However, when you mention <<"real" software>>, that changes the perspective. How would you know whether your software is "real"(-ish) or not? "Hey Claude, are we production ready enough right now?" That won't work.
TL;DR: requirements for code quality are hard to estimate, and code quality is something many professional programmers struggle to consistently recognize (not to mention achieve). Until the models get good enough to make good code a baseline, "real" software will remain hard to write, and better left to people who can recognize bad code quickly. That doesn't mean only professional programmers: plenty of hobbyist coders are "non-technical" in general. But at least for now, you still need to be a coder yourself to get consistently good results from the AI.
[1] https://pleiad.cl/research/software/gradualtalk
yeah, i kind of stretched the meaning of "real" there, i guess. i just meant software that is designed to accommodate users who are not me, and whom i don't even know.