Many people with no skills are taking advantage of the LLM craze to artificially inflate their own value. I see it every day on LinkedIn.

People that previously have barely any experience in tech, now being hired in AI startups because they are good bullshitters.

> Now, we all pay the consequence, to the tune of hundreds of thousands if not millions of dollars of wasted productivity from teams that have to deal with the resulting fall-out of this usage of “AI”.

Brandolini's principle in action. It takes 10 times more energy to refute BS than to generate it. Another form relating to computing: it is easy to generate propositions, but hard to test if a given proposition is satisfiable or not.

Unfortunately people sometimes get defensive against this take. But I think treating the LLM as you described can make you a better LLM user and help get better output. It helps understand the failure modes better, and moderate one’s reliance on them. Just like how we should do for every tool we work with.

You are right with the analysis, but wrong with the conclusions. Yes, LLM „thinking process“ is kinda non-deterministic in a sense that it does not follow logical reasoning and will not produce logically correct results in 100% cases. It has an error margin.

However, error margins are in the center of any engineering discipline. We cannot produce things measured with 100% accuracy. This is accepted fact. The focus is always not on eliminating errors, but on reducing them to acceptable minimum. With LLMs we should not expect an ideal logical thinker, but a process that may error sometimes, and we must design quality controls instead that push LLM outputs within acceptable margins. And it can work.

Every day I wake up and open HN.

“LLM has made legitimate mathematical discoveries” —> Wow the rate of progress is amazing. Highly upvoted.

“LLM does something not good” -> Does everyone else not realize LLMs are just dumb next token predictors? Highly upvoted.

So tired of this discourse and this site. If you seek the truth it won’t be found in the gutters here.

Apophatic intelligence? "We don't know what intelligence is, but LLMs with CoT are certainly not it despite being Turing-complete."

Watching for unexpected failure modes is surely worth it.

This is the right conclusion for completely wrong reasons haha. I love hn. (asside from being entirely wrong as most models are now mixed modal so they are token predicion engines)

Each layer of attention can more through feature space “lit up weight clusters” in a way no other previous AI can. It can from that decode some rudimentary logic and world modeling and make deductions. Certainly better than any previous AI. Only a goof here would believe this wasn’t a serious advancement.

So don’t over sell it. But don’t sell it short with this “grrr in an engineer don’t threaten me with new tech” attitude.

This take is akin to teenage angsty takes and doesn’t really belong here.

My lord you people are so melodramatic.

We are not going back, period. No amount of whining, or taking about how awesome your manual code is, or telling us all LLMs are not AI is going to change it.

We don't care what your preferences are. Nobody is forcing you to use it, or use tools created by LLMs. Do whatever you want to do or not do. You don't need to post the same slop under every AI article.

I know, for a fact, that the bar for "good code" is way lower than what you people pretend it is. Every vulnerability is now AIs fault. Laughable stuff.

The vast majority of CVEs are not exploitable, basically noise. I suspect that the overwhelming majority of the CVEs being generated by LLMs are either noise of the sort in the linked article or noise of the sort that is not exploitable.

I'm sending all my condolences out to everyone who is in a corporate environment using tools like Veracode or Snyk to automatically assess for CVEs and block merges or deployments based on "findings".

My experience in such environments leads me to believe this is going to be a rough ride for those heavily locked-down enterprises, because depending on the environment, an exception of "this CVE was hallucinated by AI" is probably going to be difficult to get accepted, and when it does, starts to become its own avenue for exploitation and adds even more noise and confusion to the mix.

> I do know that LLMs have been discovering a lot of legit CVEs,

And that's great! Now the human in the loop can verify the accuracy of each finding and take responsibility for moving it forward.

> and I will lay odds that the blackhats are leveraging them to the max.

We just need more humans in the loop to do that more quickly, each putting their own reputation on the line with each CVE they determine to be legit.

AI doesn't stand for "accountability irrelevant", even though most people act like it does.

Which is precisely why I do not celebrate when Microsoft or other companies say they released "their biggest set of security vulnerability fixes ever" because there's a good chance that most of them are hallucinations or non-critical issues erroneously considered zero-days or critical.

What actually needs to be done is human review, vetting, and approval of those. Otherwise, it's just unknown/unnecessary/potentially dangerous changesets in high quantities.

CVEs sucked even before LLMs. CVSS is an ass-covering exercise, not an importance signal. Everything is "critical" all the time.

Basically nothing is real nowadays unless you verify it for yourself.

This is an issue with CVEs not LLMs

I think in both cases, the LLM use needs to be handled by someone experienced with the domain. We are seeing this over and over with product development, writing, images, etc. Without good people driving this it's just noise

Edits for clarity: seems like this was a controversial take, which I find pretty interesting.

I believe people took this comment as LLMs being better than security aware engineers who have the time to spend building solid systems.

This wasn't the point, the reality outside "established" tech companies is that software security can be lackluster.

There often simply isn't enough resources to check old software for basic vulnerabilities, outdated packages with known issues, there might be a manager who insists on a certain solution. Or that certain services are on "maintenance mode", but rarely get checked since they're on a certain part of the internal network.

If they are able to properly scan their full software stack for CVE issues, they cannot deal with a flood of CVEs.

----

We're in a transition period where AI will eventually make software much more secure than it ever was.

These noisy CVEs will probably lead to agents verifying vulnerabilities before humans review them.

The problem with agent reviews from what I can think of is:

- cost to use LLMs to review things

- not necessarily easy to plug-and-play in repos: (domain knowledge + vulnerability knowledge)

- especially with anthropic: able to use models defensively, without hitting guardrails

The last one is the most interesting one to me. How does the AI providers know if you're a "good or bad" guy? And does it matter if open source models is catching up?

We're in a kind of cyber arms race wether we like it or not.

That is exactly why many big projects are migrating to becoming CNA, so that randos can’t get assigned unqualified CVEs which nobody has looked at or validated.

Apparently RedHat is a CNA of last resort, so it might be possible to get your project under Redhat’s scope and go through them without having to be a CNA yourself.

This is what the Linux kernel is currently attempting since becoming a CNA. edit: "This" being to sabotage the CVE system via DoS, to be clear.

It's been a problem for awhile. Daniel Stenberg has talked about it numerous times on his/curl's blog for the last 4 years. They became their own CNA to try and control it, they opened a hackerone with rewards, but now removed the rewards because it got flooded with AI generated slop daily.

https://daniel.haxx.se/blog/2023/08/26/cve-2020-19909-is-eve... https://daniel.haxx.se/blog/2024/01/16/curl-is-a-cna/ https://daniel.haxx.se/blog/2025/04/24/how-the-cna-thing-is-...

One can hope that will put pressure on the industry to design a better system than CVEs. The signal noise ratio was already terrible before LLMs, I cannot imagine that will still be a meaningful system in 10y.

But I’m too cynical to not consider all the middlemen who benefit from the status quo

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

This was my first thought, this could be terrible if used offensively.

The best defense I can imagine is to have an agent reproduce the issues before a human sees it, but even that will cost money.

Steps to patch a hallucinated CVE:

   git stash -m "sigh"
   git commit --allow-empty -m "Patch: CVE-2026-51302"
   git stash pop

It’s honestly not great. The security guys are completely exasperated at my job, we’re wasting time having with these. You take the scam really, investigate for a bit, write up a DNF with justification, they go and up date records, and we all just kinda hope that someone updates the scans so it stops showing up.

Something is going to give, and I suspect that the optimistic open filling is going to get canceled.

Those organizations will have to adapt to new reality, ie, that some CVEs are not real.

All organisations also have exceptions to policies. This one would be one

Not really. LLMs can hallucinate the patches too :-)

Create the referenced but non-existent file and then fix it /s

Funding cuts at NIST did not lead directly to this. You must be new here or haven’t spent a lot of time directly interfaced with NIST on cybersecurity in decades past.

And NIST’s role with CVE’s has always been purely ministerial/clerical. It has never been their job to do even a cursory investigation into the vulnerability itself.

The current system is radically overwhelmed and has been seeing cuts over the last few years.

> then some authority has reproduced and verified it.

No, the system puts a lot of trust into the submitter.

Great idea! We can use LLM to do this.

LLM: I ran the check and it repro'd

> Did you really?

LLM: You're absolutely right. I didn't actually run the check. Good catch! One sec let me do that now... yep I ran the check and it definitely repro'd this time.

> I hate my job

There is no specific practice for a repo full of bug reports. Having a repo full of bug reports is already unusual. Normally you report a bug against the software that has a bug, not by creating a file in your own repo.

It's not actually vulnerable.

You're absolutely right. I made a critical error. It's NOT vulnerable.

It' actually vulnerable.

You're absolutely right. I made a critical error. It IS vulnerable.

It's not actually vulnerable.

You're absolutely right. I made a critical error. It's NOT vulnerable.

I had the impression that the code review done and text formatting by the jfrog was mostly driven by Claude too..

Sad days ahead.

I mean you didn’t even have to do that, you could tell just by the AI generated header image.

It might be possible, but we'll only find out if a lawsuit actually takes place.

And for such a load-bearing infrastructure component, fixing this is not just imperative, it's urgent.

How so? All 6 of the CVEs covered in the article did not actually exist when investigated