petermora
5 days ago
Recently I watched this presentation of a kernel developer about the "Mythos' 79 kernel vulnerabilities": https://youtu.be/NnV_cWeoo5Q?t=257
He claims that most of these vulnerabilities are not an issue at all, or have been fixed earlier, etc. I can clearly see the improvement of the AI models, but based on this youtube video it's very plausible that these AI labs rush to announce a result without further understanding of the underlying problems. I wonder what's the quality here in this article.
ActionHank
5 days ago
That's the weird thing with LLMs, they know just enough about a topic to fool anyone without much knowledge on the subject, seemingly even frontier labs.
voidhorse
5 days ago
It doesn't help that the labs have positioned themselves on the predication that expertise is no longer required.
They are trying to sell this false myth of machines so good they can replace any human expert, and they buy this BS themselves, so, instead of actually learning something and/or confirming results with institutions of experts (what's left of them anyway) they just rush out a PR piece knowing the unwitting public and VCs won't care enough to look into the validity of anything themselves anyway.
The whole thing is a gamble on whether or not they can convince the world that a fancy word machine substitutes for deep experience and expertise.
(To be clear, the fancy word machines are still very useful tools, but they are not adequate replacements for knowing humans).
dijit
5 days ago
> they know just enough about a topic to fool anyone without much knowledge on the subject
There are humans like this and they are some of the most catastrophic people to have on a project, I actively avoid them.
I'm sure LLMs are different.
keeda
5 days ago
They've fooled me so much on subjects new to me that I actually became knowledgeable about them!
How do I know I didn't learn hallucinations?
Because based on that knowledge, I've produced code that works as intended.
jasomill
5 days ago
Yes, and before that you were probably adept at writing code based on incomplete and sometimes inaccurate documentation, using tools and technologies that were powerful but occasionally buggy.
Or at the very least, you were interested in the subjects in question and just never had the time or other resources to learn them.
keeda
5 days ago
Yeah the rate at which I'm learning is insane! I think the key is picking a project with practical goals that you can track towards, ideally backed with concrete evidence (like data) and measuring progress by how close you are getting to those goals via working code.
As such, I'm learning through motivated, self-directed, focused exploration, using the AI to answer questions, run experiments, explain findings, investigate surprises, and explore new directions to research, all grounded in concrete data related to those goals.
The data and our results on it are what keep us honest. The data is also what helps understanding nuances and building intuition because the AI can explain outcomes in the context of samples data points that you can reason about independently.
The difference between YOLO vibe-coding and engineering is understanding. I think this is the future of learning and skill-building.
chatmasta
4 days ago
The Anthropic red team is led by security professionals who wrote books on the knowledge of the subject.
smashers1114
5 days ago
What topics are you still seeing this in?
ActionHank
5 days ago
Do any of the execs you're working with seem to be overly confident that LLMs can fully replace devs?
If that's the case, that's the primary example that I've seen in many places.
dev_hugepages
5 days ago
Anthropic currently seems under AI psychosis/mania so I wouldn't use it as an argument of authority
square_usual
5 days ago
I don't get what your point is WRT this article. They aren't announcing a result? They're describing a technique they used in real projects to prevent new vulnerabilities, including a change that is merged into OpenSSL. What is the "rush to announce a result" here? What are you saying they don't understand?
Rapzid
5 days ago
Well Anthropic seems to have taken to "bragging" recently that they don't write, read, or review code anymore.
God, I'd hate to be on the receiving end of their bug/issue submissions.