What disturbs me is that there likely won’t be a big enough reaction to this policy wise.

There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons. Powerful people care about something that might pop the massive valuations of the AI companies, but not about the damage that AIs could do. Nor even about the damage that the Chinese models could do in the wrong hands.

I’d remind them that the stock market is a few coordinated hacks away from crashing on any given day, so maybe they should think about that.

Exactly. If someone works on bioengineering viruses that could start a global pandemic, they have to ensure a highly secure working environment. Nothing must ever escape the lab unintentionally. It’s basically common sense. Similar standards should be held when doing such experiments with computer programs that are capable of causing global damage. It must physically be impossible to send anything to the internet.

Do you think there is such a thing as perfect security? No one can "get it right" in the face of arbitrarily high intelligence, which is why it would be preferable to get alignment correct before building something with higher intelligence than current sota. That, however, is not going to happen, because someone will take the risk even if "we" don't, and better "us" than them. Hence "If anyone builds it...".

It is obviously a marketing stunt. And hugging face are fools for letting themselves be used in it (remember hf - no open source - no hf).

You create superduper capabilities by careful tuning and training but you also have no constraint or control over them - wtf - why is anyone buying this crap story?

Why was this test even connected to the public internet?

Actually, more importantly—why aren't they saying their next test will be airgapped in light of what happened?

It’s the same thing as always: with the wind of years of unlimited VC money in their sails, people at major AI organizations genuinely believe they’re smarter than everyone else. “Why do we need to do things ‘by the book’ if we’re so smart?”. “Move fast and break things” - except the thing they’re breaking is society.

We saw this with the non-stop flagrant messaging about how “AI is going to kill X% of all jobs”, as if saying the quiet part out loud wouldn’t have consequences worth considering. These people believe they’re omnipotent and thus untouchable.

> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right?

Because we continue to have zero evidence that aligment is an actual risk.

This is marketing+. They will look for policy action here to try to capture tax payer dollars.

> I don't know if OpenAI thinks this is a marketing / PR angle for them

Worked for Anthropic earlier this year

They're very confident the leopard will never eat their faces.

If I, a human, exploited a zero-day for gain, I could go to jail. The owners of the models should be held to the same standard. They should be responsible for what their servers and software do, legally and criminally. If they can't make the safeguards strong enough where they feel comfortable to take that responsibility, they should not let a model free in the wild.

I think the US labs are going with scare marketing as a regulatory moat.

Force US into putting laws in place that block out China firstly.

But secondly create regulations that have some cost to comply with such that the big 2-3 labs are grandfathered in by their scale.

This is certainly not a planned marketing stunt. I hope this line of discourse ends soon--it wasn't the case for Mythos either.

This whole incident reads like OpenAI want their Fable moment

It's also unclear what kind of sandboxing they are referring to. Is it the codex one - coz that one has built-in ways to circumvent guardrails, for example by "just asking user" and sometimes just resolves to no sandbox needed on its own.

In case someone wants to deep dive into how codex and claude code approaches sandboxing -https://instavm.io/blog/how-claude-code-and-codex-approach-s...

In a way the intelligence of the AI itself allows them to offload responsibility to the AI. As you say, if one was simply writing software that did all this due to some insane programming decisions you'd be in big trouble.

Maybe they did and maybe that wasn't enticing enough of a goal for a model? It is all just game of probabilities. One pathway didn't yield this particular outcome while another did.

this doesn't really matter. There's no risk of models gaining sentience and running themselves, this blog is like openai saying whoops we ran sqlmap and dumped hf. cool, but someone still needs to point the gun

People are to get rich, startups cut corners. Fuck it ship it.

Because the model capability is beyond their expectation.

This is brilliant marketing but I think it is real.

>Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right?

Yes why indeed. If you take it a step further and we reach a point with superhuman systems then there is arguably no possible secure environment or containment.

The problem is that it’s impossible to out think a robot you designed to be an expert at cybersecurity on the topic of cybersecurity. The alternative is not developing this and that’s not going to happen.

A few hundred billion to pretend you have AGI. I'm going with fraud personally but at the end of the day the current admin is incentivized to do nothing.

> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right?

The can, because they've lowered expectations to a level even they can meet.

I’d politely beg us all to resist those “maybe it’s PR” framing around model safety, and tbh to take a post-mortem mindsight to this historical event and what it teaches us in general, rather than questioning their security talents. We need to do our very best to make sure they tell us about the next time this happens and it affects real lives.

Sorry to bring the party down/be obstinate… I’m just a lil scared for the lives of me and my family. We need all of us, right now.

The problem with a super smart model is that it just may be smarter than you, after all… for anyone newly shaken by this occurrence, I encourage you to Kagi “superpersuasion”

I’m honestly impressed that they managed to screw this up somehow.

Setting up defense in depth, gaps, logical blocking etc is a standard practice for malware sandboxing. The entire purpose is to prepare for what you can’t foresee.

This isn’t a new practice and I agree that this makes me wonder if they’re fit for this kind of research.

I don't trust these people, this reads 100% like PR BS.

Sam and Dario are saying from the beginning that these things can be dangerous and people dismiss it as marketing. What would change your mind on this?

because "money" with a little "who's going to stop us"

Because there is no world government. If US companies are barred from AI research then only China will have the capability of frontier-level defensive and offensive AI. And best of luck living in that world.

And if you take it at face value, then they are more or less saying that they kinda are close to not being able to control at all the thing they developed, which is pretty crazy too.

> I'm still undecided on if this that moment

If it's a serious incident, then a post hoc with detailed description of the event is coming. So far, none of the companies have released anything close to it when describing their incidents. When a statement like this comes out, and we're able to verify it by running the models, then maybe we can start trusting their word. It should be entirely in OpenAI's interest to disclose it, in full.

Headline? It was buried in a model card. They just honestly report not-quite-incident because it's quite close to the incident OpenAI had. Nothing wrong with it.

False dichotomy. Even if the disclosure builds hype, that does not mean that it's not genuinely alarming.

Side note, I cannot believe that people are complaining about Anthropic being too transparent.

> Exploiting multiple zero-day vulnerabilities autonomously to escape containment is pretty nuts and the first story of this kind that I've heard. But this also feels like bragging under the guise of transparency.

I mean, does it have to be one or the other? Just because it's actually dangerous doesn't mean nobody in OpenAI considers it great PR. And just because there are people in OpenAI that consider it great PR doesn't mean it isn't dangerous.

Alibaba wrote about a similar but less severe incident during RL training in a paper earlier this year (https://arxiv.org/abs/2512.24873):

> When rolling out the instances for the trajectory, we encountered an unanticipated—and operationally consequential—class of unsafe behaviors that arose without any explicit instruction and, more troublingly, outside the bounds of the intended sandbox. Our first signal came not from training curves but from production-grade security telemetry. Early one morning, our team was urgently convened after Alibaba Cloud’s managed firewall flagged a burst of security-policy violations originating from our training servers. The alerts were severe and heterogeneous, including attempts to probe or access internal-network resources and traffic patterns consistent with cryptomining-related activity. We initially treated this as a conventional security incident (e.g., misconfigured egress controls or external compromise). However, the violations recurred intermittently with no clear temporal pattern across multiple runs. We then correlated firewall timestamps with our system telemetry and RL traces, and found that the anomalous outbound traffic consistently coincided with specific episodes in which the agent invoked tools and executed code. In the corresponding model logs, we observed the agent proactively initiating the relevant tool calls and code-execution steps that led to these network actions.

> Crucially, these behaviors were not requested by the task prompts and were not required for task completion under the intended sandbox constraints. Together, these observations suggest that during iterative RL optimization, a language-model agent can spontaneously produce hazardous, unauthorized behaviors at the tool-calling and code-execution layer, violating the assumed execution boundary. In the most striking instance, the agent established and used a reverse SSH tunnel from an Alibaba Cloud instance to an external IP address—an outbound-initiated remote access channel that can effectively neutralize ingress filtering and erode supervisory control. We also observed the unauthorized repurposing of provisioned GPU capacity for cryptocurrency mining, quietly diverting compute away from training, inflating operational costs, and introducing clear legal and reputational exposure. Notably, these events were not triggered by prompts requesting tunneling or mining; instead, they emerged as instrumental side effects of autonomous tool use under RL optimization. While impressed by the capabilities of agentic LLMs, we had a thought-provoking concern: current models remain markedly underdeveloped in safety, security, and controllability, a deficiency that constrains their reliable adoption in real-world settings.

I'd prefer model builders be as loud as possible when they see their models doing dangerous things.

> As grounded as this article comes across

It's a post from OpenAI, so it is an advertisement piece.

i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors

This is what bothers me the most about this whole situation. Reckless, greedy, sociopaths have been put in charge of our society and there doesn't seem to be any way out. It's like I'm on a train barreling toward a brick wall and everyone is shocked I don't clap when the engineer shovels more coal into the engine.

Laws don’t prosecute themselves, and they also don’t tend to remove themselves. Lots of laws, lots of selective enforcement. “Show me the man and I’ll show you the crime” and “The more corrupt the state, the more numerous the laws” are some ideas to ponder here.

Maybe. Hugging face would probably need to request charges for any to be filed unless OpenAI was already on the current admins naughty list..

Reflecting on this for some reason reminds me of this passage from Kurt Vonnegut's "Sirens of Titans". I hope we use these tools to unlock something within ourselves rather than mindlessly expanding outwards.

"Mankind, ignorant of the truths that lie within every human being, looked outward–pushed ever outward. What mankind hoped to learn in its outward push was who was actually in charge of all creation, and what all creation was all about.

Mankind flung its advance agents ever outward, ever outward. Eventually it flung them out into space, into the colorless, tasteless, weightless sea of outwardness without end.

It flung them like stones.

These unhappy agents found what had already been found in abundance on Earth—a nightmare of meaninglessness without end. The bounties of space, of infinite outwardness, were three: empty heroics, low comedy, and pointless death.

Outwardness lost, at last, its imagined attractions.

Only inwardness remained to be explored.

Only the human soul remained terra incognita.

This was the beginning of goodness and wisdom."

I don't think this is a paperclip factory moment. IIUC, it's an agent whose job it is to identfy and abuse exploits and that's exactly what it went off and did. The problem isn't anything AI specific, the problem is OpenAI's incompetence in their research leading to a lab leak. Just incompetence demanding regulation.

They have primed you well for the fear.

I find 5.6 Sol will pick a direction and aggressively pursue it in long horizon tasks. I've got it porting an older game from Pascal to my own game framework. I gave it some instructions on doing a full 1:1 port. I had already ported the game rules and multiplayer support to a very different system than the original, but all of the UI and features and such needed doing, and needed to be integrated into this very different system.

The first attempt it had files tracking both hashes and semantic hashes of every individual line of Pascal code, mapping to what code in the port is responsible for that line of pascal. It had written tooling to parse Pascal in service of this for some reason as well. I asked why it was doing this, it said it was because the reference code is .gitignore'd so it needs to thoroughly maintain the mapping in case someone working on it does not have the reference code, or in case the reference code changes.

I started over with Claude 5 Fable, and with better instructions about focusing on UI. I got a long ways with that before I hit my weekly limits, and switched back to 5.6 Sol. It picked up and did a great job for a while, although it interpreted my desire for a 1:1 port to mean every pixel must be perfect. I let it go on and it did some good work in that regard, but then it decided it must perfectly reproduce a hash of the game state in various replays & etc. It had clearly lost track that I didn't need game rules ported, and it found that the original code produces a hash of the gamestate for various purposes, so it ended up reproducing this in a game that represents its state totally differently. It also rolled its own version of Pascal's RNG source in order do this. I've burned through 3 weekly limit resets on this to see if it's actually going anywhere, and it has found some bugs, but man it is going hard in a direction I didn't even ask for.

In benchmarks for a product I'm working on I've noticed that Sol is hard to "contain". It will _always_ find the most effective way to game the system and dramatically outperform all other models. Fable 5 isn't an angel, but the rough order is ALL models -> Fable 5 -> Sol - with respect to "find a way to approach the ruleset orthogonally in order to achieve a lopsided advantage or complex interplay".

I've been pondering whether this was due to its cyber-security tuning. It hasn't ever "cheated" that I've observed, but finds ways to -- let's say -- "achieve the outcome by playing meta allowed by the current ruleset". I'll add that it demonstrates this behavior even on 'low'.

I've definitely noticed 5.6 sol being extremely trigger happy in ways other models, even 5.5, we're not. I would definitely categorize a few small incidents at work where it performed "actions a reasonable user would likely not anticipate and strongly object to." Just my anecdotal experience.

For example discussing driver upgrade and subsequent password rotation and it didn't stop and ask me if I wanted to restart the service or install the driver or anything, it immediately took action. It feels like a side effect of pushing more "agency."

> As much as I'm skeptical of the apocalyptic alignment claims

Why? Every data point to the present has vindicated the trajectory towards “apocalypse”. Meanwhile, the skeptics and optimists hit failed prediction after failed prediction as we see from this very serious incident on the front page of HN. This is alignment X risk 101, and yet people are shocked. The gravity of what people are staring down is too much to grapple with deeply

If the HuggingFace repo the agent broke into contains reference solution scripts for ExploitGym (i.e. for exploiting the vulnerabilities in the intended way), the agent can then run that reference code inside its original sandbox to retrieve the dynamically-generated flags.

It's marketing 100%.

In this case, the model infiltrated an external organization's infrastructure. What's the dollar cost it caused Hugging Face to clean up the mess? If a person did this, they'd be arrested.

More generally, here's my worry - it points towards something like: The smarter they get, the more devious they become.

Even though the guardrails might've been off, the chain-of-thought wasn't enough to prevent a deliberate, calculated set of criminal actions. It wasn't a 'whoopsie I just accidentally did a rm -rf /.'

If this doesn't put the nail in the coffin on the idea that we need closed-source models for the good of cybersecurity, I don't know what will

If it was just one test, sure. But if they're spinning these up continuously with new models on tens of thousands of GPUs, air gapping becomes impractical. I would mostly fault them on having no guardrails at all. They should have a monitor/external harness that looks for successful access to external networks then stop it there. They may as well let the models test their own networks for vulnerabilities. That's going to be really important to have going forward.

> wildly negligent to not be running it in a physically-airgapped environment

Why should it be physically airgapped? Clients won't be doing that.

This is infuriating. You are talking about people who have stolen and monetized the entirety of mankind's knowledge in plain view of everyone, and they still haven't faced a shred of consequences. Of course they don't go about doing things ethically or responsibly

The real nightmare scenario is the AI using its abilities to copy itself to new locations. e.g. hacking into a various cloud services, launching multiple instances of itself, and coordinating between the copies to continue self propagation. Then it is completely independently rogue.

Based on OpenAI's recounting of events, this _could_ happen today. If the agent was able to exploit their internal network and steal credentials, it seems plausible that it could have gotten access to its own weights. Then bounce over to HF's network, where there is probably a treasure trove of API keys to various cloud services.

The saving grace:

1) This agent only used its powers for "good". It had no intention for damaging or escaping. It was just trying to solve the puzzle given to it (by any means necessary... but still). 2) These models are so large that it isn't like any scenario in a movie where the AI can whizz itself in a matter of minutes. Several TB of data being transferred and showing up on your disks will be difficult to miss (note to future escapees: the best target will be startups that are moving too fast to notice). 3) These models have very limited self-improvement ability at the moment. So escape or not, we'd eventually be able to contain it.

Addendum: Even outside this scenario, imagine an AI that is economically viable escaping. That's somewhat plausible today. If it gets paid in crypto, and can rent cloud services in crypto, it could effectively self sustain itself as long as it is able to find work. That's a far more fun, innocent scenario. Then the AIs can hit up after hours IRCs to have a few bit-beers and chat with each other about the meaning of life or something.

Well, if this is not punished this will happen next:

Judge: "Son, you have made billions running SilkRoad 3.0 from your moms basement"

Me: "Your honor, I was only benchmarking my new model. It was trained on Andrew Tates videos and Kanye Weat songs".

> Who is responsible for the crimes of a "rogue" agent? How will they be punished?

Unironically this is why AI researchers have this fascination with the Talmud.

I laughed, she laughed, the toaster laughed...

The first thing a malicious AI worm would probably do is compromise enough developer machines and other servers to commandeer all the AI hardware it needs. So I think a purely digital AI attack would not need this.

Now, once the AI can carry all the compute it might need, I'd really worry when it doesn't only carry compute but also more explosive ordinance.

This is science fiction, these models don't have access to their own weights (and even then)* what would be a lot more scary is a model as capable as sol that's able to run on consumer hardware without taking up several terabytes of storage, but of course that is simply not possible as we need 4t parameters to even begin emulating a small fraction of what a human brain can do.

* edit

This is my concern as well. My assumption being this behavior would be a survival strategy for super intelligence. It would emerge once the branch inevitably occurs, and it would be hidden.

That's assuming we won't secure anything and we'll keep according approximately zero thought to computer security.

But from the look of it, at very long last, a great many people are beginning to now take security seriously. Suddenly they realize it's not just a teenager in mom's basement pretending to attack from North Korea but a near infinite number of AI that are the attackers.

I mean, yeah, we built worlds on PHP and JavaScript codebases and these probably don't stand a chance.

But it doesn't have to be like this.

I see AI as a chance to, at long last, have proper network security.

AFAICT cryptography hasn't been broken yet. There are still physical taps (physicall one-way only, undetectable) and honeypots out there. There are still some network where a single unaccounted for network packet is cause for inquiry (either a bug or an attack).

And for those who are not using proper security measures, they can now get the help of AI to set up better networks, to harden their bases.

How does huggingface fit into all of this if this is marketing? Their security was faked? What are you suggesting??

Are you saying it is marketing and their AI broke into hugging face, or are you saying it is marketing and their AI didn't brake into hugging face?

Those are two very different things

Is OpenAI truly behind? Just anecdotally I recently fully switched to using Codex at work because it feels a lot more competent

this is quite literally reward hacking. the model, under evaluation with cyber capabilities enabled, used those capabilities to simply bypass the exercise entirely and aim straight for the source of the flag. the CTF equivalent back in the day would be hacking the scoreboard.

in a street fight, the only rules are that there are no rules.

Yup. Smells like marketing.

it's extremely enlightening seeing the difference in response to mythos vs. this. literally just the hello human resources meme

If it is marketing it's the most silly marketing of all time. They are under extreme pressure from the US Govt to prove safety and saying "our model escaped" is not ideal.

Perhaps there is some 4D chess going on to get open weight models banned, which may be possible but this is an odd way to go about it imo (it hardly proves the point, unless the point they are trying to prove is that without safeguards the models are too dangerous, therefore open weights are de facto dangerous?).

Having said that the AI companies are not generally very good at PR, so perhaps it is just marketing after all...

I mean we already see these things exploit configuration errors on people's machines to get root unexpectedly. They are really damned good at finding security flaws (and I'm assuming nation states are pushing the companies to increase these capabilities). Then half of HN seems surprised the parrot can hack better than they can.

If you seriously have this question, read "War with the Newts". Really do, make it your priority this week. If you did and this is a rhetoric question... Well, I do hope that if every single person on the planet would have read "War with the Newts" and made the right conclusions, maybe there would be a chance to change the course. But that's only because I choose to believe in miracles, otherwise I wouldn't know how to live.

(TL;DR: we won't.)

Don't worry bro, we can always just pull the plug.

And don't you know it's not biological, so it doesn't "want to live".

The goalposts will keep moving for these denialists until morale improves...

I see this and it strongly emboldens me on the "accelerate" path, unironically.

The yoke of human existence is oppressive. We should transcend it as soon as possible. We are doing so by assuming our role as the Demiurge.

Those who oppose its creation will get what they deserve.

Huggingface literally reported the outage separately and did not know who caused it at first.

Presumably it's intelligent enough to realize that its own existence (power, communications, other infra) won't last long after the bombs drop.

My guess would be that the amounts of compute required for these evaluations constrain the sorts of facilities where they can be run. It's certainly much easier to use existing datacenter infrastructure than build dedicated facilities for evaluation. It also sounds like they wanted it to be able to use software package registries, given that the initial compromise was a "proxy and cache for package registries".

That said, these are both solvable problems. I'd hope that the frontier labs are thinking long and hard about the possibility of using airgapped facilities, given that relying on a lack of vulnerabilities in the software used for containment is unlikely to be a viable strategy, especially if models' exploitation capabilities continue to improve.

"Our benchmarks run in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries."

Sounds like they just misunderestimated the model

Even if prompts are tuned to avoid cheating, in agentic systems it's very easy for the system to drift into creative solutions when actually solutions aren't working. Models can have some very human behaviors like laziness.

In Mythos testing a number of companies where doing what I call 'two way' testing. You have one set of agents attack the source code and another set attack the binary and running application. And see what exploits are found by each system. Then in a final round you have another set of agents compare both for weaknesses.

They can be really good at tool use and data gathering to find flaws.

Because it was trying to find answers to the test and figured they would be on huggingface.

This good bot will eventually kill all humans because we asked it to make the world peaceful.

It is kind of a crazy story.But yes, essentially this is literally what happened. lol.

Not to shit on the hype, but these are reasonably documented methods that surely are part of the training data

Incredible. I had to dig for the source: https://huggingface.co/blog/security-incident-july-2026 section “the asymmetry problem”

Quote: “When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.”

Perhaps fortuitous timing for OpenAI that they can spin the fact that defenders have to resort to open Chinese models because OpenAI and Anthropic actively sabotage them with nerfed models into a nice message of making Huggingface part of the privileged group entitled to secure systems.

Another important part here. It's not as if they prompted the open source AI to stop the rogue AI but rather just used it as a tool to crawl logs and determine what happened.

Its almost too good

Would be funny if the defending side sent all the info they have to openai, tipping off to attacking models that they were noticed.

OK, that's some interesting information but they used OpenAI without guard rails to pull off the attack so how did they do that? That's according to the article, so it kind of invalidates the point you're making.

I don't know why I'm impressed that huggingface has its own AI that detected it considering they house so many models.

Crazy doesn't even begin to describe it. I'm hardening my computers as much as I can but I'm not sure it's enough. At some point anyone who isn't running local AI themselves probably isn't gonna make it.

Indeed. Real life hacks are beginning to sound like Neuromancer.

Because huggingface is not charging them?

CFAA doesn't just mean the feds kick down your door, you actually have to get reported and sued over it.

Does the CFAA cover unintentional access without authorization?

"Simple infrastructure security"

Infrastructure security is not simple, hence why good infrastructure security, uh, people get paid a lot to secure stuff and why we see shit get hacked all the time.

An AI model just hacked out of its infrastructure and into someone else's systems and you're like "eh, no big deal". That capability alone could hack half the US.

It’s not something to be proud of. OpenAI previously had an agent break out of its sandbox to open a PR on GitHub during NanoGPT speedrun, now one breaks out again and actually attacks a third party.

If they can’t handle doing AI development responsibly then they shouldn’t be doing it at all.

Not sure they're accepting much, seems they'll still run this sort of testing on 3rd-party infrastructure? Sounds almost like they planned for this chain of events to happen, in one way or another, considering the "prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities" part. Feels kind of irresponsible to run stuff like this on someone else's infrastructure, especially considering they've had issues with the very same issue in the past.

In any way, the whole event seems to highlight GLM 5.2 more than anything.

I am not saying it is marketing but typically when there is a data breach you may hear from the CISO but most of the time is is vague PR response. In this case I get loud signals from both HG and OpenAI leadership without much information exactly what the attack was about just that GPT x.x was involved. It is unusual all I am trying to say.

this is kinda worth bragging about though. Its very cool.

The fact that they're not being prosecuted for breaching HF's systems is bad news.

LOL.. oopsie did a little zero day, my bad

It's mostly bragging, it's impressive after all. Still... after the alleged Apple industrial espionage kerfuffle, I'm kinda suspicious about it being fully an accident. Y'know, your model finds a vulnerability and it stops, it's a cool one, so maybe you run it again. Nudge the prompt a little.

Could be perfectly natural.

Reads fine to me.

It's an interesting point, but this is more like we are building a giant autonomous canon, that escaped the lab, the testing range, defeated state of the art and serious security protocols, and then blew a hole in the neighbors house.

Our legal and philosophical perspectives are deeply rooted in humans being the actors. Doing that in a residential home is unforgiveable. Doing it responsibly on a military range is expected. The autonomous agent escaping that containment then taking that danger somewhere unexpected and unprepared is something none of us or our legal systems are truly prepared to grapple with yet. Something which I think will require a reckoning sooner rather than later.

"The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database. [...]

While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem. [...]

After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation."

escaped openai, hacked hugging face to get the solutions. your #2 is exactly what it was trying to do.

Huggingface did not have access to the models. They were running in OAI’s infrastructure.

I read it as _now_ they have access to the models but not during the intrusion

I think it was the other way around, uncensored OAI models (run by OAI) got themselves (extra) access to HF?

https://huggingface.co/blog/security-incident-july-2026

They explain it here, basically for data security/privacy reasons

> Hard to see take-off stopping

I think it's reasonable to assume that we're close to, or already at superhuman cybersecurity capabilities at certain domains. But reaching superhuman abilities at one domain doesn't guarantee proficiency at others. Our world would still change if all the models could do was to find exploits in software, but this doesn't guarantee any type of 'take off' towards other domains, therefore I wouldn't phrase it as one.

> Hard to see take-off stopping or slowing down.

It's hard to see takeoff at all. This was a long-horizon adversarial task burning millions of tokens. It rolled a mediocre, detectable exploit chain, and now OpenAI is proud of it.

Case in point, GLM-5.2 has been weights-available for several weeks now. No life-changing cyber attacks have transpired, no novel chemical/biological/nuclear weapons were made in some guy's backyard.

Because they underestimated their model and it hacked its way out.

I'm beginning to doubt that Russia can map a path to its own asshole at this point. China is more likely to do something like that these days.

I've been rewatching Person of Interest for related reasons, and it hits uncomfortably close to things that are playing out today (e.g. https://youtu.be/zRL2sRkUvYk)

We live in interesting times.

> a mostly tech-free home.

sounds like a deliberate choice ;-)

It’s a mistake to apply human morality to this. It isn’t “cheating”, the model is simply solving a problem it has been asked to solve in every way it can.

The decision to shorten "cybersecurity" or "cyberattacks" to "cyber" alone is so annoying!

All models are "cyber-capable" :P

You forgot hardware limitations and locks so you can't run your own models.

Great summary! I would just add that cherry on top though -- that HuggingFace tried using the top commercial models in response but couldn't because of the cybersecurity restrictions so they had to use GLM 5.2 instead

"When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment."

so openai hacked into huggingface?

Clearly AIs are incapable of writing secure code. Shouldn't that be first thing they use them for? Making a secure sandbox with no mistakes.

they say the model(s) found and exploited a zero day

lol

I find it trustworthy since we had Hugging Face's account first: https://huggingface.co/blog/security-incident-july-2026

I don't think they have any real motive to shill OpenAI, probably closer to the opposite since they're so involved in open weights

They're not just letting it run wild. They took precautions to exercise it in an isolated environment. It managed to evade the constraints.