I think the rule of 2 would work if it kept the 3 from your lethal trifecta. "Change state" should be not be paired with "communicate externally".

And even then that's just to avoid data exfiltration- if you can't communicate externally but can change state, damage can still be done.

Also in the context of LLMs I think model weights themselves could be considered an untrusted input, because who knows what was in the training dataset. Even an innocent looking prompt could potentially trigger a harmful outcome.

In that regard it reminds me of the CAP theorem, which also has three parts. However, in practice partitioning in distributed systems is given, so the choice is just between availability or consistency.

So in the case of lethal trifecta it is either private data or external communication, but the leg between these two will always have some risk.

I love to see this. As much as we try for simple security principles, the damn things have a way to become complicated quickly.

Perhaps the diagram highlights the common risky parts of these apps and we gain more risk as we keep increasing the scope? Maybe we can do some handovers and protocols to separate these concerns?

Totally. I think the original "Lethal trifecta" post by OP only pertained to data exfiltration and never included changing state (maybe was implied by sensitive data access).

Rule of 2 model has holes.

There was a minor scandal about exactly that a few months ago: https://asia.nikkei.com/business/technology/artificial-intel...

"Research papers from 14 academic institutions in eight countries -- including Japan, South Korea and China -- contained hidden prompts directing artificial intelligence tools to give them good reviews, Nikkei has found."

Amusingly I tried an experiment with some of those papers with hidden text against frontier models at the time and found that the trick didn't actually work! The models spotted the tricks and didn't fall for them.

At least one conference has an ethics policy saying you shouldn't attempt this though: https://icml.cc/Conferences/2025/PublicationEthics

"Submitting a paper with a "hidden" prompt is scientific misconduct if that prompt is intended to obtain a favorable review from an LLM. The inclusion of such a prompt is an attempt to subvert the peer-review process. Although ICML 2025 reviewers are forbidden from using LLMs to produce their reviews of paper submissions, this fact does not excuse the attempted subversion."

Most current consumer LLM uses are run only once or a few times, before changing prompt and task. This causes the attacker to have to move first: they put malicious injected documents onto the internet, which are then ingested by ephemeral systems, the details of which the attacker doesn't observe.

On the other hand, something like an AI mcdonalds drive through order taker runs over and over again. This property of running repeatedly is what allows the attacker to move second and gain the advantage.

The HITL is needed to pin the accountability on an employee you can fire

wouldn't that still add a lot of value, where the person in the loop (sadly, usually) becomes little more than the verifier, but can process a lot more work?

Anecdotally what I'm hearing is that this is pretty much how LLMs are helping programmers get more done, including the work being less enjoyable because it involves more verification and rubber-stamping.

For the business owner, it doesn't matter that the nature of the work has changed, as long as that one person can get more work done. Even worse, the business owner probably doesn't care as much about the quality of the resulting work, as long as it works.

I'm reminded of how much of my work has involved implementing solutions that took less careful thought, where even when I outlined the drawbacks, the owner wanted it done the quick way. And if the problems arose, often quite a bit later, it was as if they hadn't made that initial decision in the first place.

For my personal tinkering, I've all but defaulted to the LLMs returning suggested actions at logical points in the workflow, leaving me to confirm or cancel whatever it came up with. this definitely still makes the process faster, just not as magically automatic.

It kind of sounds like a weak version of airgapping. If you cant persist state, access private data, or exfiltrate data, there is not much point to jailbreaking the llm.

However, its deeply unsatisying in the same way that securing your laptop by not turning it on, is.

Yeah it's nonsense, because the author has described the standard "read, process, write" flow of computation and decided that if you remove one of these three, then everything is safe.

The correct solution is to have the system prompt be mechanically decoupled from untrustworthy data, the same it was done with CSP (content security policy) against XSS and named parameters for SQL.

Can you explain what you mean? How is Chesterton's fence applied to AI security helpful here? Are you just talking about not removing the "Non-AI" security architecture of the software itself? I think no one ever proposed that?

Sure let's just remove all security, encryption, firewalls and auth - nobody will abuse vulnerabilities if it's a crime!

What would the crime be?

If I have a web page that says somewhere on it "and don't forget to contact your senator!" and an LLM agent reads that page and gets confused and emails a senator should I go to jail?

Yeah, IT tried so hard to teach us something as basic as "don't click on links in suspicious emails" yet so many people fail that after multiple trainings and tests.

But guess what? AI! Agents! <company name> Copilot! Just let them do things for you! Who would have thought there might possibly be a giant security hole?