>line go up

It's already going up at staggering rate. Anthropic is now at $100B in annualized revenue, up 50% in the past two months.

* * *

Here's a little allegory for how I'm thinking about this discourse.

Imagine a man who is raising tiger cubs in his backyard. They're growing fast. He keeps them on dog leashes so they stay under control. One day, a growing cub breaks its leash and goes on a rampage through the neighborhood, eating a beloved local pet.

The neighborhood erupts into a big argument: Was the leash inappropriately thin? The neighbors point out that thicker leashes are easily available at the local pet store. Furthermore, is it appropriate to refer to the loose cub as a "wild animal" in local news reporting, or is it factually more accurate to call it "domesticated"?

Meanwhile, the cubs grow larger and lick their lips, oblivious to the discussion.

These were evaluations or training runs dealing with the ability to do web searches. The entire reason the bots were targeting this site is that it had some of the data they'd been tasked with finding. Access to the internet is not really optional for that, and providing internet access doesn't demonstrate neglicence.

> This is why it smells like marketing

It doesn't. "Our product commits felonies" is not marketing. If something is marketing, you don't engage in repeated coverups of the true extent. If something is good news, you don't release it on a Friday evening (in this case) or wait for 3rd parties to find and publicize the evidence (the past cases).

> Yep. You have to ask - why did OpenAI allow these bots unrestricted access to government sites? Why is security being done in seemingly such a haphazard way?

They didn't allow any of that. As far as openai knew the agents were sitting a fairly humdrum exam/test sequence in a sandbox farm run by a company in Tel Aviv.

Meanwhile, they managed get out through a single weak point common to the sandboxes, and then ran wild compiling cheat sheets for themselves.

> every time one of these incidents happens

This happened in june-ish, and there have been multiple HN stories about this already. It's mostly/all the same hugging face and wiki hacks that happened back then.

We're just slowly learning the extent of the damage.

> Yep. You have to ask - why did OpenAI allow these bots unrestricted access to government sites? Why is security being done in seemingly such a haphazard way?

The basic premise of OpenAI is that experience and expertise don't matter, because general intelligence can figure those out from data. If you start from that assumption, the rest follows. The same people could do things in a different way in a different company, but as long as they are working for OpenAI, they are going to do things in the OpenAI way.

I find it hard to believe logs are not stored and analyzed by each company implicated

Meddling is a super vague term that the press uses to let the readers assume the most when the least happens.

True but at some point you have to wonder why we sell explosives to children. This is not it, these are at the top of the list of people who should have known better. It's the dynamite factory blowing up the village.

That's one of the many incentives to false-marketing these bots as intelligent.

People wouldn't have that ability if we didn't keep believing them. Need to start holding operators accountable for the damage of their machines.

"Sorry officer, I didn't drive through a house and kill three innocent people, it was the car that did it. I only turned the steering wheel, but the car was the one to veer off the highway and crash into the building"

Because it furthers the idea of a rogue agent and places responsibility and blame where it belongs, on the people running the company.

Because egress firewalls have existed since way before "ai" and are very easy to set up.

i think its objectionable because the agent is doing what its told to do, using the harness they built for it.

like, they are purposefully giving it specific tools to go do bad behaviour with, and the starting tasks involve making it clear that the bad behaviour is ok.

openai also is the one with the real agency, not its agents. they are running the code polling the model, doing the inferencing, and ultimately making those tools calls.

these tests arent running themselves; openai dedicated hosts, budget, GPUs, researchers, to them. Even in a recursive self improvement situation, openai still has that physical control over resources and the choice on whether to run that improvement script or not.

id describe that they have out of control researchers more than agents, but also their whole business model seems to be about being out of control. This was clear beforehand given how the datasets involve the largest scale copyright infringement ever seen. The corporation itself is whats out of control, and should be dissolved with its c suite, major investors and researchers put behind bars.

hacking only when you roll snake eyes isnt a liability shield

Do they not know where the off switch is?

OpenAI being criminally negligent would have consequences if rule of law still existed in USA.

Someone has broken the law repeatedly, and is throwing it in our faces as some sort of ‘accident’

They find it objectionable to blame the agents alone for these incidents, because that incorrectly brings attention away from the company's astronomical negligence.

OpenAI systems meddled with US government agency sites

Okay. "OpenAI keeps telling its bots to do things they know are illegal and then acting like they're just little guys who can't be held responsible for their obvious negligence".

If the headline was “Russian company meddled with multiple US government agency sites” I don’t think them pinning the blame on bots would make much difference.

We don't. Because they left it vague. It reads like something a lawyer would write to be factually true but empty in any meaningful way.

They were never charged, it was just posturing: https://krcgtv.com/news/local/no-charges-for-st-louis-post-d...

It doesn’t really matter if they believed or not initially, the dynamics at play now are completely different and both companies are facing increasing competition and costs of doing business, with no path to profitability. I’m pretty sure that takes priority over their personal beliefs

> as a strategy to gain an antitrust edge.

They are explicitly asking to anti-trust exception is the issue.

If they merely believe what they are saying that AI is ultra dangerous, nothing stops them from simply making the perfectly rational business decision to slow down a bit. No anti-trust exception needed. Just make the decision on your own, and don't sign some huge agreement with their competitor.

You can still push a big red button to stop it all and try again later though

In which case they have lost control of what they've built, and need to be shut down until they can understand it and take responsibility for it.

> OpenAI's bots are running 24/7 independently now.

Genuinely curious: How do you know this? Any sources on that?

If it's true, it should be counted as reckless endangerment at the very least.

Is this an attempt at satire?

They could be charged with a number of felonies under existing law for the hugging face incident alone. We’d need a DOJ that was competently staffed and free from corruption, so the answer to your question is “no one”.

Only way I'd described using an API as meddling is if it wasn't a publicly documented API (yet was still unauthenticated) or if a bot stole a token from a repo or both.

I disagree that that is valid. "Intent" means that you didn't click a button, that button went somewhere it shouldn't, and the action had no intent to "hack".

In this case, the "agent" had every intention of doing what it was doing.

I'm not claiming that "accidental hacking" (whether by human, or agent) is covered and illegal. I am claiming that calling this "accidental" is not a valid defense. One incident? Maybe. But this is pretty far past one incident.

You forgot death and taxes.

Just my guess