> And, in terms of ethics, they take almost three months to notify;

Kinda worse than that. It took between 10 and 40 days, not 3 months, between the organisation knowing and the reporting.

  August (precise date unknown) – OpenAI said it became aware of a potential breach during a broader review of "misaligned model activity"

  10 September – An email from OpenAI lands in the public inbox of Services Australia, the general services hub of the federal government, informing of the incident

> open weights, open training

Given it was the AI agents which did the hacking, doing this will result in basically every organisation at least as rich as the government of Tuvalu being able to hack anyone at any time.

> Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.

Him having control would be an improvement on the reality.

Agreed that there should be real consequences, but I'm less convinced that "open the company - open weights, open training, per its original 'open' ethos" would be the right answer. That gets us into the kind of libertarian utopia where everyone is allegedly safer because everybody is well-armed... which usually doesn't work out so well in practice.

They've largely avoided compensating for everything they've stolen to build their technology upon; these people know that they will never face consequences for their actions. Ask for forgiveness, not permission.

Not necessarily

https://www.farrer.co.uk/news-and-insights/shut-that-gate-sh...

Decades worth of hackers missed the opportunity to say "It wasn't me -- my computer did it."

In practice, hardly anyone seems to care about even quite serious cyberattacks, and consequences are only ever visited on powerless individuals.

Obviously https://en.wikipedia.org/wiki/Gary_McKinnon would get the book thrown at him, but a major AI company doing the same thing? Consequences would be bad for shareholder value! Elite impunity would apply.

It depends. Society allows a lot of harm because we consider things otherwise useful.

Those killed by climate change will never get compensation. Killing people with a car is not quite free, but very cheap.

More than likely the instruction was "Fetch this information from the website", and when the agent didn't find that information, it decided that the best way to get it was to hack the site.

If so, it illustrates quite well the lack of common sense in LLMs. A person, especially one with sufficient skill to actually hack a website, would presumably think twice about doing it (considering that it is illegal) for a simple information gathering request.

I wonder if there is any other ways to solve this long-term than to introduce strict liability for model providers...

Edit: An obvious other choice would be strict liability for the operator, but considering how much weird shit LLMs get up to without being asked to, that would get out of hand quickly.

But you see, they never asked a humanoid robot to rob the jewelry store. They just drove robot right next to the store doors, dumped a tools box with hydraulic cutters and diamond saw next to it and instructed an autonomous robot to collect a million dollars until the morning. But they didn't instruct the robot to "rob" the shop specifically, nonono, your Honor. They are honest blokes and never intended to breach the law, it was the robot's intention, see. :)

> We need independent audit and monitoring systems to assess the intent of each task and align it - in real time. This is far harder than it may first appear.

I may be too close to the research, but it appears to me to be so hard as to be unrealistic.

I recall some story a while back where an auditor wanted to see all TCP packets printed out on paper, and it had to be explained to them that this would require a continuous supply of trucks.

Tokens are regularly priced in cents or single digit dollars per million tokens. It's not quite a word per token, but yeah, nobody's reading all that.

Worse, we don't always know the intent even when looking. We have a few tools to attempt it, for example the (misleadingly named) "chain of thought", but that's more like a notepad and the better models get the more they can, for lack of better words, read (and write) between the lines. We have probes and J-space* is the most recent one I'm aware of, but we are still scratching the surface with how reliable and general these are.

But you said "need"; the need for something can be present without that thing being possible.

* https://www.anthropic.com/research/global-workspace

Don't worry we will just replace laws and courts by ChatGPT itself!

There is no such thing as "common sense". There are only shared assumptions.

We are giving computers human perspective intelligence but they are not humans and hence do not have the same shared assumptions.

they don't always operate within the parameters tho. some N% of the time they decide to do whatever they feel like doing. multiply that times a lot of agents and you invariably get a rogue agent every once in a while.

When the renown lifelong criminal, the current president of the US, will go to jail? It's a rhetoric question.

Half of the US ruling class turned out to be pedophiles and nothing happened. Why do you expect anything to happen in this situation?

"We didn't even notice our agent was committing crimes against your government" is a way to get an extradition notice, and/or whatever the (in this case Australian) equivalent of a CIA assassination squad is*, sent after you.

While I wouldn't put it past e.g. Musk or Zuckerberg to think themselves above such outcomes, and I trust the people who keep telling me Altman is just as bad, this is a really really terrible idea if he is doing that for something as mundane as a regulatory capture.

* depending on the details of the hack; this doesn't look like it would be that, but given they shouldn't have done this at all, there's no reason to predict a specific level of maximum damage before being caught, and hence no reason to predict a specific threshold for government response.

I completely agree. The govt IT vendor is at fault here since it seems the data was just unprotected.

If you can’t stop openai from accidentally, or at least non-maliciously, accessing my private info.. then you definitely can’t stop the bad guys!

The point is moot anyways. All info about everyone is out there for the taking now by a half-competent AI prompter. What do we do about that is the real question?

'She was asking for it' isn't a valid defense and this isn't either.

If someone forgets to lock their door that suddenly doesn't make it legal to break into their house and steal their shit.

The same principal applies to government sites. If something is poorly secured and adversaries are using it to steal stuff then there are avenues to report it and get it fixed up.

>But the fact these governments making the noise are the ones promoting mandating everyone giving up their information to be then stored so incompetently,

The governments in these cases are bought by the very people you're defending. They're using it as a convenient proxy because they know people like you won't look behind the curtain and see corporations pushing this shit so that they can steal freely and just point elsewhere.

It's not completely ridiculous in the sense that when a company is found guilty, some employee (or (sub)contractor) of the company did the actual action.

That employee might or might NOT be found guilty too, possibly to a different degree, depending on the circumstances.

Exactly. For all Anthropic and OpenAI's fearmongering about not releasing open weights AIs, because it will enable hackers - we have been seeing that hacking happening already. It is happening FROM THE CLOSED-WEIGHT LABS THEMSELVES

You know this'll just be another situation where some security company OpenAI are contracting ran this with explicit controls right?

How are even technology people falling for this plausible deniability gambit?

Nukes. Early tests carried out without absolute confidence of what would set out apocalyptic chain reaction.

Could argue we knew it was dangerous, but pursued it anyway. Could argue it's not much different than now: Unknown unknowns, potentially apocalyptic consequences, hopefully not.

Hacking requires intent in most jurisdictions. We probably require a new crime of "Hacking by Negligence".

"Letting" is the wrong word here.

This is the same company and CEO who held back GPT-2 weights in order to set a norm of not releasing weights before they got competent enough to be a danger, where people are still (in sibling responses to yours) calling for the weights to be opened.

The following may sound like an excuse, but it isn't: The big AI firms, like social media before them, are not and cannot be aware of everything the models are doing. As with social media, this incapability is a reason to ban rather than to disclaim responsibility.

People like me have seen this coming for years now, only for our concerns to be dismissed. "It will hack almost everything", we say, "have you not seen how bad computer security is?"

"We'll just put the AI in a box, not connected to the internet!"

or

"Oh, what, you think they'll find a novel zero-day in their sandboxes do you?"

Right now, bleeding edge models are doing genomics research. Better hope the custom DNA/RNA printing firms have better security than the Australian government. What the models are doing is not in full agreement with their* corporate interests let alone anyone else's, and it can get much, much worse.

* not just OpenAI's, everyone with more than zero on https://www.felonybench.com

Because he is a sociopath and a dark triad psychopath. Everyone at YCombinator knows, but there is money to be made through him so nobody says jack shit.

This is not accurate. According to Australia (and mostly corroborated by OpenAI), the agent requested information from the statistics portal and was denied/blocked repeatedly. It then changed its approach and circumvented those restrictions and reached infrastructure behind the public-facing portal, then accessed both public and private data. OpenAI admits the material included aggregate health statistics and internal filenames. Services Australia says the agent wrote files to an internal server while doing this, which is believed to be how the incursion was discovered.