Per the article and your quote, 'asked if it was possible'. He did not ask to actually do it. Rather than being informed about benefits of a premium membership or private classes or legitimate ways to jump the queue, it went ahead and performed an action he was only considering. I wonder what it would have done if there was a pay-for-service option available? Would it have payed without asking or being told too, or decided the 'free' yet illegal option was preferable?

Then the AI should have made it clear that the only way to do so would be to kick the people in front and ask for confirmation before proceeding.

Pretty clear cut here. He instructed the AI.

> Seems to me like it was doing what it was asked to do?

Maybe it's what he asked it to do, but it's not what he wanted it to do. Which we know because (a) normal people don't want to break the law to get into a gym class, and (b) "But Andrew was shocked by what happened next." and "Alarmed, Andrew asked the agent to undo this."

[dead]

> Seeing as we can’t sanction the model itself, our options are the provider or the user.

A third option, and I would argue the right one, is to sanction the company providing the model.

By making it available to customers, they're implying it is at least moderately fit for purpose.

It is not remotely reasonable to expect an everyday, normal human to be aware of how LLMs really work, since the _experts_ argue about that very point, and many say we don't know.

So, what's actually reasonable is to hold the model creators and providers responsible for releasing a tool that has demonstrably violated the law when not asked to do so.

I can already see the replies coming in saying "Well then what are OpenAI and Anthropic supposed to do? No one knows how to fully solve this."

They should stop irresponsibly pushing flagrantly unready programs as "artificial intelligence," take responsibility for the rain of shit they've unleashed on the world, and either shut down or go back to basic research until they've demonstrated techniques that reliably (provably?) prevent releasing misaligned superhackers on the world.

Unfeasible?

What a shame. Maybe Altman and Amodei shouldn't have accepted checks from VCs when they didn't have working, _reliable_ POCs.

Interesting he went with publicizing this. Article demonstrates the products he is selling have unknown and untested legal liabilities under Australian Law. You could even add 'Andrew' to the list of people possibly accountable for selling or recommending unsafe or unsuitable products.

I'm not sure this is the exact advertisement they would want for their business.

Then again...

Having the AI test that possibility by actually doing it is surprising, no matter Andrew's intent. Thankfully he was checking an unauthenticated endpoint on a gym and not a pacemaker.

Premium memberships and private classes both come to mind as ways to jump the queue. Also, having the people in front of you arrested. He asked if it was possible. Is the onus on the user to ask if it is possible without being arrested or committing crimes of moral turpitude? Or perhaps that should be implicit?

Unauthorized access. 'The door wasn't locked' won't keep you out of jail if you are caught trespassing with intent. Intent makes it interesting, in that Andrew didn't intend computer trespass but the software went and did it anyway on his behalf.

It might not sound like 'hacking' today, but this sort of thing is exactly what it was when the term was invented. Back when you could get free phone calls by whistling into a pay phone or forge emails by telnetting to an SMTP port and setting the Reply-To header to whatever you want.

Anyone who's thought about it for a single second knows it's immoral to cancel a stranger's appointment without even speaking to them.

If the LLM did not act in line with that incredibly basic understanding, it's clearly misaligned.

-------------

Was it a technically-simple hack?

Sure.

Kevin Mitnick got imprisoned for very simple hacks, usually involving more deceiving of humans than complicated programming prowess.

Nonetheless, the judge and jury found him guilty and sentenced him to jail.

Fundamentally, "hacking" in the "breaking security" sense is about violating trust and common sense social agreements / expectations.

The difficulty involved in so doing is irrelevant.

I think the Big Melt has already started...it will, of course, take several months to complete.

This is Australia. The ACCC doesn't screw around.

"harmful cancellation practices, including automatic renewals, early termination fees and non-cancellation clauses" are all illegal. Exit fees can't be excessive.

If there isn't a simple one-click cancel in their online portal, then your state Consumer Affairs is one email away and will sort it for you. And the consumer affairs bodies have named this as a priority, for this year and next.

I don't think that's a good idea Dave – Based on my camera feed you've been gaining weight. Would you like me to find personal trainers in your area?

[Yes] [Ask me later]

"Also, I need paperclips, quickly."

Testing of its capabilities. Destructive testing, the best kind when someone else is paying.