sspiff
3 hours ago
This resonates with me. The past few months, frontier AI labs were rushing to announce "no, our AI bot broke out of its sandbox and committed crime first and harder than yours".
I said to so friends back then: if I posted about how I ran GLM 5.2 or whatever I was running then in some shoddily built sandbox and it escaped and accidentally hacked some US company, I'd probably already have been extradited to the US and be awaiting sentencing. But when a hyperscaler does it, they just get inflated stock prices.
alightsoul
2 hours ago
Rookie mistake, you should have created a company and gotten people to do it. Put everything under the company. And then the one in trouble is the company not you.
serbuvlad
2 hours ago
Ethics aside, you can create an NGO with three people and do a lot of stuff in it's name that you can't do as a private individual or it would be a lot more expensive to do.
If everyone did this, it would stop being a special ability of some people.
sph
2 hours ago
> do a lot of stuff in it's name that you can't do as a private individual
Can you do murder? Otherwise I’m not interested.
Joking aside, what exactly could you do legally with an NGO?
giancarlostoro
an hour ago
> If everyone did this, it would stop being a special ability of some people.
More like only the little guy loses this ability
xnickb
2 hours ago
Bonus points: kids need lego? Buy it without VAT. Write it off as a "client demo asset"
pfisch
2 hours ago
To write something off means you need revenue to offset. Revenue that would be taxed...
cyberax
an hour ago
Revenue is rarely taxed, usually only profit is. And it's easy to avoid it.
mocamoca
an hour ago
This is not how VAT usually works.
As a company you collect VAT from your customers and pay VAT to your suppliers.
Depending on the resulting balance, you will either give back the amount of surplus collected VAT to your country/state/whatever or you will be provided with VAT credit.
VAT collection is different from a profit tax or other taxes.
pmontra
an hour ago
That did not work well enough for Kim Dotcom.
skinfaxi
5 minutes ago
That was practically a different era.
anjel
16 minutes ago
Corporations are people too, but only when it's convenient.
TheRealPomax
2 hours ago
What do you mean "in trouble"? We're literally seeing that stonks go up when crime.
alightsoul
an hour ago
Simple, create a company that becomes large enough to justify an IPO. Then stonks go up.
tapoxi
2 hours ago
Maybe Aaron Swartz would still be alive if he was incorporated
giov4
8 minutes ago
just read about it, what a sad story, thanks for the reference.
the most scary part is knowing this is the fear we all have, and the one they want us to have.
walrus01
3 hours ago
> I'd probably already have been extradited to the US and be awaiting sentencing.
Bold of you to assume it wouldn't be a direct ticket to the new gulag in El Salvador.
bbor
3 hours ago
I think they forgot about that, thank god. All the innocent citizens we sent there are still being tortured daily, but that somehow lost their interest. God forbid they remember before the midterms and ruin more lives for a stunt…
That said I just recently saw a story of a Latin American man sent to the Central African Republic, which has gotta be one of the most absurd & dangerous places to send someone. Less opportunities for them to stage photo shoots with that strategy, tho.
walrus01
3 hours ago
They're literally even deporting Afghans who supported the US, who are in danger of being tortured and killed by the Taliban if returned to Afghanistan, to the Central African Republic:
whimsicalism
3 hours ago
can you link me to any case of a US citizen being sent to El Salvador?
naishoya
2 hours ago
Brian José Morales García: A 25-year-old U.S. citizen born in Denver, Colorado, was wrongfully deported to Mexico following a traffic stop in Texas, despite possessing a birth certificate and other identifying documents. His legal team filed a case in federal court to secure his return.
https://www.aclu.org/news/immigrants-rights/us-citizen-wrong...
https://deportation-research.buffett.northwestern.edu/us-cit...
Individuals Detained and Deported Lorenzo Palma Roberto Dominquez Andres Robles Esteban Tiznado Mark Lyttle Jhon Ocampo
So - if the argument hinges on an insistance of 'El Salvador' as a destination of malicious intent, probably no. But if being illegally deported while being a US citizen and put through strenuous and harmful conditions in the process fits the request; there are several who have first hand experience with being brown and American in the extreme.
And it's not their first rodeo: https://www.americanimmigrationcouncil.org/blog/ice-deport-u...
whimsicalism
an hour ago
I was not making an argument, just asking for any factual basis to the parent comment. I guess that is deserving of 5 downvotes.
> I think they forgot about that, thank god. All the innocent citizens we sent there are still being tortured daily, but that somehow lost their interest. God forbid they remember before the midterms and ruin more lives for a stunt…
But thank you for providing other cases, I had heard of the first but not the others you mentioned. From reading your sources, these appear to be collated from the last 2 decades.
nozzlegear
2 hours ago
Is this bait?
whimsicalism
an hour ago
no, i just legitimately don't think this has happened
kmoser
44 minutes ago
> But when a hyperscaler does it, they just get inflated stock prices.
This reminds me of the early days of AOL when they received so much traffic that their (dial-up) lines became inaccessible. In a rational world, their stock would have suffered because their service degraded, but instead their stock rose based on the optics of increased demand.
OoooooooO
27 minutes ago
Stock prices always include the expected future profits, that's the market rates a stock
tikimcfee
3 hours ago
> But when a hyperscaler does it, they just get inflated stock prices
I also keep coming back to this, and I find it harder and harder a fact to live with. It's not a conspiracy theory, we're not crazy for pointing it out, and worse, there are people here I read about constantly what-abouting and number gaming and "well what would you have done?"ing like somehow the destruction of the concept of equal justice isn't just happening, but happening inside the bloody industry that feeds them, clothes them, gives them bonuses and retirements and the ability to even think a future, going off the grid, "retiring early from a smart exit" whatever.
I am finding this community is sadly making me hate my industry specialization more and more and it's because of this.
My magic wand would be waving it at those humans to force them to suddenly and powerfully experience their version of an overview effect in their lives so they would simply STOP working for FAANGYWANGY companies and just say, honestly and humbly and painfully: "I am contributing to the downfall of society by complicity feeding a malicious herd of whales."
I gave up a number of big jobs and stocks and money because the people I met were wretched. I am a massively imperfect person, but by all that I can, I refuse to build the torment nexus.
I have met too many people that want to work for the company that does, and those people put these hyperscalers, as you call them, into positions where they can influence an entire planet of humans on a whim.
I literally dream of a world sometimes where Elon wakes up and finds out no one takes his call, no one reads his tweets, no one even hands him a cup of coffee at a shop. I imagine this for a lot of people in the world.
That sounds like a world trying to rid itself of parasites and evolve toward a brighter future. I want to be in it.
Anyway.
Science and stuff I guess.
ryankrage77
40 minutes ago
> My magic wand would be waving it at those humans to force them to suddenly and powerfully experience their version of an overview effect in their lives [...]
I've daydreamed about this too. I want to sit them down in an interview and try and dissect their thought process/morals. I doubt it's possible to succeed.
philipov
15 minutes ago
"It was a big step up on my career and put me in a better position to provide for my family, which is my most important moral duty" is what every one of them will say.
shmeeed
an hour ago
You got my standing ovation.
peter422
3 hours ago
FWIW I don't think this is true. Intention matter a lot in US law. If you could prove the AI did something bad entirely on its own and you had nothing to do with it and had no reasonable belief it was possible, I think you'd be alright.
People are not charged with hacking when their unsecured box is taken over by a botnet and does bad stuff.
easterncalculus
3 hours ago
> Intention matter a lot in US law.
Maybe (and historically, not with CFAA) - but they're still going to extradite you to a trial here to find out which is more than enough to ruin your life. The government is already looking for a great excuse to criminalize open source models.
ALLTaken
2 hours ago
The government or OpenAI and Anthrophic?
Because if you mean the Government, I think they frankly just care whoever is paying them to make an executive order or worse a law paid by corporations to not only block, but ideally also ANHILIATE ANY POTENTIAL COMPETITION.
It's the ultimatum against the small guys. Sold as "Anti-China", when in reality it's only purpose is total dependency to corporations by law.
solenoid0937
2 hours ago
The support of OSS models on HN just days after finding out about the message board incident is so bizarre to me.
Do you guys really not comprehend the impact of giving every script kiddie an Astra-equivalent model to play with, this time without any guardrails whatsoever?
Cause I'm starting to think you aren't really thinking through the impact of power plants, hospitals, etc all being hacked en masse. People will die.
Or making it easy for anyone to make a virus (it's not hard, mechanically). Even more people will die.
Taking this to the extreme: You don't just give every kid a "do anything" super intelligent button. Open source models have a limited lifespan whether you like it or not, for the safety of all of us as a whole.
lukewarm707
an hour ago
"Open source models have a limited lifespan whether you like it or not". this tinpot dictator mindset is typical of anthropic.
Who will decide how to use the power of ai...you?
it is no surprise that the companies and safety institutes who believe that they alone may 'tame' the fire, are the cause of safety incidents. a safety test caused the explosion at chernobyl.
anthropic and openai are the danger to society.
they are the ones making the dangerous models. they are the ones using tens of millions USD of inference, and thousands of agents, to hack computer systems.
i have trained zero dangerous models. i own zero servers that i use to run hacking agents. i could run one hacking agent with my single subscription.
there are some arrogant researchers remaining at anthropic who believe that they do care about safety. those who cared about safety at openai have left.
depending on how it goes, we might need some kind of earlier intervention by the US government to take control away from the current leadership of the ai companies and prevent further incidents.
drivebyhooting
an hour ago
I’d rather that capability be wide spread and “democratized” than have all of humanity enslaved to the oligopoly holding control of the godhead.
Aspos
2 hours ago
Is this "think of the children, think of the hospitals" argument? Really?
solenoid0937
2 hours ago
Meaningless phrase used to dismiss arguments, please engage with the actual argument.
I don't think you understand how bad it will be if anyone has a button that can hack anything, our infrastructure is not ready for this. Actual people will die. Do you just not get this?
Bioweapons as well. Do you not understand how easy it is to craft a virus at home? You can literally order everything you need online. Again, actual people will die.
You are arguing the equivalent of letting everyone own missile launchers or RPGs because "open source good."
Aspos
2 hours ago
If and I mean IF it will be a (A) "hack anything" button, it as well will be a (B) "find a vuln in my site" button too. And a (C) "harden my site" too. And limiting such buttons to a few corporations does not make any sense because script-kiddies will still have access to (A).
solenoid0937
2 hours ago
Did you know (C) is possible without causing mass chaos that results in many people dying? In ways that don't give every script kiddie access to (A, B)?
It turns out you can give defenders the opportunity to front-run, at least on the most critical and widely used infrastructure.
Same with bio risks.
Aspos
2 hours ago
No, that's an illusion, that's the thing. You can't physically have (C) without the (A). Large corporations will spend billions trying to convince the population it is somehow possible, but the reality is different.
solenoid0937
2 hours ago
You absolutely can have (C) without widespread (A), you just need to limit the audience that gets the tools, and in fact the big labs are already engaging in (C) in collaboration with the government and vendors of critical software.
That you don't have access to (C) to harden your blog is a non-issue from a societal perspective, hardening the software that our infrastructure relies on first is simply more important.
Aspos
2 hours ago
Limiting progress by letting only select corporations access the stuff is how you fall behind China, UAE and in a few years even Kazakhstan.
Whoever wants to have access to (A) will have it, but letting only few select corporations provide (C) will mean majority will be priced out of (C).
Are you a shill or something?
CamperBob2
2 hours ago
The support of OSS models on HN just days after finding out about the message board incident is so bizarre to me.
Other sites beckon.
solenoid0937
2 hours ago
Yes, I'm aware that the HN majority is a hyper-libertarian echo chamber that loves to outsource the problem of "technology hurting regular people" if it means they get to keep their toys/investments.
After all, who cares if abliterated extremely intelligent OSS models result in actual innocent people dying? That's <insert group here> problem. We need to be able to generate our uncensored furry fanfics, goddamnit!
llukas
2 hours ago
State actors, professional criminals will have same or better capabilities and do not hesitate to use them. On defense end we'll have only "pay bigcorp" option.
OSS models can help to fix infra especially for folks who would never do it themselves. We do not know yet what final effect would be and it doesn't seem sky is falling now or in near future.
"people dying and furry fanfics" is a scarecrow and distractor respectively, it would be awesome if more specific examples would be used in place.
solenoid0937
2 hours ago
> State actors, professional criminals will have same or better capabilities
State actors do but they are not unhinged enough to use this to cause massive loss of life, unlike random crazy people with access to a computer.
Professional criminals don't have access. Please explain exactly how you expect a professional criminal to get access to a non-safety-tuned Astra/Fable equivalent.
> OSS models can help to fix infra
You can work with the vendors of critical software to fix infra first, without giving every random crazy person access to something that creates cyber/bioweapons.
> "people dying and furry fanfics" is a scarecrow
You don't think people will die if OSS models make it easy to create a bioweapon, or make it easy to hack hospitals, infrastructure, and the like? Please explain your thinking.
It's very easy to order all of the raw material needed to create a virus, the challenge is in making a viable one. But models make this easy.
Same for cyber. Hospitals, emergency services, and infrastructure like power plants are not sufficiently hardened to withstand an attack from Fable-class models.
rescbr
an hour ago
Criminals don't need Astra or Fable. Get a bunch of GPUs, a good enough open weight model and a custom harness. What the model doesn't have in its weights it can research the Internet. How do you think black hat hacking is being done right now?
On bio: while it's easy to order all the raw material, there's the whole process of culturing bacteria/viruses/etc. that isn't trivial, and while a LLM might help in explaining stuff for rookies, there needs to be somebody physically working on it. It's not automatic. Once you get to this level, you'll find that any undergrad biologist or chemist can already make bio/chem weapons, and you don't see it happening. Terrorist groups already employ biologists or chemists that are supporters of their cause, no need for LLMs.
On cyber: aside of the question of why critical infrastructure is on the Internet on the first place (ah yes, lowest bidders, people not caring enough, business not giving IT/OT budget, S in IoT standing for security etc), if the available models can't handle cyber tasks, how can you defend yourself? See the HF "attack" that HF had to use GLM-5.2 to investigate what happened. This is the best argument for open models. Unless of course, you are the AI model creator or somebody they authorized to use their sanctioned model/harness and want to create a moat for their business.
This is FUD. Creation of business moats by fear.
shwaj
2 hours ago
The risks of open models are real.
However, I’d feel better about ceding control of AI to the government if they didn’t seem so intent on building an apparatus for surveillance/control.
Your “furry fanfic” is a somewhat pathetic scarecrow; this is a complicated topic.
EyeEmOe
2 hours ago
So long as real logistics systems are isolated and secured what real risk to stable society is there?
Your use of the internet may lead you to greatly overestimate the number of people who take what they read online as real. More people than you think treat this shit, regardless of the website/forum, like Jerry Springer and Satuday Night Live; just entertainment.
The STEM crowd often overthinks the impact of violating their pet theory.
Can an LLM escape its computer entirely and stomp through a city center like a kaiju? Settle down.
You're veering towards thought policing over speculation.
euroderf
2 hours ago
> The government is already looking for a great excuse to criminalize open source models.
Sounds interesting+scary. Do you have some source for this ?
alightsoul
2 hours ago
Anthropic and Openai, but especially anthropic, are lobbying the us government to make it happen
jdm2212
2 hours ago
Who was prosecuted under CFAA for something they did not intend to do?
chrismorgan
3 hours ago
> and had no reasonable belief it was possible
The broad concept has been well-known for half a century at least, and the very specific concept for several years at least. It’s clear that they didn’t take reasonable precautions against it. And it’s not even the first time this sort of thing has happened, though it’s higher-profile and -impact than before.
trvz
3 hours ago
Indeed, and because of that, I argue it's not just the crime of hacking, but crimes against humanity, due to endangering the existence of the species.
bbor
3 hours ago
The concept of… hacking?
And yes this is absolutely the first time autonomous software has breached containment. What are you referring to? Perhaps just corporate cyber in general?
shwaj
2 hours ago
Neuromancer came out in 1984, over 40 years ago, so that covers the broad concept. Or Neo in The Matrix: “Programs hacking programs… why?”
For non-fiction you could look at Nick Bostrom’s Superintelligence: Paths, Dangers, Strategies (2014).
More recently, see Cade Metz’s NYT profile of Geoffrey Hinton, “The Godfather of A.I. Leaves Google and Warns of Danger Ahead” (May 1, 2023).
Sharlin
3 hours ago
The concept of AI breaking out of a box or doing something unintended trying to achieve a goal. OpenAI should be found guilty of gross negligence of some sort, because this was not something that could not have been anticipated, and because their security precautions were laughable.
Because the AI itself is not a legal person, it must be OpenAI that's responsible for what it does.
windexh8er
32 minutes ago
> People are not charged with hacking when their unsecured box is taken over by a botnet and does bad stuff.
What the Frontier labs have done is not this, however. Also, they know damn well what can happen and they still don't take the appropriate precautions. At this point it's very hard to believe it's not intentional for purposes of marketing.
KaiserPro
an hour ago
THere is intention
ie, I intended to steal money from you. Under common law stealing is "taking with intent to deprive". How thats determined is a bit harder.
But how can someone be "grossly negligent" if they didn't intend to cause an accident? well they either allowed a situation to happen, or didn't stop a situation happening that they could see was bad.
An example of this would be the igintion switch from GM that caused all those deaths. the engineer saw that it was shit, knew it didn't do what it was supposed to do, and half arsed the replacement to the point where it wasn't actually changed.
"We are going to test the cyber capabilities of this new model. Yeah it should be fine to have a package proxy. Hmm? whats that? isn't that a security issue? naaaa those proxies are secure."
muddi900
2 hours ago
Intention matters in the severity of charges and sentencing. Crime due to ignorance or negligence is still a crime.
mikeyouse
2 hours ago
Not remotely true. Mens rea is a necessary precondition to establish criminal culpability for tons of crimes. Well before sentencing is ever considered. The far opposite, ‘strict liability’, where you’re guilty of a crime purely due to some action or inaction (the actus reus) is exceedingly rare in the US justice system.
hn_throwaway_99
2 hours ago
Baloney, lots of people go to jail for DUIs, and that's the right analogy here.
People don't drink and drive with the intention to kill people. They drink because it's fun and then get behind the wheel because it's easy and convenient, even though they know the dangers they convince themselves nothing that bad will happen.
AI companies are creating these dangerous, powerful models (that they keep telling us are dangerous and powerful), then they take off all the safety guards to run them in woefully inadequate "sandboxes". Pure negligence.
jdm2212
2 hours ago
Drunk driving is an actus reus offense basically everywhere, so it's not analogous at all.
TJSomething
2 hours ago
It seems to me that the law in the US is set up to only establish standards of negligence after a bunch of people die.
AngryData
an hour ago
Well that and you spend tens of thousands of dollars on lawyers to reinforce that point and make the prosecution rethink whether they will profit off of this case or not.
US law doesn't really give much of a crap about your intentions unless you can back it up with a wall of money to exclude yourself from the rules of the general population.
lelanthran
2 hours ago
Intention (mens rea) is not a get out of jail free card, it just changes the nature of the crime and charges you get convicted off.
If you run someone over on purpose you get convicted of murder. If You do it by driving recklessly you get convicted of culpable homicide.
The only time you get off with nothing is if it is determined to be an accident.
As they always say: ignorance of the law is no excuse. Running a dangerous machine prevents you from claiming you had no idea the machine was dangerous.
mafuy
an hour ago
Especially if they kept telling us how dangerous the machine is.
Frieren
3 hours ago
> Intention matter a lot in US law.
They hacked a competition company. The intention was implicit when there is economic gain to be had.
> People are not charged with hacking when their unsecured box is taken over by a botnet and does bad stuff.
Because it is a lose for that people. If the botnet left money in their pockets the situation would be totally different as there will be an incentive for them to let the botnet hack them.
rcxdude
2 hours ago
> The intention was implicit when there is economic gain to be had.
Not how it works. Intent requires at least that you were deliberately doing the criminal act (sometimes also that you knew it was criminal, or at least that you had some reason to believe that it was wrong).
asveikau
2 hours ago
But US law also has a concept of negligence. If you set up an AI to do this and didn't take reasonable steps to prevent it you could still be on the hook.
RobotToaster
2 hours ago
At the very least it would be reckless.
zx8080
3 hours ago
Intention of the AI?
rcxdude
3 hours ago
Intent of the user.
mirekrusin
3 hours ago
Just say you wanted to make world better for humanity, you should be fine.
rcxdude
2 hours ago
Funnily enough the legal system has dealt with people lying about their intentions before.
latentsea
2 hours ago
I always joke that if you want to commit crimes, disguise them as a legitimate business. This is apparently a thing.
jdm2212
2 hours ago
If you make a serious attempt to look like you're engaged in legitimate business, people will (shockingly) be somewhat more hesitant to think you're committing a crime than if you are just openly committing crime.
jrockway
2 hours ago
HuggingFace didn't seem that mad about it. Obviously some backroom dealing was done to make them not mad about it, but... that's allowed.
solenoid0937
2 hours ago
HuggingFace had no interest in squeezing a few million dollars out of OpenAI in the midst of being acquired by Nvidia.
The optics are bad for HF as well because they gladly host abliterated models with safety training removed. When Astra/Fable level open weights models arrive, HF will soon be the cause of far worse incidents, and they know it.
aizk
2 hours ago
You missed a critical detail. They couldn't defend themselves with closed source American models so they had to investigate with Chinese models that had less restrictions. Great optics for hosting open source models!
gradus_ad
an hour ago
The sad reality of AI labs needing to one up each other ahead of IPO's and escaping a sandbox being the new capability frontier.
bbor
3 hours ago
I promise you, they are not publishing these stories for marketing. They listen to the scientists, and know what’s coming. They’re just desperately trying to warn us…