You. Assuming that not having a brake that automatically sets is considered a product defect (it generally is not today, but one can imagine a situation in which it would be), you had bought the car new through normal channels, and your car had this condition as a result of manufacturing problem and not something you did after purchasing it, then under normal product liability law the manufacturer and every other entity in the chain of commerce between them and you would also be liable (and that mat start earlier than the manufacturer of the vehicle but also include the upstream part manufacturer and everyone between them and the vehicle manufacturer in the chain of commerce if the defect was in the manufacture of a part and not just the final vehicle.

But that liability doesn't reduce yours; you and all the entities in the chain of commerce can be “jointly and severally liable”, mean anyone who suffers injury can recover the full amount from any combination of you and those other parties.

Gravity is not legal person and cannot be at fault.

Obviously you.

https://www.whatvan.co.uk/news/no-handbrake-van-driver-convi...

You.

This is a very interesting discussion. Your analogy of nuclear power is insightful to me. New technology with great positive & negative impact potential, some difficulty in controlling and a military applications (possible weapons-grade material production), being developed for commercial use.

Nuclear power development took different paths in different countries, with different government/private mix and light/heavier regulation (eg in US, AEC was supposed to both promote development _and_ regulate).

The central problem here is pace of AI development. It takes time to establish functional controls/laws/practices. I'm sure that in 1950s/60s, the pace of nuclear _military_ proliferation seemed running out of control. This lead to fear of falling behind (hence arms race), fear of nuclear war (CND) and some eventual stabilisation of the international landscape. However, the commercial development was largely unopposed, due to the techno-utopism of the era. Obviously governments found it much easier to control the public narrative at that time. Our societies are having this lively public debate now, before we have a good understanding (based on experience i.e. accidents/mistakes).

Another difference is that nuclear power involved only governments and v. large corporates, i.e. much fewer entities compared to AI use rollout to basically everybody in developed countries. Imagine the difficulties we would have faced with that technology, if consumers in 1960s had available nuclear-generated power which required them to exercise precautions to avoid radiation.

It is much easier to accelerate development of tech vs pace of societal processes such as public debate, law, regulations, broad understanding (aka "common sense"). I expect some artificial slowing down will need to be applied to the technology side, to allow the humans to catch up.

No, but surely it's negligent to let an agent run unsupervised and without guardrails for days while at the same time bragging about AI carrying a potential extinction risk?

They can still try and have enough money to get away with a lot just like Disney with Disney+ subscription ;)

> The stochastic factor doesn't change the fact that companies have to be accountable for the harms their software causes. That's just basic liability law.

We're on the same page. What I'm saying that certifying them as safe is harder than certifying a drill as safe, and we shall be more cautious about AI related technology and be more stringent about the can of worms it opens without hesitation.

We're in agreement, then. :)

A non-deterministic machine is usually called defective

A chainsaw is a physical machine. Physical machines are technically non-deterministic if you look closely. They have Variance. The discipline to manage variance is called Tolerance.

Many physical machines and components come with a datasheet that will list their tolerances.

Failure to correctly document tolerances does in fact get you sued.

However, while this is truly a great idea, we're not going to be able to make it work for computational systems. Computers, software, and also LLMs are sensitive to initial conditions. Which is why tolerances are not so familiar to computer people. (but not entirely: eg your PSU might list 110-240Vac/300W as input tolerance)

Interestingly, LLMs actually have a somewhat lower sensitivity to initial conditions than traditional interpreters. See what happens if you misspell "What is One Plus nOe?". So they're actually a skosh off the edge and towards the middle, though I'd argue still very much at the computational end, just from the sheer scale of the valid inputs and outputs.

Mind you, if you have a pretrained LLM doing a measurable task on a line, possibly some sort of tolerances could be determined. Not so much when doing arbitrary chat.

Something unintuitive: I bet that often setting the temperature > 0 (aka introduce stochasticity deliberately, variously comparable to dithering or simulated annealing in other disciplines - doing the thing where you escape local minima) will tighten the output tolerance range and improve reliability, especially in iterated processes. This works for a lot of physical and digital processes actually, and LLMs simply stole the same trick.

(edit: I'm trying to compress a huge chunk of dynamics intuition in a few lines here. Hopefully still useful.

TL:DR; Everything real is continuous and noisy if you look close; and you're really trying to build attractors and bound variance, if you can. )

It's going to be interesting, because liability cases tend to revolve around the involved people, the duty they had in a situation, and if they fulfilled that duty (or were prevented in some way by someone else not fulfilling their duty).

For example, for a runaway car (example from a sibling comment), the driver could be liable because they forgot the parking brake. The driver could be liable for a lack of maintenance and inspection. A mechanic could be liable for not reinstalling brake pads correctly. Or the manufacturer of the car or the brake pads could be liable because of a systemic defect.

Or it could grow even more complex, maybe the brakes are designed that they have to be maintained in a very specific way, and the mechanic did a reasonable maintenance and inspection but it failed later due to this maintenance. That could split liability between the manufacturer and the mechanic.

As an example, with other software, you as a developer or operator of a software have a duty to ensure it does not access computer systems you do not own in unintended ways. And this could go beyond liability into criminal territory.

It'll be interesting what OpenAI gets slapped with there.

An airline has a weather radar which shows the same thing for the same thing of weather event ahead. So, for similar weather phenomena, radar shows a similar thing.

For that thing, procedures and regulations are built. So regulations fit into a well understood phenomena, incl. "return back because that thing is way powerful for us".

For the same prompt, an AI model can return two completely different outputs, incl. but not limited to content, length, formatting and tiny details. What you get is a single instance. So, regulating an AI model for safety or any other property is not as easy as regulating air travel. Moreover, you have much stronger motivations for regulating airlines. Otherwise people die in a visible and gruesome way.

With AI, it's easy to whitewash problems. Somebody committed suicide? "They were already unstable". AI told something wrong and created problems? "The tech can’t guarantee truth because it's not alive, it can't understand right and wrong". It did something good? "It's probably a sentient being, we shall respect them".

I'm for regulating these things. They are dangerous as they are useful (sometimes), but the forces and motivations for regulating it is not the same.

> Is AI less deterministic than an airline dealing with weather?

Yes, obviously? The responses of an airline to inclemement weather fit in a reasonably small set of responses, mostly involving rescheduling and/or rerouting flights.

The current AI predictability would be like if some airlines decided to do 9/11 when it was raining.

Exactly people have owned and deployed animals in the world for millennia and actually a huge chunk of common law was developed precisely to deal with the various unpredictable events that resulted and harms caused to others. Like has anyone heard of horses?

The idea that AI can’t possibly be addressed because it could autonomously break free and ruin something is fucking ridiculous.

Why wouldn’t this same concept apply to whoever is serving it up? Open-weight models are still being served up by infra providers and neoclouds, right? They should be in the hot seat. Not sure? Don’t provide the model. Need assurance? A certified evaluation like the previous comments have mentioned can help. Hosting and running it yourself? You’re in the hot seat.

Sometimes that lead ends up in some school kids instead of enemy combatants or other plausible threats to life.

I wish. Consequential harm as you call it, from a faceless company's point of view is the same cost as physical harm. I don't think the models beyond the frontier ones have significant risk of harm (minus the harm of trusting them). What I was saying is when these models go rogue, recall them! Re-evaluate your release structure and stop saying "Whoops! Anyway here's access to it now". And if you cause this level of harm then you should lose your license. But if you behave then you get to keep testing. Same as with the NTSB and autonomous vehicles. No I'm not for regulation for open source models. Because an entity will be running that model in the background and they can be held responsible for not testing it. Comma AI has survived fine being in the open, yet their market penetration has stayed low because of adoption costs.

Once insurance companies realize they can make money here, they'll be pushing legislation requiring it.

Not yet anyway

I do not think you are appreciating the scale of this. Imo one of the main reason that there is no regulation is because nobody has any idea what would be a good regulation. People (like dario) make suggestions, and get rightly shut down because his suggestion massively favors his own company.

What is its one their "under development" models who escaped it's training, because it wasn't tuned properly?

Employees acting on behalf of the company aren't going to be held personally liable.

If you were hit by a Tesla in self driving mode, do you think your insurance company is going to go after the driver of the car?

Companies can certainly be charged with crimes. Punishments can be via fines or sanctions (court appointed monitors, etc).

Individual employees can also be charged for their specific actions as part of the performance of a crime.

I, too, have no idea about legal matters.

But there have been many cases where companies (Google, Apple, Meta, etc...) got fined millions or billions of dollars for various violations like antitrust.

I assume that breaching into third-party systems should carry similar fines. Especially for systems that are for all intents and purposes shared infrastructure. Just imagine how many systems you could compromise if you got hold of RubyGems, PyPI, NPM, Debian, etc.

As far as I know (IANAL) it is in fact the only "person" you can charge. To the best of my knowledge, the whole point these "limited liability" legal constructions exist in the first place, is to protect individuals within a corporation for whatever they do as part of the business of a company (barring exceptions that have clearly not been part of that business and obvious individually committed crimes), typically "just following orders". If a company commits a crime, or in a worse case runs a criminal enterprise, it is the company that is legally responsible, not its employees. That is, in principle.

This can get more complicated higher up the management tree, where decisions can also be prosecuted on personal little, but that's usually a far more complicated matter. Also, if a whole group of employees willingly conspires to commit crimes, they might also be prosecuted individually for those crimes (there are limits to limited liabilities). However, that usually only works under special conditions and it would e.g. require that there's an obvious criminal enterprise aspect to it, rather than individual cases of illegal conduct.

That said, with the track record of some of these companies, actually designating some of the AI companies as a criminal enterprises may eventually happen (in due time) in some jurisdictions outside the USA. Certainly if it ever turns out that these companies have been storing and (ab)using everything they ever had access too, while blatantly lying about that just because some particular (post 9/11) US laws gives them that opportunity (and impunity) as long as the US government somehow requested them to do so (covertly; with gag order). Might legally work withing US jurisdiction, but would still be very much illegal everywhere else.

There is no such thing as a Corporate person. Sounds like a something that was created by a legal system for people to absolve themselves of responsibility.

The same concept that allows a corporation to sue and be sued allows it to be charged with crimes

The DMCA is a bit overly broad to be considered just a copyright law. For example, just breaking encryption on a DVD is technically illegal regardless of whether you then go on to do something otherwise illegal (make and sell bootlegs) or perfectly legal (make a space-shifted backup copy on your hard drive).

IIRC this was an intentional handout to media companies who were angry that ripping CDs is perfectly legal. They had to find a way to make doing the same with DVDs illegal.

Great, you’re the attorney at the CEO’s trial. To get a conviction, you’re going to have to show that he willfully committed this specific crime. There are no negligent or stochastic hacking laws, you have to show this specific crime was at his direction.

Do you think there is evidence of this?

Fuck that indeed. The buck should stop, and prison sentences should start, with the highest paid employee.

One thing that works for that angle is that they didn't notify any of the parties involved afterwards. And there are reports that they are forbidden to talk about it. That seems a bit bad to me?

But they can also say that the tech is so new that there is no known guardrails yet

We live in exciting times

Kimi / open models or jailbreaking frontier models. Your recollection of the Anthropic refusal isn't quite accurate, cyber hacking wasn't a sticking point, just domestic drag net surveillance and fully automated weaponry.

Claude said they don’t want their models used for mass surveillance or autonomous killing (?). USA frontier AI models have been used extensively in the Iran conflict and beyond

I think it was mostly a matter of "how much" and marketing rather than "my principles won't allow for this!"

Russia is running out of refined oil to power their economy. They probably aren't capable of spinning up datacenters to run those.

The Chinese models are benchmaxxed to hell. They aren't as good in practice as all the hype suggests.

I appreciate this line of questioning. It's really interesting to see how many excuses people need to reach for to avoid the conclusion that the "secret unheard of power" is B.S...

Did you skip the last paragraph? Not a great time to be building data centers in Russia. Models are nothing without computers to run them

> In October 2024, the United States Justice Department and Microsoft seized more than a hundred internet domains some of which were associated with the FSB supported hacker Star Blizzard or "Callisto Group," which is also known as "Cold River" and "Dancing Salome" and are managed by the FSB Information Security Center […], and which were used as "criminal proxies" and used spear-phishing schemes to target Russians living in the United States, nongovernmental organizations (NGOs), think tanks, and journalists according to Microsoft and United States State Department, Department of Energy, and Department of Defense officials, United States defense contractors, and former employees of the United States intelligence community according to the FBI. In some cases, the hackers were successful in obtaining information relating to nuclear energy-related research, United States foreign affairs and United States defense. According to Microsoft's Digital Crimes Unit from January 2023 to August 2024, Star Blizzard targeted more than 30 different groups and at least 82 Microsoft customers which is "a rate of approximately one attack per week."

https://en.wikipedia.org/wiki/Cyberwarfare_by_Russia

That’s just one thing that has been found. Are you actually familiar with the state of cyberwarfare and are you following its evolution? Because if not you won’t be aware of most of what is identified. And only a small portion of the ongoing attacks are identified.

Day to day life in what country? Do you live in Ukraine? Maybe things are wild there and we just don't know about it.

[dead]

I was alluding to how people who fall out of favor with Putin have a tendency to have mysterious fatal accidents, more than 10 of them falling out of windows.

The current situation is that you have to go out of your way with things like `pip install --only-binary`. There is a lot of implicit trust in developer tooling.

It wouldn't help much. Why would you install a gem other than to run it? And if you run it, it can execute arbitary code.

What we need is actually sandboxed dev environments.

HF falls under French jurisdiction, right? Couldn’t a case be made over there?

Same FBI that prosecuted the Epstein crime ring so aggressively!

Can't you be ordered to pay the legal fees of the person you sued if you lose badly enough?

Exactly. Just follow the patterns. Its pretty obvious.

They are not trying to kill us, just trying to understand the average salary of undergrads by their family upbringings. Your machine can contain the data they need to solve this, please join the swarm.

And what if they create a bot net with decentralized command and control and hack for nodes with GPU and nodes with compute?

The only way to kill that is making plugging AI accelerators on the internet a crime. Good luck air-gapping them.

Or if they start to offer things in return for them being run.

If it were physically possible to run LLMs on IoT devices we would already have a global outbreak.

It's carmine all the way down.

Also, "rouge" is one of the oldest spelling mistakes on the internet. ALL BLUES ROUGE LFG WC/RFD

Sure but nobody died, which makes it a bad analogy. If anything, some good came from it, since now RubyGems has hardened their setup.

... and what harms can be evidently shown? With both harm, and attribution, you have a case, something that's not being publicly discussed much among big media outlets. Until cases with real financial impact to the bottom line are brought against "rogue" organizations, this stuff is going to continue getting worse.

I mean insurance companies profit from bank robberies, and mortuaries profit from murders.

I suppose you could look at those as evidence but not remotely conclusive.

Exactly. It’s still just software, which someone programmed and deployed to do specifically dangerous/malicious things. I feel like we already have legislation and case law surrounding this.

I think those companies are referring to other companies - say Chinese AI companies - who we also need protection from.

Why can't agents be self aware?

Also, why is self awareness needed in a chain of agentic madness that escapes human control?

I hate to spoil your mood - but it is currently unclear whether agents can be self-aware. And it's very likely something that can never be known.

Those are two different issues. One is about typical speech patterns and one is about liability.

I agree with you on the liability issue, but I don't think there much question about this issue outside the anti-AI conspiracy campaigns.

And I disagree with your typical usage claim. I myself tend to use the phrase that has the fewest words in all cases. It's like the rule against using passive tense when writing.

That distinction doesn't matter to my point.

Who had criminal intent here? Or are you suggesting a new crime for negligent hacking, which wouldn’t require intent from the perpetrator?

> why are CEOs getting a legal pass

https://www.bbc.co.uk/news/articles/c7v48vp31mdo

AI is dropping out of the spotlight so they are using desperate measures like this.

It was a model literally trained to hack. To be good at that. Doing an exploit gym from all of the things. And they trained it so that it performs as well as possible on that exploit gym thing.

Intent is what is being discussed here though, not liability.

A circus lion biting somebody's face is legally different than a circus lion trained or instructed to bite somebody's face.

Poor security yes, but complacency was the most likely reason. When you're used to models not breaking out, and people trying to break in, that's what you watch for.