hackernews client

SwellJoe

17 hours ago

Without backups, odds are good it was going to happen eventually, regardless of Claude's involvement. Yeah, it sucks, but humans make mistakes, too. And, sometimes disks fail.

Anyway, people are far too trusting of their own ability to be perfect and the models' ability to be perfect. Which is why you have to keep backups if you value your data.

fnordsensei

17 hours ago

> he asks who is accountable when an AI agent causes autonomous damage

He is. On the other hand, when a tool of massive power becomes increasingly easy to use, it becomes increasingly easy to misuse, intentionally or not.

The question is if self-regulation is enough. Not that I necessarily trust legislators to know any better. But like we have observed from the world of agentic coding, sometimes two stupid things can behave as a unit of higher intelligence than the individual parts.

graemep

16 hours ago

This looks like privately funded educational not for profit. How would that be appropriately regulated?

fnordsensei

14 hours ago

Right, I'm not saying that they would be regulated, but rather the provider of the instrument, and the market adoption of it.

Take the emotions he experienced after the event: "Who do I talk to, how do I undo this, what do I do" etc., and the fact that he's met with a wall of silence from Anthropic.

Perhaps there needs be more concerted preparation, guidance, direction and less trial and error for how society adopts it.

The imperfect comparison that sometimes comes up is to that of nuclear power, where international regulation and guidance quickly was adopted. I'm not necessarily endorsing the literal comparison, but an attitude of "well, let's just see how people choose to use nuclear power" would have yielded a significantly different world, I'm sure.

graemep

11 hours ago

Its far more difficult than physical safety criteria with things like nuclear energy. Will the ban be on providing tools that can delete things? In this case it looks like it was writing software that can delete things. SO maybe require that coding tools do not run code without a check? So things like Claude auto mode are not allowed? The result will be people ignore warnings.

The question of how people choose to use nuclear power does not arise, because most people use the electricity it generates rather than operate their own nuclear power stations. With AI people need to operate it themselves. Its fundamentally different. Its more like trying to stop people from writing bad code.

Planktonne

14 hours ago

The user is of course responsible for this, but I don't think all the blame rests with him.

He's using a tool that he pays for, that has been sold to him as hyper-capable and autonomous (even with the small-text warning about it being able to make mistakes). The providers of the tool are also responsible, even if only for the misleading advertising.