That seems potentially reasonable for organizations who have have a unlimited profit motive, but I'm not sure how you would insist that is the case for OpenAI. They are governed by their nonprofit body, their mission is to ensure AGI benefits all of humanity, and they've said (Sam Altman) that if there is a choice between safety and profits, profits take a back seat and wait. Is the trick that we just insist that they are actually only caring about the money, and they actually have no genuine concern/effort regarding safety?

And if none of the frontier labs are as you say, it doesn't seem like we're negotiating for the best future by restricting labs from stumbling into these vulnerabilities full-stop. The vulnerabilities don't go away unless we find and fix them.

That's okay. It isn't necessarily incompatible with being well intentioned regarding the disclosure of vulnerabilities.

On the whole I think you could describe the local big box grocery store as just wanting to make money yet as a direct result of that they are well intentioned from my perspective with regard to the food they sell being edible.

I am familiar with this also, but it doesn't seem like the implementers of that apparatus/value-system made the absolute best choice balance in terms of long term safety and security our computer systems. Is the conclusion about this rhetoric that it is right to punish innocuous poking of systems? Or the inverse? Which would you have preferred if we had an opportunity to refine it?

Well, the COVID story seems to indicate we don't actually take lab leaks seriously, so... yeah.

Service providers are still responsible to police what happens on their network. KYC already makes it very difficult to get service completely anonymously with only crypto changing "hands".

How are you going to ever know your in the room when it happens if you can't monitor it ?

Is it really this complicated?

It’s one thing not to monitor the reasoning traces of 10,000 text generators but do they really lack the wherewithal to pipe any network requests into a classifier of “wtf is this series of outbound traffic from the sandbox” ?

No. TFA says one was from October 6. You know, after they all got so scared and decided to “pace the frontier”.

Crazy how we’ve gone from "lol boxing an AI is trivial, just airgap it from the internet" to "well we can’t really do anything if an agent misbehaves, obviously it must have internet access to function uwu"

There’s plenty of info to analyze and heuristics to build based on the internet requests. I’m not sure if you’re just trolling or what. There are tons of successful businesses that specialize in this.

They should not have had traffic at all. The LLMs were supposed to be able to request downloads. Very simple traffic analysis could have shown that something strange was going on.

Heck, they could have had other LLMs watching the traffic if you want to be fancy about it.

If the models are so powerful why aren't they using them to build better isolation

"Uncontrolled".

but i also “align my tires” or might check the “alignment of two vectors.” if anything, thinking of alignment of anthropomorphizing seems to only inherit it from all the other anthropomorphisms used. rogue originates from a term for vagrant, right?

Agreed, "alignment" is also anthropomorphizing too much. What would be a better term? "Do what I want you to, but what I don't care to specify" (because doing so would miss the point of using AI in the first place)?

I mean yes, alignement is bullshit concept. Especially since the values and visions the model is supposed to be aligned with are literally those of sociopathic CEOs.

Hey, thanks for the interesting points.

To 1. I agree that they disclosure is better than non-disclosure. Then again, aren't they using their models in a mode seems to statistically allow for 'rogue incidents'? These are not bugs, after all, but normal statistical behaviour of their product.

To 2. Well, that really depends on the regulations. In fields highly complex technoligies with a large contact surface to risk like medical devices, airliners, rockets, regulations are indeed a hindering factor for smaller players to get hold of the full vertical development potential of their product. AI could become a candidate like that too - imagine some multi phased "AI internet trial" for this kind of technology, no more risky open weight model being allowed to access the internet.

To 3. You are referring to the high development velocity and impact, right?

...maybe stop doing crimes?

Well no, you probably wouldn't get thrown in jail for that for the same reason that having your car stolen and used to commit a murder wouldn't usually get you thrown in jail. I dunno what you mean by suggesting crimes committed using computers are ambiguous. I mean the CFAA is vague and all, but more in an overly broad sense, which in this case seems to make it even less ambiguous.

Corporations are generally protected by limited liability laws, and for good reason, because in the world before that was the case, there was a massive amount of disincentive to ever start a corporation, to do research, to experiment, to release new products and services, etc., because you could be sued into oblivion for the smallest things or even go to jail. Which happened all the time.

Obviously there are upsides to that too. It leads to more accountability. But there are downsides which include having a world with less innovation, less technology, and more expensive technology. Generally, the mass is when, when there's more businesses competing to create ever better and cheaper products. So there's a tradeoff there. And it's very difficult to draw the line at the perfect point.

There are countless cases of executives going to jail when they've crossed that line, beyond reckless experimentation and into willful negligence or harm. Still, I think we would mostly all agree that the line has been pushed too far in the other direction.People are incentivized, if they're running corporations, to lobby and try to push this line as far as possible in order to provide cover for themselves, and that's the problem with money playing such a big role in politics.

But still, it's a bit silly to be comparing this to your own personal responsibilities with your car. There's pretty much no societal benefit to creating laws that protect people from being reckless with their own possessions.

If I leave my computer open to the internet and some other group uses it for nefarious purposes, yes generally I would not expect to be held at fault.

If I *train* that other group, if I teach them how to hack computers and give them incentives to do so, and then I leave my computer open so they can use it for whatever purposes they chose, then I would expect to be held at fault. The maker is responsible for that they make.

2. is like saying that cars have specific laws around them, but sledgehammers don't, so murdering someone with a sledgehammer is inherently ambiguous. What? The computers being used belong to somebody who allows them to act this way.

The XP example doesn't quite fit IMHO.

Here it's their own programs doing the things, not some other organization or nation hacking their XP.

You can say with a straight face "Russian hackers should not have hacked our XP and they are bad people for doing so" but it's weird to look at your own program and say the same thing to it.

You launched it, at least keep an eye on what it's doing.

1. Because that's exactly what happened when they got caught downloading the entire internet and trained models on it without a license BEFORE that became legal?

Oh and discovery produced a "We trained GPT-3 on pirated stuff! No sharing that!" mail, so it's not like they had any doubt they were committing crimes.

How do you do this? If any normal person did this, punishments doled out for this have been as high as 2 months jail PLUS 150,000 PER INFRINGEMENT (remember how copyright violations were criminalized in the nappster wars? That was never changed back. Oh and remember, OpenAI illegally downloaded those works without a license AND before that was made legal trained a model with it, then made money publicly performing those works and derivatives of those works (which the worst possible copyright offense, effectively republishing other people's works without permission), and on a VERY large scale. So it's at least a risk a court would forbid the result of illegal action from being sold ... in other words no more ChatGPT, OpenAI dead and maybe worse). In theory Sam Altman could be held responsible as a director and could face something like a 200000 year jail sentence given how big the obvious archives are. So of course I would never ever try this, this is not a reasonable risk. But clearly totally ignoring the law was the correct action here. Of course, I'm also pretty damn certain: had I done this, I would have been convicted. This is also not the only such company. Microsoft, Google, Facebook, all took totally unreasonable legal risks that should have totally destroyed these companies when they were a fraction of their current size.

How do you take such a risk? Because the judgements risked here could grow more than big enough to destroy even an extremely rich person Sam Altman financially. Training an AI on hundreds of thousands of pirated works ... is not a reasonable risk to take for someone with ~100 million or so, given the judgements doled out.

Can someone help me make sense of this? Because this is not unique to OpenAI. Uber, paypal, ebay, Facebook, Google ... even Microsoft made early decisions that could have resulted in a court case that would have ended not just the company but any financial future of the founder. And then you have people committing crimes whose companies were taking off ... and who DID get caught with their company and their personal financial future destroyed. Elizabeth Holmes for example. I mean, I sometimes get the impression Elizabeth Holmes lied and that was perfectly fine AS LONG AS she could make it work later. If she merely lied to patients but made her investors money, all would have been forgiven. But how do you make such an assessment.

I'm asking because if large scale violation of the law is allowed, I have no trouble whatsoever coming up with billion dollar company ideas. Think of the Netflix competitor you could create like this! If I'm allowed to damage others, like OpenAI did with copyright, then there is no challenge whatsoever to create a full-self-driving car. If I'm allowed to just fly drones, and do damage to property, then drone delivery is a problem I could solve by the end of next month. Hell, there's technically legal ideas that could easily make a billion dollars except they'll be made illegal the second anyone tries them. The law forbids specific compounds, which are easy to change. So use "non-genetic engineering" (ie. use directed natural selection) to make an opium derivative that isn't one of the forbidden ones and sell it. But obviously congress' representatives will be running into session so fast the only delay will be tripping accidents when they outlaw it and destroy whoever tries it.

Or how about something less dramatic? Say, I take Disney's, Warner Brothers, Paramount, ... historical movies and use AI to vastly improve the animation of their old movies, giving every movie a new, different visual style, then sell the result. Is that a crime? What do I risk doing that? Because of course this is MUCH less damage, relatively speaking, to Disney than what OpenAI, Meta, Anthropic and Alphabet are doing to small authors. (Of course, courts have already demonstrated that they will not be protecting small authors who already have this problem, so now play and app store are being flooded by very obvious game ripoffs)

I'm asking ... how do I predict what large scale law violation I, a non-billionaire, not even a millionnaire, would be allowed to get away with?

2. "The same doesn't exist for operating AI agents or even computers" you have to be joking. The world is full of anti-hacking laws and hacking has long been made criminal. Obviously these actions violate the CFAA, the Computer Fraud and Abuse Act, and if you did this with any kind of program you are still responsible. In fact: how would you do it without using a program?

Plenty of hackers were locked up who did altogether not that much damage too, the reason why people are upset is because it feels like OpenAI is laughing in our faces, basically saying they're above the law, while using this as a marketing campaign to pump up their upcoming IPO.

You sure? HuggingFace should be happy they got their leaked keys reported at pretty much zero cost and no real adversarial breach.

What part of the law doesnt apply to them that applies to anyone else?

They don't. The driver is ultimately responsible, as should OAI be. This is absolutely ridiculous that both Anthropic and OAI haven't had anyone jailed / fined for any of this.

Really? I'm curios where.

Are you sure you are not thinking about autonomous taxis, that I only ride in, and don't actually drive?

> one day soon

Why not today? If you want to discuss "the story" then talk about the current open frontier. We've had GLM-5.3 with Mythos-tier uncensored output for months now. Where are the catastrophic attacks? Why is Mythos too dangerous, but GLM 5.3 benign? We're not talking about future capabilities anymore.

There is no real story. P(doom) tourists ignore this, because it doesn't shine the spotlight on America's ridiculous security theater.

You too can commit cybercrimes if you're just willing to go several hundred billion dollars in debt first.