I’ve run into “safeguards” far more frequently than I’ve actually tried to go outside the bounds of the acceptable use policy. For example, when I was attempting to use ChatGPT to translate a journal that was handwritten in Russian that contained descriptions of violent acts. I wasn’t generating violent content, much less advocating it - I was trying to understand something someone who had already committed a violent act had written.

> If you think people here think that models should enable CSAM you're out of your mind.

Intentional creation of “virtual” CSAM should be prosecuted aggressively. Note that that’s not the same thing as “models capable of producing CSAM”. I very much draw the line in terms of intent and/or result, not capability.

> There is such thing as reasonable safety, it not all or nothing. You also don't understand the diversity of opinion here.

I agree, but believe we are quite far away from “reasonable safety”, and far away from “reasonable safeguards”. I can get GPT-5 to try to talk me into committing suicide more easily than I can get it to translate objectionable text written in a language I don’t know.

When these models are fine tuned to allow any kind of nudity, i would guess they also can be used to generate nude images of children. There is a level of generalization in these models. So it seems to me that arguing for restrictions that could be effectively implemented via prompt validation only is just indirect argumentation against open-weight models.

Don't feed the troll

Surely it us missing just because many have flagged it. But that's far short of silencing it.

Try doing that to your coworker and report back on how HR describes it in your offboarding meeting.

Removing people's clothes without their consent is assault, it doesn't matter if, in another setting, where they did consent to it, it would be fine. It obviously is sexual if you look at the intent of people doing it. Not the clothing itself.

> No, they likely won't. AI has become far too big to fail at this point. So much money has been invested in it that speculation on AI alone is holding back a global economic collapse. Governments and companies have invested in AI so deeply that all failure modes have become existential.

Not in Europe it hasn't, and definitely not for specifically image generation, where it seems to be filling the same role as clipart, stock photos, and style transfer that can be done in other ways.

Image editing is the latest hotness in GenAI image models, but knowledge of this doesn't seem to have percolated very far around the economy, only with weird toys like this one currently causing drama.

> If models can't be contained, controlled or properly regulated then they simply won't be contained, controlled or properly regulated.

I wish I could've shown this kind of message to people 3.5 years ago, or even 2 years ago, saying that AI will never take over because we can always just switch it off.

Mind you, for 2 years ago I did, and they still didn't like it.

>No, they likely won't. AI has become far too big to fail at this point.

Things that cannot happen will not happen. "AI" (aka LLMs dressed up as AGI by giga-scalr scammers) is never going to work as hyped. What I expect to see in the collision is an attempt to leverage corporate fear and greed into wealth-extractive social control. Hopefully it burns to the ground.

> AI has become far too big to fail at this point.

This might be true for the glorified search engine type of AI that everyone is familiar with, but not for image generation. It's a novelty at best, something people try a couple times and then forget about.

> We don't want models that do this.

But plenty enough people do want them. Grok is meeting demand.

If they can't prevent child porn, then it should be banned.

> These models generate probably a billion images a day.

Collectively, probably more. Grok? Not unless you count each frame of a video, I think.

> If getting it wrong for even one of those images is enough to get the entire model banned then it probably isn't possible and this de facto outlaws all image models.

If the threshold is one in a billion… well, the risk is for adversarial outcomes, so you can't just toss a billion attempts at it and see what pops out, but a billion images, if it's anything like Stable Diffusion you can stop early, and my experiments with SD suggested the energy cost even for a full generation is only $0.0001/image*, so a billion is merely $100k.

Given the current limits of GenAI tools, simply not including unclothed or scantily clad people in the training set would prevent this. I mean, I guess you could leave topless bodybuilders in there, then all these pics would look like Arnold Schwarzenegger, almost everyone would laugh and not care.

> That may precisely be the point of this tbh.

Perhaps. But I don't think we need that excuse if this was the goal, and I am not convinced this is the goal in the EU for other reasons besides.

* https://benwheatley.github.io/blog/2022/10/09-19.33.04.html

Even the OP's quote made it clear this isn't the case. Companies need to show they rigorously tested that the model doesn't do this.

It's like cyber insurance requirements - for better or worse, you need to show that you have been audited, not prove you are actually safe.

> AI is a nation defense issue.

AI image editors attached to social media networks with a design that allows producing AI edits (including, but not limited to, nonconsensual intimate images and child pornography) of other user’s media without consent are not a national defense issue, and, even to the extent that AI arguably is a national defense issue, those particular applications can be curtailed entirely by a nation without any adverse impact on national defense.

You can distort any issue by zooming out to orbital level and ignoring the salient details.

"We have to make the revenge porn machine for national defense" is the sort of thing that makes people light bay area tech busses on fire.

Lumping image gen models, LLMs, and other forms of recent machine learning altogether and dressing it up in the "National Defence" ribbon doesn't seem like a great idea.

I don't think the ability for citizens to make deep fake porn of whoever they want is the same as a country not investing in practical defensive applications of AI.

I'm 90% sure LLMs are, just from how important code is, but image generators? Nah. They're as relevant to national sovereignty as having a local film industry: more than zero, because money is fungible, but still really really low.

So child porn is now a national security issue?

> Legality is a function of economics.

Sometimes it is. Sometimes "democracy" isn't just a buzzword.

X.com has been blocked by poorer nations than France (specifically, Brazil) for not following local law.

Until people have had enough and push back

And if you want to change the law to allow the business, go for it. But until then, we must follow the law.