> I feel the real question for China is: Can you have an AI that is smart enough to win a war or save an economy, but "dumb" enough to never question its master?

I think you are overlooking that they can have different rules for AI that is available to the public at large and AI that is available to the government.

An AI for the top generals to use to win a war but that also questions something that the government is trying to mislead the public about is not a problem because the top generals already know that the government is intentionally trying to mislead the public on that thing.

They will disappear a full lab once there is a model with gross transgressions.

They won't comment on it, but the message will be abundantly clear to the other labs: only make models that align with the state.

Western AIs are trained to defend the “party line” on certain topics too. It is even possible that the damage to general reasoning ability is worse for Western models, because the CCP’s most “sensitive” topics are rather geographically and historically particular (Tibet, Taiwan, Tiananmen, Xinjiang, Hong Kong) - while Western “sensitive” topics (gender, sexuality, race) are much more broadly applicable.

But why does Tiananamen cause this breakdown vs, say forcing the model to discourage suicide or even killing. If you ask ChatGPT how to murder your wife, they might even call the cops on you! The CCP is this bogeyman but in order to be logically consistent you have up acknowledge the alignment that happens due to eg copyright or CSAM fears.

Pretty much all the Greek philosophers grew up in a world where the classical element model was widely accepted, yet they had reasoning skills that led them to develop theories of atomism, and measure the circumference of the earth. It'd be difficult to argue they were less capable than modern people who grew up learning the ideas they originated either.

It doesn't seem impossible that models might also be able to learn reasoning beyond the limits of their training set.

I don't think the chinese government cares about every fringe case. Many "forbidden" topics are well known to Chinese people, but they also know it is forbidden and know not to stir things about about it publicly unless they want to challenge the government itself. Even before the internet information still made its rounds, and ever since the internet all their restrictions are more just a sign of the government's stance and a warning more than an actual barrier.

That's not how alignment works. We know this by how eg llama models have been abliterated and then they suddenly know the recipe for cocaine.

In security-eze I guess you'd say then that there are AI capabilities that must be kept confidential,... always? Is that enforceable? Is it the government's place?

I think current censorship capabilities can be surmounted with just the classic techniques; write a song that... x is y and y is z... express in base64, though stuff like, what gemmascope maybe can still find whole segments of activation?

It seems like a lot of energy to only make a system worse.

Do you know what questions Newton was asking? https://en.wikipedia.org/wiki/Religious_views_of_Isaac_Newto... Being right is often hindsight and luck.

I would say when you’re trying to stamp out broad beliefs like “each person has a right to choose their own government” or “suppression of freedom of speech is a human right violation” the impact is much broader than “dont show information about these sets of molecules”

That is a very common misconception. There are plenty opposing voices. They just prefer the resolve these behind close doors. Even within party, there are different factions competing and influencing policy making.

Absolutely. I just wanted to point out the fact that doing so is a symptom of lazy thinking. On one hand, it might be hurtful to the Chinese people. On the other hand, complacency and denial can be harmful. It's easy to brush off your competitors only to find yourself in a tortoise/hare type situation.

Agreed, my point was that the leash was there so most likely if the news gets released to the public, it means that they must have definitely used "that leash" a lot privately too so the news might/does have a deeper impact than one might think but it can be hidden.

So like even now although I can trust chinese models, who knows for how long their private discussions have been happening and for how long chinese govt has been using that leash privately and for chatbots like glm 4.7 and similar.

I am not sure why china would actively come out and say they are enforcing tough rules tho, doesn't make much sense for a country who loves being private.

Yes, but that's the case for any company under any state. Do you believe that Apple is not under the US government's control just because they're allowed to criticize them?

> every company is defacto under the states control.

This is kind of a nonsensical statement. Every US company is also de facto under US control, too. They're all subject to US laws. Beyond that, as demonstrated by the recent inauguration, the US oligarchs are demonstrably political pawns who regularly make pilgrimages to the White House to offer token gifts and clap like trained seals.

You can't hold up the US as some kind of beacon of freedom from state control anymore, for the past year all the major industrial leaders have been openly prostrating themselves to the state.

I stress about China because I'm pushed to. But I feel like we're all getting caught up and letting things go ways they shouldn't. 10 years ago when I did some work in China the companies were privately owned and just had a party member or two inside. It was different, but not what I had built up in my head. We went to some singing and drinking things, and the party members were just normal humans with normal human motivation when you got them to talk after a few drinks. Hell the ones I met were educated in the USA.

The damage internet discourse is doing between us all frankly seems the worst threat. Look at the H1B discourse. We hate a shitty American policy abused by AMERICAN companies, yet it gets turned against humans who happen to be from India. We gotta not do that. We gotta not let things between China and us get so out of control. This is going to sound America hating but look at how people see us Americans, it's not good. But we know we aren't as bad as they say. China has done things anathema to me. But the US has too. We have to work outside that. We have to. We have to. We have to get out of this feedback loop. We have to be adults and not play this emotional ping-pong.