Because they're likely just parroting whatever their AI tells you.

These people are not in charge of their faculties; havne't developed the intelligence to rationalize their own actions; and lastly, are high on their own supplies.

That said, these models are as powerful as the security systems you bypass to test them. So the danger is more this is a private company well funded and could hold people hostage with demands.

Looks like "this weird trick feels illegal" marketing

> This one is triggered when AI is doing enough of Anthropic's research to roughly double how fast its models improve. The policy says that pace would pack two years' worth of AI progress, at its 2018-2024 rate, into a single year. The filing says AI "has already been accelerating our research and development." Crossing the line means more safeguards, not a shutdown.

We're like, sooooo dangerous! It's scary! Give us money to make the scary AI stop!