Anthropic irks White House with limits on models’ use

246 pointsposted 5 months ago
by mindingnever

116 Comments

impossiblefork

5 months ago

Very strange writing from semafor.com

>For instance, an agency could pay for a subscription or negotiate a pay-per-use contract with an AI provider, only to find out that it is prohibited from using the AI model in certain ways, limiting its value.

This is of course quite false. They of course know the restriction when they sign the contract.

bri3d

5 months ago

This whole article is weird to me.

This reads to me like:

* Some employee somewhere wanted to click the shiny Claude button in the AWS FedRamp marketplace

* Whatever USG legal team were involved said "that domestic surveillance clause doesn't work for us" and tried to redline it.

* Anthropic rejected the redline.

* Someone got mad and went to Semafor.

It's unclear that this has even really escalated prior to the article, or that Anthropic are really "taking a stand" in a major way (after all, their model is already on the Fed marketplace) - it just reads like a typical fed contract negotiation with a squeaky wheel in it somewhere.

The article is also full of other weird nonsense like:

> Traditional software isn’t like that. Once a government agency has access to Microsoft Office, it doesn’t have to worry about whether it is using Excel to keep track of weapons or pencils.

While it might not be possible to enforce them as easily, many, many shrink-wrap EULAs restrict the way in which software can be used. Almost always there is an EULA carve-out with different tier for lifesaving or safety uses (due to liability / compliance concerns) and for military uses (sometimes for ethics reasons but usually due to a desire to extract more money from those customers).

axus

5 months ago

A classic:

THIS SOFTWARE PRODUCT MAY CONTAIN SUPPORT FOR PROGRAMS WRITTEN IN JAVA. JAVA TECHNOLOGY IS NOT FAULT TOLERANT AND IS NOT DESIGNED, MANUFACTURED, OR INTENDED FOR USE OR RESALE AS ONLINE CONTROL EQUIPMENT IN HAZARDOUS ENVIRONMENTS REQUIRING FAILSAFE PERFORMANCE, SUCH AS IN THE OPERATION OF NUCLEAR FACILITIES, AIRCRAFT NAVIGATION OR COMMUNICATION SYSTEMS, AIR TRAFFIC CONTROL, DIRECT LIFE SUPPORT MACHINES, OR WEAPONS SYSTEMS, IN WHICH THE FAILURE OF JAVA TECHNOLOGY COULD LEAD DIRECTLY TO DEATH, PERSONAL INJURY OR SEVERE PHYSICAL OR ENVIRONMENTAL DAMAGE.

dgfitz

5 months ago

Look up DoD (DoW?) 882 and LOR ratings. This is a fancy way of saying “Java can’t do that because we haven’t certified a toolchain for it”

And for bonus points, go find the last certified compilers for LOR1 rating that follow 882 guidelines.

Now you’ve scratched the surface of safety-critical software. Actually writing it is a blast. I think most web developers would weep in frustration. “Wait, I can’t allocate memory that way? Or that way? Or in this way not at all?! There’s no framework?! You mean I need to do all this to verify a button click??!!”

0x457

5 months ago

Most web developers don't know what "memory allocation" is and let alone how to manually allocate it.

I think people that don't write safety-critical software in general will weep in frustration, not just web developers.

_9ptr

5 months ago

I never knew Java was so dangerous

gowld

5 months ago

Everything is dangerous by default. That's the point.

1vuio0pswjnm7

5 months ago

This excerpt does not disclaim liability for death, personal injury or physical damage (b/c generally they can't)

Nor does it prohibit use or resale of "Java Technology" for any particular purpose

It does suggest they are aware of Java's shortcomings

m463

5 months ago

reminds me of jslint

author added "must be used for good, not evil" to the license

...and IBM asked for an exception.

https://en.wikipedia.org/wiki/JSLint#License

https://news.ycombinator.com/item?id=5138866

note that restricting use of software makes it non-free gpl-wise.

RMS said the GPL does not restrict rights of the USER of software, just that when the software is redistributed, the rights are passed along.

kentonv

5 months ago

Good old Douglas Crockford. He also put the "must be used for good, not evil" restriction on JSON, which he invented. Obviously JSON is used for all kinds of evil, though.

A much younger, more naive me (~20 years ago) actually emailed him to complain about the ambiguous terms and he replied saying something to the effect of "It's obviously unenforceable, get over it."

user

5 months ago

[deleted]

salynchnew

5 months ago

Could also be an article placed by a competitor + a squeaky wheel.

giancarlostoro

5 months ago

> due to a desire to extract more money from those customers

If it gives you high priority support, I dont care, if its the same tier of support, then that's just obnoxiously greedy.

matula

5 months ago

There are (or at least WERE) entire divisions dedicated to reading every letter of the contract and terms of service, and usually creating 20 page documents seeking clarification for a specific phrase. They absolutely know what they're getting into.

darknavi

5 months ago

I have a feeling in today's administration which largely "leads by tweet" that many traditional "inefficient" steps have been removed from government processing, probably including software on-boarding.

dannyisaphantom

5 months ago

Can confirm these teams are still around. There is now an additional "SME review group" that must comb through any and all AI-related issues that were flagged, sends it back down for edits and must give final approval for before docs are sent over to provider for response. Turnaround has gotten much slower (relatively)

gowld

5 months ago

Or you can use personal accounts to bypass red tape for government business.

anjel

5 months ago

I have a legal education but reading TOS and priv policy docs at account creation is purposefully too time consuming by design.

One my fave new AI prompts: you are my Atty and a expert in privacy law and online contracts-of-adhesion. Review the TOS aggreement at [url] and privacy policies at [url] and brief me on all areas that should be of concern to me.

Takes 90 seconds from start to finish, and reveals how contemptuously illusory these agreements are when SO MANY reserve the right to change anything with no duty to disclose changes.

bt1a

5 months ago

Perhaps it's the finetune of Opus/Sonnet/whatever that is being served to the feds that is the source of the refusal :)

andsoitis

5 months ago

Don’t tech companies change ToS quite frequently and sometimes in ways that’s against the spirit of what the terms were when you started using it?

ajross

5 months ago

This is a contract, not a click through license. You can't do that.

(Legally you can't do it with a click-through either, but the lack of a contract means that the recourse for the user is just to stop buying the service.)

user

5 months ago

[deleted]

jdminhbg

5 months ago

Are you sure that every restriction that’s in the model is also spelled out in the contract? If they add new ones, do they update the contract?

mikeyouse

5 months ago

The contracts will usually say “You agree to the restrictions in our TOS” with a link to that page which allows for them to update the TOS without new signatures.

PeterisP

5 months ago

All the US megacorps tend send me emails saying "We want to change TOS, here's the new TOS that's be valid from date X, and be informed that you have the right to refuse it" (in which case they'll probably terminate the service, but I'm quite sure that if it's a paid service with some subscription, they would have to refund the remaining portion) - so they can change the TOS, but not without at least some form of agreement, even if it's an implicit one 'by continuing to use the service'.

giancarlostoro

5 months ago

Usually, contracts will note that you will be notified of changes ahead of time, if it's a good faith contract and company that is.

impossiblefork

5 months ago

Here in Sweden contracts are a specific thing, otherwise it's not a contract, so agreeing to conditions that can be changed by the other party simply isn't a contract and therefore is just a bullshit paper of very dubious legal validity.

I know that some things like this are accepted in America, and I can't judge how it would be dealt with. I assume that contracts between companies and other sophisticated entities are actual contracts with unchangeable terms.

mindcrime

5 months ago

I know that some things like this are accepted in America

Not really. Everything you said about contracts above applies to contracts in America last time I checked. Disclaimer: IANAL, my legal training amounts of 1 semester of "Business Law" in college.

impossiblefork

5 months ago

In theory yes, but you also have this stuff where people agree to get medical treatment and the price isn't specified.

This would be a non-contract in Swedish law, for example.

mindcrime

5 months ago

One thing about the US, is how we handle settings where one could conceptualize a contract as being needed, but where it would be way too inefficient and impractical to negotiate, write out, understand, and sign, a written contract. In those cases, which includes things like retail sales, restaurants, and may other cases, the UCC or Uniform Commercial Code[1][2] applies. Not sure offhand if that relates to the medical example or not, but I expect that at least some similar notion applies. So there are binding laws that cover these transactions, it's just not done the same way as a "full fledged contract".

[1]: https://en.wikipedia.org/wiki/Uniform_Commercial_Code

[2]: The UCC also covers other things, but these cases are a lot of what it's best known for.

watwut

5 months ago

Knowing medical prices up front would be entirely possible and practical. In most situations, you should be able to sign contract up front.

adastra22

5 months ago

It is also illegal in USA, although that only changed recently.

mikeyouse

5 months ago

Yeah, I’ve signed dozens of contracts for services and some are explicit in the way you expect but a lot of software or SAAS type contracts have flexible terms that refer to TOS and privacy policies that are updated regularly. It’s uncommon that any of those things are changed in a way that either party is upset with so companies are generally okay signing up and assuming good faith.

gowld

5 months ago

The contact's restriction is on the usage of the model, not the behavior of the model.

owenthejumper

5 months ago

This feels like a hit piece by semafor. A lot of the information in there is purely false. For example, Microsoft's AI Agreemeent says (prohibits):

"...cannot use...For ongoing surveillance or real-time or near real-time identification or persistent tracking of the individual using any of their personal data, including biometric data, without the individual’s valid consent."

cbm-vic-20

5 months ago

There's nothing stopping Microsoft hammering out different terms for certain customers, like governments.

gowld

5 months ago

Semafor is Ben Smith's blog, trying to imitate a reputable newspaper like Financial Times.

saulpw

5 months ago

Gosh, I guess the SaaS distribution model might give companies undesirable control over how their software can be used.

Viva local-first software!

nathan_compton

5 months ago

In general I applaud this attitude but I am glad they are saying no to doing surveillance.

saulpw

5 months ago

Me too, actually, but this is some "leopards ate their face" schaudenfraude that I'm appreciating for the moment.

_pferreir_

5 months ago

EULAs can impose limitations on how you use on-premises software. Sure, you can ignore the EULA, but you can also do so on SaaS, to an extent.

ronsor

5 months ago

With SaaS, you can be monitored and banned at any moment. With EULAs, at worse you can be banned from updates, and in reality, you probably won't get caught at all.

MangoToupe

5 months ago

Are EULAs even enforceable? SaaS at least have the right to terminate service at will.

LeoPanthera

5 months ago

One of the very few tech companies who have refused to bend the knee to the United States' current dictatorial government.

jimbo808

5 months ago

It's startling how few are willing to. I'm rooting for them.

chrsw

5 months ago

Can we trust this though? “Cooperate with us and we’ll leak fake stories about how frustrated we are with you as cover”.

And I’m not singling out Anthropic. None of these companies or governments (i.e. people) can be trusted at face value.

astrange

5 months ago

They don't do that. They're not capable of cooperating with anyone, it's maximum punishment all the time. It's unclear if they can keep secrets either.

TheCoelacanth

5 months ago

Yeah, and a private capitulation doesn't accomplish their goals either. They want the spectacle of public submission.

user

5 months ago

[deleted]

sitzkrieg

5 months ago

because of this they're probably on borrowed time in this political climate

jschveibinz

5 months ago

[flagged]

9dev

5 months ago

I don’t get this notion. Politics has a place on HN like any other interesting topic does, whether you like it or not

tene80i

5 months ago

Which part?

baggy_trough

5 months ago

I should think that it was very obvious that America does not have a dictatorial government; this is hyperbole.

SpicyLemonZest

5 months ago

It's very obvious that America has a dictatorial government, and I'm baffled why you would deny this. The dictator-in-chief has argued explicitly and repeatedly that the written laws of the land don't constrain him; he can shut down departments Congress ordered him to run, levy taxes they didn't authorize, and overrule or rewrite any statute he feels isn't correct. He seized 10% of Intel Corporation without even a fig leaf of legal basis!

Perhaps you're confused that the normal system of laws is still operating? That's just the nature of dictatorship in a large country. The dictator only has so much time in the day, and if he has to delegate anyway he might as well use the preexisting courts and civil servants. He just has to put supervisors on top who can credibly threaten to invoke his wrath if people step too far out of line.

baggy_trough

5 months ago

The reason that I would deny it is that your claims are greatly exaggerated.

SpicyLemonZest

5 months ago

Now I know you're lying, but I don't understand why. It's widely known, for example, that USAID is shut down. You can go to their site and see that they're shut down. What do you think you're going to gain by lying to me about such easily verifiable facts?

baggy_trough

5 months ago

[flagged]

SpicyLemonZest

5 months ago

Trump just announced that broadcasters who are against him should have their network license revoked! My patience with you has run out, so don't expect any further response, but I remain utterly baffled by this phenomenon of people who think it's "paranoid" or "hyperbole" to believe the president's official announcements about what he wants and what he plans to do. I understand that you don't perceive yourself to be an apologist for fascism - like other people I've met, you seem to genuinely believe that it's paranoid to accurately report what the President is doing and savvy to instead make up a fictional president who does less alarming things for more sympathetic reasons. But why? It's just absurd.

curt15

5 months ago

Does threatening to prosecute office supply store workers unless they print certain flyers count as the behavior of a dictator? That doesn't sound like the behavior of a government respecting the First Amendment.

baggy_trough

5 months ago

I think it would count as a (very) small element of that.

Loughla

5 months ago

Any single element of dictatorial rule is dictatorial rule. There is no space for any amount of that bullshit, regardless of how inane seeming it is.

A) Boundary testing. Small bites end up being large portions after enough are taken.

B) If I shit in 10 gallons of chocolate pudding, would you want to eat a bite of that pudding?

baggy_trough

5 months ago

[flagged]

vkou

5 months ago

You're being plainly absurd by splitting hairs over a dangerous destruction of the rule of law, complete breakdown of checks and balances, and an executive that is behaving like it is both above the law, and will never lose power.

Loughla

5 months ago

If I steal just a little from you, are you okay with that?

tene80i

5 months ago

No need for that. I was just asking someone to clarify what they were saying.

Regardless of what you think about the government, that wasn’t a statement in the above. The statement was about tech companies. So it wasn’t clear.

mjparrott

5 months ago

[flagged]

dcre

5 months ago

The fact of being elected is not relevant to the question, and neither is the nominal existence of constitutional checks. Extrajudicial murder of alleged criminals, abuse of criminal prosecution to target political enemies, armed thugs yanking innocent people out of their cars, steamrolling firms and universities into administration-favorable policy changes and extracting hundreds of millions of dollars — and that's just the first few things I could think of. There are dozens of examples. It is not inflammatory to describe simple reality.

dcre

5 months ago

New ones every hour! TV shows preemptively taken off the air in _anticipation_ of state backlash.

MadnessASAP

5 months ago

I would be hesitant to call the US democratic process "free and fair." And the powers held by the president certainly make them more dictatorial then the heads of state of other democracies, particularly as wielded by the current administration.

So you have a democratic process of dubious quality that elected a government that is dictator-ish.

Don't accept that your countries elections are free and fair as a axiom.

tclancy

5 months ago

Arguing de jure instead of de facto is fine until the reality of the de facto affects you.

FortuneIIIPick

5 months ago

Dictatorial suggests a "ruler with total power". The US has three branches of government. That hasn't changed, ever.

izzydata

5 months ago

The definition of dictatorial government is either a single person or a small group of people. So there being three branches of government doesn't necessarily prohibit a government from being a dictatorship if they are all working together to enact their authoritarian control without constitutional limits.

But really this is just pointless semantics. It doesn't matter what it is called it is still a problem.

TheServitor

5 months ago

"Eventually, though, its politics could end up hurting its government business."

Good? What if, and I know how crazy this sounds, not using AI to surveil people was a more desirable goal than the success of yet another tech company at locking in government pork and subsidies?

Terretta

5 months ago

Here's an entertaining example from 20 years ago:

By using the Apple Software, you represent and warrant that you ... also agree that you will not use these products for any purposes prohibited by United States law, including, without limitation, the development, design, manufacture or production of missiles, or nuclear, chemical or biological weapons. -- iTunes

No production of missiles with iTunes? Curses, foiled again.

user

5 months ago

[deleted]

tracker1

5 months ago

What is the govt expecting to do in combination of surveillance and Antrhopic models? I'm not convinced this is any kind of valid job function.

SilverbeardUnix

5 months ago

Honestly makes me think better of Anthropic. Lets see how long they stick to their guns. I believe they will fold sooner rather than later.

sfink

5 months ago

First, contracts often come with usage restrictions.

Second, this article is incredibly dismissive and whiny about anyone ever taking safety seriously, for pretty much any definition of "safety". I mean, it even points out that Anthropic has "the only top-tier models cleared for top secret security situations", which seems like a direct result of them actually giving a shit about safety in the first place.

And the whining about "the contract says we can't use it for surveillance, but we want to use it for good surveillance, so it doesn't count. Their definition of surveillance is politically motivated and bad"! It's just... wtf? Is it surveillance or not?

This isn't a partisan thing. It's barely a political thing. It's more like "But we want to put a Burger King logo on the syringe we use for lethal injections! Why are you upset? We're the state so it's totally legal to be killing people this way, so you have to let us use your stuff however we want."

gowld

5 months ago

> The policy doesn’t specifically define what it means by “domestic surveillance” in a law enforcement context and appears to be using the term broadly, creating room for interpretation.

> Other AI model providers also list restrictions on surveillance, but offer more specific examples and often have carveouts for law enforcement activities. OpenAI’s policy, for instance, prohibits “unauthorized monitoring of individuals,” implying consent for legal monitoring by law enforcement.

This is unintentionally (for the author) hilarious. It's a blatant misinterpretation of the language, while complimenting the clarity of the lanuage. Who "authorizes" "monitoring of individuals"? If an executive agency monitors an individual in violation of a court order, is that "authorized" ?

FrustratedMonky

5 months ago

Wasn't a big part of AI 2027 that government employees became overly reliant on AI and couldn't function without it. So guess we are still on track to hit that timeline.

SanjayMehta

5 months ago

So a private company sanctioned the US government? And now the US government is upset?

I do love the smell of hypocrisy early in the morning.

user

5 months ago

[deleted]

Filligree

5 months ago

[flagged]

docdeek

5 months ago

What would be an example of such criminal charges being brought by this administration? Is there a case that stands out as clear retaliation?

Edited to add: I can think of the mortgage fraud cases being discussed/brought against some high-profile people, but can’t think of any corporate world leadership being charged.

Filligree

5 months ago

I wasn't thinking of this administration necessarily. There's been cases going back to IIRC the early 00s, for instance Joseph Nacchio.

sandworm101

5 months ago

Saying no today also means you can say yes tomorrow. Then you are a hero, a dealmaker, as opposed to the "weak" who never put up a fight. This is schoolyard rules.

pineaux

5 months ago

Yes! Everybody goose-step in unison as to not irk the administration? /s

People should behave more like the invertebrates we are and show some semblance of a spine. Now most have more semblance with snails and jellyfish. Yes they will survive but only because there are so many of them.

stevage

5 months ago

This isn't a principled stand, it's just a negotiating tactic. They'll allow it when the price is right.

chatmasta

5 months ago

Are government agencies sending prompts to model inference APIs on remote servers? Or are they running the models in their own environment?

It’s worrying to me that Anthropic, a foreign corporation (EDIT: they’re a US corp), would even have the visibility necessary to enforce usage restrictions on US government customers. Or are they baking the restrictions into the model weights?

bri3d

5 months ago

1) Anthropic are US based, maybe you're thinking of Mistral?

2) Are government agencies sending prompts to model inference APIs on remote servers?

Of course, look up FedRAMP. Depending on the assurance level necessary, cloud services run on either cloud carve-outs in US datacenters (with various "US Person Only" rules enforced to varying degrees) or for the highest levels, in specific assured environments (AWS Secret Region for example).

3) It’s worrying to me that Anthropic, a foreign corporation, would even have the visibility necessary to enforce usage restrictions on US government customers.

There's no evidence they do, it's just lawyers vs lawyers here as far as I can tell.

itsgrimetime

5 months ago

Anthropic is US-based - unless you meant something else by "foreign corporation"?

jjice

5 months ago

> It’s worrying to me that Anthropic, a foreign corporation, would even have the visibility necessary to enforce usage restrictions on US government customers.

"Foreign" to who? I interpretted your comment as foreign to the US government (please correct me if I'm wrong) and I was confused because Anthropic is a US company.

chatmasta

5 months ago

Ah my mistake. I thought they were French. I got them confused with Mistral.

The concern remains even if it’s a US corporation though (not government owned servers).

toxik

5 months ago

Anthropic is pretty clearly using the Häagen-Dasz approach here, call yourself Anthropic and your product Claude so you seem French. Why?

mcintyre1994

5 months ago

According to Claude, it’s named after Claude Shannon, who was American.

astrange

5 months ago

But it might also be the albino alligator in the California Academy of Sciences in SF.

chatmasta

5 months ago

Hah, it was indeed the Claude name that had me confused :D

bt1a

5 months ago

Everyone spies and abuses individuals' privacy. What difference does it make? (Granted I would agree with you if Anthropic were indeed a foreign based entity, so am I contradicting myself wonderfully?)

jjice

5 months ago

Ah yes - Mistral is the largest of the non-US, non-Chinese AI companies that I'm aware of.

> The concern remains even if it’s a US corporation though (not government owned servers).

Very much so, I completely agree.

user

5 months ago

[deleted]

g42gregory

5 months ago

No judgement here, but a US-based corporation refusing services to the US Government?

While the terms of service are what they are, the US Government can withdraw its military contracts from Anthropic (or refuse future contracts if they don't have any so far). Or softly suggest to its own contractors to limit their business dealings with Anthropic. Then Anthropic will have hard time securing computing from NVIDIA, AWS, Google, MSFT, Oracle, etc...

This won't last.

LastTrain

5 months ago

I am of an age where I read comments like this with my mouth agape. It is (was) perfectly normal to choose whether or not to do business with the government.

Gigachad

5 months ago

A lot of abnormal things happen when your government becomes a dictatorship.

e_i_pi_2

5 months ago

I'm sure this sort of unofficial blacklisting is fairly common, but it does seem very opposed to the idea of a free market. It definitely doesn't seem like Anthropic was trying to make some sort of point here, but it would be cool if all the AI companies had a ToS saying it can't be used for any sort of defense/police/military purposes

g42gregory

5 months ago

I am not even sure what free market is, aside from Economics textbooks and foreign policy positioning. Whatever it may be, I don't think we had it for quite some time.

SpicyLemonZest

5 months ago

Your grandchildren will disown you when they learn what you supported in 2025.

g42gregory

5 months ago

First of all, no need for personal attacks. Second, I am not supporting this, merely stating the reality.

Would you rather pretend that things like that aren't happening?

SpicyLemonZest

5 months ago

There's absolutely a need for personal attacks. By suggesting that people ought to just give in and accept this as the new reality, you are supporting it, and you need to know that this represents a severe moral flaw. You're not just being pragmatic, or realistic, or whatever polite fiction you tell yourself to avoid having to take a stand. You're being a bad person, intentionally doing your part to make the country worse in the hopes that you can be a little more comfortable. Change your ways now or you'll bear the shame the rest of your life.