ghrl
an hour ago
My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ...
So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illicit means.
Any university using AI detection in their submission pipeline, or lawyers, editorialists, proofreaders that check for AI marks will be sending significant amounts of text like unpublished research, books, potentially internal documents and more, most of which is high quality human written, to dozens of AI companies, blindly trusting they won't train on any of that.
chii
32 minutes ago
> blindly trusting they won't train on any of that
being allowed to train on any data that you can legally obtain ought to be a right for anyone.
After all, i am allowed to learn off anything i can legally read (and perhaps even illegally read). The only thing not allowed (rightly so) is to produce a copy with enough similarities that it can be replacing the original.
saghm
a minute ago
At the risk of stating the obvious, there are a lot of legal rights that are human-specific (voting, holding office, filling lawsuits, etc.). It's not at all obvious why you think that you as a human being legally allowed to learn from something implies that it should be legal to train an LLM on.
bonzini
22 minutes ago
It's a bit different when "training on any data" means basically storing a lossily-compressed copy of that data, that could be spit out years later if the model decides to do so.
vanviegen
13 minutes ago
> The only thing not allowed (rightly so) is to produce a copy with enough similarities that it can be replacing the original.
But LLMs are replacing the original, just in different words.
And what does 'legally obtain' mean in this context? Copyrighted content is usually licensed for specific purposes. So if a license is given from training your LLM, then by all means do! But what if the license is 'for personal use'... ?
Gud
27 minutes ago
Why would that be a legal right?
Why should we hand over even MORE power to the owner class?
In a fantasy world this could be possible yes.
juggle-anyhow
14 minutes ago
Make it a right, then companies/universities will think twice before using said APIs. Instead of this grey area where we will never know.
rcxdude
20 minutes ago
Copyright (or any other such restriction on free use of information) creates power for owners by the simple fact that it turns information into something that can be owned.
m12k
19 minutes ago
We don't hand over more power to the owner class by making fewer things ownable.
josephg
39 minutes ago
I think this is a very real concern. But I’m not sure of any way around it. Any stenographic system that you have the code for can be trivially defeated.
I wonder if this would be a good use for homeomorphic encryption. There might be a way to let anthropic check some text without actually giving them access to the source text. Any experts around? We could use your skills!
zrm
11 minutes ago
> Any stenographic system that you have the code for can be trivially defeated.
They're giving you an oracle regardless, which is almost as good. Take LLM output, make some modification, ask the detector if it's LLM output, repeat until you learn what kind of changes you have to make to defeat it.
Or don't even bother learning what to do, just make arbitrary changes until it says it's not, so when the person they're submitting to does the same check it says the same thing.
piker
37 minutes ago
Won't we just be able to fine tune OSS models to detect these patterns across providers? It will be cat-and-mouse but my bet is it converges to a central detector that isn't affiliated with any model provider.