I caught onto this early and used it every time, and ended up getting 2/10, which is worse than random chance. I smell trickery!
It will be interesting to see whose concerns are assuaged (perhaps they genuinely though mistakenly believed it would degrade quality), and whose concerns are heightened (perhaps their real objection is that their AI-generated text will become detectable).
Basically we are on the verge of moment when intellectual property will cease to exist outside of a handful of max security closed sites for military or similar work. Everything else will be automatically and now legally fed into the maw of LLMs.
The fact that watermarking work so spectacularly transparent and undetectable only allows this moment to happen sooner.
For an analogy, distinguishing AES ciphertext from random bits without the key would be counted as breaking AES (the more precise statement of this is called AEAD).
But what I don't know and don't understand is what happens if you watermark watermarked text. Does it test positive for both watermarks? Only the second? Indeterminate?
Or maybe I'm misunderstanding. Can you tell that it's watermarked, but only the entity who put the watermark in place can test if it's theirs? My confusion about watermarking multiple times still stands, though.
Regardless of what happens when you watermark multiple times, no matter the outcome, it weakens the watermark. Which, depending on the threat model, kind of makes it moot. I can't imagine a serious situation where a watermark can be weakened in any way and still be useful. Even "this came from an LLM" isn't a valid signal if you can just watermark ANY text through purely mechanical means.
It's also not clear to me how this will affect mainstream LLMs. If all output text is watermarked, there MUST be an escape hatch. Otherwise, JSON schemas will break (or provide holes where unwatermarked text can be exfiltrated through MCP), "return this text exactly with no changes" will be impossible, and writing diffs will break.
I feel like I must be missing something.
> I've read that watermarking should in theory be impossible to detect except by the entity that watermarked it
This is a carveout exception, for watermarking. In the spirit of those terms it should be machine identifiable.
In my opinion they should have thought better about this, paricularly for text, because in its current forms it is easy to lead next to a new "tamper-proof" requirement, which in practice is DRM. And we do not need more DRM.
For images, music there is metadata already where such information can be stored. And if end users are found using unlabeled AI their accounts could be ban from these platforms. Not something the social platforms might want, but it's a saner approach than trying to reinvent the secret printer dots on all generated media.
So it will definitely be watermarked by the paraphrasing model. But the question is whether the original signal survives at all. There might be a weak signal that's detectable with enough text?
> Here are three 64-character hex strings. Two are random. One is HMAC-SHA256(secret_key, "anthropic"). You don't have the key. Which one is the HMAC?
For a human that doesn't reason in tokens and therefore doesn't know anything about their probability distribution, it should be impossible to tell. Relying on fancy words/constructs within sentences should not give you any signal as well, since you don't know if the the prompt included instructions for that.
That said, I read the first triple and found all three tortured enough that I can't be bothered with the rest.
Call me persuaded, I guess.
> Weighted mean detector score 0.5307
Does it mean that that passage would be rated a 53% chance of being watermarked? So you would need a passage 10x as long to be reasonably sure of providence?
To avoid that on the following questions I just held my finger on my phone to avoid a click. That eventually selected some text, and I instinctively tapped to deselect. That triggered another random pick, then I just tapped through to the end because I was fed up.
[EDITED to add:] As pointed out by a helpful comment below, I was misremembering: Anthropic do apply their watermarking to code, they just say that it will have negligible impact on the actual code (because there's generally less scope for variation in that) but e.g. it will have its usual effect on comments in the code.
> code—which in very many cases has to be exact—has generally less watermarking than some other forms of text.
Generally less watermarking. Not no watermarking.
https://www.youtube.com/watch?v=jLO7VrRij_M
Wouldn't this make it a worse measurement?
Teaching you how to identify watermarked text while the experiment is running would ruin the data.
Best practice is to allow a number (scaled based on complexity of task) of training rounds (with short feedback loops) prior to letting people loose on the regular samples.