I'd like to know a lot more about how that works.
A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know exactly what I want to happen, it will NOT be OK if those refactors have some kind of weird pattern baked into their text to act as a watermark.
I guess this may be covered by this:
> Content generated by Claude may not carry a detectable mark if, for example: [...] The passage is very short, leaving too little text for a reliable signal;
My guess is that it works like Gemini's SynthID: by altering the logprobs of the next token.
Like, for every 10th token, instead of outputting the most probable, it outputs the 17th most probable, or something. (Obviously it's way more complicated but I think conceptually this is how it works.) No human will notice this, but a classifier trained on Claude's output will.
So it's not like the watermark is the words "le epic bacon" and Claude will output "le epic bacon" in everything. That would be extremely annoying (and easy to defeat).
Odd variable naming? Stylistic choices that are watermarked?
Or as someone else noted further down in the comments, it could be more subtle:
Between the first and second most likely choice, in certain positions it will consistently choose in a certain way.
Whatever it is, I'm sure it's load-bearing.
https://www.pcmag.com/news/genius-we-caught-google-red-hande...
Count load-bearing words using two different algorithms in a belt-and-braces fashion
The mechanism seems to survive editing. The extreme probabilities get a little less extreme, but are still extreme enough to be distinctive.
But it wouldn't survive paraphrasing, because the output would be entirely human and the token correlations would disappear.
It might not survive referencing if only a sentence or two is used.
The practical issue is how true the claims are. It's one thing to create a proof of concept, another to see how it works in use.
And this is potentially catastrophic for code, because the grammar and word choices of code are completely different and more fragile than standard English.
Similarly, if you quote someone word-for-word, you wouldn't anticipate their words to be flagged as Claude content, but if someone memorized Claude output word-for-word. That would still be classified as a Claude output.
Going forward you could categorize the influence of Claude on a population based off a percentage match between their spoken words with the LLM prose.
public abstract class BaseAnimalBeanFactoryGeneratedFromClaudeFactory
The bias is different for each position and follows a defined RNG, seeded somehow predictably.
Can be either an open algorithm, or not. If not open, then an API could be provided to determine if text is watermarked or not.
How it applies to code - maybe it could be a subtle nudge to symbol names, etc, I'm just speculating (I only read about this in passing very recently).
Gotta be hard to tune that.
Computerphile on YT has a video explaining how models can fingerprint the text they produce. Essentially they modify the probabilities of word choice slightly in a predictable way.
It’s just not a reasonable ask.
Thus, if a news article, research article, book, student paper submission, blog post, HN comment, etc, bears the mark, it could be automatically flagged as such.
It helps detect low effort slop.
---
Caveat. If you write your own creative work and send it to Claude for "cleaning up grammar", it might insert the watermark.
It seems it would get as simple as:
outputText = promptLLM(prompt)
scrubbedText = scrubWatermark(outputText)
That's not necessarily the same thing as a markov-style fingerprint but it could be a correlated factor.
If Claude was the only model family they could ship a change like this and users who want to cheat (or don't like watermarks for other reasons) would just have to put up with it.
In a world with many different competing models, the risk of losing customers to other providers over this is much more real.
Maybe they've looked at the numbers and the portion of people who clearly use Claude to cheat on examples etc is so tiny that losing them to other providers isn't a problem?
I guess whoever is the policy maker is assuming that some protection is better than none and that most people will not reach for such tools.
Given that text is, well, text, and not some kind of binary format, I don't see how any watermarking can work unless you insert characters which are invalid under Unicode. I further don't really understand how this won't be perceivable by assistive technology (the "watermark" will just appear as either unreadable characters, or if the watermark is mixed thoroughly enough into the text, it will scramble the text to any speech synthesizer and will make it really really obvious). Thus, I don't see how this wouldn't be insanely trivial to remove. And this is before we get into things being put on the clipboard. Sure, I can press the "Copy" button at the end of each response, but what I can also do is manually select the response and copy it, or only copy partial selections, or any number of other things. How does this "watermark" (or any "watermark" technology) take into account this?
So, really, to summarize this: I see no way of this actually being technologically achievable unless we revise the very core of how computers work and encodings for textual information. So I'm very curious as to how this is actually supposed to work.
At first I thought this approach was just the "LLM flavour" of writing, but it's way more subtle, especially as the bias is applied uniquely for each token position.
you can then consistently like figure out if it was claude that wrote the sentence. it is easy as you noted if you just get another ai to read it and then rewrite it.
This seems to be similar in execution to Google's SynthID. I hope they release actual code the technically proficient can use, unlike SynthID which can only (afaik) be queried with Gemini's UI.
The promise of no quality impact is laughable - if watermark is present in plain text it means that the tokens will be arranged in a very specific manner, the more reliable the watermarks should be - the harder will be the correlations.
Don't forget how annoyingly bad Anthropic products have become in recent releases - low adherence, annoying alignment, annoying guardrail false-positives, unwarranted checkpoints - all that shit. Now they deliver more crap.
This should make it easier to catch cheaters who use Claude, right? Unless everyone runs their artifacts through some watermark and metadata sanitizer?
> Regions. Marking will apply to output from supported models wherever Claude is offered, worldwide.
It will happen if Claude tampers the text. Guaranteed.
One example I've seen are junior employees at my company deliberately adopting a lowercase/less punctuation writing style so as to stand apart from AI.
No Anthropic model has been launched in August.
I am cancelling my Claude max 5x subscription and moving to ChatGPT pro. I have difficulty enough trying to ensure my meaning comes through correctly, along with everything else; to now have to look out for/analyse watermarks too?
I feel shamed enough by society, thanks Anthropic.
Unless with "proofreading" you actually mean having the LLM write your content for you.
Can it be circumvented? Of course. Will most people go through the trouble to circumvent it? No.
Hell, it'll probably happen no matter how sophisticated their watermark is. There's no watermark in text that can't be detected and removed, and no text that can't be converted to generic keyboard ASCII.
U+2800 or U+3164 would be nice.
But as I remove unwanted characters with grep before layout in InDesign, someone will make a skill for removing such space characters.
EU regulation does it again!
"When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response.
Because the watermark is part of the text, it will travel with the text when it’s copied and pasted elsewhere, and may persist through some editing. Watermarking will be applied at the model level, which means it will be present no matter which Claude product or surface the text comes from."
- "Ensure distribution of vowels is in >99th percentile of human work"
- "Ensure the distribution of the letter "s" is within 99th percentile of human work"
- "Ensure the distribution of the letter "L" is periodic with periodicity within 5% of 1/N characters.
- "Ensure there is a cross-linguistic 'typo' (colour vs color) at 1/N words, where N: 1000 = Model1, 2000 = Model2, 3000 = Model3.
- "Ensure the distribution of tense error is within 99th percentile of human work"
If more than 3 dimensions have a score >99% percentile of human, let's call it watermarked...
Yes they can do this, but it's more likely closer to the original "red token, green token" paper: https://arxiv.org/abs/2301.10226
i.e. take half of your LLMs vocabulary, and upweight its probabilities by ~55% to the other half's ~45%, and scan for overuse of this half of all tokens. You can even choose a different half/slice for every individual user, for every individual action. You can implement this under the hood cheaply with logit-biasing.
That's probably an over simplification. Also a solid defence that can be used against complaints about the way AI writes text.
They should make it easier, to detect slop so we can ignore it quickly.
I hope Pangram makes an API or an extension to analyze a page to detect slop on a page and then closes the tab immediately.
Nobody should be wasting time on garbage LLM output in code, text, image and videos.
https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...
What about on Pangram 4?
Is their research also a scam too?
https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...
https://www.pangram.com/blog/pangram-4-technical
If so, what is the best one out there other than Pangram then?
Today, the best way is probably Pangram. Tomorrow, it might not be, especially if they try to push their recall up.
You might have to make peace with the fact that there may not always be a tool that does what you want.
Thanks!
> But the burden of proof is on them...
I mean is this enough proof?
https://www.pangram.com/blog/pangram-4-technical
https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...
Or is this marketing, a public stunt or not real research?
I think this is enough for me to know they are actually improving their AI slop detector.
Keen to see if they are doing something SynthID-esque?
What about "watermarked long-form code"? I'm having a hard time understanding how a model could watermark not prose, but functional/semantic text like code, that actually has meaning. You can't switch our the characters, you can't use various types of whitespace, you can't add arbitrary code comments, and a lot of other restrictions. Is there any state of the art methods for watermarking code without affecting the quality/correctness?
The way I use LLMs (and I'd advice everyone to do the same) there really isn't, the agent implements things exactly how I want them, or I use the agent to massage it into the exact bit-by-bit version I imagined when I first sent the prompt afterwards. I honestly don't know what the point would be to let the agents compose worse code than what I'd do manually, although I know it's a popular approach taken by many.
> And of course you can add arbitrary comments; my Claude-generated code is very verbose.
So watermarking for all users who allow code comments from agents, no watermarking for us who force the agents to never write a single code comment? Alright, I'd be fine with that.
And the reason to let Claude make worse code than a professional would by hand is basically suppressed demand. Since programmers are expensive, previously code mostly got written when a large number of dollars were on the line, or when an individual programmer did something not economically optimum (e.g., hobby project).
That left a whole lot of somewhat less valuable software unwritten. It's the economic space that no-code tools have been nibbling on for years. One way to think of things like Claude Code is as effectively no-code tools. Pre-LLM no-code tools would produce data structures that got executed by special environments without ever being seen or tuned by a human. Claude Code can be used just like that, with text as the input and python as the intermediate representation that nobody ever looks at.
That approach probably isn't sustainable for what we professional programmers would call a serious project. Claude can easily get in over its head and I expect that its code decays over time, in a fashion similar to how many human teams get in a state where they just have to rewrite everything. But faster, I'd expect.
But there are a lot of unserious projects that previously would have never been created. E.g., a quick app to manage your little league team, or a bit of in-house business stuff in the "a little hard to do with a spreadsheet" range.