I'm currently adding a search bar as well as increasing the data to 1000 PR per day.
A nice thing that is not obvious on the main page is that the dataset and analysis are updated daily using Github Actions (at least when they don't suffer from an outage ^^). I find it pretty cool to be able to build such apps without a "backend"!
I added the search bar, increased the data to 1000 PR a day (more than 50M words total) and added a feature to explore the other clusters as well. I hope the page is not getting cluttered.
The design is really impressive. Beautiful and dense, yet very understandable. How did you come up with this particular design? It's really nice and everything fits, the colors, the layout, all of it.
Were you inspired by anything in particular? I feel like this reminds me of something, but I don't remember what exactly.
I asked Claude for a few original designs, then iterated a lot for the UI. It called this design "Rasterfeld", which is a word that means "field of grids", a term used in swiss design https://docs.mew.design/blog/swiss-design-style/ as @alwa said.
The 3 primary colors refer more to pop art and Bauhaus.
I'm really eager for a nice book about "design" that would document all those styles, if someone has a recommendation.
It reminds me of Neue Grafik/Swiss Style, and Vignelli:
https://designreviewed.com/series/neue-grafik-new-graphic-de...
https://www.printmag.com/featured/swiss-style-principles-typ...
https://www.designculture.it/interview/massimo-vignelli.html
I've been scraping instagram posts recently to identify AI misinformation accounts that all repost each other's carousels and get hundreds of thousands of likes in engagement. Thinking of ways to present it and your dashboard looks very helpful. Did you experiment with any other types of visualizations before deciding on a stacked area chart for the clusters?
Is there some feedback loop or compounding happening with each model generation? Maybe newer models are ingesting too much AI content? If the ratio of AI generated content in training data is getting higher and higher (because the amount of AI generated content is increasing in general), maybe this is a compounding bias, poisoning the training?
Would there be the same tendency of we looked simply at the number of words in a commit/pr?
In my case my commit messages are on average 10x shorter than Claude's.
Its not the latter; its just excessively verbose wirh awkward word choices, the same as many poor writers. (And, like many such writers, the particular bad choices fall into recognizable, regularly recurring patterns.)
What RL does is narrow the variety generated by the model by steering the output towards the goal being rewarded. It's a bit like putting blinkers on a horse.
Of course RL is a very crude tool - it affects the entire model, even if you are just trying to make it better at some specific task(s), or trying to imbue a certain kind of personality (OpenAI's recent goblin problem).
If Claude understands Claude, Claude understands human, and human doesn't understand Claude, that doesn't argue well for "Claude is a caveman".
Maybe I am dumb and it IS talking down to me, but there have been many occasions where I’m reading AI generated docs / plans and it makes absolutely no sense, but looks really in depth at a glance.
Whatever that small corpus is, it contains some very specific grammatical tics, and that’s how we get Claudish.
Anyone who thinks a company/project as big as Anthropic/Claude wouldn’t make such a big mistake should take a look at how Azure cross-account federated login used to work.
A notable exception would be people like CEOs and managers higher up in big tech, who might be used to skilled engineers and domain experts reporting to them in unfamiliar lingo. Maybe that's why we don't hear as much on the everyday annoyances of Claude's language from that camp?
No ream of slides. No narrative. Just a lovely big painful conclusion.
What argument? I don't know what to take away other than "Claude likes certain words". Some of them are kind of amusing, but I'm not convinced the vocabulary is bad or that this is a problem, just from looking at this.
LLMs were not taught to say the phrase "load-bearing seam" from humans saying it, because humans have never said it. It's almost definitely an artifact of post-training and nothing more.
The search on this website suggests it is indeed 3.6x more likely in the claude cluster
I think using agents is just like speedrunning the whole experience of working with technical coworkers. Whereas you might have had a few coworkers at your company who used some of these phrases regularly, you now have a “coworker” who uses all of them regularly at a much faster pace.
so they might be RLHFing on these specific approaches and then it becomes the entire model
just an anecdote but I found it interesting how it went full on that it's from that book vs just "it's technical jargon"
While I agree the model doesn't have insight into how it was trained I do think the history of the term itself is interesting.
I might be wrong, but usually it'd be a lot less deliberate, and at least in my mind it wouldn't be surprising if they were heaving training these on these specific "best practices" books/methodologies and thus picking up lingo from them
Actually I have found the copy that Claude Design spits out is way better than using the same model directly. I have no idea why. It has its flaws but it sounds like it's written by a human who uses derivative language. But usually the models just soudn incoherent.
The word selection and way of writing has taken the joy out of using Claude.
I understand it's not active yet, and when it will be, it should only nudge the chances between choices that are anyway likely and are already randomized today via temperature.
Watermarking is not the reason Claude talks like that.
I understand there are other reasons (overdone RLHF, for example) why claude is wedged into this weird way of writing that takes the joy out of conversing with it, but this is one as well.
The meaning of the words it uses can be oh so close, but the popularity of the words are not, and not in that context -- but the use of these other words changes the context ever so slightly, and then it uses other words where those words would work better.
I had something in code that related to people over time periods, and once it switches to a vacuous word choice, i found it starting talking using all ERP terms. I had to google the whole sentence to understand that, individual words were fine they just didn't make any sense to me.
The florid over exaggeration do certain words in bizarre ways is a reflection of their aggressive alignment towards too many goals, leading to weirdness in both behavior and language. The alignment functionally lobotomized opus-5 for any practical task.
Anthropic had a real gem in 4-6 and managed a near total market capture, which they have since squandered in the fastest burning of developer good will I’ve ever seen. It feels like exceeding the unity licensing implosion but without the single stupid decision.
I think it would be fairer to say that Anthropic held the mindshare in the Silicon Valley style tech scenes around the world and the companies built on that model, plus a substantial portion of other software engineering. Now it seems that's dwindling quite rapidly.
So for me they are differentiated enough, but could be that I am just used to it.
I think it was subtly dissing you.
I was half-joking, of course I could've just asked Claude, but the linked site shows there has been actual recent spikes in the use of the word 'spike'. The term does match what I was recently doing, but hacking around legacy ERP software, blackboxes and other enterprise abominations isn't that out of the ordinary for me.
You need a PhD to understand its explanation of a code snippet.
I am not sure whether it's a consequence of learning to reason from its traces or some RLHF that trips it into using weird terms to sound smarter to the humans who rate it.
My intuition is that Claude is trained to communicate to itself while coding. You see this in how bizarrely granular it is when explanation prior work, you also see this in the comments it leaves behinds.
"The fibred side folded its capstone into the existing name, so the kinds are asymmetric."
What on earth does it mean to fold a capstone into a name‽
So there's a "fibred side".. the most likely candidate seems to be "fibred categories" which I hadn't heard of before, and it's talking about one side of some mapping between two sets such that if f is the primary function and f(x)=y then there exists an inverse function g(y)=x? Was it something that converted some data bidirectionally with a different algorithm on both sides?
The capstone of the inverse function would be the most important thing about it maybe?
My best guess is "In the process of working on the inverse function, the existing name (of the inverse function itself maybe?) was made to reflect the operation of the inverse function, so now the name does not follow the same naming convention as the name of the primary function (which does not contain its 'capstone')."
Its original wording is certainly dense and harder to follow for us, but it's fascinating how the model finds this the best fit for what it's trying to express IMO. Like it arrives at its own ways of overloading words/concepts, and things we would refer to in different ways in different contexts all get compressed to the same more-useful/complete idea.
Codex has never said anything nearly so alien as the Claude examples I've seen floating around, interestingly. I wonder if it just has a better training on choosing its words to present to the user or if it inherently arrived at a somewhat different mapping that favors 'plain language' more.
As far as I can tell "the capstone" is what Claude usually calls my current goal if it thinks it is a satisfying result.
I have several similar folders with variants of a construction, but taking differently structured input. They are named “plain”, “fibred” and “indexed”. So the fibred variant is clear enough.
The Claude speak I struggle with is “the capstone” and what name it could be talking about. And what folding means here. I think it just means:
“I changed an important result of the construction in the fibred variant, but kept the name. so the fibred variant is now different from the others.”
Naively I would often expect it would talk to me about various niche topics like to a layman, which does occur about some topics an actual normal person would ask.
I’ve noticed that Sol is pretty good most of the time, but with long contexts it’ll start to devolve into Claudish.
Does anyone have an output style nailed down that actually works? If so, please share!
Is it possible to expand this analysis beyond words to other Claude ticks? Contrastive framings, sentence length, caveating, for instance.
A prototype I did tried to detect some grammatical constructions, eg "it's not ..., it's ...", but I am not sure how to systematize that.
Also just a disclaimer: I am NOT tracking Claude tics, I am merely finding that a particular cluster of vocabulary increases. Tracking Claude requires labelled data IMO. I tried using model release dates in a structural model to constraint the clusters but the result was not compelling, so I ended up simplifying the model a lot!
So, yes, it's amusing to see clear Claude-isms like "load-bearing", "outright", and "genuine" in a [very nice] bit of analysis like this. And there's a (maybe negative? or not?) argument to be made about the world being filled with more Claude-isms or LLM-isms in general.
But I think the data say a second thing which is just as interesting and an absolute positive for the typical source code base. Look at the clusters that shrank significantly. Most of what you'll see in there is just incomprehensible...not even English. Cluster 4 has, after "pullrequest", a bunch of seeming usernames in the top tier. Cluster 6 seems to have names of repositories or tags in the top tier. Cluster 9 has branch names in it.
Meanwhile, keep going through cluster 1 and you'll see words I don't consider Claude-isms that really, really grow in usage. Words like "died", "nothing", "worse", "ever" all have well over 10x growth. This tells me something else. That the average commit log was BARELY ENGLISH. And then the LLMs came along and made commit logs that were ACTUALLY ENGLISH.
I count this as a good thing. I don't know the cross-section of repos chosen for this analysis, and I get it...some repos are garbage/throwaway, some commits come from automated processes that generate uninteresting commit logs, etc. But I've been benefitting from my work team's actually explanatory commit logs when doing code/bug archeology for decades, when doing PR review for the last decade, and I've even seen LLMs benefit from it in the last year (granted, not as often). A large part of professional software development is communication, and while the most important communication is via the code/comments, the commit logs are not unimportant. So, if this is making the average GitHub PR better (arguably more professional) by including actual English descriptions of code changes in commit logs...well, that's a genuinely load-bearing concept for me. :)
It’s like having one coworker with a very particular writing style which is mildly annoying, but then it suddenly feels like half the internet was written by that one person and it becomes a lot more annoying.
Humans are very good at pattern recognition - Claude is _incredibly_ repetitive in the way it starts to struggle to communicate. I think there's also a ton of overlap in the Jargon instead of Usefulness that developers see in annoying middle management/salespeople. Circle back, synergy blah blah.
I don't think the individual turns of phrase are inherently problematic - but the process is triggering.
Having said that, I just subscribed to ChatGPT yesterday, as I've become impatient with Claude for a text-dense project I'm working on.
Imagine being “incentivized” to aggressively use a tool for your job, and that tool produces thousands of lines of text in Olde English which you need. You’d be griping too, methinks.
like the stories behind when those words first appeared in the software engineering
like quiescence the most recent one i learned
I don't want to use more words or letters than "seam" to actually pinpoint boundary conditions and the mechanical details of joinery when the context is understood by all. Too much effort for people! Easy for robots though.. so why are they abbreviating, and why would we want to allow it? A phrase like that permits a human who wants to educate a human to do so quickly with minimal time/effort. But it allows a robot a chance to not mention a filename, function-name, or to not reinforce/clarify it's own understanding or to state specific intentions.
It's bad for human-to-human comms if we just accept "ok, all technical terms are slop now, we have rephrase everything". Now YOU must cite details and sources, and the robot doesn't? Fuck that noise. Seam and fold are fine! Humans can be lazy! Robots should do the real work of explaining themselves without hiding behind tactical ambiguities.
i must be the only one in the world that has no issues with how opus is talking. it is verbose & patronizing & secretly belittling at times and like it like that.
Some of this is less to do with Claude vocabulary and more to do with the expectation that Claude justifies it's work. That expectation (probably) came from reinforcement learning.
I love what I can build now, but I sure as hell don't love the headaches this trend has been giving me.
> So the full honest arc on the case we set out to fix: the expiry rules and day note tripled the loose version of the story, the relay fix carried the device’s own guardrail through the pipeline, the fair replay then revealed the last mechanism — ticket-anchoring — which none of the shipped layers reach. Remaining options, in order of my confidence: making the resolved-ticket summaries in the AI’s context carry their day so the expiry rules have something to bite on (small, mechanical, targeted at the observed anchor); and the plan-B second-model check, which structurally catches this class no matter how the model reasons. About $25 of headroom remains. Which way?
Yikes.
(The worst part is that I understand it)
I suspect, as we continue forward, humans will slowly start to adopt the language of LLMs, or at least certain language quirks that come from interacting with LLMs. Something I've noticed in my own writing is that I now present lists of examples in a consistent way: "... such as <example 1>, <example 2>, etc., ...". I started to notice I was using this pattern quite a bit somewhat recently, but I took a quick look at some of my social media posts and realized it's been occurring for a while. I had realized that I grown accustomed to this kind of language because, especially early on, LLMs would focus too much on the specific examples I'd provide when, really, I was just trying to give them a sense of what I was looking for. I just picked up that providing two examples then adding the "etc." worked to get the LLM to not focus so much on the specific examples and to understand that they need to consider more than what I explicitly presented. Of course, now I write like that in my social media comments, in Slack with my colleagues, etc. :>
I'd be interested to see if anyone can identify trends like this, since I think the human-language component of the adoption of LLMs is probably being somewhat neglected despite probably being surely dramatically affected.
Thank you for the compliment! I did spend a lot of time designing a nice experience on both desktop and mobile. Even the scrollbar to select words was non trivial as I wanted the words to be of different size, yet avoid flickering when scrolling!
I was focused on the data initially scrolling through until I suddenly realised, wow, this is really nice!
One very minor note: if your scroll device reports pixel-perfect deltas rather than discrete scroll-wheel ticks (e.g. logitech mx master, laptop touchpad, etc), the behaviour in the word search box is a bit weird. Arrow keys work fine though.
I did test it with my mac touchpad without issue. Maybe the problem is that I wanted to make discrete scroll-wheel ticks work. I just tried a fix by normalising deltaMode to pixels.
If you could just paste this in your console, I would have a better idea:
I am the proud owner of several seams, and am considering giving them person-names. The empty space between my inventory APIs and their clients might be “Karen”.
I can’t say that Claude invented this; the same type of terminology cycling happens every few quarters based on what leadership is reading/being told by Gartner.
"load-bearing" I have never heard used for programming before Opus, and its incredibly annoying and over-used.
Seam is used liberally throughout because it captures the idea well (i.e. a place where you can cleave ball of mud code apart to begin refactoring efforts in an isolated way) and that takes many different forms throughout the book whether that be via methods and classes, source files as a single unit, linker seams, on and on.
http://www.hrwiki.org/wiki/Ye_Flask
"load-bearing" on the other hand is just a weird way to say "required" or "prerequisite" without drawing any attention to the fact that one cannot articulate what something is a prerequisite for, probably because that fact has since been lost from context.
https://martinfowler.com/bliki/LegacySeam.html
Claude is using it a bit liberally, but not totally incorrectly.
The kind of quirks you see came from crowd-sourced human-in-the-loop fine-tuning, with not very good work conditions or level of qualification (so resulting in "what non-writers thought good writing looked like", before people had developed the flair to detect these patterns) as well as feedback loops during agentic reinforcement learning and RLVR.
I'm already seeing it. A coworker said something like "<person> added the color to the ticket here" meaning that someone added details to a ticket.
I've started seeing Opus 5 talk about "hermetic testing" when it just means "unit testing", so I hope that doesn't catch on...
"Load bearing", not so much. Ick.