Claude also loves to describe things as being "real", particularly saying "X is real".
In this case,
> The reallocations were real, but they were never the bottleneck.
There was never any indication or setup in the text that they weren't real, but it's how it justifies wasted effort, it insists that some phenomenon it corrected but failed to solve the problem "was real".
Another giveaway are nonsensical analogies:
> The predictor is like a barista who starts making your usual order the moment you walk in. If you are a regular, this is fantastic: the coffee is ready when you reach the counter. If you order something random every day, the barista keeps pouring drinks into the sink.
If you order "something random every day", then you don't have a usual order for them to be making, it's an analogy that doesn't work.
And of course, the smoking gun is:
> The smoking gun
It probably won't be a good indicator forever as it has been noticed so much, but it's a particular favourite of the current generation of anthropic models.
I wouldn't be surprised though if (especially) non-native speakers unconsciously start adopting the Claude writing style when they stare all day long at Claude generated text at work.
An awful lot of the open-weight models also talk in the Claude-y style. Not sure if an artefact of distilling anthropic models, or just a preponderance of slop in the training set...
That's one of the main tells that AI wrote this. All the stylistic tics that people usually point out combine to make the writing seem more important than it is.
I find it really annoying when the LLM says “good instinct” as if I’m an animal barely able to think.
I'm glad my internal AI detector doesn't win over my curiosity to learn.
Using a highlevel language construct like "y += (x > 0) as usize;" doesn't "switch on" branchless code just because the source code looks branchless, compilers are not that dumb anymore.
E.g. I bet that writing
if (x > 0) {
y += 1;
}
The only reliable way is to mostly bypass the optimizer via simd intrinsics, or drop down to assembler, everything else is just cargo culting.
(fwiw I can't shake the feeling now that the article is recycled, I'm pretty sure I saw those exact same code examples in another "branchless" blog post, but maybe for a different language - because the next question was ineviatably "then why is the code using "if" slower? answer: because it also behaves differently). Or maybe I'm just having a strong dejavu ;)
> fwiw I can't shake the feeling now that the article is recycled
Ok, I remembered wrong. The article I remembered was this: https://tiki.li/blog/blqsort
HN link: https://news.ycombinator.com/item?id=48375445
It's peddling the exact same myth though.
The blog posts from 2010s are in a completely different style and written by a human: https://www.greyblake.com/blog/vim-preview-plugin/ https://www.greyblake.com/blog/how-to-install-firefox-icewea... https://www.greyblake.com/blog/unexpected-ruby-behaviour/ ...
This new blog post is clearly AI edited (probably 'improved' with AI), the old ones are not.
[0] https://www.greyblake.com/blog/vim-preview-plugin/
[1] https://web.archive.org/web/20220516225844/https://www.greyb...
It's "fake corporate enthusiasm" style. LLMs were just trained in it.
Something like that anyway. There are some very clear AI tells (smoking guns if you like), but most of it does not read like the prose AI produces by default.
Author if you are here I am curious about your writing process, and why you didn't remove the obvious AI tells.
I guess that's fine, but after awhile I get a spidey-sense reading something that feels like a Claude session.
I'm moving on to evaluating articles with a modified lie detector test and tarot cards, I'm sure that'll help my credibility and give my public rejections more authority.
1: https://bfi.uchicago.edu/insights/artificial-writing-and-aut... 2: https://arxiv.org/pdf/2501.15654
- "The reallocations were real, but they were never the bottleneck."
- "Note that the villain is not the branch itself. It is the branch that [..]"
- "Same million floats. Same threshold. Same function."
- "Notice the price we paid though."
Interesting topic but why destroy your own credibility and reputation by shoveling llm-assisted slop to us here at hn?
The post should be flagged, and in general, i wish hn would adopt a no-tolerance policy to enhanced posting like this.
So what if the original text, if it existed in a human written form at all, had weird textual quirks and prose issues the author wished to hide. That texture's what makes humans interesting to engage with in the first place.
https://github.blog/engineering/architecture-optimization/do...
Discussion: https://news.ycombinator.com/item?id=49127983
I didn't know about branch prediction or pipelined CPUs back when I was profiling the code I wrote - honestly it probably would have helped.
I do wonder, would the performance characteristics of branchless vs branching be consistent across different CPUs/architectures? If you had a CPU that wasn't trying to be fancy with branch prediction, would the regular algo be faster?
If you want to treat the CPU as a black box, trust me you do not want to use a CPU with out a branch predictor, your slow code will run like molasses frozen in antarctica.
The regular algo will be lightyears slower on any CPU that does not have a branch predictor.
If you're running on a very old CPU, yes, the regular algo should be faster.
PowerPC 601 arrived at more or less the same time, and the Alpha 21064 was a year earlier. There were a few minicomputers and mainframes before that with branch predictors.
Arguably the 486 could have done with a branch predictor (even a single entry loop predictor would have helped), and maybe the 386 too. But microcoded CISC designs didn't benefit much from predictors because they have multiple cycles to work it out.
And RISC cpus were in their "branch delay slots are awesome" phase throughout most of the 80s. With a bit of trickery (very simple branch conditions and a 2 phase clock), your classic 5-stage MIPS design can fully hide all branches with just a single branch delay slot, so they were a little slow to adopt predictors.
I get the impression that CPU designers in the 80s and early 90s massively underestimated just how beneficial even a small predictor can be.
Before dynamic branch prediction, where the prediction for every branch is updated after each branch execution, depending on its history, static branch prediction had been used for decades, since around 1960, typically using the rule that forwards branches are unlikely to be taken, but backwards branches are likely to be taken. An alternative was to have an instruction bit where the compiler stored its prediction about the probability of a branch being taken.
Dynamic branch predictors began to be used since the mid seventies.
I do not remember now if any notable monolithic CPU had a dynamic branch predictor before Pentium, but prior multi-chip CPUs certainly existed.
It's got a lot to do with how cpu clock speeds were getting way faster, but ram wasn't. That's what makes deeper pipelines attractive, and if you give a cpu a deeper pipeline, it's gonna want a good branch predictor.
They were shipping the high-performance R4000 and R4400 with 8 stage pipelines and no branch predictors.
They could have really done with a branch predictor, each branch took three cycles (and the branch delay slot could fill only one instruction, often a useless NOP).
The Pentium only had a 5 stage pipeline and massively benefited from its branch predictor. IBM was slapping branch predictors on PowerPC designs with 4 stage integer pipelines. You simply don't need a long pipeline to justify the benefits of a branch predictor.
I say this as someone who is interested in a special type of processor architecture that has no branch prediction at all and would need a branchless subset of Rust to meaningfully program it at high performance.
Or am I missing something?
I note the hazard 3 on the pi Pico rp2350 only predicts a branch if it's the last branch and was taken, ie a single loop. Which seems weird to me, so I'm probably lacking understanding somewhere.
They have a stupendously fast version and it is also branchless, but it just required more than branchless alone.
I'm fuzzy on the details but I think one of the ifs was an early exit, and without that the loop does a memory assignment on every byte instead of skipping most.
The really fast version was also vectorized. The branchless makes it possible to vectorize, but it was the vectorization that actually made it fast.
When a branch alternative will be taken much more frequently than the other, then branched coding with an "if" becomes superior.
So neither is better in general than the other, whenever the program must choose between alternatives, you must think about whether one is more likely than the other, or if both have similar probabilities.
For instance, when sorting an array, the optimal algorithm is not the same when you expect the input array to have a random order and when you expect it to be already almost sorted.
If the branchless code didn't transform to vector instructions, it would be strictly slower. But if it does, it allows the CPU to work on 16 bytes at a time instead of 1 at a time.
https://github.blog/engineering/architecture-optimization/do...
"Oh, this was mostly already sorted, done"
If you meant exactly rather than almost then you can still squeak a small win from having an algorithm which is optimised for this case but the vast bulk of your runtime is eaten by the unavoidable work of checking. "Don't check" is faster but then you're not a sort algorithm at all.
I laughed out loud reading this. Interesting writeup. I wonder what kinds of tricks like this exist for computation graph compilers like JAX.
(def find-shared-gcd
(comp
(fn [e] (max ;(map (fn [d] (* d ;(map |(- 1 (min 1 (mod $ d))) e)))
(range 1 (+ 1 (min ;e))))))
|((juxt* max min) ;$)))
(defn max-diff `where elements increase` [& numbs]
(reduce max
-1 (filter |(< 0 $) # strip 0s and add -1 in case (= true (apply > numbs))
(map - numbs (accumulate2 min numbs))))) The worst case became almost 4 times faster. And look how flat the branchless column is: the running time does not depend on the data anymore, exactly as we wanted.
Notice the price we paid though. At 1% kept the idiomatic version wins, because an almost always correctly predicted branch is nearly free, while the branchless version always pays for one million writes. Branchless code is not faster in general: it trades the best case for the worst case.I discussed this with a coworker earlier this week and the best they were able to come up with was
for &x in input {
out.push(x);
n += (x > threshold) as usize;
out.truncate(n);
}
Why do people not write their own blogposts on their own?!
Also, I can tell that you are a good writer. You didn't need the LLM to "polish" your text.
LLMs love this pattern. Whether one put it into this text, or the author soaked it up and now used it himself, who knows. But it is one of the few things in the post that gives me the ick.
And then, there's the verbosity.
If I had to guess, an LLM was involved, but the author did a good job with manual writing and editing, too.
but.... running PGO is just too much pain.
We can't do it "incrementally", can we? How about combining with LTO?
edit: I was thinking profiling individual module on a test driver and link them after PGO
You can do even a bit better if you're willing to use intrinsics. In particular this kind of operation is well-suited for compress-type operations, available as a first-class operation in at least AVX512, SVE and RVV; you can also emulate them reasonably quickly on NEON and AVX2.
Here's an example, building on the OP's work:
For me it's about 25% less time than the branchless version with 1,000,000 elements, and 60% less with 10,000 elements where memory bandwidth effects are less relevant.https://deplinenoise.files.wordpress.com/2015/03/gdc2015_afr...
Would probably have to pass `-C target-cpu=native` to cargo so that llvm is allowed to use AVX512.
For example, you'll notice here that we perform a full vector store of 8 elements unconditionally, even if only a few of the elements are active. This is safe, though, because the output buffer is as large as the input buffer, and we're chunking by 8, so we'll never trash memory past the end; but this is a tricky analysis. Performance-wise, we rely on the CPU's store buffer to make these overlapping stores cheap.
Instead, you might think that you could just store the elements which are actually active, using a masked store. In fact there is also an intrinsic for this purpose (_mm512_mask_compressstoreu_pd), but it is extremely slow on some CPUs, namely Zen 4, so it's dangerous to use unless you know exactly what CPU you're using. (In my testing, there also seems to be some weird hazard on Zen 5 where multiple memory-destination compress instructions to nearby, even non-overlapping, addresses are serialized. But I haven't looked closer at this.)
So this divides up the problem, the compiler can vectorize your 8-at-a-time code without needing to handle edge cases where N isn't a multiple of 8, and if a later pass notices we actually never end up using those edge cases they're dead code, if it doesn't they're just a rarely-taken branch once.
Edit: actually looks like autovectorization is in play here [1]. Doesn't look like the bounds check gets in the way at all.
[1] https://godbolt.org/z/af4qGba5o
> Executing code compiled with target features that the current thread of execution does not support
I.e. calling AVX512 on Neon architecture.
You need to wrap it in target attributes to even dream of it being safe.