I Hate Compilers
50 points by xena 3 hours ago | 47 comments

jdw64 2 hours ago
Reading this, I think low level engineering is actually more dependent on specific environments. Hardware also has its own points of change. Usually, when you think at a high level, environmental changes are less significant than you might expect. But low level thinking tends to be tied to specific environments, which is what makes it difficult. The reason low level is hard is that even if the code itself is short, the hidden assumptions inside it are difficult and place a heavy cognitive load on the programmer. For example, even a short snippet in C like `int value = (int)buffer` requires a lot of implicit knowledge about the 4 byte alignment of the buffer, or whether int is exactly 32 bits. LLMs do not seem to be very good at knowing these things. Rather, they are strong at high level wrapping, but at the low level, they seem surprisingly difficult and somewhat useless. Hardware has CPU generation changes, and in the case of PLCs, where I mainly work, the protocol differences between vendors are far too severe. There does not seem to be any technology with a very long lifecycle.
reply
jstimpfle 58 minutes ago
Depends on what you mean by low level I guess. Compared to web application framework churn rate, simple procedural programming without many dependencies is remarkably stable. You tend to program in a way that works for most platforms (all targetted platforms). How to best do that you learn over the years. To me personally it's very refreshing if the environment around you does not constantly change. That affords learning a bag of tricks and a list of gotchas to avoid.
reply
jdw64 13 minutes ago
I think you're right too. So I also think that maybe I'm viewing the changes as bigger than they actually are, based on my own standards
reply
ComputerGuru 2 hours ago
These seem very reasonable, the workarounds used are natural, and overall the article is not at all congruous with the conclusion in the (clickbait?) title?

Compilers literally made your project possible!

reply
crvdgc 2 hours ago
Nix also needs the build output to be deterministic to calculate the hash. It also has the problems of timestamps etc. The build environment tries to be hermetic by setting the time to be epoch among other things.
reply
lloeki 32 minutes ago
SOURCE_DATE_EPOCH is not a Nix thing

https://reproducible-builds.org/docs/source-date-epoch/

(although Nix sets it as a default)

reply
swiftcoder 2 hours ago
The Birth and Death of Javascript really had the gift of prophecy, eh
reply
znpy 7 minutes ago
> What do you do when the client has WebAssembly disabled?

> I decided to take inspiration from the legendary talk The Birth and Death of JavaScript and just recompile the WebAssembly to JavaScript.

So what do you do when the client has Javascript disabled ?

reply
biglost 2 hours ago
Time date env variables and random address... Is also input data, maybe not as a flag but still
reply
RyanSquared 2 hours ago
Time and date are... tolerable. There's SOURCE_DATE_EPOCH which should always be set to whack it into submission when used. ASLR of the _compiler being invoked_ resulting in a difference in the _program being compiled_ is nuts and would break any self-hosting compiler with consistency checks.
reply
yjftsjthsd-h 2 hours ago
Explicit is Better than Implicit.
reply
Animats 17 minutes ago
I hate proof of work code running on my machine for the benefit of someone else. It's like planting a crypto miner.
reply
pertymcpert 2 hours ago
If Clang generated non-deterministic output due to pointer addresses then that's a bug (happens regularly) that should be fixed. The most common way this happens if it some code path is iterating over a DenseMap which is non-deterministic. Sometimes that's fine and sometimes that's not depending on how that map is used. The common way to fix that is to switch to a MapVector which pays some additional runtime/memory cost to guarantee deterministic iteration order.
reply
xena 2 hours ago
I'll try and make a minimal reproduction case and file a bug. Do you know if any tooling that can take a binary and fuzz it down to a minimal reproduction set?
reply
amatria 2 hours ago
cvise. Here is a link to a mini tutorial I wrote for a user in an LLVM issue: https://github.com/llvm/llvm-project/issues/108827#issuecomm...
reply
glandium 2 hours ago
Claude code is actually rather good at this. If your initial testcase is not too big, you can use creduce or cvise.
reply
sylware 56 minutes ago
To avoid all those grotesque and absurd compilers and runtimes, more for those of computer languages with a ultra-complex syntax (c++ and similar), I now design "binary specifications" which I "design" and "validate" with RISC-V assembly coding.

Here, since any whatwg cartel web engine is an issue, the author should not bother.

reply
mathisfun123 3 hours ago
[flagged]
reply
xena 3 hours ago
reply
heavensteeth 2 hours ago
I'm surprised by the amount of heckling this post received almost immediately! And a lack of constructive input.

I for one enjoyed the article and understand what you're getting at.

reply
yjftsjthsd-h 2 hours ago
> This is the goofiest I've seen written unironically in quite a long - the C preprocessor is not part of the compiler. The pre in preprocessor should probably give it away.

This is true but doesn't seem relevant; does replacing the word "compiler" with "build chain" change anything? Because that seems like the clear meaning.

reply
LPisGood 2 hours ago
Re: source code producing different binaries: things like ASLR, stack canaries, optimization levels, linking, etc all lead to different binaries.
reply
charcircuit 3 hours ago
As long as the program is equivalent there isn't an actual problem here. Requiring the output to always be the same is an arbitrary restriction.

If you want to have users trust that someone else hasn't modified it, then sign it with your identity.

reply
yjftsjthsd-h 2 hours ago
We'd like to verify, not trust.
reply
charcircuit 42 minutes ago
The whole point of a signature is that you are able to verify what was signed was in fact a message that was signed by signer.
reply
robinsonb5 36 minutes ago
Sure, but a signature doesn't prove that a particular binary came from a particular codebase - merely that a particular human (or other trusted entity, for varying degrees of "trusted") has vouched for it.

Being able to reproduce the binary from the source code and being able to verify that it's the same as the original is quite important in some contexts.

reply
charcircuit 32 minutes ago
>Being able to reproduce the binary from the source code and being able to verify that it's the same as the original is quite important in some contexts.

I disagree. The contexts that people come up with are purely theoretical, and are not practically important. Please do try and convince me otherwise by sharing such a context. From my view the juice of trying to accomplish this is no where worth the squeeze.

reply
dyauspitr 3 hours ago
LLMs should be trained on and directly output binary.
reply
klodolph 3 hours ago
On the off chance that you’re serious, that would result in disastrously bad output. The difference between “jmp $+15” and “jmp $+16” is inscrutable and the LLM would not be able to pick the right one without tooling.

That tooling is a compiler. The higher level, the better chance the LLM can be steered to good output. Machine code is hopeless, don’t bother.

reply
pjmlp 3 hours ago
That compiler does wonders with languages that have UB on their specs, especially when having optimizations passes with heuristics.

Also there are dynamic compilers were the shape of machine code changes as the code executes, and each single execution will certainly generate different sequences, depending on the program execution and where it is running.

Deterministic JIT compiler code generation, at least on optimising ones, is not a solved problem.

reply
faangguyindia 3 hours ago
What about AOT optimization? whuch brings aot closer to JITs performance? Isn't that something LLM + Harness can easily do?
reply
klodolph 2 hours ago
I think the idea that AOT is inherently faster than JIT, or vice versa, is a thoroughly debunked idea.

You can have LLMs help you optimize code but I don’t think you can do this unattended for non-trivial code.

reply
jenadine 3 hours ago
> The difference between “jmp $+15” and “jmp $+16” is inscrutable

I don't see why that's the case. LLM trained on binary would totally see it, not?

Also the tool can also be running the test and a debugger.

reply
klodolph 2 hours ago
> I don't see why that's the case. LLM trained on binary would totally see it, not?

It would not. You find the correct version by counting the number of bytes to the destination. LLMs are famously bad at this kind of problem (counting).

> Also the tool can also be running the test and a debugger.

The test needs to provide a good amount of signal. That’s too hard if you are throwing machine code at the wall.

In order for debuggers to work, you need some kind of model that describes what the code should do and what state the computer should be in after each instruction. That model is high-level code.

I can understand the intuitive appeal of training LLMs with machine code, but all of my experience with LLMs suggest that they are incredibly ill-suited to the task, and we just don’t have the capacity to train them to make useful machine code.

reply
zx8080 2 hours ago
Can "LLMs are bad at counting" be generalized to "LLM are better in complex stuff but make more mistakes in simple"?
reply
ozlikethewizard 2 hours ago
Its more LLMs are better at vague problems with multiple non perfect solutions, and struggle at problems that require precision.
reply
fluoridation 2 hours ago
I would phrase it as "LLMs are good at big picture stuff and bad at fine detail", or to put it another way, they're accurate, but imprecise and with low reproducibility.
reply
klodolph 2 hours ago
No, I don’t think so. LLMs are good at a lot of simple tasks, but bad at certain simple tasks. Moravec’s paradox in a new iteration.

It applies to humans too. Calculus is “simple” but it takes something like sixteen years to train a human to do it, if all goes well. Meanwhile, most humans think that inverse kinematics is, like, the easiest thing in the world (it’s a super complicated task).

reply
fluoridation 2 hours ago
Calculus is definitely the harder task, considering it took a species developing the cognitive capacity for symbolic reasoning for it to show up, whereas any animal can figure out how to position its limbs. Yeah, we figured out how to make CAS programs before inverse kinematics software, but that's because computers were made to solve numerical problems, not to replace the cerebella of chordates.
reply
bandrami 2 hours ago
Generative algorithms have been studied for decades now and while they have led to some interesting results they're a bad fit for LLMs because there's no such thing as a "plausible" binary: a small perturbation yields an unusable result.
reply
xiaoyu2006 3 hours ago
It should not. Abstraction in software engineering brings intelligence. (compression correlates to intelligence)
reply
shshshjaja 2 hours ago
runApp()

Done! Excellent abstraction. High intelligence.

reply
frwrfwrfeefwf 2 hours ago
people don't get this
reply
dyauspitr 2 hours ago
Why? I mean this is all emergent, right? And it’s not like humans ever work at this level. It would be very interesting to see what sort of outputs and abstractions an LLM comes up with.
reply
fulafel 2 hours ago
Technically they are, just a subset. But still a practical one, they're frequently used to produce executable files.
reply
junior44660 59 minutes ago
You should go back to elementary school
reply
wahnfrieden 2 hours ago
What other recent Musk talking points do you like to repeat?
reply
rvz 3 hours ago
I think you forgot the "/s"
reply