and in practice I find it _very_ effective to express key ideas with some formal language where applicable. a little ebnf here, some snippets of python or prolog there or other languages there. and I find llms wonderful at filling in the dots and, and that's the topic here, to translate our semi formal conversation into rigorous, formal language.
and I dare one step further: I also find llms useful in formulating and refining technical proposals, for human peer review and llm driven implementation, and of course these proposals mix natural language prose with semi-formal and formal detail. think PEP-12 and PEP-1 style. and these then get work packages which each loop in a spec as in sdd.
seems to work just fine so far :-)
I've realized that the agent doesn't write 'good' code, instead my vague understanding of what I expect makes me think it does. When I look at the AI code, I see that there are more nuances and decisions that needed to be taken before writing the code. One would say that a spec would suffice, but then I remember TLA+, developed by Leslie Lamport to help developers think critically about their programs/algorithms. Sitting down and formally writing what I want to do helps me better understand the problem and come up with a real -not vaguely defined- solution.
Apart of natural evolution of meanings, look at Orwellian limitations some (cloud) models impose on is that our natural language program become illegal. Due to topic or combination of words the program needs to be flagged and soon likely reported to three leter agency.
Next, RMS would ask: can you make your own natural language program compiler from scratch? I know we have (very expensive) local models, but wait until GPUs get a propper online subscriptions to work and surveillance taps.
With specification driven development, the prompts need to be very detailed in order to produce exactly what is required.
Also gave me plenty of opportunities to go above and beyond. I miss when that felt safe.
You didn’t collaborate with other engineers? Or what if the spec you were given was unclear in places? Forgive me, but the style you describe sounds incredibly collaborative, super transactional.
Massive? My introduction to programming (from noob to beginner level) has been like only 10 books worth of content. Including some alternative pathways like 3D, security, system administration,... To this day, I don't think I've reached 50 books of materials. And If I counted everything it would be still small.
Human learning is very efficient (not time wise, but cost wise, and even training data wise).
Ah, yes, I can fondly recall reading and internalizing every word available on the internet when I was young...
But for mature software eventually every single detail matters.
I have observed several samples where the business, when provided with a natural language configuration opportunity, will wind up staring blankly at the textarea until the original developer gives them a complete starting point.
I don't think "natural" language makes much of a difference unless you are already skilled in the art of developing useful engineering specifications. I often have to babysit the business through very basic boolean algebra concerns despite everyone having 3+ decades experience dealing with this stuff. Making everyone speak python could only make this worse, not better.
I use HN to keep myself warmed up on natural language expression. The difference between developers who post here prolifically and those who barely write one email a month is fairly shocking in my experience. If you intend to use a specific instrument to accomplish a complex task, you need to practice it constantly. Ideally every single day. There is no substitute for this. There never will be.
The experiment in designing a language that resembled natural languages (English and Japanese) was not successful. -[1]- William R. Cook, AppleScript, HOPL IV
Now our new LLMs are different, because they don't use "language the resembled natural languages", but actual natural language. But the lesson is instructive nevertheless, because it gives us some empirical evidence to compare to Dijkstra's theoretical arguments.
And this is that natural language is at best a two-edged sword. Yes, there are benefits, but there are also huge drawbacks. And one problem is that the benefits are front-loaded and the drawbacks are long-term problems.
To me, one thing we are missing is that what LLMs do for us actually has 2 parts.
1. It allows us to use natural language, allowing us to be fuzzy.
2. It allows us to specify what we want at a much, much higher level than usual.
With our LLMs, these two are intrinsically linked. I think the whole would be much, much more valuable if we could separate them.
He's still quite correct. I generally see a machine as being "programmed in" whatever is tracked in source control. So as long as your git repo is tracking changes in Java, that's still programmed in Java, even if the Java was generated with the assistance of LLMs. I may consider the thesis of this article invalidated only when the majority of repos start tracking natural language rather than programming languages as the primary artifacts defining behavior of the software.
[1] For many people, the result is actually the mean not the best, but it's another topic.
It seems Dijkstra is describing potential calability of LLM model trained on published collective human knowledge enabled by the Internet
>and, in view of the history of mankind, it may not be overly pessimistic to guess that to do the job well enough would require again a few thousand years
Since the article was written in 1978 (about 50 years ago) and from the time the computing resources of CPU and RAM has increased exponentially until the LLM first appeared in 2020s, it seems the thousand of years prediction is merely several decades accelerated by the rapid advances of computer and communication systems technology spearheaded by companies like Nvidia.
In order to make machines significantly easier to use, it has been
proposed (to try) to design machines that we could instruct in our native
tongues. This would, admittedly, make the machines much more complicated,
but, it was argued, by letting the machine carry a larger share of the
burden, life would become easier for us. It sounds sensible provided you
blame the obligation to use a formal symbolism as the source of your dif-
ficulties.
transcribed by Tristram Brelstaff
revised Fri, 19 Nov 2010
It shipped as a free PostScript font with NeWS:
https://www.donhopkins.com/home/NeWS/fonts/Dijkstra.ps
https://lucacardelli.name/indexartifacts.html
(You have to click on the "Fonts" tab, there's no direct link to it.)
Dijkstra [.ttf (2013 updated version by Reuben Thomas)] [.zip (old PC)] [.hqx (old Mac)] [Preview.pdf]
A beautiful handwriting font of unknown origin, first popularized by E.W.Dijkstra. Excellent for slides that appear to be carefully written, and for short, deep technical notes.
https://lucacardelli.name/Artifacts/Fonts/Pc/dijkstra.ttf
https://lucacardelli.name/Artifacts/Fonts/Pc/Dijkstra.zip
https://lucacardelli.name/Artifacts/Fonts/Mac/Dijkstra.sea.h...
> "The virtue of formal texts is that their manipulations, in order to be legitimate, need to satisfy only a few simple rules; they are, when you come to think of it, an amazingly effective tool for ruling out all sorts of nonsense that, when we use our native tongues, are almost impossible to avoid."
AFAIU this particular chain is not compromised.
How many times have we seen that there's been a spec for something and the programmers write something that is exactly like the spec said but was completely wrong?
It's excruciating to write, excruciating to read, and even legit geniuses with decades of hard won experience still leave room for essentially infinite interpretation. laws and contracts are all still swiss cheese and can be made to demand or prohibit whatever the party with the most money or other influence wants.
The law is rather a complex set of guidelines which need to leave some space for interpretation, so that additional relevant information for the situation at hand can affect the decision.
It is perhaps for the best that Djikstra did not live long enough to witness this problem become orders of magnitude worse.
This argument makes the implicit assumption that humans are uniquely able to convert between this "customer wants a program that solves their problem" level and compileable code. That was a reasonable thing to believe in 1978, but is not a reasonable thing to believe in 2026.
Dijkstra's arguments tend not to change their minds. I think the analogy takes hold because there is a kernel of truth in it. AI models now give us the magical ability to program in natural language, but the implication that an AI model is a form of compiler is deeply flawed. AI models don't replace compilers, they replace software developers. The reason software developers rarely look at assembly code is that compilers (and emulators and VMs) are able to precisely preserve the semantics of the original program.
A natural language spec is not a program, and there are many possible programs that can meet the spec. Software engineers and AI models turn specs into code, but the code is not the spec I think the analogy clouds our thinking about the value of programming languages as a tool for formally describing what the computer should do and enforcing properties of the program that should not be left to either humans or AI models to uphold.
'Precisely' is doing a lot of work here. Depending on your language some things might be preserved and some not. Eg if you write stuff in Haskell, you don't really preserve much of anything about memory usage, you leave that to the runtime system to figure out, and exactly how the GC works isn't really specified in the language standard.
You can interact with AI and say "code me a calculator" or "now declare a array of booleans to map to each of the keys in the UI and set the corresponding array position to true when the key is pressed".
These commands are written in natural language at such different levels of abstraction that is hard to put them in the same bucket. It would be fair to call the latter an imperative language with the LLM in the role of a compiler. That's very similar to how I use AI.
I think "I'm working in a higher level of abstraction" is easier for some devs to say than "I asked somebody else to do the thinking and the work." I don't think there's anything wrong with asking an LLM to do the work. But pasting jira tickets into claude is not abstraction lmao.
I guess arguments on either side of this can get difficult for a reason that another commenter already mentioned: you can use LLMs on a wide spectrum of granularity. You can one-shot a whole project, or you can tell it to rename some vars in a single file. I guess there are uses that are probably closer to programming. But if you're not even looking at the code, I'd say that's closer to somebody else doing the work
No, they replace the need for a precise description of what you want the computer to do by a mechanism that's really good to fill in the gaps. But guesswork only takes you so far.
Guess what wasn’t?
The nonfunctional requirements.
Also: my app needs to do LLM classification. But instead sometimes the coding assistant wrote not an LLM api call but it wrote a regex to classify instead. This was despite the fact that I mentioned to not use regexes to classify things with.
Non-technical people don’t know what a regex is. Usually they also aren’t knowledgeable enough about nonfunctional requirements.
All of this may improve over time of course. But right now, I am not impressed (in terms of replacing SWEs).
This stochastic nature existed before AI. Humans are stochastic too. Two engineers convert specs to code in two different ways. I would argue the distribution is wider among humans than among agents.
while both LLMs and humans are not 100% deterministic, I don't think that's the interesting part. It's both true that there are many assembly language programs that implement a C program and that there are many valid programs (in any programming language) that correctly implement a natural language spec. But there's a significant qualitative difference between the two types of translation and that would be true regardless of whether the translations are stochastic.
It would be interesting to state sane measurments that determine a "distance" between a spec and software that claims to implement it
Please go ahead and do.
So? Two compilers convert source to binary in two different ways.
I would argue that, in the absence of such evidence, we cannot conclude that the English-to-code conversion process is stochastic; it is merely not deterministic, which is altogether different.
Well, then, their interpretations of the spec are stochastic. Good enough for me, anyway.
In my opinion, LLMs don't challenge anything.
And like with a team, many programs could meet what I asked for, but I check the behavior of what they built, not the code directly. I think of it as working on the level of an engineering director, who also doesn't review code directly. Each agent shows me its change running in the app, in a short narrated screen recording, like in a launch review for the director. When it's not what I meant, I say what, and it goes another round.
However you can't prevent some other people from working as software developers who write and read code.
Even with cars, you still walk. Even with factories, someone still craft handmade bags.