As a modern array language D4M is the natural successor for SQL [1].
D4M is based on mathematics like SQL, specifically associative array algebra but not relational unlike SQL. It's more generic since can it caters to most modern data abstractions including spreadsheets, database tables, matrices, and graphs [2].
You can achieve 100M database inserts per second with D4M and Accumulo more than a decade ago back in 2014 [3].
[1] D4M: Dynamic Distributed Dimensional Data Model:
[2] Mathematics of Big Data: Spreadsheets, Databases, Matrices, and Graphs:
https://direct.mit.edu/books/monograph/5691/Mathematics-of-B...
[3] Achieving 100M database inserts per second using Apache Accumulo and D4M (2017 - 46 comments):
Applying the Lindy effect [1]: after half a century of SQL we can expect it to survive for at least as long.
Disruption/displacement of SQL is like attempting to replace email. It's not going to happen. At best an alternative technology can carve out a small niche (and there's nothing wrong with that).
This is the power of low-level reasoning.
Today, even for a junior developers, even if they have AI that solves syntax problems, SQL teaches you to reason and approach problems logically. Without any wrapper masking low-level logic.
It's something like the letters of the alphabet that form concepts: why should they change?
I'd say SQL is a very high level language.
"SQL teaches you to reason and approach problems logically" -- I kind of agree here. It teaches relational data mgmt. I think it is better to attack most software design challenges at a higher level, and --once settled at that level-- consider how to "serialize" those solution to an RDBMS (is that's the tech that you've chosen for persistence; still a very solid choice after 50+ years)
I understand the point you're trying to make, and yes, it does seem like SQL is "low-level" from the perspective a wrapper like ORMs or a GUI db browser tool with menus for filtering data.
But it's also worth remembering that SQL itself is a high-level wrapper that hides the lower-level C/C++ code of the db engine that has the loops that iterate through b-trees, 8k data pages, memory blocks of the buffer cache, etc.
And C/C++ itself is a high-level wrapper that hides the logic in lower-level Linux o/s system calls that manages RAM and disk i/o.
And Linux itself is a high-level wrapper that hides low-level device drivers like SATA/SSD memory-mapped IO ... and so on and so on.
Depending on the type of app, you can ignore all the lower levels and just work at the abstraction level of higher-level wrappers.
- Comparing SQL to React weakens the argument. SQL is the language, React is a piece of software. You certainly can run 30 year old JS today in modern browsers.
Had to reread the title again since I thought I opened a different article about TLA+.
As for SQL, if you're referring to DBMS systems, here's what E.F. Codd, inventor of relational algebra, had to say about them and the departure from his work: https://thaumatorium.com/articles/the-papers-of-ef-the-coddf...
It was a great foundation and has served me well to this day.
Although SQL is of course not relational Algebra (and others like Datalog and D4M are better), it's still cool. It inspired kSQL like Lil uses https://beyondloom.com/decker/lil.html#lilthequerylanguage , which inspired the code I'm most proud of: https://codeberg.org/veqq/declarative-dsls A common query language, a common idiom, for many data structures (arrays, hashmaps, datafremas) is liberating, permitting you to e.g. solve sudoku, make mandelbrot sets or calculate primes directly:
(def n 40) # to reach primes up to, left is sqr of n, right n/2, then multiply them for rows
(def composites
(df/select :from (range 2 (+ 1 (math/floor (math/sqrt n))))
:cross (range 2 (+ 1 (/ n 2)))
:where |(<= (* ($ :value_left) ($ :value_right)) n)
[[:value_left :value_right] :value
|(* ($ :value_left) ($ :value_right))]))
(df/select :from (range 2 (+ 1 n)) :exclude composites)
(import declarative-dsls/dataframes :as df)
(def people (df/dataframe :name :age :job))
(df/dataframe? people)
(df/insert! {:name "Bob" :age 30 :job "Developer"} :into people)
(df/insert! {:name "Alice" :age 27 :job "Sales"} :into people)
(df/update! :set {:job "Engineer"}
:where |(= ($ :job) "Developer")
:from people)
(df/save-csv people "people.csv" :sep "\\t")
(def people2 (df/load-csv "people.csv" :sep "\\t"))
(-> people2
df/dataframe->rows
df/rows->dataframe
df/print-as-table)
That and SQLite seems to be able to scale to almost any problem, is disgustingly fast and with litestream incredibly resilient.
It's not that I like or dislike SQL, it is just that it has such raw power and mature tooling/resources, I wonder what an alternative could even offer me.
It's like C. It does such a great job at being structured assembly that it is hard to displace it for similar reasons.
Helps simplify complex SQL queries and no need to waste network traffic on data that client side is never going to use, and waste CPU cycles processing it.
Yes, what about database portability?
I am on my 50s and it only mattered on a single project, which was anyway a middleware for application servers.
For sure, but have a solid grounding in set theory to go with it.
I've dealt with so many poorly-performing stored procedures that ended up being written as iteration over a CURSOR when they could have been done with sets. Programmers who don't grok set theory reach for iterative constructs which, while they work fine, are an impedance mismatch with SQL.
If the solution you find is longer and not much faster than the procedural alternative, you throw it away and fall back on procedural code.
Stored procedures are not advanced SQL. Most of them are not SQL at all. There are a few legitimate reasons for using SPs such as reducing roundtrips to the database and writing little pure functions for use in SQL statements.
But many uses of SPs are just laziness or a symptom of organisational dysfunction.
I think stored procedures - or anything that goes beyond storing / looking up data - had a place when a database had multiple different clients, but with modern day systems that's less likely to be an issue.
1. C language.
2. *nix tools (shell and friends).
3. SQL.
4. Basic IPv4 networking.
These things I learned around 20 years ago, they didn't change much and they are useful for me to this day.
The point being that sometimes the tools themselves don't need to survive because you take the lessons from one thing to another (e.g. move semantics and rust/modern c++)
[1] - https://pragprog.com/titles/btlang/seven-languages-in-seven-...
I know I'm in the minority in places like this, but I've spent all my life using ORMs, and never once regretted it. And I'm the kind of person that actually likes low-level C from time to time. SQL just feels like a poor abstraction layer: either go higher or lower.
The only difficult part in arguing this is that RDBMS != SQL != RelationalAlgebra, and it’s very often forgotten
Rivalled only by Linux, shell scripts, and Cron!
The value of this stuff is difficult to overstate. Batching allows for you to rapidly load the RDBMS. The first few times you test, it will probably go so fast you won't believe it loaded anything at all. Set operations allow for you to bring this newly loaded data to visibility in production tables nearly instantly. Your OLAP & OLTP workloads should be dominating the compute. ETL ops (loading/set ops) should be a ghost in terms of cpu time and memory. None of this is vendor specific knowledge. Every major engine has a reasonable way to bulk load and perform quick merging of records.
Please, preach your gospel more loudly and frequently. It always feels like people complain about RDBMSs being slow because they run insert queries one at a time.
SQL is not a programming language. You do not write programs in SQL. It's a declarative language (or set-of-sublanguages).
> a working developer can learn once and > use for 30 years without rewriting their mental model.
There is any number of long-living languages which satisfy this.
Plus, SQL it's not even really a single language, because the spec changes, and is huge, and few people know it fully; and the dialects have non-trivial differences; and if you switch DBMSes, you often switch SQL dialect. In that sense, it is very much like other programming languages which evolve, like C++ or Fortran or even C.
Javascript is actually fully backwards-compatible, to not break the Web. Any javascript from 10 years ago works in the browser. This is good but also a bit of a burden, since the language can only expand but not shrink. React is a library, and like all libraries it has breaking versions. Not understanding the basic difference between the two kinda undermines the credibility of the article.
Also, in a similar way, core, ANSI SQL is largely backwards compatible, but all the SQL dialects linked to various DBMS implementation are generally incompatible. Obviously that's not mentioned in the article.
> Not a tutorial. Not an ORM. Actual SQL: joins, subqueries, window functions, query plans.
Not text written by a human. Not a style that an real writer would ever use. Actual AI slop: Short sentences. Incorrect facts. Not X, Y.
Yes, SQL is based around relational algebra, but all programming languages are built on a theoretical foundation.
And SQL is very much a "fad" language - it just somehow managed to stick around. The goal was not some sort of mathematical purity, but rather to built a natural language data interface (sounds like something currently very hyped?) and it failed spectacularly at that goal.
It is so far from natural language that English speakers with statistical understanding won't be able to read it, but it is also inconsistent enough in its grammar design that it is unreasonably difficult to learn and needs large refactoring every time you want to query into the result of a query.
To continue my rant: Sometimes '=' is an identity test, sometimes it is `==`. Sometimes groups are called groups, sometimes they are partitions.
When creating a CTE, you put the name before "AS", but when creating a column, you put the name after "AS".
SQL is great because it is everywhere and it is definitely good enough, but it is not something great, that transcends other programming languages.
But then it's only a query lang (DDL you still do in SQL then I guess).
Bottom line for me now is that I dont write much of my SQL by hand. AI does a much better job at it. I just read it back and point out mistakes and/or inefficiencies.
1: https://prql-lang.org/