It’s hard to overstate the reputation of the author of this post. Everything good about GitHub’s internal systems seemed to have his name all over it (I realize that today this statement hits different than a few years ago). Our times at GH didn’t overlap much but hearing the fact that he’s working at cursor increases my estimation of their engineering org by leaps and bounds.

> What about consensus? Elections? Which server is the primary for a given repository? It also doesn't matter! There's no state and no consensus here. Any server can be the primary. All updates to the write-ahead log are synchronized with an atomic compare-and-swap (CAS) operation on S3, so it's always safe for any instance of a repository to receive a push.

Again reminded of what an amaizing piece of engineering S3 is (99.999999999% - 11 nines of durability) [1]

1: https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDu...

There's nothing Cursor can do that GitHub/Microsoft can't in 2026... And vice versa... after several years of Cursor vibecoding a GitHub clone while catching up with GitHub's new features. Git's performance it's not the issue; it's GitHub Actions, PRs, etc. The moment Cursors steals paid GitHub customers and gets the same workloads, they will start having similar issues! Stealing engineers who couldn't fix this at GitHub won't make much of a difference. GitHub is not just source control; everybody can set up Gitolite and have an amazingly configurable and secure Git setup. I did this 10 years ago, and CODEOWNERS, protected branches, and tags can't do 1% of what I had done before. Cursor is going all over the place as it can't compete with their model. So, this is a defeat, a pivot, not something glorious!

https://cursor.com/en-US/blog/git-at-any-scale to avoid automatic translations (couldn't find a button to switch)

There’s a trend of doing impressive things by pushing many of the hard problems into S3 and assuming S3 “just works”, then not bothering to explain how S3 works. I guess we do the same thing all the time with other miraculous solutions to hard problems, like assuming that superscalar out-of-order processors “just work”, but in cases like this it makes for an unsatisfying explanation. Especially because S3 is a proprietary product, not an algorithm.

This is an excellent post that explains why their version of git hosting is more scalable. But it seems like most people won't need it, assuming Github remains free to use and they get their availability problems fixed.

Will free Git hosting continue to be sustainable, though, without architectural improvements like this?

It's time to look back on S3's 20-year history

https://blog.fnil.net/s3-at-twenty/

> The "fan-out" is synchronized with a classic consensus algorithm called 3PC (three-phase commit) so that a push is only accepted if a majority of the nodes acknowledge it.

Doesn't 3PC require all nodes to agree, not just a majority?

Took me some thinking to figure out where this design is not optimal.

The first step is to separate the problems. The content addressable object storage and storing the refs. libgit2 has two backends for those already.

Then use appropriate distributed dbs for the two backends. With large scale something like Cassandra for the objects (AP) and FoundationDB (CP) for the refs (or the equivalent Amazon services).

For performance the objects need to be cached locally and requests routed to nodes where the objects are cached locally.

The key part: they decided to use the unchanged git codebase as their building block. Likely assuming that the people who wrote the Linux kernel probably knew what they were doing. That explains some features that look bizarre from the generic distributed-database perspective. I think the author has had this argument many times, because he reiterates it several times in the post. The rest follows from it.

Fantastic write up, but I still have doubts. As the author states: Serving git at scale is hard. I can only concur.

I feel like the article is a bit light on the design of the WAL and maintenance operations, but maybe it's me.

Anyway, I'd love to have a friendly chat with the author.

Any ideas how these interactive animated diagrams were made?

> The productivity cost of five minutes of downtime in your CI system is hard to quantify in dollars, but it is, by any measure, a humongous amount.

Not sure the last month GH had fewer than five minutes of downtime - but I think it's been a while

That was a very well written and interesting article. I really enjoyed it. Some good background for people telling GitHub to "just scale".

I don’t see a button to switch to English and the German text is a pain to read, comparable to Reddit auto translate..

WAL on S3 is the new normal.

Related:

Cursor Origin Code Hosting

https://news.ycombinator.com/item?id=49334209

Remember, don't put large objects in cnt, it makes it a real pain later.

Reading this, I can't come to a conclusion other than "git rapidly starts sucking balls for use cases outside of Linux development". Github started with ze_current_fastness but then ended up recurrently smashing into walls because of it.

They should have just failed to scale git and leave place for the less idiosyncratic bazaar or mercurial to take the lead. These were slower, but would probably have scaled much better in the long term.

Github's core innovation was re-centralizing a decentralized SCM (as per article) by giving it nice web UI and features such as PRs. Oh and also free code hosting space. None of which relied on git's peculiarities.

Very cool write up. But I do have to wonder, is Git the right solution for version control given where we are heading? The commands are all muscle memory for me now at this point, but all the projects I've been working on lately have just all moved to trunk based development, rebasing, and squashing merging feature branches onto main.

I've been working on another project where I have binary files checked in, assets that move or are moved and then changed that git "loses track of" due to the significant change. This just may be me doing silly things like moving a file and then editing it and then committing it and git thinks I deleted it and created a new file.

how does git lock the repo on extreme concurrency? let's say 1000 devs writing to the same repository? eventually you'll have saturation of the resource locking everyone out.

When I've worked at big companies they had bespoke SCM that resolved this. But i'm curious about mid size teams that are big enough to overwhelm git but not so that they can rewrite it.

It doesn't matter how many technical blog posts they put out, I will still never host my code with Elon Musk.