It’s hard to overstate the reputation of the author of this post. Everything good about GitHub’s internal systems seemed to have his name all over it (I realize that today this statement hits different than a few years ago). Our times at GH didn’t overlap much but hearing the fact that he’s working at cursor increases my estimation of their engineering org by leaps and bounds.
Is this enough to counteract the reputation of his new boss, Elon Musk?
<edit>: there is lots of demand for a GitHub alternative due to poor uptime, poorly implemented components (CI), and Microsoft stewardship. Switching, especially at this early stage requires a huge component of trust.
how does git lock the repo on extreme concurrency? let's say 1000 devs writing to the same repository? eventually you'll have saturation of the resource locking everyone out.
When I've worked at big companies they had bespoke SCM that resolved this. But i'm curious about mid size teams that are big enough to overwhelm git but not so that they can rewrite it.
> What about consensus? Elections? Which server is the primary for a given repository? It also doesn't matter! There's no state and no consensus here. Any server can be the primary. All updates to the write-ahead log are synchronized with an atomic compare-and-swap (CAS) operation on S3, so it's always safe for any instance of a repository to receive a push.
Again reminded of what an amaizing piece of engineering S3 is (99.999999999% - 11 nines of durability) [1]
There’s a trend of doing impressive things by pushing many of the hard problems into S3 and assuming S3 “just works”, then not bothering to explain how S3 works. I guess we do the same thing all the time with other miraculous solutions to hard problems, like assuming that superscalar out-of-order processors “just work”, but in cases like this it makes for an unsatisfying explanation. Especially because S3 is a proprietary product, not an algorithm.
The key part: they decided to use the unchanged git codebase as their building block. Likely assuming that the people who wrote the Linux kernel probably knew what they were doing. That explains some features that look bizarre from the generic distributed-database perspective. I think the author has had this argument many times, because he reiterates it several times in the post. The rest follows from it.
> The "fan-out" is synchronized with a classic consensus algorithm called 3PC (three-phase commit) so that a push is only accepted if a majority of the nodes acknowledge it.
Doesn't 3PC require all nodes to agree, not just a majority?
<edit>: there is lots of demand for a GitHub alternative due to poor uptime, poorly implemented components (CI), and Microsoft stewardship. Switching, especially at this early stage requires a huge component of trust.
https://blog.fnil.net/s3-at-twenty/
When I've worked at big companies they had bespoke SCM that resolved this. But i'm curious about mid size teams that are big enough to overwhelm git but not so that they can rewrite it.
Again reminded of what an amaizing piece of engineering S3 is (99.999999999% - 11 nines of durability) [1]
1: https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDu...
How many startups and corporations built on top of S3 semantics?
Snowflake and the big lakehouses are one of them. They use S3 as giant disk in the sky.
Doesn't 3PC require all nodes to agree, not just a majority?
Cursor Origin Code Hosting
https://news.ycombinator.com/item?id=49334209