GitHub's new Git storage architecture

(github.blog)

157 points | by DAlperin 16 hours ago

15 comments

  • yakkomajuri 11 hours ago
    If I'm reading this right, this is (really solid) future planning but won't help with the current outages?

    I'd have thought so from the headline + initial content but then they say that the horizontal scale techniques are basically carrying over from the existing architecture (where it seems the problem is?) and this is mostly to help with what they call "vertical scale" (intra-repo scalability). They also admit right after that the most active repos today will receive short bursts of at most dozens of writes.

    So this seems like really useful planning for a near future that likely helps prevent more issues but isn't really meant to help current issues. Am I reading this right?

    Of course improvements like this are always useful either way. Not dunking here at all.

    • rockwotj 10 hours ago
      personally I see more issues with actions over pushing
      • Walf 3 hours ago
        What? 7.2k CI runs per second is a piece of cake! ;-)
    • AbhiDunna 6 hours ago
      [flagged]
    • pksunkara 10 hours ago
      Since they wouldn't need to actually have the inactive repositories on disk anymore, I reckon it should solve the horizontal scale issue too by reducing the compute/disk capacity that needs to be maintained.
  • pksunkara 10 hours ago
    Most of this is similar to what https://enroute.sh already did scaling Git. https://enroute.sh/blog/everyone-is-doing-git-on-s3-wrong
    • esafak 9 hours ago
      Looks cool. Anybody here with experience to share?
    • cedel2k1 10 hours ago
      At this point GitHub should likely just try snatching the Enroute guys :)
  • dwm 1 hour ago
    This looks a lot like Cursor's Continuity, though Github are reporting an order of magnitude greater write throughput.

    https://cursor.com/blog/git-at-any-scale

  • bob1029 1 hour ago
    I think perforce might be the better VCS if you intend to have a bunch of autonomous agents attempt to collaborate over one big pile.

    Multiple terabytes is no problem out of the box. You get exclusive locking of files. You can easily take partial checkouts of the monolith. Centralization is a feature in this particular context. Git works great if the need for coordination is moderate. If you've got thousands of automated developers with their hands in the cookie jar, exclusive locks start to look really compelling. Perforce sucks with a highly distributed team, but in the case of an AI agent swarm we presumably have the ability to instantly nuke an agent's checkout if needed. I don't have to contact the agent and wait for them to get out of bed.

  • jxramos 7 hours ago
    Agent scale is definitely on the mind around industry, I wonder what github folks would say in response to https://www.anthropic.com/research/redesigning-source-contro.... It's an interesting read to step back and consider source control from first principles and rebalance the tradeoffs for what qualities matter in this new regime.
    • jason1cho 3 hours ago
      Thousands of changes per day means no human reads all these changes.

      There're really security issues, and a black box creates black boxes.

      • mdbug 1 hour ago
        Humans are also black boxes
  • rochacon 12 hours ago
    So that is why I got a bunch of 500's during push this week :)

    Hope it helps improve reliability, it was getting hard to defend GitHub with all the recent outages.

  • petervandijck 3 hours ago
    I wonder if they’re working on a non git version forward looking for agents as well
  • mitxela 8 hours ago
    You could also just ban people who spam you and consume immense amounts of server time. Or make them pay for the server time.
  • crote 9 hours ago
    Sorry, but thousands of pushes per second, per repository?! In what world could that possibly seem like a sound idea?

    Sure, it's nice that you can technically handle it, but to me it sounds like at that point something has gone absolutely horribly wrong in the way you set up your agents. Last time I checked, GitHub was intended as a platform to upstream finished work items, not a realtime cross-agent chat channel.

    Handling thousands of git pushes per second implies thousands of issues, thousands of PRs, tens of thousands of comments, thousands of Actions runs, and dozens of releases. How's that part of their infra going to scale?

    How about introducing rate limits instead, nudging people to not have agents push every individual single-line changes to GH, but instead have them collaborate as a local cluster and only upstreaming PR-sized chunks of finished work?

    If you still need thousands of pushes per second you'd be looking at millions of lines of code, and at that point you're looking at an insane monorepo which is growing at a rate of terabytes per day, and which basically instantly becomes unusable due to being impossible to clone or check out.

    • clivedup 9 hours ago
      My understanding is that forks on GitHub share the same storage in the GH backend.

      It's not unreasonable to imagine thousands of commits per second to the kernel, especially if you consider that bots might be watching LKML and applying patches to various forks.

      > Handling thousands of git pushes per second implies thousands of issues, thousands of PRs, tens of thousands of comments, thousands of Actions runs, and dozens of releases. How's that part of their infra going to scale?

      For each issue my agent might raise 5 independent PRs as experiments.

      Further, the more changes there are, the more important it is that they don't conflict, which means smaller changes, which means even more changes.

      Bots interleaving typos, single line whitespace fixes, single dependency bumps, etc.

    • MisterMunchkin 2 hours ago
      Yeah it’s just total slop. If your code is slop anyway, why commit it? Just push it at major milestones. It’s not like it matters if you lose it.
    • locknitpicker 1 hour ago
      > Sorry, but thousands of pushes per second, per repository?! In what world could that possibly seem like a sound idea?

      The only conceivable scenario that I can imagine leading to this volume of writes is a push-based GitOps repository for a huge cluster holding a large volume of apps, each one with many services. Each commit in one if these services triggers a commit to the gitops repo to update the reference to the new container image, which in turns triggers a deployment update.

      There is no software project on earth that justifies a rate of changes in magnitudes of dozen of commits per second. This is a symptom of a huge problem, not a sign of times. Even in monorepo mania this sort of thing is inconceivable.

      But that's also what you get with vibecoding: lots of output from people who do not have a clue about what they are doing.

    • mitxela 8 hours ago
      Welcome to the new AI future, where everyone is insane and reality doesn't matter.

      I'm starting to think AI isn't a productivity machine, but rather a machine for hypnotising people into the false belief of productivity.

      • __s 4 hours ago
        Yesterday's "web scale" is today's "agentic workload"
      • conartist6 1 hour ago
        You weren't supposed to notice
  • usr1106 6 hours ago
    Based on Azure... Isn't that one of the more unreliable clouds in the market?

    Edit: https://assets.theregister.com/2026/04/04/azure_talent_exodu... The blog post that story is based on was also on HN.

    Edit2: https://news.ycombinator.com/item?id=47616242 (1297 points)

    • landoncrabtree2 3 hours ago
      Define unreliable? Azure has a 99.97% historical uptime, AWS at 99.95%. Azure is #2 in market share, right beyond AWS. Engineer anecdotes about internal strife doesn’t equate to external reliability. I’m sure you can find AWS engineers complaining about AWS, or GCP, or insert company. (Also, Microsoft owns Azure and GitHub, so what did you expect?)
  • mococa 11 hours ago
    So we can have one more 9?
    • cr125rider 10 hours ago
      It will be nice having maybe a few at the beginning of the number!
    • UltraSane 9 hours ago
      Maybe every other month.
  • fifjcj38384 8 hours ago
    it’s pretty ironic we got all the complexity of distributed source control with all of the drawbacks of centralisation
    • locknitpicker 1 hour ago
      I don't think this is ironic at all. The source control system is decentralized, in the sense that you can checkout a repo and use it as the central source or truth for everything.

      For example, you can clone a repo from GitHub into your workstation,and then clone that repo into multiple folders in your workstation where you unleash agents to implement changes. The you pull those changes to your local repo or even directly to GitHub or GitLab or whatever. That's pretty decentralized.

      Now, compare this to the god awful mess that was subversion.

  • fnord77 7 hours ago
    Which repo were they referring to that was growing too big vertically?
    • qrobit 2 hours ago
      The biggest very active repo I know of is nixpkgs, and I heard it was a stress test for GitHub already.
    • jacques_chester 6 hours ago
      Possibly Shopify's monolith.

      Or alternatively someone is doing something very silly.

      • byroot 3 hours ago
        Nah, they replaced GitHub with some in-house videcoded stuff a few months ago, and from what I hear from the grapevine it's not going well.

        But there's a number of other big private repos with a high rate of merge on GitHub, and indeed it's currently a bit painful, e.g. the subroutine that check whether a PR can be merged tend to take ages. So my read is that they want to better support these big customers out of concern that they might leave too.

  • pentagrama 8 hours ago
    [dead]
  • chjail-11 12 hours ago
    As if more developers means better code. Wasn't it Bonhoeffer who said stupidity is when smart people don't question authority?