Software is breaking down at a systemic level

(breck-mckye.com)

67 points | by zero_shift 33 minutes ago

29 comments

  • doug_durham 11 minutes ago
    This narrative isn't consistent with reality. Cloud computing is much older than 10 years ago. We were doing production grade cloud computing over 16 years ago. The OSS revolution dates to the late 90's. The author seems to think that SWE was in a golden age 10 years ago. Sorry, but it's about the same. We have the same problems in SWE as we had 40 years ago. The scale and ambition of the problems we are solving have increased as compute capabilities have increased.
    • cmiles74 1 minute ago
      AWS is 20 years old, before that people did rent machines from places like Rackspace. Maven Central (Java) is 25 years old, RubyGems is 22 years old, CPAN (Perl) is 31.

      I find the arguments about decoherence and the sort of snowball-of-complexity more compelling than this timeline.

    • nosefrog 4 minutes ago
      Yep.
    • nullsanity 4 minutes ago
      [dead]
  • CharlieDigital 11 minutes ago
    It doesn't have to be this way.

    AI is great at following patterns. AI is great at writing tests. AI is great at doing work that humans would consider drudgery and then applying "enough" judgement to it.

    The problem is that most software teams and most engineers are very, very bad at the thing that AI needs to succeed: technical writing. Documenting the patterns, the rules, the heuristics, the SOP, the judgement; developers could barely be bothered to write comments before (which in the AI age are free, infrastructure-less memory that is always "line-of-sight" for the agent and can be used to enrich with keywords).

    It's not that AI is the problem, it's that we humans are good at resolving these constraints in the real world at a human pace. AI makes that pace impossible for a human to keep up as a trailing activity. Rather teams need to invest more in technical writing and producing the corpus that AI relies on to learn the operational constraints and rules.

    • logarhythm 0 minutes ago
      I completely agree, and I’m building a tool to solve exactly that challenge.

      Pathfinder lets you understand your product’s building blocks. So you and your team pilot your agents at the product level, not the code level.

      It’s augmented product engineering: https://pathfinder.com.ai

    • cmiles74 4 minutes ago
      The urge to one-shot everything isn't helping. Getting people to push back on assumptions the model is making, and to have an actual conversation with it about the solution, would lead to more maintainable output.

      Maybe part of it is laziness but for sure part of is low morale and hopelessness.

    • oulipo 3 minutes ago
      "AI is great at following patterns", sure but if some attackers can "inject attack patterns" at a high rate in your training set (check "dataset poisoning") then it's going to be an issue...
    • Forgeties79 5 minutes ago
      SOP and documentation is everything. You don’t want it going out of bounds and coming up with its own solutions, which are often just inefficient facsimiles of random stuff it finds that seems to makes sense. You also run the risk of not understanding the solution it puts in front of you. With an ironclad SOP and quality documentation, you know when it is straying from the best paths, because they are laid out for both of you.

      Plus your documentation can often work very well alongside whatever skills you have created over time, because you basically have two sources it is drawing on telling it to do the same thing. I find it leads to less off book behavior.

      I’ve always been perplexed by how people see AI as this creative tool or something to innovate with. That’s not how I see it at all. I see it as a device that can follow a template very well if you lay good tracks. They can archive and check hundreds of pieces of media way faster than I can. But I would never trust it to craft a story

  • ljm 5 minutes ago
    Software wasn't exactly hard 12 years ago either, the barriers to entry were being lowered at a rapid pace. 2014 was the dawn of the software bootcamp where JS became more and more pronounced in web architecture.

    I feel this piece is either more alarmist than it deserves to be, or it is late to the party.

    It uses various incidents in 2026 to prove a point while dismissing those that did take place in the decade or so previous. For example, heartbleed, the log4j RCE, Rails' mass assignment DOS, Cloudflare updating a WAF rule and breaking the internet, Atlassian deleting almost 1000 of its customers' accounts and data, the Ashley Madison breach, Crowdstrike bricking their customers computers, Okta getting owned multiple times, etc.

    My own point is that software became like this when we started becoming pragmatic. Nothing to do with AI or management really, we just realised, especially with web tech, that you can release something and then fix it. In the old age of getting CDs or floppies printed and distributed, that was not even remotely feasible.

    Standards have dropped and continue to do so. It's not suddenly breaking down.

  • LPisGood 24 minutes ago
    Software has always been broken at a systemic level. It’s a complicated continuously changing web of abstractions and abstraction breaking hacks attempting to survive prolonged contact with reality.
    • kerblang 7 minutes ago
      It was less broken before open source allowed an architectural explosion, because the dependencies were too expensive. It took a lot of arguing to get management to splurge on a single library or other tool. A lot of astronaut-architect wannabes were constantly grumpy about mgmt that wouldn't pay for the latest whiz-bang thingie.

      The actual programming logic has certainly always been complex, but the integrations weren't so bad. You could learn your very small toolset quickly and focus on ideas. Now, you have an ultra-massive toolchain to come to terms with, made up of free software and "cloud" architecture that starts off cheap and gets expensive later on.

      Dependencies have always been the problem, but that problem has expanded in every direction, at every layer, approaching our own limits of comprehension. AI has allowed us to catch up, temporarily, but given the collective penchant for upping the ante, I expect the real supernova is yet to come.

    • psychoslave 8 minutes ago
      Well, yes, but the proportion of systemic level broken stuffs might have change a bit. Just like, ok, there was always rule34 lurking to make dirty everything innocent that remained of the untainted child memory one had.

      Now it’s somehow guarantied that the volume of dirtied quickly prompted version of anything anyone ever cherished far exceed the scarce outcomes of the original.

    • SV_BubbleTime 20 minutes ago
      I do embedded. I’ve always marveled at how ludicrous that “real software” is. I have truths. I have registers that will never be or act differently, I can make pretty damn good assumptions that no one will put this code somewhere else’s etc.

      In the past few years though, it’s been creeping into my domain too.

      • bumby 14 minutes ago
        >I have registers that will never be or act differently

        Even this needs a qualifier. Radiation exposure can cause bit flips. Certain aerospace applications must address this, and even some earth-bound applications at high altitude.

        • marcosdumay 5 minutes ago
          Besides, chip bugs are famous since the Pentium, and unreliable circuits are the normal way to describe circuits, you have to add notice when you get a reliable one.
        • skydhash 1 minute ago
          That's why we do requirement gathering and analysis. Engineering is always about tradeoffs and things that matters in certain context don't in others. The issue is that in a lot of orgs, while there's a record of decisions, we don't usually track assumptions and the overall system is full of contradictions. Also there's no overall keeper of knowledge (either one person or a team) that periodically ensure that contradictions are kept at minimum.
        • LPisGood 8 minutes ago
          At high altitude or high deployment numbers. Hyper scaler data centers measure these “one in a million” anomalies in thousands of times per houe.
        • leftnode 7 minutes ago
          Yeah, don't we have error correcting RAM for this reason, here on Earth?
    • add-sub-mul-div 8 minutes ago
      I don't get this thing where people are allergic to the idea that change can happen.

      No matter what's trending worse there's always this thought terminating phantom law of conservation of everything. The fact that something is not an entirely new bespoke phenomenon always turns into denial that it could be happening more often or in a newly qualitatively worse way.

  • dividedcomet 25 minutes ago
    This feels a little hyperbolic, the grounding was a result of a cable swap and the ssh cert incident they cite was so human caused it’s not even funny. I think things are a little more shaky, but not massively so. I had the town I was living in during college lose its internet and phone lines due to a cable being cut on a mountain pass and it was fine. If your business is sensitive to an email outage, prioritize high reliability over integration advantages that outlook offers. The php2shell vuln is bad, but we also had log4j years ago.
    • sharkbot 0 minutes ago
      I’d like to analogize the situation with the Kessler syndrome: the increasing density of satellites in low Earth orbit increases the probability that a cascading failure occurs, which would deny us access to LEO for centuries. The inciting incident could be intentional or could be accidental, but the result is the same.

      The analogous situation is brewing. The depth and complexity of our computational infrastructure is increasing, and the impact of an incident is less and less likely to be contained solely to the affected system. Eg, a router replacement goes awry, and the entirety of the world’s aviation industry is broken for weeks.

      Intentionality isn’t necessary: this is a systemic problem.

  • spiralpolitik 14 minutes ago
    There have been parts of the software industry that have been mission critical from day one and have built processes, languages, and tools to make sure things are 100% reliable. Building software that way is slower and more expensive and in the past has been an overkill for most systems.

    The problem software engineers are facing is that now that software is ubiquitous, almost everything needs to be 100% reliable (or close to it). This is coming into conflict with the move fast and break stuff everything needs to be done this quarter at minimal cost mentality.

    Hence why we are at the breaking point.

  • bob1029 5 minutes ago
    It feels like we are currently experiencing a soft version of the sewing machine riot.

    Perhaps no one is intentionally destroying anything in this case, but malicious compliance and simple negligence can go a long way when it comes to disruption of complex systems. Destroying a sewing machine requires actual violence. An air traffic control system is an abstract concept. You don't need to touch it to break it.

  • skeledrew 5 minutes ago
    Things are breaking down because they were weak to begin with, and demolition is needed in order to rebuild better, stronger. This is just a period of trial by fire: iron has to be heated and beaten numerous times (with carbon) before it becomes steel. All that's happening is AI is accelerating the evolution of industries into something which all eventually have the strength of steel.
  • anonreplier 21 minutes ago
    There are some decent points, but this is collating a lot of separate incidents with wildly different causes that just happened to occur in a close timeframe.
    • xg15 13 minutes ago
      Let's see how many more incidents will just happen to occur in proximity to those events.
    • llm_nerd 14 minutes ago
      I don't even know if there are decent points. This flails around in a way that's just...not super useful to spend the time trying to digest.

      Some things happened. AI is also happening, therefore AI is to blame (in the way that some people really believe a problem isn't a problem unless you test for it) and here's a lesson about organizational coherence or something.

      These essays are the real AI slop of 2026. Not authored by AI, but always trying to get attention by tangentially being related to AI and getting the anxious upvotes of the "can I turn back the clock?" crowd. And there's a humorous commonality to all of these pieces that they all start with some sort of "don't think I'm not with it. I'm with it! I hand slung an LLM off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate." call to authority, then pivoting to the "everything has gone to shit" spiel.

      We've always had a tenuously connected, barely operated network of systems. We've always had bugs and engineers doing shitty things. Every organization goes through periods of upheaval where all of the institutional knowledge is invalidated or discarded regardless.

      None of this is new.

  • tflinton 7 minutes ago
    I’ve had AI running to auto upgrade our infrastructure and it’s successful twice as much as humans. Failure rate of a human managed upgrade is 1/10, failure rate of an AI agent is 1/20 in my small set data.

    Problem is I’ve had 5x more incidents. AI is operating at 10x the speed of a human so even though it’s failing less, it’s causing more incidents per month.

    I wonder if people are ready for this kind of paradox where using AI may be safer and faster but less reliable for “human time”.

  • keithnz 11 minutes ago
    To talk abstractly about all software, you'd also need to analyze all the software that stays working. This article is just cherry picking a few recent incidents and try's to make some conclusions without really looking at causes and whether that can be generalized to "Software".
  • jstimpfle 18 minutes ago
    My impression is it's always been a mixed bag. The level of bugginess has been relatively constant. In the meantime, scope and capabilities have gone way up. I hate Teams and Outlook and the virus scanner on my company's laptop though. Those are keeping me from doing my work. Windows + anti-virus makes file access and process spawning 10x to 100x slower compared to a Linux machine.
  • tippytippytango 3 minutes ago
    Is defect rate increasing or is the amount of, and use of, software increasing? What is the defect rate relative to that?
  • drooby 20 minutes ago
    Most folks here didn't even know, but a Bitrise security outage hobbled our entire CI this weekend. Friday to Sunday night.

    The security issue motivated all Bitrise customers rotating their secrets. this was a big deal for mobile deployments but didn't get much attention on HN..

  • SoftTalker 16 minutes ago
    > You can find programmers who know how to manage the complexity, but they want to be paid lots of money.

    This wasn't always the case, by the way. It started with the "dot-com" mania and VC investment in the late 1990s. Before that, a programming job paid decently well but it was on the level of other cost-center professional jobs such as accounting. A comfortable living, but nothing extravagant.

  • travisgriggs 14 minutes ago
    > About ten years ago, two small revolutions happened in our industry ... And then, we had the Open Source boom.

    This seems a bit myopic. Or naive. Open Source "happened" a lot longer than 10 years ago. Microsoft didn't shit itself about Linux just 10 bare years ago. And it's not like prior to 10 years ago, the narrow focus of npm world wasn't cobbling web sites together with bits and pieces garnered from wherever.

    That said...

    > But AI is both a cause and a symptom

    This I agree with 100%. The rise of AI and necessity of AI to continue to make progress is a massive indictment against an industry that prioritized more features and uptime over solid first principles for years, taking out promisory note after note in complexity, in hopes that the stuff papered over would quietly just fade away.

  • _pdp_ 3 minutes ago
    On the flip side, once you hit rock bottom the only way is up.
  • chalmovsky 5 minutes ago
    > I believe that tech companies risk losing their “organisational cognition” - their ability to think and respond as a coherent unit.

    LMAO. They can't lose something they never had in the first place.

    This article has other pearls like this:

    > Reddit and Bluesky speculated that even MSFT engineers had no real handle on the complexity of this outage.

    Like, with all due respect, what the fuck do they know?

    Also this:

    >It’s the software that controls your flights, your banking, your petrol pumps, your TV, your payroll.

    Dude, how are you gonna sneak TV software into the same sentence as flights and banking? :D

    This is not a serious article, and I don't recommend reading it. But to finish on a positive note, since I've already (regretfully) read it, here's a sentence I like:

    > This is the real threat: a scenario where nothing in the system is done with intent, and nobody in the system has control. We generate complexity faster than we can tame it.

    I would just add that this was the case long before AI. And also, just as AI generates complexity at such a great pace, it can just as easily help you fix it and fight that complexity.

  • herval 23 minutes ago
    This reads like nostalgia way more than reality
    • wvenable 11 minutes ago
      I agree. In my life time there hasn't been a minute of software reliability. There's nothing special about this year or even this month.

      I remember the first time I installed Linux in the early 2000s and was and owned within 5 minutes of putting it on the Internet. Desktop computers of any OS crashed regularly. Servers crashed and needed total rebuilds.

      If anything software is way more reliable than it appears because the scale of computers and software is thousands of times larger than it was 20 years ago. More things can fail because there are just more things.

  • rurban 19 minutes ago
    He obviously is preparing another doomsday conference talk. Nothing interesting to see here.

    On the other hand software is improving dramatically at a systemic level, even without safe system languages arriving on the horizon.

  • titzer 9 minutes ago
    Not everyone is using AI to make shittier software. Most big important software projects I know of are frantically fixing an avalanche of AI-found bugs. Important software is getting more robust and resistant to attacks, while still adding functionality.

    There's a lot of slop, and it will almost certainly grow quicker than non-slop, but there is a big opportunity for those of us who still work at the bottom to make the pillars more solid, and take on re-engineering projects that would be infeasible otherwise.

    Case in point: Before Claude a port of my compiler to a new architecture might take anywhere from months to a year. After AI, now that there are multiple backends as examples and an even larger--massive even--test suite, this can take days or less. It took Claude about 3 days to come up with an arm64 backend for Virgil (around April time frame). Then in August it took just 3 hours to build a complete RiscV backend. Both of them pass the entire test suite. And they shook out other bugs elsewhere. Overall, more targets, and more robust.

    Things can have rocket boosters, you just gotta know where to place them.

  • adamddev1 18 minutes ago
    People keep believing that the big LLM players with somehow disprove Rice's Theorem. "Somehow we'll be able to get AI to check if a program does X or Y." "We'll be able to have the machines verify things automatically." But this is is of course as hopeless as solving the Halting Problem. Theoretically, we know this is impossible.

    Empirically, we can observe the breaking effects of this broken thinking. Code quality goes down, and apps go from being 99% to 95% reliable. But that's not a big deal, they say, that's good enough... But when the quality of all the building blocks further down get degraded in the same way, you can't keep track of which parts are broken, and which parts are dependable, and the whole thing comes crumbling down.

    People keep saying, "I can build an app with AI, therefore we can build everything with AI." This article talks about some potential anecdotes that could help to show that doesn't work.

    • habitue 16 minutes ago
      Rice's theorem is across arbitrary turing machines. We intentionally limit the kinds of programs we emit with type systems, knowing they will prevent us from writing some programs, but we'll be able to analyze the ones we can write much better.

      We prove properties of programs all the time, that's what a type system is for.

      Can we prove all properties about our program that we're interested in? Maybe, maybe not. But Rice's theorem is not the tool for that argument.

      • adamddev1 13 minutes ago
        Are vibe-coded apps being being written in Rocq programs that perfectly match dependent types to every possible behavior? Is anyone anywhere writing software with type theory at the level where it perfectly limits things and saves our programs from semantic mistakes?
    • ginthian 1 minute ago
      [flagged]
  • tcoff91 28 minutes ago
    There were package managers long before npm. Maven, rubygems, etc...
    • noosphr 22 minutes ago
      Apt and yum.

      It was a very long time ago indeed that you needed to manually edit popular programs to build in your environment.

  • AndrewKemendo 18 minutes ago
    Software was mostly always broken but it was hidden and invisible, now it’s visible because the cost of making it visible is so low

    This is classic surveillance/detection bias: https://healthjournalism.org/glossary-terms/surveillance-bia...

    It’s basically free to black/grey/whitehat penetration test anything and everything without having to even know what ghidra or metasploit so everything gets penetrated

  • tracerbulletx 16 minutes ago
    When I first started my career we used to ship like once a month and that was considered good, everything was extremely convoluted and worse than it is today, whole segments of the industry didn't even use Version Control, half the government websites you used stored passwords in the clear and emailed them to you for resets, e-commerce was a cgi script that sent an email for fulfilment and it took weeks to get your shipment, sites went down at night and weekends for maintenance regularly, servers fell over with traffic that would run on a raspberry pi now, but for some reason everyone keeps saying how bad things are now. They're delusional or don't actually remember the past.
  • jmclnx 22 minutes ago
    Well there was this saying "Do One Thing and Do It Well", seems once again we are seeing the affects of ignoring that.
  • TacticalCoder 13 minutes ago
    It's because amateurism has been hailed as the second coming of Christ.

    Leftpad, mentioned in TFA, is a case in point: not because of leftpad itself but because of all those defending leftpadism.

    "People can't write a proper leftpad".

    Yeah, well... If a dev cannot write a proper leftpad, then it should never have been a dev in the first place.

    Someone who cannot write leftpad is not qualified to use leftpad as a dependency (and, no, leftpad is neither the Linux kernel nor a cryptographic libray: it's fucking leftpad, which is the entire point).

    We've had "devs" whose only qualification was writing glorified "punch the monkey" ads in JavaScript: the JavaScript surveillance/ad complex. Devs who only ever knew the mono-threaded JavaScript ecosystem and suffered from heavy leftpadism.

    And those very same surveillance/ad complex bros are now generating countless crap with LLMs, in languages they don't understand.

    They're just mediocre.

    That's what happened: the veneration of mediocrity.

  • alazsolutions 3 minutes ago
    [flagged]
  • ginthian 7 minutes ago
    [flagged]