28 comments

  • keyle 2 hours ago
    I've been in those companies where "struggling departments" ended up getting all the praises and raise in budgets the following quarter because of the heroic saves they did, and raising awareness on how important they are... For stuff they totally caused on themselves.

    Meanwhile, my perfectly purring department was struggling to keep the lights on.

    It's a serious problem in this industry due to the disconnect between non-technical management (who understands how to double click) and engineering (who holds the company standing).

    <insert IBM story about IT department cost cuts>

    I'm not sure how we solve this, other than having management come from engineering.

    • dchevell 17 minutes ago
      By building pain into the system. If your hands dealt with injury directly without sending pain signals up to your brain, you'd never change the behaviour that led to that harm or reconsider your priorities. Like it or not, sometimes the best thing for an organisation isn't to just fix every problem and prevent it from bubbling up; it needs to be treated like a learning opportunity for org leadership, which means sending the pain signals upward before just repairing it.

      Building the right incentives around that can be tricky, those incentives need to ensure the highest levels of management aren't themselves disincentivising their directs & their departments from surfacing pain & problems - but it's also pretty common for people to mask those signals purely out of a well-intentioned desire to help. It's important to coach people on the idea that in large group sizes, it's more efficient to let certain kinds of problems play out and not be so reactive to them.

      Too many companies ground their performance incentives & processes around oversimplified ideas that don't match the reality of human behaviour

      • antonvs 14 minutes ago
        The problem is it's systemic. Ultimately, pain needs to come from outside. As long as society rewards incompetence, we'll have incompetent organizations.
    • willXare 2 hours ago
      The tragedy is that “nothing broke” looks like “nothing was done” to people far enough away from the system.
      • _carbyau_ 1 hour ago
        Things keep breaking - "What are we paying you for?"

        Nothing ever breaks. - "What are we paying you for?"

        Management can choose their burden.

    • al_borland 1 hour ago
      I think a good place to start is tracking all the proactive things being done and reporting them. At least then maybe someone will see why it’s quiet, because you’ve anticipated the problems and stopped them before they start.

      When things come up with other teams, you’ll have a catalog of tasks that were done to show why you didn’t have the same issue. The work was done, just at a better time to avoid downtime.

      • qurren 1 hour ago
        > start is tracking all the proactive things being done and reporting them

        Speaking from experience, this does nothing. If you're at a company that is okay with average performers, then absolutely, 100%, fix all the bugs in advance, make the system rock solid and stable, prevent downtime, be a good engineer.

        If on the other hand if you're at a company where 10% of people must get stack ranked and PIP, or at a company where "meets expectations" actually means you're going to get the stick, and you're supposed to be "redefining" expectations every year ... then yeah, don't do anything preventative. The optics are better when you take the 3am on-call and fix the issue (that you secretly knew in the first place would happen some time in the future in your coworker's code, and already knew how to fix -- but don't actually fix it until it surfaces). Be the savior that the VPs praise in the next meeting, that's your insurance against the PIP.

        They set the rules of the game, you just play the game. The rules were their choice. They could have chosen different rules.

        • nrightnour 33 minutes ago
          I'm sorry about your experience.

          Personally, I only rehire people from projects that went smoothly, not ones where I had to make the urgent phone call.

          Teams that "just work" are highly valued. They clear up my attention for other things.

          • fgonzag 27 minutes ago
            Teams that just work can't exist in stack ranked companies. You can't keep the team as a whole, you always have to cut someone.

            Which means that everyone is playing the game to not be cut.

            • bruce511 4 minutes ago
              True, stack ranking is a terrible management approach, and if you work at a company that does it, then playing the game is the only way. But frankly, I'd be looking to get out anyway. The best way to play thr stack ranking game is to be job hunting.

              But I'm not sure the author of this thread works in such a place. In that case the game is different.

              In the case where the "urgent midnight fix" is important, it's necessary to promote the visibility of your (just working) team. If visibility is the game, then be visible.

              You know how test-driven-dev was always "write the test first"? In that environment a test is always written before any code.

              Well in the "ticket closing" scenario it's important to open a ticket, regardless of how trivial, for every code action taken. For every meeting attended. For every scenario dodged. If tickets are the way to score then write tickets.

              If "being a hero" is the valuable thing, then be a hero. Be prepared to champion your team every chance you get. Every time you interact with management stress the emergency you just fixed (before it became an emergency.) Tomorrow do it again with the next thing.

              Management needs visibility. Be visible. I know, this seems stupid and beneath you. But that's why they call it a job, not playtime.

        • al_borland 42 minutes ago
          I refuse to play those games. If they want to fire me for avoiding problems instead of sacrificing my sleep, fine. I’ll go stock shelves at Walmart.

          If someone is constantly playing the hero, I see that as incompetence. If the boss can’t see that, they are also incompetent. I have no respect for “leaders” who don’t know how to get out of the firefight.

          I’ve made some high profile appearances, working 18 hour days on 4 day long outages, from vendor issues I was no part in causing. I figure that gives me some good will on playing hero without willingly creating problems for myself. I’m too old to manufacture stress for the optics.

          For what it’s worth, with the right boss, I have had proper reporting work. Everything ran smooth and work was relaxed. My boss would regularly tell me I should take 3 months off because we were so far ahead of everyone. He would occasionally get bored and lob a grenade into the works to cause some chaos, but since everything else was running so smooth we were able to sort them out and keep going. People who couldn’t explain what they were doing were always getting yelled at and assumed to be doing nothing.

          • qurren 38 minutes ago
            > I’ll go stock shelves at Walmart

            Yeah, but then I wouldn't have been able to pay for my healthcare. A certain toxic company's health insurance paid for my care, though. Prior to joining said toxic company I'd be racking up $6000+ in healthcare bills a year with shitty startup-sponsored insurance.

            After 2 years, it was decided I didn't play the hero well enough though, and ended up having to leave. I work for a less toxic company now, but the next time I need a heart-related surgery (likely in ~5-10 years) I'll join a toxic company in the months leading up to pay for it.

            The rules of the US, I guess.

            > I’m too old to manufacture stress

            My point was less about manufacturing artificial stress. I don't do that. But many times I see issues in coworkers' code. If the company will value and praise me for catching and fixing them early, then by all means I'll do that. But if fixing issues in the codebase early for prevention only gets me criticism of "you haven't met expectations, we expect you to exceed expectations every performance cycle" then hell, I don't feel like fixing anything proactively. In that world I'd rather be the hero that fixes it when it surfaces, that's more likely to nail the rating.

            • al_borland 23 minutes ago
              Health insurance does complicate things. I hope your heart is doing well now.

              I will say my motivation for helping other people avoid issues has dropped. If they want to make problems for themselves, they can. Me helping them hasn’t worked so far, so maybe some sleepless nights will be a better teacher.

              I had a former boss call me Brent after reading the Phoenix Project. That made me step back and stop helping so much. Everything seems worse, but whatever… if that’s what they want.

    • HerbManic 1 hour ago
      This thinking eventually results in The Scream Test. When the screams come as a system fails that is when they act on it.

      Alas, for many parts of society there is a large amount of people that would rather be reactive than proactive. It means it is easier today but harder long term.

    • markvdb 2 hours ago
      > It's a serious problem in this industry

      s/in this industry//

    • ChicknNuggt 2 hours ago
      I feel that disconnect is everywhere, when the suits dont see anything and act on reports
    • kshacker 2 hours ago
      lol. I hate presentations. I like to run a tight ship. But that does not shine, so they made me do presentations every quarter. If you do some work, you must "take" credit. It is kinda a need when you manage people since you need to build their careers.

      I finally moved on to be an IC. Same story, same pressure :) You need to present to directors not because they need to know, but because your managers have a quota of N presentations per quarter, and if you back out, someone else needs to step up.

      Needless to say my productivity reduces by half and sometimes to almost zero during the week or fortnight of presentations every quarter.

    • kamaal 1 hour ago
      >>I'm not sure how we solve this, other than having management come from engineering.

      Given the whole point of management is to work to ensure their own survival and growth, it would in their interest to kill genuine competition when its coming up.

      Who wants to raise their new competition and lose to them, no one!

    • esafak 1 hour ago
      Track leading indicators, pricing them if possible.
  • timmg 2 hours ago
    There are a lot of things like this.

    My favorite is how elegant solutions often look simple in retrospect. So if you noodle on a problem for a while and then come up with a clever solution: once you explain it to someone they'll be like, "yeah, of course."

    Meanwhile the guy next to you that overcomplicates the problem ends up getting kudos for building something so difficult :D

    • npunt 2 hours ago
      I feel like AI coding is accelerating everyone's work toward greater solution complexity and I think it's pushing people to build defenses and be more averse to someone else's complexity rather than being impressed by it. Bigco's are probably well behind the curve on this and are still impressed by complexity, but for people on the receiving end of AI stuff either directly via your own hand or indirectly via others, it seems like complexity is not as impressive as it once was.
    • fugaziboutit 1 hour ago
      "Je n'ai fait celle-ci plus longue que parce que je n'ai pas eu le loisir de la faire plus courte."

      ("I have made this longer than usual, only because I have not had the time to make it shorter.")

      Blaise Pascal

    • HerbManic 1 hour ago
      I have said it for decades - Basic is easy, Simple is hard.

      But when someone comes up with something simple but effective, it always looks so obvious in retrospect.

    • nullhole 2 hours ago
      The passage that comes to mind for me whenever this idea comes up, from the Brett version of the Holmes story "The Dancing Men":

        H: So, Watson.
        W: Hmm.
        H: You do not propose to invest in South African securities?
        W: How on earth do you know that?
        H: Now, confess, you are utterly taken aback.
        W: I am!
        H: I should make you sign a paper to that effect.
        W: Why?
        H: Because in a few minutes you will say it is all so absurdly simple.
        W: I should say nothing of the kind!
        H: You see, my dear Watson, it is not really difficult to construct a series of inferences, each dependent upon its predecessor and each simple in itself. If, after doing so, one simply knocks out the central inferences and presents one's audience with the starting point and the conclusion, one may produce a startling, though possibly a meretricious, effect.
        H: I can tell by an inspection of the groove between your left forefinger and thumb, that you have decided not to invest your small capital in the gold fields.
        W: I can see no connection.
        H: Very likely not; but I can quickly give you a close connection.
        H: Here are the missing links in the very simple chain: You had chalk between your forefinger and thumb when you returned from the club last night. You put chalk there when you play billiards, to ease the cue. You never play billiards except with Thurston. Now, Thurston, you told me, four weeks ago, had an option on some South African security which expired in a month, and which he desired you to share with him. Your checkbook is locked in my drawer, and you have not asked for the key. So, you do not propose to invest your money in that manner.
        W: How absurdly simple!
        H: Quite so. Every problem is absurdly simple when it is explained to you.
    • johnthescott 2 hours ago
      "elegance and speed go hand in hand" - d. mcilroy
  • whatever1 1 hour ago
    I never gave credit to my electricity company for delivering electricity to me. I only get mad when there is an outage.
    • tikhonj 2 minutes ago
      Counterpoint: I absolutely give credit to Sonic for being a great ISP and recommend them to everyone. I got my parents to switch when Sonic finally rolled out to their neighborhood.

      If online comments are anything to go by, I'm not alone.

      If you're in the Bay Area and you can get a Sonic fiber connection, I would highly recommend them over AT&T/Comcast/etc.

    • m463 10 minutes ago
      I think of electric vehicle fires and jet airliner crashes.

      Also, telsa self-driving. yes, we know about the greatly publicized accidents, or the tweets of the founder, but the avoided incidents ones not so much.

    • HerbManic 1 hour ago
      Just remember, the power grid fails in theory but works in practice.
  • smath 1 hour ago
    I'm looking for some data -- if anyone has it -- on the fraction of companies that are led (CEO) by a technical person, over the years/decades. I have the (anecdotal) impression that this fraction has been falling (stories like Boeing), but it would be cool to support or refute this with hard data. Anyone know where to find/assemble something like this? Also, if this trend is true, then why?
    • doctorwho42 1 hour ago
      I don't have the data, but I think a good case study is the company MITRE.

      Originally it was engineers from the top down, but over the last 15-20 years those leaders with engineering backgrounds have retired and been replaced by non-engineer MBA's. And the more I look around, the more I see that as a common trope is the US.

    • fsagx 1 hour ago
      If this has been true, perhaps at some point the pendulum will swing back the other way. BTW the current CEO of Boeing is a mechanical engineer.

      https://www.boeing.com/company/bios/kelly-ortberg

  • didgetmaster 1 hour ago
    We all learned this back in first grade. The kids that behaved in class and did their homework did not command most of the teacher's time and effort. It was the problem children who refused to follow the rules and needed constant praise for every bit of actual effort that they put into their studies; that got the teacher's attention.
  • throwawa1 3 minutes ago
    Its called doing your job.
  • throw0101c 1 hour ago
  • sandeepkd 1 hour ago
    Its really hard to measure effectiveness, problem becomes even harder when a non-engineering person has the job to measure effectiveness of a engineering person.

    On other hand. for software engineering some of the signals that can be used to measure such a management itself can be

    1. On call requirement, outages and team burnout - A well written software should not require on-calls from the dev team

    2. Ask them about the "concrete" roadmap for next 6 months to a year - Absence of concrete items is a bad sign

  • jacques_chester 2 hours ago
    You'll see capability traps everywhere once you learn about them.

    Sterman, Repenning and other collaborators wrote several papers after this one. All fascinating and almost entirely depressing.

    Especially since MIT's Sloan school, where system dynamics first became a discipline, is just around the bend from Harvard Business school, where system dynamics first became ignored.

    • macrocosmos 1 hour ago
      One thing I don't get about the concept of capability traps is why is it expected that a company which is good at one thing would be capable at the new thing? What exactly makes a capability trap a trap?
      • jacques_chester 1 hour ago
        The trap is that you can't get better without first getting worse. You can't get out of the destructive cycle of production pressure and decaying productivity without removing the pressure. Many managers expect, or at least behave as if they expect, improvement to be monotonic and costless.
  • ChicknNuggt 2 hours ago
    This is exactly the problem with the nature prevention. When it's well done, it seems like nothing was done.
    • HerbManic 1 hour ago
      When Covid started, our local government was very clear from the start in saying "If people think we have over reacted, then that means we have done a good job."

      Alas, that doesn't always fly with the populace.

  • mdmabatj 2 hours ago
    There is something I saw on a reddit post of all places, about how every manager who doesn't predict a baseline of "3 annoying problems every month, 1 awful problem every 3 months" is essentially a bad manager. The reasoning being that, if your number of problems is under that threshold, then someone is doing a 'good job'.
  • random3 2 hours ago
    Like nobody gets credit for avoiding problems or unnecessary things/complexity altogether. In fact the opposite may happen.
  • Jtsummers 3 hours ago
    Two significant prior discussions:

    https://news.ycombinator.com/item?id=8940820 - 24 Jan 2015, 50 comments

    https://news.ycombinator.com/item?id=39472693 - 22 Feb 2024, 434 comments

  • tjmc 1 hour ago
    This is why I'll never be a fire protection engineer
  • rmunn 2 hours ago
    Article published in the Summer 2001 edition of California Management Review, yet it never mentioned Y2K, the first thing I thought of when I read the line "fixing problems that never happened". Perhaps it was actually written in 1999 and took a while to get published, because otherwise that seems a very strange omission. The Y2K problem was very much over-hyped by the American news media at the time (no, at no point would airplanes have been falling out of the sky — I literally heard someone say that would happen once — even if no effort had been put into fixing the bug).

    But in recent years I have seen people (elsewhere, not on HN) claim that Y2K was a big nothingburger, and all the money spent on fixing the bug was wasted. No, that's not true either. All the money spent on fixing the bug was why it turned into a big nothingburger. Sure, some of that money was wasted, by executives who wanted an "official" Y2K-certified certificate, issued by a consulting firm that had nothing "official" about it except their own say-so. And so they spent $2 million learning what their own employees could have told them for $2,000. THAT money was wasted. But a lot of banks were running old COBOL code that used 2-digit years, and needed to be fixed. The fact that in January 2000, everyone's bank interest was still calculated correctly, and not calculated as if it was January 1900? THAT was entirely due to the vast amounts of money spent paying old COBOL coders to come out of retirement and fix the 2-digit years.

    The lesson I learned from that is that it's possible for a problem to be overhyped, even massively overhyped, and yet still be a serious problem. The other lesson I should have learned is that people rarely get credit (I won't go so far as the article authors and say "nobody ever gets credit") for fixing problems that never happened.

    • armada651 2 hours ago
      The problem is that a lot of people have a very binary view on life. Either something is a complete success or a complete waste of money, rarely do we accept that most projects fall somewhere in the middle.
      • cortesoft 1 hour ago
        And even worse, they don't think probability is a thing. If something happens, it was certain to happen and we just failed to predict it correctly.

        So when someone predicts something will happen with a 90% probability, and then the 10% chances happens and the predicted event does not happen, people will talk about what a bad prediction that was and how they were clearly wrong.

        It's the same logic that causes people to say vaccines don't work because they don't stop a disease with 100% effectiveness, or that there is no point to wear a seatbelt because people still die while wearing one.

    • takinola 2 hours ago
      My issue with this version of explaining the lack of severity of Y2K is that there were lots of countries that were being derided for not taking the issue seriously but did not seem to suffer any ill effects.
      • akoboldfrying 2 hours ago
        This is interesting, do you have any links?

        A couple of possible confounding factors I can think of:

        1. Plenty of countries use software developed elsewhere.

        2. I suspect that the more recently you computerised your economy, the less likely it would be to have code vulnerable to Y2K.

    • tjwebbnorfolk 2 hours ago
      Y2K is especially interesting because the fact that the year 2000 would one day occur was entirely foreseeable, and no less probable in 1990 than in 1999. I can hardly think of anything with closer to 100% probability of happening.
      • alduino 2 hours ago
        To be fair, there was a non-zero chance that society could have ended (or your company, or the tech became obsolete) before 2000, which would be higher the earlier before 2000 you were.
        • rmunn 2 hours ago
          The tech being obsolete is why Y2K was a smaller problem than it would have been otherwise. Most places were no longer running much COBOL code. But banks are famously slow to upgrade their tech, and for good reason much of the time, so most of the world's remaining COBOL code (and other code too, COBOL is just what I'm most familiar with, not that I'm all that familiar with it) was in banks and other financial institutions.
      • Dwedit 1 hour ago
        Year 2038 says hi.
  • hedora 2 hours ago
    Two counter-examples:

    - Arnold bought a fleet of mobile hospitals that would have been perfect for covid response, but the next governor didn’t want to pay 1% the fleet cost per year to maintain it, so he scrapped it.

    - Under Obama, SARS v1 was stopped by US health workers that Trump fired because it was a “bad deal”. In the absence of that team, we got SARS v2, which was renamed to COVID 19.

    There’s also the related category of “never blamed for fixing problems poorly, creating even bigger problems”.

    Thanks to 9/11, plane cockpits can now be locked from the inside. Now, we have examples of commercial passenger airline pilots locking the doors and committing mass-murder-suicide by plane crash.

    For some reason, these stories don’t make the news.

    • arcanemachiner 2 hours ago
      You might want to double check your dates on that SARS claim. Are you talking about the swine flu outbreak?
  • cm11 57 minutes ago
    I'd guess a lot of people here consider this "reality" at this point. Has anyone come up with a response—not a fix for the company or leaders behaving this way, but a response for their own path?

    Did you change from a quiet diligent one to manipulating and playing the game (now that you know the game)? Did you go from quiet and diligent to quiet and not diligent (why do good work when meh work does the trick)? Another path?

  • erelong 2 hours ago
    So let's create moments or days of observance to make people aware of preventative measures taken
    • alok-g 1 hour ago
      I align as such.

      Then we soon see non-technical people start leading the same, pushing for some people to be recognized for this every sprint. Meaningless recognition starts coming in. The process fades out in just a couple weeks.

      The problem is there are these people in the mix, often leading, who do not understand.

    • al_borland 1 hour ago
      A moment of silence to appreciate the silence the proactive measures gave us.
  • tinyhouse 30 minutes ago
    This is very true and applies for everything in life.
  • Guestmodinfo 2 hours ago
    Human civilization runs on personal sacrifices but money bags will never care about that.
    • N_Lens 2 hours ago
      Good thing we’re converting human civilization into money bags at maximum speed, then. Solves all problems elegantly.
  • nxy 2 hours ago
    Very true! Along with it comes with peace/quietness at work so it’s not too bad.
  • jmyeet 1 hour ago
    This is the real problem with performance reviews in companies, which then feeds into opportunities, promotions and compensation. It's just a popularity contest. And this is particularly harmful to people who are neurodivergent, particularly if they're on the autism spectrum, because neurotypical people, who end up making all these decisions, view such people negatively for literally no reason.

    You could spin up a team of 6 engineers and have them go away and try some greenfield project. They could come up back in 6 months having shipped nothing. Which of these descriptions fits the facts?

    1. The team learned a lot and ultimately decided there was no product-market fit and decided it was best to reallocate resources elsewhere. The learnings from that project will help a whole bunch of other projects across the division; and

    2. They failed to ship and get subpar performance ratings for having no impact.

    The answer is... both. Or either. How you are treated will depend on how you are viewed by your management chain and that's a social function. We've all encountered people who never shut up about how hard their job is. Often they end up solving problems that they created, often by not listening to anyone that those problems would occur. And they get credit for it.

    You could say to people who anticipate problems to stop because it gets you nowhere. Let people fail. If only it worked that way. Instead you'll get blamed for not seeing a problem someone else created because you're viewed as competent but you aren't liked through no fault of your own.

    Google seems to be the posterchild for a company that briefly solved this problem and then forgot what made them successful. I am referring to Project aristotle [1], which ultimately determined that psychological safety was the key ingredient in a team's success.

    Now amplify all of this with constant rounds of layoffs where the environment isn't just for pay bumps and opportunities but where the cost of failing is losing your income. What you've created is an environment where office politics is everything.

    [1]: https://psychsafety.com/googles-project-aristotle/

  • lstodd 3 hours ago
    > The combined expenditure of U.S. companies on management consul- tants and training in 1997 was over $100 billion

    erhm, if this figure is close to true i can see what market ai companies is after.

    • jacques_chester 2 hours ago
      AI slots quite neatly into the capability trap model, actually.

      Which loop it belongs to in the model is left as an exercise for the reader.

  • senectus1 2 hours ago
    sounds like my day to day job experience.
  • sublinear 2 hours ago
    Making critical decisions without oversight is just as bad, or maybe worse.

    If you frame it this way in a meeting, you will get the attention you want. Don't say I didn't warn you because that comes with a lot of scrutiny you might not want.

  • everyone 1 hour ago
    The jokes around Y2K being a nothing burger always annoy me. Nothing happened because a lot of talented people worked their asses off fixing it.
  • Viacol 1 hour ago
    [dead]
  • dang 2 hours ago
    Related. Others?

    Nobody ever gets credit for fixing problems that never happened (2001) [pdf] - https://news.ycombinator.com/item?id=39472693 - Feb 2024 (424 comments)

    Nobody Ever Gets Credit for Fixing Problems That Never Happened (2001) [pdf] - https://news.ycombinator.com/item?id=8940820 - Jan 2015 (50 comments)