22 comments

  • mentalgear 1 hour ago
    It's coming for everyone: there's a startup that introduced robot arms that promise to learn anything 1 shot: if it holds - and there's little wiggle room for one shot - it would be a good indicator that manual labor is also soon gone.

    The only question that remains is if 99.9% of the world population will let themselves be fooled by UBI & VR to keep the 'masses content; while the 0.1% hold all the wealth and AI power.

    And this is decided now, because it is too late once AI is consolidated under one bi/tri-llionaire.

    • root_axis 31 minutes ago
      UBI is a political impossibility and VR still sucks. The old ways are not going anywhere.
      • lll-o-lll 8 minutes ago
        The “old ways”? France 1790ish? Or do you mean the really old ways; sacrifice.
    • noosphr 29 minutes ago
      AI doesn't need the 0.1% either.

      Humans amassing money in an AI world is like squirrels amassing nuts when humans are logging the forest.

      If we want to have a planet left we need to make it so laughably inefficient to run AI ground-side that no one bothers.

      Extremely high temperature semi conductor research to run datacenters in dyson swarms around stars/black holes is fundamental research for human survival at this point.

    • jstummbillig 56 minutes ago
      I feel you skipped the step, that explains how this works in democracies. Musk was just not able to buy an election, and decidedly so.

      It might all end very badly, but we should not be so prone to presume bad outcomes without explanations, that are just as good, as we would expect for good outcomes.

      • UncleOxidant 14 minutes ago
        > Musk was just not able to buy an election, and decidedly so

        This time he wasn't able to. But the time before (2024) he seemed to have some effect. I'm sure that he and the other oligarchs are going to learn from this midterm election and try different approaches in the future. Think of the 2026 election as one of his crashed rockets. They learn from those crashes - iterative improvement. They'll figure out ways to be more sneaky next time.

    • spwa4 1 hour ago
      Not a chance: if there was even a 1% chance of this being true you'd see something entirely different happening. If AI research self-reinforces, as Anthropic publishes (but doesn't actually believe, it can't), then it will win from all the economic activity on the planet. The major consequence of that is that money (as a source of labor, ie. to get people to do stuff for you, to get stuff for you) is now worthless.

      Why? It is now easy to get a few motors to do that stuff for you. It is now easy to get whatever manufactured ... except for raw materials, and energy if you see that as separate.

      Think about that again: if superintelligence exists the huge change is that money is now worthless and raw materials have all just turned more valuable than their weight in pure gold (assuming they're more useful than gold, which is all of them). Steel, just raw ingots, will increase in price BY A FACTOR ONE HUNDRED THOUSAND, minimum. At this point your dog can probably reason towards the investment opportunity, no?

      You would buy farms, mines, mining rights, water, energy (and not for DCs, actual energy sources), rivers, raw materials ... that sort of thing. A superintelligence is not going to be exempt from the means of production. Raw materials are mostly monopolized in the current economic system, so this would in fact be absolutely critical to your success. And hoards of raw materials, anything, will be worth more than Gold (gold being pretty useless except for looking pretty and having a somewhat favorable electrical resistance, but not super great. When it comes to strength it is beaten by 2-day old chewing gum. So if economic activity is 100x-ing every raw material actually used for something will go into extreme shortage)

      So no, sorry. Obviously this cannot be how Amodei, Altman, Google, Meta, and others are seeing things: they're going after money, not something else. So they are either stupid or they know 99%+ sure they do not have superintelligence. Because if they do let superintelligence loose, mining countries will have them by the balls and will squeeze. Hard. Yet the money all these people are collecting boatloads of will be useless. And yet, they want the money, not the raw materials! That tells you what they believe: that the money is more valuable than raw materials ... AND more valuable than superintelligence.

      In other words: AI is a bubble, a hype, according to the management of the big AI firms. It's a way to get people to give them money, not an actual path to superintelligence. Either that, or these people are complete idiots.

      • ryeights 47 minutes ago
        If you believe in ASI but you spend all your money on farmland, mining rights, and power plants, instead of compute, you will lose the race to ASI.

        Your argument only makes sense in a world with one AI lab. Labs are pouring all of their money into scaling because there is no alternative.

        • spwa4 39 minutes ago
          Not really. You are assuming that ASI will just win irrespective of anything else, 2 seconds after it starts. How would it do that? By launching the nukes? And if it takes time, everyone else has time to catch up ... and raw materials will be the determining factor.

          So that's not what happen. Yes, someone will be first, but the second will not be far behind.

          Once you have ASI, raw material owners and the governments they're in will PICK the winner from the choices available. Just like they did in the 20th century.

          To say nothing of the consequences of what is currently happening with Chinese models only 2 months behind frontier labs. Even if it's 8 months or 2 years or whatever. ASI will not build an insurmountable advantage in the world economy in 2 months, especially not if some raw materials players sabotage it (which seems likely in the extreme, and is simply what raw materials players are already doing, so they don't even have to realize what's happening and start doing it)

          • ryeights 6 minutes ago
            You are assuming that ASI won't just win, simply because you can't imagine it? Examples:

            - Sophisticated cyberattacks to disable or quietly sabotage competing labs

            - Undermining nuclear second-strike capability by locating submarines and infiltrating command-and-control networks

            - Fast-replicating nanotechnology

            - Superpersuasion (I am dubious on this)

            etc., backed by global surveillance to preempt sabotage efforts. If you haven't read Superintelligence you should, all the leaders of the labs certainly have.

      • morkalork 53 minutes ago
    • dominotw 1 hour ago
      This comment makese sense if we do extreme amount of reality suspension "think what will be possible in future..."

      But we've all seen robots fold clothes to do that sort of thinking easily. Oh maybe we need to hangout on ai twitter more.

    • reducesuffering 1 hour ago
      > the 0.1% hold all the wealth and AI power

      Is the equivalent of ants trying to outsmart Google that wants to build a datacenter on their land. ASI will unfathomably outsmart the wealthy and intelligent humans like Stockfish 13 does Magnus Carlsen

      • Avicebron 57 minutes ago
        This is jumping the real world concerns into the realm of science fiction. How about we engage with the premise that before the AGI god comes into being and annoints the Rationalists as it's chosen people, there will be people who will try and use this to enrich themselves and immiserate others.
        • reducesuffering 42 minutes ago
          Every year we are making the "science fiction" of the previous year come true. The trajectory has been clear, the insider info from lab employees themselves are that the scaling laws continue unabated. The coding of the last 40 years is gone in a flash. Unanswered math problems are being solved every week and the top mathematicians of the world see crisis. The only "science fiction" is hyperbole about anointing rationalists, it's a strawman no one holds, X-risk concern is about disregarding everyone.

          "there will be people who will try and use this to enrich themselves and immiserate others."

          This is already happening from Big Tech and VC's, all the big money. You should try listening to the people that have shirked enormous fortunes in order to tell the truth about things. Some examples:

          https://x.com/hilbertspaess/status/2097476196791709843?lang=...

          https://fortune.com/2026/09/26/geoffrey-hinton-godfather-of-...

          https://yoshuabengio.org/en/blog/faq-catastrophic-ai-risks

          https://ai-2027.com/

        • rcoveson 27 minutes ago
          Sarcastically quipping that the AGI god would anoint the Rationalists as it's chosen people is like saying that the global warming god is going to come along and anoint the Climatologists as its chosen people. It's like saying Al Gore is a reptile and is excited for the universally hot, humid future.

          Everybody who thinks they're "the chosen people" predicts the end of the world is coming, but not everybody who predicts the end of the world is coming thinks they're "the chosen people". Maybe the end of the world is coming and it's Just Bad.

  • throw-away-mle 46 minutes ago
    I work as a research engineer within an Alphabet company, and I have been in the industry for almost 15 years. I am in the same place as Dwarkesh. I have seen some things that have absolutely shocked me within the last two months. A team of agents rewriting a model from scratch that had had many years worth of R&D spent on it, beating the original in just a few days. An agent that anyone in the company can just message an experiment id and say 'improve this', and it will come back a few hours later with like 20 issues, having examined checkpoint weights and gradient steps.

    These agents know every paper, every trick, have amazing instincts, and can code up 20 ablations and start them running before you finish your commute. I'm finding guiding them is becoming less and less useful. I have never felt my obsolescence rushing towards me as much as now.

    • lousken 2 minutes ago
      Tell me more about how AI is able to fix security bugs within code.
    • atleastoptimal 43 minutes ago
      What options does humanity have? All I can think of is

      1. Pause AI development and allow humans to stay in control a bit longer while we work on making AI safer, allowing the field to advance bit by bit (and hoping this plan even works)

      2. Allow AI to progress to superintelligence and hope that it is aligned and will treat everyone fairly

      3. Stop AI development altogether and create a world government panopticon which ensures it never progresses towards human obselesence?

      • oconnor663 19 minutes ago
        > create a world government panopticon

        This seems to be technically possible for uranium enrichment, but I'm not sure it's technically possible for ML research. You can do it on any CPU just really slowly, right? Or you could hide small numbers of GPUs in any basement. I think AI-2040 has an estimate of how much time we could possibly buy ourselves this way (even ignoring all the political problems), and I don't think it's especially long.

      • credit_guy 23 minutes ago
        We are somewhat lucky that your option 1 will just happen by accident. And by this I mean the fall from grace of Jensen Huang will put a bit of brakes on the AI movement. It will be temporary, for sure, but it will give us some time to adjust.
      • reducesuffering 33 minutes ago
      • asa123 40 minutes ago
        You reckon there are maybe a few things in between those three points?
        • atleastoptimal 38 minutes ago
          Obviously there is the general gradient between full speed ahead <-> full stop, but what's more important is the communication and assurances between all relevant parties (US/Chinese governments, major AI labs). If everyone is forced to make a deal without being aware of the risks, then there is no point as they will imagine that these safeguards are overly stringent.
    • wslh 36 minutes ago
      > These agents know every paper, every trick, have amazing instincts, and can code up 20 ablations and start them running before you finish your commute. I'm finding guiding them is becoming less and less useful. I have never felt my obsolescence rushing towards me as much as now.

      For math/cs research I found that agents alone doesn't replace a harness that gives them a strict sense of reality to have a better "compass". It's not only Lean but a plethora of tools like Mathematica, SageMath, SAT/SMV solvers, and very niche ones.

    • ant6n 42 minutes ago
      I am not a research engineer and have only a vague idea what you are talking about. I doubt me plus a model can replace you.
    • noosphr 36 minutes ago
      I am not in an alphabet lab, feel free to hire me though, and using off the shelf models with a few tweaks to the harness I used the same models to build I beat all the published benchmarks by substantial margins.

      We need people at this point to tell the AI to do better and to stop disastrous side quests that kill its productivity.

      And yes, rsi is here already. And its small enough that it can fit on a local gpu.

      The tsunami is coming for everything, I now think that AI is the first democratising technology since the musket. And the results will probably be the same.

  • aabajian 1 hour ago
    I think the next step for developers is to combine existing technologies. For example, in the medical imaging space that are thousands of published algorithms that were implemented once (by the researcher who published the paper), but never used. AI can now read the paper and one-shot an implementation. I do wonder if there are breakthroughs to be had by combining such algorithms.
    • throwaway219450 56 minutes ago
      Almost certainly yes - I’ve tried this for some imaging problems - but what tends to happen is you find that the method is fragile and only works well on a narrow task, or you need some specific data that’s a pain to label, or the results are strongly overfit. Reproducibility crisis type stuff. Which makes sense when you think about how narrowly scoped many papers are, and the fallibility of the average grad student. Also survivorship bias is strong. Lots of papers are forgotten about because the methods didn’t really work for other things. Sometimes it doesn’t matter, maybe it’s fine to have a method that only works on some niche type of data that you have.

      But models are really good at being told “here is a paper, check the implementation against the claims in the paper and munge my data to work with it”. On the plus side, many seminal papers have really simple implementations. Like the meat of many CVPR papers is a derivative repo + one function modified. Often they can fix things or backport improvements from more recent work. And they can definitely re-implement things that ought to work, but the authors never published.

  • zarzavat 1 hour ago
    If AI is a verifiable domain, how do you get AIs to construct benchmarks to apply to themselves?

    It seems that the only benchmarks that could be automatically constructed are those for tasks that are themselves verifiable (theorem proving, coding, etc).

    Non-verifiable tasks require humans to define what exactly a good job is.

    The RSI hypothesis is dependent on an assumption that progress on verifiable tasks generalizes to non-verifiable tasks.

    • atleastoptimal 52 minutes ago
      There are hundreds of verifiable tasks in AI development as simple benchmarks on the models themselves, such as

      1. Performance engineering ability 2. Ability to reconstruct visuals in code alone 3. Ability to orchestrate cheaper models towards a given task 4. Pretty much any verifiable engineering task

      The loop is pretty clear, have AI's work on pre and post-training, then benchmark their outputs against these tasks to determine how to steer AI progress. The non-verifiable stuff isn't on the critical path towards these accoutrements anyway.

    • OtherShrezzing 52 minutes ago
      I think more accurately, it rests on (lots of) non-verifiable tasks sitting in latent space between verifiable tasks.

      It’s a small distinction, but I think it creates a meaningfully less challenging requirement.

  • lavendermon 43 minutes ago
    What I find fascinating about AI predictions is the human nature reflected behind them. There seem to be two camps: the first includes the author of this essay, people who are beginning to believe that unknown capabilities yet to exist will soon become possible and are impossible to predict. This camp seems to be winning an increasing number of people due to previous unknowns being solved very rapidly. Therefore, belief has been increasing in this group that AGI and other big AI questions such as massive human job losses can't be far around the corner.

    On the other side of the spectrum we see people who still want evidence and kick their AI predictions out to a more distant future by pointing at current limitations. This group seems to move the goalposts as needed to support their argument with each new advancement. As the author of the article pointed out, he has moved from this camp to the first one. I have also moved to the first one.

    • popkkksidhd 24 minutes ago
      Anthropic would not be IPO’ing if the first case were coming anytime soon. They’d have no need. Their current behavior looks more like a company who wants to cash out while they can.

      The behavior of these frontier labs is much more telling than anything else.

    • thewanderer1983 32 minutes ago
      It's human nature and all to common to move goalposts when a belief is entrenched or ideologically motivated. It can be as simple as the belief is aligned with the interests of that person.

      String theory is one that comes to mind.

    • Dzugaru 35 minutes ago
      Three camps. "Impossible to predict" one doesn't throw around numbers like 2030.
      • lavendermon 31 minutes ago
        Future capabilities are impossible to predict, therefore the timeline must be accelerated to an attention grabbing year such as 2030. All the same camp
    • newyankee 41 minutes ago
      the author bet everything on AI explosion and it paid off for him. Sometimes it is easier to be a believer and hope you end up right than endlessly discuss.
  • j2kun 1 hour ago
    What has this author done that would suggest we should listen to his opinion?
    • wnmurphy 26 minutes ago
      It's the internet, you're not forced to read anyone's opinion.

      If someone has interesting ideas, or a reasoned argument, do you require their credentials before considering what they've written?

    • drcode 1 hour ago
      Most people here will know who Dwarkesh is, he hosts a tech podcast
      • avanwyk 56 minutes ago
        That's right, most people know that. Why does popularity make his opinion valid? Citation: the US president.
      • platinumrad 1 hour ago
        What has he done other than host a podcast?
        • timschmidt 47 minutes ago
          What have you done?
          • viklove 37 minutes ago
            Not much but I've been a S. Eng for 10 years and I'm wondering where Dwarkesh's claim to expertise comes from.
            • timschmidt 17 minutes ago
              > I'm wondering where Dwarkesh's claim to expertise comes from.

              Seemingly from you, as a strawman. He doesn't seem to make such a claim in the article.

              • viklove 13 minutes ago
                Not from me, although I suppose I did assume.

                So he's not an expert. Just a guy off the street. Got it, I'll pass on the podcast then thanks.

                • timschmidt 13 minutes ago
                  And absolutely nothing was lost.

                  “Great minds discuss ideas; average minds discuss events; small minds discuss people.” -- Eleanor Roosevelt

                  • viklove 12 minutes ago
                    That could only be true if there was nothing to gain in the first place
                    • timschmidt 9 minutes ago
                      Just keep telling yourself that. ;)
          • jdjdjdnddjdj 33 minutes ago
            [dead]
    • dumberquestions 58 minutes ago
      He's making an argument and you're free to engage with it or not, if you don't read anything by anyone who doesn't meet your credentials quota then save your time and skip it.
  • Dzugaru 42 minutes ago
    It's not only about verifiable or not. It's also about how high the fruit hangs vs how good your ladder (search heuristic) is. NP stuff is readily verifiable for example.

    Absolutely plausible that this math avalanche was 1 cm deep, and harder problems are not solvable without new math theories that have to be constructed with a lot of understanding needed and taste (human heuristic).

  • SirMaster 45 minutes ago
    Why is there a limit from the chips? If AI is better than all the humans in every field, then it should also be able to design a better chip no problem. It should be able to figure out how to make new manufacturing processes etc to make chips smaller and faster and overall more capable.
  • zaler 1 hour ago
    I feel like this is why more distributed control or open ownership of AI models is such an important goal for regulators. As AI accelerates its own development, the private companies that are initially maybe only a few months ahead of open-source will widen the gap more and more.

    It seems likely we end up with an even more extreme tech oligarchy than we already have when superintelligence arrives. And that's only if we manage to solve alignment and it doesn't do something more horrible.

  • qw34ahg 1 hour ago
    The machine of AI influencers is running at overdrive. A true avalanche of bullshit.
    • aerhardt 1 hour ago
      Dwarkesh has been skeptical about AI claims for a long time. In his interview with Dario earlier this year, he pressed him hard. So this is a fairly recent change of opinion.

      What he says still doesn't map to my daily practice. I use Astra and Fable extensively and they need an incredible amount of guidance and harnessing to produce fairly simple systems. Any decent code is produced with tons of static tooling, agent files, harnessing, and even then, there are myriad mistakes. My main point of skepticism is whether software engineering is verifiable at all, once you factor in architecture, code design, let alone the human element in usability, requirements, etc.

      As far as I'm concerned Dwarkesh could be prey of AI psychosis... I don't think a proxy metric like the average daily spend of AI researchers going up proves anything.

      The technology has undoubtedly advanced but for me it's still a coin toss on whether we're in an exponential or an s-function.

      • jeremyberemy 54 minutes ago
        Thank you for saying this. Sometimes I'm like, wow, this thing is way smarter than me, and others I'm wondering how it can be so dumb.

        In my experience, s-curves are much more prevalent in nature than exponentials.

        • timschmidt 45 minutes ago
          Counterpoint: A human lifespan on a thousand-year S-curve would see only exponential.
        • aerhardt 38 minutes ago
          That also matches my observation, particularly in economics once you factor in a scarce resource. But it's not like I have a convincing metamodel to say whether that or the other distribution is more common. But neither do people who fervently believe in the exponential...
  • dyauspitr 1 hour ago
    Why is dwarkesh using the Trump term superintelligence. This is a bad look for him.
    • dumberquestions 56 minutes ago
      He's trying to predict when it could happen, not using it as a replacement for 'AI'.
      • dyauspitr 55 minutes ago
        We already have a term for that- AGI that everyone has been using.
    • hackermailman 1 hour ago
      It was a commonly used term for AGI in 2014 hence this book ttps://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies
    • wnmurphy 24 minutes ago
      I understood that the author was using the actual term, before it was hijacked by a malignant narcissist as a perversion of the meaning of words and an indicator of fealty (which only applies to Federal employees in the Executive Branch, by the way).
    • wilkystyle 1 hour ago
      Since when do we give Trump the credit for the term superintelligence? An absolutely bonkers comment to make.
      • avanwyk 52 minutes ago
        The insane part is mandating the term instead of AI.
      • dyauspitr 1 hour ago
        He literally mandated that everyone used the term Superintelligence instead of artificial intelligence like two weeks ago

        https://www.politico.com/news/2026/09/29/trump-signs-executi...

        • baq 58 minutes ago
          Just so his friends can sell you those sweet .si domains they got 100x cheaper
        • wilkystyle 33 minutes ago
          So? Are we all supposed to stop using the term, despite it being around since like the late 90s? Sorry, I don't let politicians control my life like that.
          • dyauspitr 11 minutes ago
            No one was using it. Everyone was using AGI.
  • HarHarVeryFunny 15 minutes ago
    So if AI is shortly (Dwarkesh suggests maybe next year) going to be better than any expert at anything, I guess I'll just wait until it's there and ask it to make me rich.

    Seems far simpler than asking it to make super-human intelligence since nobody appears to know how to do that, and tons of people already know how to successfully start companies and make money.

    My backup plan is to still be here in a years time, not rich, reading about the latest forecasts about how the singularity is coming real soon now.

  • orphereus 1 hour ago
    When is superdupermegaintelligence going to be announced?
    • itronitron 1 hour ago
      Well, we already have superdupermegalomania.
    • Ridius 1 hour ago
      Just before hypergigaintelligence
    • blooalien 55 minutes ago
      > When is superdupermegaintelligence going to be announced?

      Isn't it already here? Infallible God-King Trump and all his ultra-genius tech-bro buddies delivered superintellgence already, and due to exponential growth it should already exist by this time, no? By this time next week it will have solved all problems and made humans obsolete.

  • root_axis 43 minutes ago
    "The avalanche that hit math is coming for X" is already a tired meme. It's telling that the impressive success of applying LLMs to the entirely abstract domain of math is being used to make extrapolations about all kinds of domains that aren't amenable to 10k agents (intelligently) throwing shit at the wall.
  • afgfaa 1 hour ago
    Who sponsors this advertisement? Many AI boosters.

    https://sponsorradar.com/channels/dwarkeshpatel

    And that does not include Substack bulk subscriptions.

  • iaanix 1 hour ago
    A lot of AI research can be already automated. The experimental part of work can be tonally outsourced to agents (at least for the topics that I am interested in). I had some success with outsourcing theoretical work as well. These days I use quite advanced orchestration. Interestingly, the most difficult part still is writing good stories for research papers. There I can still beat any AI. And when it comes to research we have been already covered by an avalanche of AI slop. There was always slop in AI research but now it is the fully automated slop. A few years ago there was some human-made slop, now it is being factory produced. AI can do good research on its own, but good research is rare in general.
  • FLeXMurphy 1 hour ago
    At last, he's talking his book.
  • jazzyjackson 1 hour ago
    Hype hype hooray
  • dominotw 1 hour ago
    >majority of this acceleration today is likely attributable to Anthropic’s extensive use of AI to do R&D

    Lots of companies think they are reaping similar rewards from ai coding and research but it has failed to lift bottomlines or create any new products. My last two employers are gungho on ai and ppl have been producing unfathomable amount of ai output. To a causal observer it would look like we have really accelerated but this has had no visible impact on any of the products that employers produce other than make them shittier.

    Whats different about product development at athropic that ai is particular suited for not generating slop.

  • rsoto2 1 hour ago
    love reading about idiots that swallowed trumps "superintelligence" drivel. Altman said we had AGI years ago, now we wait for the "superintelligence"
    • bananaflag 1 hour ago
      Superintelligence is not Trump's concept, it was invented years ago.

      Trump's mistake is using the word to denote AI in general when it already had an established meaning.

    • spwa4 1 hour ago
      Isn't Sam Altman accused of being a habitual liar by a lot of people, including the richest man on the planet, the arguably best (certainly longest-lasting) AI researcher, the most famous AI ethics researcher, and half of his own board?

      Strange how this guy can be CEO of anything. All you hear management types ever blubber about is integrity ... Clearly there's a problem here.

  • mequelkramer 59 minutes ago
    [flagged]
  • quicklywilliam 1 hour ago
    TLDR; someone used some fake graphs and probabilities pulled out of thin air to justify an unwarranted conclusion based on a false analogy to a situation that is far from clear.