The AI Superintelligence Debate Has Serious Flaws

(spectrum.ieee.org)

19 points | by rbanffy 39 minutes ago

4 comments

  • jokoon 7 minutes ago
    We don't even know what intelligence is.

    I just want science to at least build a notion of intelligence, in terms of evolution, biology, psychology, neurology. We don't really know what animal intelligence is, and we only limit AI to human intelligence, which is very narrow.

    If we use AI to do human stuff, it's just an assistant for human goals. If humans don't have a specific goal, super intelligence will not guess or decide it for us.

    What does humanity want? Health? Mental health? Colonize Mars? Reduce income inequality? Solve climate change? World peace? Immortality? Happiness?

    AI can help for those things, but that's it.

    We tend to romanticize super intelligence like it would be some new life form, external to humans, a form of omniscient deity that could be a threat to humanity. There are worse threats: antibiotic resistance, nuclear war, climate change, dictatorships, mental health epidemics, obesity.

  • saagarjha 9 minutes ago
    I don’t necessarily agree with the labs’ policies and predictions on RSI but it’s worth noting that the points raised here are something they do have responses to.

    On the idea that training better models requires more data, I think the current thought here is that we are not getting all the “meat off the bone” of the data we have, so there is still quite a ways to go with training that AI can improve. After all, it’s not like Anthropic is doubling their data set every year (?) and yet their models definitely improve by more than that.

    For computability, yes, obviously the models cannot violate mathematics. But there’s a lot of things that are very computable and very desirable, or correspondingly very computable and very bad. This is enough to be worth thinking about.

    For alignment: it’s not actually a question of alignment at all, but rather one of “can we ignore alignment and design systems where the AI cannot cause harm?” Obviously, we can. However the issue is that doing this takes time and very few people (especially the frontier labs!) seem to be doing so. It’s much easier to hook your LLM up to the world and ask it to go than ask, “is this safe?”

    • soco 3 minutes ago
      Training a human brain needs and uses much, much, less data than training the AI models. So maybe the expectations on the AGI are, dunno, just optimistic? I mean you can put tons and tons of gas in a car but that won't make it drive faster.
  • shinojcm 6 minutes ago
    [flagged]
  • amelius 5 minutes ago
    The debate will quickly change when we have 8ft tall humanoid robots marching through the streets.