Henry Shevlin (@dioscuri), Dan Williams (@danwilliamsphil), and I discuss AI and "superintelligence."

You get all of these maladaptive outcomes—all this cruelty and waste and competition and aggression—precisely because there was no designer who wanted to minimize any of these things. Natural selection did its thing.

If AI were simply allowed to evolve without any kind of human intervention, then that would be a risk. But in the case of AI, there is intelligent design. I mean, us. These are gadgets. I think it’s illicit to anthropomorphize AI as being like human beings, which are products of natural selection, when AIs are not products of natural selection. They’re products of intelligent design.

And so the engineers can think, “Well, these are the goals that we want to optimize. These are the kinds of side effects that we want to minimize. These are the kinds of things that we wouldn’t in a million years allow it to do.”