Assuming that it is attainable, and that it’s no more than a century away (a conservative estimate by Bostrom’s standards), my argument is that humans ought to be less focused on what we might gain or lose from the advent of artificial intelligence and more preoccupied with who we might become and––most importantly––what we might give up.Ĭlever and capable as they are, I believe thinkers like Nick Bostrom suffer from a kind of myopia, one characterized by a zealous devotion to particularly human ends. I’ll try not to rehash those same arguments here, and will also put aside for the sake of expedience the question of whether or not ASI is actually attainable. Some of these I also took up in my review of James Barrat’s Our Final Invention, which should be viewed as a less detailed but more accessible companion to Bostrom’s work. Instead, I want to hone in on some of the philosophical assumptions on which this book and others like it are founded, with the goal of exposing some key ethical issues that are too often minimized or ignored by technologists and futurists. It’s a worthwhile read for anyone passionate about the subject matter and willing to wade through a fair amount of jargon.īostrom demonstrates an impressive grasp of AI theory, and a reader like me has neither the professional standing nor the basic knowledge to challenge his technical schemas or predictions, which by and large seem prudent and well-reasoned. Nick Bostrom’s Superintelligence: Paths, Dangers, Strategies is a short, dense introduction to our most cutting-edge theories about how far off superintelligence might be, what it might look like if it arrives, and what the consequences might be for humanity. The idea of artificial superintelligence (ASI) has long tantalized and taunted the human imagination, but only in recent years have we begun to analyze in depth the technical, strategic, and ethical problems of creating as well as managing advanced AI.

Superintelligence book review