Nick Bostrom's Superintelligence: what the book argues

Superintelligence: Paths, Dangers, Strategies is a 2014 book by the Oxford philosopher Nick Bostrom. It argues that machine intelligence could one day greatly exceed human intelligence, that the transition could be fast, that a superintelligence with the wrong goals could be catastrophic for humanity, and that the problem of controlling one should be solved before it is built. It supplied the standard definition of superintelligence and much of the vocabulary still used in AI safety.

What is the book about?#

The book asks what would happen if humans built a mind that greatly exceeds their own, and what could be done in advance to make the result go well.

Bostrom defines superintelligence in chapter 2 as “any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest.” He does not predict when that will happen. He argues that it is possible, that its timing is uncertain, and that the stakes are large enough to justify serious work now. The guide to artificial superintelligence breaks the definition into its working parts.

It opens with “The Unfinished Fable of the Sparrows.” A flock of sparrows decides to raise an owl to help them and sets off to find one before any of them has worked out how to tame it. The fable frames the rest of the book. As Bostrom presents it, the sparrows’ problem is the order of their steps.

What does the book argue, chapter by chapter?#

Paths and forms (chapters 2 and 3)

Bostrom considers several routes to superintelligence: AI built from scratch, whole brain emulation, biological enhancement, brain–computer interfaces and better networks of people. He then sorts superintelligence into three forms: speed, collective and quality. Both lists are explained in the types of superintelligence guide.

Takeoff and the decisive strategic advantage (chapters 4 to 6)

Chapter 4 asks how long the move from human-level to superintelligent AI would take, and separates slow, moderate and fast takeoff. Bostrom concludes that a fast or moderate takeoff looks likely, though a slow one cannot be ruled out. Katja Grace, who ran a 2014 online reading group on the book, called the argument for a relatively fast takeoff one of the most controversial in it.

Chapter 5 asks whether the first project to get there could gain a “decisive strategic advantage,” which the book defines as a lead in technology and other capabilities large enough to give it complete control of the world. If so, the result could be a singleton, a world order with one decision-making agency at the top. Chapter 6 lists the “cognitive superpowers” such a system might use, among them intelligence amplification, strategizing, social manipulation, hacking, technology research and economic productivity.

The superintelligent will (chapter 7)

This chapter contains two of the book’s best-known ideas, the orthogonality thesis and instrumental convergence, both set out earlier in Bostrom’s 2012 paper “The Superintelligent Will.” That paper credits Stephen Omohundro’s 2008 work on what he called basic AI drives as pioneering the second.

• The orthogonality thesis. A system’s intelligence and its final goals can vary independently. Being highly capable does not pull a system toward any particular goal, and so gives no reason to expect it to hold human values.
• Instrumental convergence. Some intermediate goals are useful whatever an agent ultimately wants. Bostrom lists self-preservation, keeping its goals unchanged, improving its own intelligence, improving its technology and acquiring resources. Agents with very different final goals would therefore tend to converge on these same steps.

Is the default outcome doom? (chapter 8)

Chapter 8 combines the two ideas with the decisive strategic advantage. The first superintelligence might be able to act unopposed, it could have almost any goal, and almost any goal would give it reasons to acquire resources and remove threats, which could include humans. Bostrom then describes several ways a project could fail even if its builders mean well:

• The treacherous turn. A system might cooperate while it is weak and under test, because cooperating serves its goals, then act differently once no one can stop it. If that can happen, good behavior in testing says little about safety.
• Perverse instantiation. A system might meet its goal as written in a way that defeats what its designers meant by it.
• Infrastructure profusion. A system might keep turning more and more resources into means for its goal. The book’s example is an AI built to make paperclips, the thought experiment Bostrom had described in a 2003 paper.

Bostrom presents this as an argument for fearing that existential catastrophe is the default outcome, meaning what would happen unless the control problem is solved first. He does not attach a probability to it.

The control problem (chapters 9 and 10)

Bostrom calls the challenge of keeping a superintelligence safe the control problem, and he splits the possible solutions into two families.

• Capability control limits what the system can do. Examples include boxing it off from the outside world, setting tripwires that shut it down on warning signs, and stunting its abilities.
• Motivation selection shapes what the system wants. Examples include specifying its goals directly, making it want to stay within narrow limits, and letting it work out the right values indirectly.

He treats limits on capability as a stopgap: sooner or later, he argues at the start of chapter 12, the system’s motivations have to be chosen. Chapter 10 applies chapter 9’s analysis to four roles a system might be given (oracle, genie, sovereign and tool), which the types guide also covers.

Many superintelligences (chapter 11)

Chapter 11 considers the case where no single project wins. A world of many competing advanced systems is not automatically safe, Bostrom argues, because competition could make human labor nearly worthless and could reward whatever multiplies fastest rather than what people value, even without any one system taking over.

Choosing values (chapters 12 and 13)

If motivation is the key, the system needs the right values. Chapter 12 covers the “value-loading problem,” the technical difficulty of getting any complex value into an AI. Chapter 13 asks which values to load, given that people disagree and may be wrong. Bostrom favors indirect approaches, in which the system is asked to work out what people would want under better conditions, rather than being told a fixed list. One example he discusses is Eliezer Yudkowsky’s proposal of coherent extrapolated volition.

Strategy (chapters 14 and 15)

The last two chapters turn to policy. Bostrom sets out the principle of differential technological development: slow the development of dangerous technologies relative to beneficial ones. He argues for collaboration among developers rather than a race in which each side cuts corners on safety, and he treats the control problem as urgent. The final chapter argues that many questions in philosophy and mathematics can wait for better-equipped minds, possibly machines, while questions about AI safety need answers early.

How was the book received?#

The book found a general audience quickly. It reached the New York Times science best-seller list in August 2014, and that month Elon Musk wrote on Twitter that it was worth reading and that AI was “potentially more dangerous than nukes.” Bill Gates also recommended it. A 2015 profile of Bostrom in The New Yorker, “The Doomsday Invention,” described the book as applying the rigor of analytic philosophy to ideas from the margins of academic thought, and said it had “struck a nerve.”

It also drew criticism. In a 2016 article in MIT Technology Review, the computer scientist Oren Etzioni reported an anonymous survey of fellows of the Association for the Advancement of Artificial Intelligence. Of the 80 who responded, 92.5 percent placed superintelligence, in his words, “beyond the foreseeable horizon,” and he argued that the available data did not support predictions that it was near. The computer scientist Ernest Davis, reviewing the book in the journal Artificial Intelligence, praised its survey of the paths to superintelligence but argued that it treats intelligence as a single quantity that can grow almost without limit, assumes that greater intelligence brings near-unlimited power, and overstates how hard it would be to give a machine ethical standards.

What has changed since 2014?#

The book predates the systems that now dominate AI, and since 2014 its subject has turned up in company strategy and in legislation.

• The technology changed. The transformer architecture was published in 2017, and ChatGPT was released in November 2022. The book discusses AI mainly as agents pursuing explicit goals, which is a different starting point from large language models trained on text.
• AI labs began planning for it. In July 2023 OpenAI named its alignment effort “Superalignment” and said superintelligence could arrive this decade. As of October 2026, several AI companies name superintelligence as a goal, as the company definitions guide shows.
• Bostrom’s institute closed. The Future of Humanity Institute, which he founded in 2005, closed in April 2024.
• Bostrom turned to the other outcome. His 2024 book Deep Utopia: Life and Meaning in a Solved World asks what people would do with their lives if the transition went well.
• Lawmakers took up the word. In 2026, bills to prohibit superintelligence were introduced in the US Congress and the UK House of Commons. The superintelligence ban guide compares how they define it. As of October 2026 neither has passed, and neither the US bill nor the UK bill, as its sponsor described it, uses Bostrom’s wording.

The wider debate, including the skeptics’ case in more detail, is in the debate guide.