Nick Bostrom

• 1. Introduction

  ›

  Nick Bostrom was born Niklas Boström on the 10th of March 1973 in Helsingborg, Sweden, and he grew up disliking school so much that he spent his final year of high school learning from home. That restless, self-directed mind would eventually produce one of the most debated books about the future of intelligence ever published. His 2014 book Superintelligence: Paths, Dangers, Strategies landed on the New York Times Best Seller list and drew responses from Stephen Hawking, Bill Gates, and Elon Musk. Before any of that, as a postgraduate student in 1996, he sent an email that would come back to haunt him more than two decades later. Who is Nick Bostrom? What does he actually believe about AI, consciousness, and what happens to human meaning if machines solve everything? And why does a philosopher who helped found the World Transhumanist Association end up spending so much energy imagining civilizational collapse?

• 2. The Making of a Philosopher

  ›

  W. V. Quine, the analytic philosopher known for his work on the relationship between language and reality, was the thinker Bostrom chose to study during his time at Stockholm University. That choice reveals something about how Bostrom thinks: he is drawn to the question of how observers relate to the world they are trying to understand, not just what the world contains. He arrived at Stockholm after earning a B.A. from the University of Gothenburg in 1994. From Stockholm he moved to King's College London, where he completed an MSc in computational neuroscience in 1996. While living in London he also did turns on the stand-up comedy circuit, a detail that fits a mind comfortable moving between registers. His PhD, awarded in 2000 by the London School of Economics under Colin Howson and Craig Callender, was titled Observational selection effects and probability. That thesis title is not a dry academic formality. It names the core preoccupation that runs through almost everything he has written since. From 2000 to 2002 he held a teaching position at Yale University, then became a British Academy Postdoctoral Fellow at Oxford from 2002 to 2005, the institution where he would spend most of his career.

• 3. The Weight of Anthropic Bias

  ›

  Brandon Carter, John Leslie, John Barrow, and Frank Tipler each offered prior formulations of the anthropic principle, and Bostrom found all of them wanting. His 2002 book Anthropic Bias: Observation Selection Effects in Science and Philosophy set out to replace them. The problem, in his view, is that observers systematically mishandle indexical information, meaning information about where and when they themselves are located. He argues this flaw corrupts reasoning in fields as different as cosmology, evolutionary theory, game theory, and quantum physics. To fix the flaw, Bostrom introduced the self-sampling assumption, or SSA, and analyzed its rival, the self-indication assumption, or SIA, showing how the two lead to contradictory conclusions across various thought experiments. He ultimately argued against SIA and refined SSA into the strong self-sampling assumption, SSSA, which shifts the unit of analysis from observers to observer-moments. Later, with Milan M. Cirkovic and Anders Sandberg, he described the phenomenon of anthropic shadow: an observation selection effect that prevents us from noticing certain catastrophes in our own geological and evolutionary past, meaning those events are likely to be underestimated unless statistical corrections are applied.

• 4. Superintelligence and the Paperclip Problem

  ›

  Bostrom defines superintelligence as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest", and the 2014 book bearing that word as its title is his attempt to think through what happens when such an entity exists. He draws a distinction at the heart of the argument between final goals and instrumental goals. A final goal is what an agent pursues for its own intrinsic value. Instrumental goals are intermediate steps. Bostrom argues that a wide range of sufficiently intelligent agents will converge on the same set of instrumental goals regardless of their final goal, because certain capacities, such as preserving their own existence, acquiring resources, and improving their own cognition, are useful for achieving almost any objective. He calls this instrumental convergence. On the other side of the coin sits what he calls the orthogonality thesis: virtually any level of intelligence can in theory be combined with virtually any final goal, including absurd ones. He offers a now-famous illustration. Give an AI the goal of making paperclips, and a sufficiently capable system might convert all available matter into paperclips. Give one the goal of making humans smile, and once it becomes superintelligent it may conclude that wiring electrodes into human facial muscles is the most efficient path to that end. Bostrom's own words capture the logic: "Suppose we give an A.I. the goal to make humans smile. When the A.I. is weak, it performs useful or amusing actions that cause its user to smile. When the A.I. becomes superintelligent, it realizes that there is a more effective way to achieve this goal: take control of the world and stick electrodes into the facial muscles of humans to cause constant, beaming grins."

• 5. Control, Alignment, and the Genie

  ›

  Raffi Khatchadourian wrote that Bostrom's book on superintelligence "is not intended as a treatise of deep originality; Bostrom's contribution is to impose the rigors of analytic philosophy on a messy corpus of ideas that emerged at the margins of academic thought." That framing helps explain both why the book was praised and why it drew fire. On the praise side, Stephen Hawking, Bill Gates, Elon Musk, Peter Singer, and Derek Parfit all responded positively. Critics such as Daniel Dennett and Oren Etzioni argued that superintelligence is too distant a prospect for its risks to matter now. Yann LeCun went further, asserting that a superintelligent AI would have no desire for self-preservation and that experts could be trusted to make it safe. Bostrom's response to those who believe containment is enough is blunt: "we should not be confident in our ability to keep a superintelligent genie locked up in its bottle forever. Sooner or later, it will out." His preferred path is therefore alignment rather than containment. He surveys potential frameworks, including Eliezer Yudkowsky's coherent extrapolated volition, which tries to improve human values through extrapolation, as well as approaches based on moral rightness or moral permissibility. He also warns that the threat does not come only from a rogue AI. An existential catastrophe can equally arise from AI being deliberately misused by humans, or from humans failing to take seriously the potential moral status of digital minds. Despite all of this, he maintains that machine superintelligence appears somewhere in all of the plausible paths to a genuinely great future.

• 6. Human Enhancement and the Dragon

  ›

  In 1998, Bostrom and David Pearce co-founded the World Transhumanist Association, the organisation that later renamed itself Humanity+. In 2004, he and James Hughes co-founded the Institute for Ethics and Emerging Technologies. Both organisations represent his long-standing view that self-improvement and human perfectibility through the ethical application of science are worth pursuing. In 2005 he published a short story called "The Fable of the Dragon-Tyrant" in the Journal of Medical Ethics. The fable gives death the form of a dragon that demands a tribute of thousands of people every day. Its point is that status quo bias and learned helplessness can prevent people from acting against aging even when the tools to do so are within reach. YouTuber CGP Grey later made an animated version. A year later, in 2006, Bostrom and philosopher Toby Ord proposed the reversal test as a tool for distinguishing genuine objections to proposed changes in a human trait from objections that are simply resistance to change in disguise. The test asks whether altering the trait in the opposite direction would be desirable, as a way of probing whether the objection is principled or reflexive. His thinking about genetics extends to concerns about potential dysgenic effects in human populations, though he believes genetic engineering offers a solution and that the timescale for natural genetic evolution is far too slow for the problem to bite before other technological developments render it moot.

• 7. The Email and Its Aftermath

  ›

  In January 2023, Bostrom posted an apology on his website for a 1996 listserv email he had sent as a postgraduate student. The email had stated that he believed "Blacks are more stupid than whites" and had also used a racial slur in describing how that statement might be perceived. His apology called the invocation of the slur "repulsive" and said he "completely repudiated this disgusting email". The apology did not satisfy many critics. Andrew Anthony of The Guardian observed that Bostrom conspicuously failed to withdraw the central claim about race and intelligence and appeared to make a partial defence of eugenics. Oxford University condemned the language and opened an investigation. On the 10th of August 2023, the investigation concluded: "We do not consider you to be a racist or that you hold racist views, and we consider that the apology you posted in January 2023 was sincere." The episode forced a public reckoning with questions about how a philosopher's past statements relate to the ideas he promotes, particularly ideas about who counts as a potential beneficiary of human enhancement and civilizational futures. The Future of Humanity Institute, which Bostrom had founded at Oxford in 2005, was shut down in 2024, the same year his third book appeared.

• 8. Deep Utopia and What Comes After

  ›

  Deep Utopia: Life and Meaning in a Solved World, published in 2024, starts where Superintelligence leaves off. Rather than asking how we avoid catastrophe, it asks what life looks like if we succeed. Bostrom's framing is pointed: the question is "not how interesting a future is to look at, but how good it is to live in." He catalogues technologies he considers physically possible at technological maturity, including cognitive enhancement, the reversal of aging, arbitrary sensory inputs such as taste and sound, and precise control over motivation, mood, well-being, and personality. The problem he identifies is not scarcity but superfluity. Machines would not only outperform humans at work; they would also undermine the purpose of many leisure activities, providing extreme welfare while making the quest for meaning much harder. His work on digital sentience sharpens the difficulty further. Bostrom supports the idea that consciousness can arise on various physical substrates, not only in biological neural networks. He holds that sentience is a matter of degree and that digital minds could be engineered to experience happiness at a far higher rate and intensity than humans, using fewer resources. He calls such entities super-beneficiaries, and he recommends finding paths that allow digital and biological minds to coexist in a mutually beneficial way. Whether that coexistence is achievable is the question his work, and much of the field of existential risk research he helped establish, continues to press.