Global catastrophic risk

• 1. Introduction

  ›

  Global catastrophic risk names a category of hypothetical events severe enough to damage human well-being on a planetary scale, or to end the human story entirely. The term does not have a sharp definition, but researchers use it to mark off a special class of disaster: one that is not merely local or regional but truly worldwide in reach. What makes these risks philosophically difficult is a peculiar feature of observation. Every civilization that has ever experienced an extinction event has gone unobserved, because no one was left to record it. That silence tells us almost nothing about how likely such events actually are.

  The distinction between a global catastrophic risk and an existential risk matters here. Global catastrophic events could kill vast numbers and shatter civilization without ending the species entirely. Existential risks go further. They either cause outright human extinction or lock humanity permanently into what researchers call a drastically inferior state of affairs, with no path back. A permanent totalitarian dystopia from which no recovery is possible counts as an existential catastrophe just as much as extinction does. Philosopher Bryan Caplan has put it bluntly: perhaps an eternity of totalitarianism would be worse than extinction.

  The questions this documentary will pursue are less about any single disaster than about the machinery of risk itself. Why do humans struggle to take these threats seriously? What institutional structures have emerged to study them? And what does a credible defense against civilization-ending events actually look like?

• 2. A Taxonomy of Disaster

  ›

  Not all catastrophes are created equal, and researchers have worked hard to draw meaningful lines between them. The Black Death may have killed a third of Europe's population, around 10 percent of the global population at the time, yet Europe survived without anything resembling civilizational collapse, even after losing between 25 and 50 percent of its people in some estimates. The 1918 influenza pandemic killed an estimated 3-6 percent of the world's population. Both were catastrophic; neither was existential.

  The framework that emerges from this history distinguishes among three types of irreversible failure. The first is extinction itself, plain and final. The second is unrecoverable civilizational collapse: a disaster severe enough that humanity never rebuilds any complex society. The third is what researchers call unrecoverable dystopia. George Orwell's novel Nineteen Eighty-Four is cited as a literary illustration of this last scenario, in which the catastrophe is not a body count but a permanent narrowing of human possibility.

  All three share a defining feature. Before the catastrophe, humanity faces what researchers describe as a vast range of bright futures to choose from. After it, that range collapses forever. Whether the agent is a comet, a pandemic, or a totalitarian regime, the loss is the same: all the futures that will now never happen.

  Anthropogenic risks, those caused by humans, include artificial intelligence misaligned with human goals, biotechnology, nanotechnology, nuclear holocaust, biological warfare using genetically modified organisms, cyberwarfare targeting critical infrastructure such as the electrical grid, and radiological warfare using weapons like large cobalt bombs. Non-anthropogenic hazards range from asteroid impacts and supervolcanic eruptions to a lethal gamma-ray burst, a geomagnetic storm from a coronal mass ejection, or, on the longest timescale imaginable, the Sun eventually transforming into a red giant star and engulfing the Earth billions of years from now.

• 3. Why We Look Away

  ›

  Eliezer Yudkowsky has argued that large numbers trigger a fundamentally different mode of thinking than smaller ones do. People who would never dream of hurting a child, he writes, hear of existential risk and say, "Well, maybe the human species doesn't really deserve to survive." That psychological jump is not a reasoned conclusion. It is what researchers call scope insensitivity at work: the tendency for moral concern not to scale with the magnitude of a problem. Studies show that people are roughly as willing to prevent the deaths of 200,000 birds as they are to prevent the deaths of 2,000 birds. The number changes; the emotional response barely moves.

  Sociobiologist E. O. Wilson traced this tendency deep into evolutionary history. He argued that close attention to the near future and early reproduction was advantageous during nearly all of the two million years of existence of the genus Homo. Disasters occurring only once every few centuries were forgotten or converted into myth. Natural selection did not build humans to model extinction-level events. It built them to find food and raise children.

  Nick Bostrom adds another layer. All past predictions of human extinction have proven false, and to many people this makes future warnings feel less credible. But Bostrom argues this track record is weak evidence either way. Survivor bias distorts the record entirely: humanity is here to notice that no extinction has occurred, but humanity would have no record at all if one had. The absence of catastrophe in the past is not the same as evidence of safety in the future.

  Hyperbolic discounting, the availability heuristic, the conjunction fallacy, the affect heuristic, and the overconfidence effect all compound the problem, pulling individual and collective judgment away from the long-run and the large-scale toward whatever feels immediate and personal.

• 4. The Coordination Problem

  ›

  Even when a government understands global catastrophic risk clearly, the economics of responding to it work against action. Risk reduction is a global public good: if one large nation invests heavily in preventing a pandemic or stopping an asteroid, every other nation on Earth benefits without contributing. Markets systematically undersupply global public goods because no individual actor captures enough of the benefit to justify the full cost.

  The problem compounds across time. Most of the people who would benefit most from successful risk reduction today do not yet exist. Future generations might be willing to pay substantial sums for measures taken on their behalf, but no transaction mechanism exists to transfer those resources to the present. This makes existential risk reduction what researchers describe as an intergenerational global public good: the hardest possible variety to fund.

  The result is chronic underinvestment. Nick Bostrom has noted, with comparisons critics called high-handed, that more research has been done on Star Trek, on snowboarding, and on dung beetles than on existential risks. The Biological Weapons Convention organization offers a concrete data point: as of 2020, its annual budget stood at US$1.4 million, a modest figure for an organization tasked with monitoring biological weapons threats worldwide.

  Governance mechanisms, the source text notes, develop more slowly than technological and social change. Governments, the private sector, and the general public have all expressed concern about the absence of coordinating structures capable of negotiating between the diverse and conflicting interests that any serious global response would require.

• 5. Layers of Defense

  ›

  Researchers working on mitigation have converged on a framework called defense in depth, which divides protective measures into three distinct layers. Prevention aims to reduce the probability that a catastrophe occurs at all; one example is measures designed to prevent the emergence of new highly infectious diseases. Response aims to keep a small-scale disaster from scaling to the global level; one example is measures to prevent a limited nuclear exchange from escalating to all-out war. Resilience focuses on humanity's ability to survive and recover if the first two layers fail; stockpiling food for a nuclear winter scenario falls into this category.

  Human extinction becomes most likely when all three layers are simultaneously weak: when a risk cannot be prevented, the response fails to contain it, and surviving populations lack the means to endure. The framework is straightforward, but applying it to existential risks is genuinely hard. Unlike with ordinary engineering failures, there is no track record of previous events from which to learn. Humanity cannot iterate toward a better response by studying past extinction events, because there are none to study.

  Robin Hanson, an economist, has argued that even a self-sufficient, permanently occupied settlement housing as few as 100 people would significantly improve the odds of human survival across a range of global catastrophes. The Svalbard Global Seed Vault takes a parallel approach for agriculture: buried 400 feet inside a mountain on an Arctic island, it is designed to hold 2.5 billion seeds from more than 100 countries. More speculative proposals include growing mushrooms on dead plant biomass left after a catastrophe, converting cellulose to sugar, or feeding natural gas to methane-digesting bacteria to maintain calorie supply during a prolonged absence of sunlight.

  Stephen Hawking advocated a longer horizon still: colonizing other planets within the Solar System once technology progresses sufficiently, specifically to reduce humanity's dependence on Earth's continued habitability.

• 6. Institutions Built to Watch

  ›

  The oldest organization in this space dates to 1945. The Bulletin of the Atomic Scientists was founded in the aftermath of WWII, when the public first grasped the destructive potential of atomic weapons. In 1947 it created the Doomsday Clock, which has become the field's most recognizable symbol. The Foresight Institute, founded in 1986 by K. Eric Drexler, was among the earliest groups to study the unintended consequences of technology at a global scale; Drexler postulated the "grey goo" scenario in which self-replicating nanotechnology runs out of control.

  After 2000, the pace of institution-building accelerated. The Machine Intelligence Research Institute, established in 2000, focuses specifically on the risk of catastrophe caused by artificial intelligence, with donors including Peter Thiel and Jed McCaleb. The Nuclear Threat Initiative, founded in 2001, works on nuclear, biological and chemical threats and maintains a nuclear material security index. The Lifeboat Foundation, established in 2009, funds university-based research into preventing technological catastrophe. Seth Baum and Tony Barrett founded the Global Catastrophic Risk Institute in 2011 as a non-partisan think tank covering artificial intelligence, nuclear war, climate change, and asteroid impacts. Laszlo Szombatfalvy established the Global Challenges Foundation in Stockholm in 2012; it publishes a yearly report on the state of global risks.

  The Future of Life Institute, founded in 2014, pursues grantmaking and policy advocacy in the United States, European Union and United Nations. Its donors include Elon Musk, Vitalik Buterin and Jaan Tallinn. On the university side, Nick Bostrom founded the Future of Humanity Institute at Oxford University in 2005. Cambridge University hosts the Centre for the Study of Existential Risk, founded in 2012, which focuses on four man-made risks: artificial intelligence, biotechnology, global warming and warfare. Huw Price, one of its founders, told the AFP news agency that when machine intelligence escapes the constraints of biology, "we're no longer the smartest things around," and risk being at the mercy of "machines that are not malicious, but machines whose interests don't include us."

  Georgetown's Walsh School of Foreign Service established the Center for Security and Emerging Technology in January 2019, with an initial emphasis on artificial intelligence policy; it received a grant of 55 million US dollars from Good Ventures as suggested by Open Philanthropy.