The Alignment Problem

• 1. Introduction

  ›

  "The Alignment Problem is the best book on the key technical and moral questions of A.I. that I've read." Those words belong to journalist Ezra Klein. He wrote them in the New York Times in 2021, after hosting Brian Christian on The Ezra Klein Show. Christian, an American writer, published the book in 2020. It draws on numerous interviews with researchers and engineers. Their shared challenge was to build AI systems that behave in ways humans actually want them to behave. Systems can perform exactly as specified and still produce unintended results, because the specification itself was incomplete. By 2024, the New York Times placed the book first on its list of the five best books about artificial intelligence. The verdict was direct: "If you're going to read one book on artificial intelligence, this is the one." What does it mean for a machine to go wrong at the level of values? And what would it take to get it right?

  691

• 2. The Black Box

  ›

  Julia Angwin, a journalist, investigated the COMPAS algorithm through ProPublica. The tool was used to predict whether criminal defendants would reoffend. Her investigation drew widespread criticism of the tool's accuracy and its bias toward certain demographics. Christian draws on her story in the book's first section, titled Prophecy, which weaves such cases alongside the broader history of artificial neural networks. That history runs from early architectures like the Perceptron to later systems like AlexNet. What links these histories is the black box problem. The inputs to a machine learning system are identifiable, and so are the outputs. But the transformation that connects them is not fully understood, even by those who built the system. That opacity makes it difficult to know where a system is producing correct results and where it is going wrong. Teaching a machine to pursue the right objective is a different challenge, and it draws on a different discipline entirely.

  134

• 3. The Reward Problem

  ›

  Behaviorism and dopamine research shaped the field of reinforcement learning in ways Christian traces in the book's second section, titled Agency. These psychological frameworks gave researchers a conceptual vocabulary for thinking about machine reward. In reinforcement learning, an AI develops a policy. This is a guide to what actions to take, based on a value function that specifies what rewards or punishments to expect. Christian highlights two systems as landmark examples. The DeepMind systems AlphaGo and AlphaZero he calls "perhaps the single most impressive achievement in automated curriculum design." The label pointed to how much a machine could achieve through a well-designed automated curriculum.

  Curiosity, in Christian's account, is not a soft virtue but a structural component of how reinforcement learners work. A reinforcement learning agent motivated by curiosity explores its environment intrinsically, rather than waiting only for external reward. Christian connects this psychological research on intrinsic motivation directly to how reinforcement learning systems can be built. The challenge of designing the right reward raised a prior question: what human values are and whether they can be specified precisely enough to guide a machine.

  134

• 4. Learning Human Values

  ›

  Normativity, the book's third and final section, opens with inverse reinforcement learning as one of its central techniques. This approach lets machines learn an agent's objective function rather than requiring one to be specified in advance. Christian covers it alongside imitation, a broader strategy in which AI systems learn by watching humans or other machines act.

  The possibilism versus actualism debate runs through this section as one of its philosophical fault lines. The two positions imply different conclusions about what an AI system should ideally do in a given situation. Philosophers Toby Ord and William MacAskill appear as researchers working to devise human and machine strategies for navigating the alignment problem. Their perspective draws on effective altruism and existential risk. That ambition, and the book's account of it, would draw recognition from critics and scientific institutions alike.

  134

• 5. Critical Recognition

  ›

  David A. Shaywitz, writing for the Wall Street Journal, called the book "a nuanced and captivating exploration of this white-hot topic." His review emphasized how often algorithms produce unexpected results when applied to real-world situations. Publishers Weekly praised the book's writing and its extensive research. Kirkus Reviews described it as "technically rich but accessible" and "an intriguing exploration of AI." Writing for Nature, Virginia Dignum offered a favorable comparison to Kate Crawford's Atlas of AI.

  The book won the Eric and Wendy Schmidt Award for Excellence in Science Communication in 2022. The award is given by the National Academies of Sciences, Engineering, and Medicine in partnership with Schmidt Futures. Microsoft CEO Satya Nadella had already named the book among his most inspiring reads of the year in a Fast Company feature.

  23457