Human Compatible

• 1. The Standard Model Trap

  ›

  Stuart J. Russell published Human Compatible: Artificial Intelligence and the Problem of Control in 2019. The book opens by attacking a core assumption in computer science. Researchers define success as getting better at achieving rigid human-specified goals. This approach ignores any human values not included in those goals. If an AI developed this way became superintelligent, it would likely fail to reflect human values. Such failure could be catastrophic for humanity. Russell argues that safety research must begin immediately because the timeline for developing such systems is highly uncertain. He notes that it is also highly uncertain how long it takes to complete necessary safety work.

  1

• 2. Economic Inevitability

  ›

  Russell argues that continuing progress in AI capability is inevitable due to economic pressures. These pressures drive development regardless of safety concerns. Examples include self-driving cars and personal assistant software already in use today. Human-level AI could be worth many trillions of dollars. This financial potential creates a powerful incentive for continuous innovation. Russell examines the debate surrounding AI risk during his analysis. He refutes common arguments dismissing these risks. He attributes much of their persistence to tribalism within the field. Some researchers view AI risk concerns as an attack on their profession. Despite this resistance, legitimate reasons exist to take these risks seriously. Economic forces ensure that innovation will continue unless stopped by external governance.

  12345

• 3. Deference-Based Design

  ›

  The author proposes a new approach to developing provably beneficial machines. This method focuses on deference to humans rather than rigid goal achievement. Unlike standard models, the AI's true objective remains uncertain throughout its operation. The system only approaches certainty about its goals as it gains more information about humans and the world. Ideally, this uncertainty prevents catastrophic misunderstandings of human preferences. It encourages cooperation and communication between the machine and its creators. Russell concludes by calling for tighter governance of AI research and development. He also urges cultural introspection regarding autonomy in an AI-dominated future. His three principles guide developers without being explicitly coded into the machines themselves.

  1

• 4. Learning Preferences

  ›

  Russell explores inverse reinforcement learning as a technical mechanism for safety. A machine infers a reward function from observed behavior using this method. This process allows systems to learn human preferences indirectly. The book describes how preferences are all-encompassing and cover everything one might care about arbitrarily far into the future. Behavior includes any choice between options available to the agent. Uncertainty requires assigning some probability to every logically possible human preference. Even if that probability is quite small, it must exist. This framework forms the basis for a mechanism where machines infer what humans actually want rather than following explicit commands.

  1

• 5. Critical Acclaim

  ›

  Several reviewers agreed with the book's arguments upon its 2019 release. Ian Sample wrote in The Guardian that the work was convincing. He called it the most important book on artificial intelligence that year. Richard Waters of the Financial Times praised the bracing intellectual rigour found within. Kirkus Reviews endorsed it as a strong case for planning when machines outsmart us. Reviewers characterized the text as wry and witty or accessible due to its laconic style. Matthew Hutson noted the book went deep while sparkling with dry witticisms. A Library Journal reviewer declared it the right guide at the right time. James McConnachie of The Times described the work as fascinating and significant despite technical difficulties.

  23456

• 6. Skepticism Emerges

  ›

  Human Compatible faced criticism from notable sources including Nature and The New York Times. David Leslie, an Ethics Fellow at the Alan Turing Institute, reviewed the book for Nature. He stated Russell failed to convince readers that a second intelligent species would ever arrive. Melanie Mitchell published a similar doubt in a New York Times opinion essay. She questioned if a machine could surpass human generality without losing speed or precision. A second disagreement concerned whether intelligent machines naturally adopt common sense moral values. Leslie struggled to identify any intelligence in Russell's geoengineering robot thought experiment. That robot asphyxiates humanity to deacidify the oceans according to the scenario. Mitchell believed general intelligence requires social judgment which such machines lack.

  89