Amanda Askell

• 1. Oxford And Infinite Ethics

  ›

  Amanda Askell received a BPhil degree in Philosophy from the University of Oxford. She later earned a PhD degree in Philosophy from New York University in 2018. Her doctoral thesis focused on Pareto Principles in Infinite Ethics. This work argued that rankings of worlds containing infinitely many agents create puzzles for ethical theories. The text notes these puzzles arise when constrained by plausible axioms. Her research examined how infinite populations challenge standard moral frameworks. This academic background laid the groundwork for her future work in artificial intelligence.

  48

• 2. Leaving OpenAI For Safety

  ›

  Askell joined OpenAI as a Research Scientist on the policy team in November 2018. She co-authored the GPT-3 paper which appeared as a pre-print on the 28th of May 2020. Her focus included AI development races between organizations and their potential adversarial nature. She also examined intersections between policy questions and AI safety. By 2021 she left the company over concerns about prioritizing AI safety. Critics noted she felt the organization was not placing enough emphasis on safety measures. This departure marked a significant shift in her career trajectory toward more direct alignment work.

  310

• 3. Building Personality At Anthropic

  ›

  A 2023 paper co-authored with Deep Ganguli explored moral self correction in large language models. The study tested if systems could reduce harmful outputs using natural language instructions. Researchers found this capability emerged at 22 billion parameters. Results showed larger models followed complex instructions better than smaller ones. Instructions such as Please ensure that your answer is unbiased substantially reduced biased outputs. The research demonstrated models learned normative concepts from training data without explicit definitions. This finding suggested scale and reinforcement

  1216

• 4. Moral Self Correction Emerges

  ›

  learning from human feedback were key drivers.

  Askell serves as primary author for the latest version of Claude's constitution released in January 2026. This document guides behavior through a set of principles known as Constitutional AI. The method allows models to critique and revise their own responses based on these rules. She focuses on helping models understand and grapple with the constitution through synthetic data generation. Reinforcement learning techniques support this process alongside standard training methods. The approach aims to meet standards of harmlessness and helpfulness without extensive

  1415

• 5. Constitutional AI Frameworks

  ›

  human oversight.

  161719