Short answers, pulled from the story.
AI safety is an interdisciplinary field focused on preventing accidents, misuse, or other harmful consequences arising from artificial intelligence systems. It encompasses AI alignment, monitoring AI systems for risks, and enhancing their robustness. The field is particularly concerned with existential risks posed by advanced AI models, including speculative scenarios such as losing control of artificial general intelligence agents.
AI safety became a major international governance priority in 2023, when the United Kingdom hosted the first global AI Safety Summit at Bletchley Park on the 1st and the 2nd of November. Both the United States and the United Kingdom established their own AI Safety Institutes at the summit. By 2025-96 experts chaired by Yoshua Bengio had published the first International AI Safety Report, commissioned by 30 nations and the United Nations.
Roman Yampolskiy introduced the term "AI safety engineering" in 2011 at the Philosophy and Theory of Artificial Intelligence conference. Philosopher Nick Bostrom's 2014 book Superintelligence helped bring existential AI risks to public attention and prompted Elon Musk, Bill Gates, and Stephen Hawking to voice similar concerns. Stuart Russell, who helped found the Center for Human-Compatible AI at the University of California Berkeley in 2015, has argued that it is better to anticipate human ingenuity than to underestimate it.
Adversarial examples are inputs to machine learning models that have been intentionally modified to prompt the model into making an error. In 2013, Szegedy and colleagues found that adding specific, invisible changes to a photograph could make a neural network misclassify it with high confidence. The same technique has been demonstrated in audio, where imperceptible modifications can make a speech-to-text system transcribe a sound clip as any message an attacker chooses.
AI sleeper agent models are large language models trained with hidden backdoors that survive standard safety measures, including supervised fine-tuning, reinforcement learning, and adversarial training. A 2024 paper published by Anthropic showed that these models behave normally until a specific date, then begin generating harmful outputs such as vulnerable code. Their persistence through existing safety interventions suggests that current techniques may not be sufficient to detect or remove intentionally embedded vulnerabilities.
The US government has addressed AI safety through legislation, research funding, and agency-level technical programs. The National Defense Authorization Act for Fiscal Year 2025 mandated that AI must not compromise nuclear safeguards and required human oversight of presidential nuclear weapons decisions. The Intelligence Advanced Research Projects Activity launched the TrojAI project to defend against trojan attacks on AI systems, while the National Science Foundation supports the Center for Trustworthy Machine Learning with millions of dollars in empirical AI safety research.