Mechanistic interpretability
Mechanistic interpretability, nicknamed mech interp, mechinterp or simply MI, treats a trained neural network the way a reverse engineer treats a finished program. Nobody hands over the original plans. The analyst opens the thing up and works out what it does from the inside. This research area belongs to explainable artificial intelligence, and its target is the computations running within a model. Three questions drive what follows. Who gave the approach its name, and what were they pushing against? What does it mean to find a mechanism inside a network? And once such a mechanism is found, what can anyone do with it?
Inception v1, a vision model, was one of the networks studied in the work that came before circuit analysis. Those earlier efforts mixed feature visualization, dimensionality reduction and attribution with human-computer interaction methods. Then Chris Olah gave a label to his own line of research: mechanistic interpretability. He used it for his circuit analysis, and he meant it as a contrast with the usual methods in interpretable AI. Circuit analysis tried to characterize individual features and circuits in full. The wider field tended to rely on gradient-based approaches such as saliency maps. That split in ambition is why the new name was needed.
The weights of a machine learning model are the place where mechanistic interpretability goes looking for answers. Researchers hope to find structures, circuits or algorithms encoded there. Earlier interpretability methods mostly concentrated on explanations that linked inputs to outputs, leaving the middle untouched. One idea about that middle is the linear representation hypothesis. It proposes that high-level concepts appear as linear directions in the activation space of a network. Evidence from word embeddings and large language models points toward it. The source also says the hypothesis does not hold everywhere, so it is a guide and not a law.
A circuit, in this field, is a causal chain of feature activations. Mapping which circuits lead to which downstream effects is the central task. Researchers can also switch circuits on or hold them back and watch what changes. By doing this on a large language model, they can trace how a particular input produced a particular result. The methods are causal ones, and they often borrow formal tools from causality theory. The next chapter turns to a different tool that aims to pull those features out of the tangle in the first place.
A sparse autoencoder, shortened to SAE, is a model trained to separate the activations of a neural network into sparse representations. Its learned dimensions often correspond to simple concepts a person can recognize. Anthropic applied the technique to the interpretability of large language models. The appeal is practical. Raw activations are hard to read, while a dimension that tracks a recognizable idea gives researchers something to name, test and connect into circuits.
AI safety is the setting where mechanistic interpretability is used to understand and verify how complex AI systems behave. A central worry is AI misalignment, and the field tries to spot potential risks of that kind. Looking at what a model computes, instead of only what it outputs, is what makes such checks possible. Each circuit mapped and each feature recovered gives that effort another piece of evidence to examine.
Continue browsing
Updated
Common questions
What is mechanistic interpretability?
Mechanistic interpretability is a research area within explainable artificial intelligence that tries to understand the internal workings of neural networks by analyzing the mechanisms in their computations. It is often shortened to mech interp, mechinterp or MI.
Who coined the term mechanistic interpretability?
Chris Olah coined the term to describe his work on circuit analysis. He used it to set that work apart from the usual methods in interpretable AI.
How is mechanistic interpretability different from earlier interpretability methods?
Mechanistic interpretability looks for structures, circuits or algorithms encoded in a model's weights. Earlier methods focused mainly on explanations linking inputs to outputs, and the wider field leaned on gradient-based approaches such as saliency maps.
What is a circuit in mechanistic interpretability?
A circuit is a causal chain of feature activations in a neural network. Researchers map which circuits lead to which downstream effects, and they can activate or inhibit circuits to see how a model reaches a result.
What is a sparse autoencoder in mechanistic interpretability?
A sparse autoencoder is a model trained to separate neural network activations into sparse representations. Its learned dimensions often represent simple, human-understandable concepts, and Anthropic applied the technique to large language models.
What is the linear representation hypothesis in mechanistic interpretability?
The linear representation hypothesis says high-level concepts are represented as linear directions in a neural network's activation space. Word embeddings and large language models offer supporting evidence, though it does not hold universally.
Why does mechanistic interpretability matter for AI safety?
In AI safety, mechanistic interpretability is used to understand and verify the behavior of complex AI systems. It also helps attempt to identify potential risks such as AI misalignment.
All sources
17 references cited across the entry
- 1Mechanistic Interpretability for AI Safety -- A ReviewLeonard Bereska et al. — 2024-08-23
- 2Mechanistic?Naomi Saphra et al. — 2024
- 3JournalZoom In: An Introduction to CircuitsChris Olah et al. — 2020-03-10
- 4JournalThe Building Blocks of InterpretabilityChris Olah et al. — 2018-03-06
- 5JournalAn Overview of Early Vision in InceptionV1Chris Olah et al. — 2020-04-01
- 6Towards automated circuit discovery for mechanistic interpretabilityArthur Conmy et al. — Curran Associates Inc. — 2023-12-10
- 7MagazineWhy AI Breaks BadSteven Levy — 2025-10-27
- 8JournalExplaining AI through mechanistic interpretabilityLena Kästner et al. — 2024-10-11
- 9Linguistic Regularities in Continuous Space Word RepresentationsTomas Mikolov et al. — Association for Computational Linguistics — June 2013
- 10The linear representation hypothesis and the geometry of large language modelsKiho Park et al. — 2024-07-21
- 11Investigating gender bias in language models using causal mediation analysisJesse Vig et al. — Curran Associates Inc. — 2020-12-06
- 12This startup wants to reprogram the mind of AI—and just got $50 million to do itMark Sullivan — 2025-04-22
- 13NewsResearchers are figuring out how large language models work2024-07-11
- 14JournalCausal Abstraction: A Theoretical Foundation for Mechanistic InterpretabilityAtticus Geiger et al. — 2025
- 15JournalBridging the Black Box: A Survey on Mechanistic Interpretability in AIShriyank Somvanshi et al. — June 2026
- 16On the Biology of a Large Language ModelJack Lindsey et al. — Anthropic — 2025-03-27
- 17Emergent Linear Representations in World Models of Self-Supervised Sequence ModelsNeel Nanda et al. — Association for Computational Linguistics — December 2023