Superposition made neurons uninterpretable. A sparse autoencoder pulls a single-layer transformer apart into thousands of clean, causal, human-readable features — the first convincing proof that superposition is resolvable.
Bricken, Templeton, Batson, Chen, Jermyn et al. — Anthropic — October 2023 Transformer Circuits transformer-circuits.pub
Presentation 03 (Toy Models of Superposition) showed that a network with limited width packs more sparse features than it has dimensions by storing them as overlapping, non-orthogonal directions. The practical consequence: an individual neuron does not correspond to a single concept. The same neuron fires for, say, academic citations and HTTP headers and Korean text. This is polysemanticity, and it is the central obstacle to interpretability — you cannot understand a model neuron-by-neuron if each neuron means several unrelated things at once.
The right unit of analysis is not the neuron but the feature: a direction in activation space — a specific linear combination of neuron activations — that does correspond to a single human-understandable concept. The question this paper answers: given a trained model's polysemantic neurons, can we mechanically recover the underlying monosemantic features that superposition smeared together?
Interpretability had been stuck because it was asking the wrong question. The unit of computation is not the neuron the architecture hands you — it is a learned direction. Superposition is the reason those directions are hidden, and recovering them is now a concrete, testable engineering task rather than a philosophical one.
The model studied is deliberately tiny: a one-layer transformer whose MLP has 512 neurons. The paper trains a separate model — a sparse autoencoder (SAE) — to reconstruct that MLP activation vector. The SAE is a shallow encoder/decoder:
This is classic dictionary learning: learn an overcomplete dictionary of directions such that every observed activation is a sparse linear combination of them. Sparsity is what disentangles superposition — it pushes each feature to capture one coherent thing.
Going wider than the data feels backwards — an autoencoder usually compresses. But superposition is compression: the 512 neurons already hold more features than dimensions. To undo it you need a layer wide enough to give each crammed-together feature its own slot, and an L1 penalty to make sure it claims exactly one.
From a single MLP layer of 512 polysemantic neurons, the SAE recovers more than 4,000 features (in the headline run) that are individually monosemantic and human-interpretable — and larger dictionaries pull out tens of thousands more. Where a neuron meant five unrelated things, each feature means one. The features the paper surfaces read like a catalogue of crisp concepts:
Crucially, these are not just "fires on token X" detectors. Many are contextual: a feature for a particular token only in a particular surrounding context, exactly the kind of structure a single neuron could never expose cleanly because superposition had it sharing that neuron with unrelated concepts.
This is the payoff. The decomposition turns an opaque 512-neuron blob into a labelled dictionary you can read. The number of clean features exceeds the neuron count by an order of magnitude — direct, concrete evidence that superposition was real and that the model genuinely computes with far more features than it has neurons.
A wide-enough autoencoder can fit almost any pattern. How do we know these features are genuine properties of the model and not artefacts of the SAE — convenient labels we projected onto noise? The paper supplies three independent lines of evidence.
A feature fires when and only when its concept is present. The Arabic-script feature activates on Arabic text and is near-silent otherwise — far sharper than the most-correlated raw neuron, which also fires for unrelated inputs.
Features are not just correlated — activating one steers the output logits in the expected direction. Pin on the DNA feature and the model raises the probability of A, C, G, T tokens. The feature is doing the prediction, not watching it.
An automated pipeline writes a plain-language description of each feature, then is scored on how well that description predicts the feature's activations. Features score markedly higher than the model's neurons.
The causal check is the strongest: it intervenes rather than observes. Because the SAE decoder gives every feature an explicit direction in activation space, you can add that direction back into the running model and read the change in the next-token distribution. A feature that "is" DNA must, when amplified, push the logits toward nucleotide tokens — and it does.
Specificity alone could be coincidence; causality cannot. By moving from "this lights up when we see DNA" to "forcing this on makes the model predict DNA", the paper crosses from description to mechanism — the bar interpretability has to clear to be trustworthy.
How many features are there? The paper's answer is that there is no single number — the dictionary size is a resolution knob. Train a bigger SAE and a single feature splits into several finer-grained ones. A generic feature for the token "the" in a small dictionary becomes, in a larger one, many context-specific "the" features — mathematical "the", legal "the", "the" before a proper noun, and so on.
| Dictionary size | View of the model | Effect on a concept |
|---|---|---|
| Small (coarse) | Few features, easy to read end-to-end | One broad "the" feature |
| Medium | More structure becomes visible | "the" begins to split by context |
| Large (~131k, fine) | Subtle properties exposed | Many specialised "the" features |
Features live on a continuum of granularity set by how many you ask for. This dissolves the "how many features does the model have?" question into a more useful one: at what resolution do you want to read it?
Feature splitting reframes interpretability as zoomable. There is no canonical decomposition — you choose coarse for an overview or fine for detail, the same way you pick a zoom level on a map. This is also a warning: any claim about "the" features must state the dictionary size it was measured at.
If features were quirks of one training run, they would be of little scientific interest. The paper finds the opposite: train the SAE on two independently initialised and trained transformers and you recover largely the same features. The same concepts — DNA, base64, the script detectors — reappear, matchable across models.
Features: Arabic, DNA, base64, citations, legal…
Features: Arabic, DNA, base64, citations, legal…
The match is close enough to pair features between the two dictionaries. That implies the features are not artefacts of one model's idiosyncratic weights but reflect structure in the data and the task that any competent model of this kind learns.
Universality is what makes a finding a science rather than an anecdote. If different models converge on the same features, then a feature dictionary describes something real about language modelling — and interpretability lessons learned on one model have a chance of transferring to another.
This is a proof-of-concept on a toy: one layer, 512 neurons. The obvious question is whether it survives contact with a real, production-scale model. Presentation 05 — Scaling Monosemanticity takes exactly this recipe and runs it on a deployed Claude-class model (Claude 3 Sonnet), extracting millions of features — including abstract, multilingual, and safety-relevant ones — and showing they can be used to steer behaviour.