"Neural Networks Are Already Clean": Goodfire's Cammarata Says Most Neurons Are Legible

A Goodfire researcher is arguing that one of the field's central pessimistic assumptions about AI interpretability may be wrong. In an essay titled "Neural Networks Are Already Clean", published October 15, 2026, Nick Cammarata writes that "the cause of polysemanticity in early model organisms wasn't superposition, but several ordinary, mostly fixable issues," and that "we can now understand most neurons in models up to trillions of parameters." The essay is explicitly labeled "a working hypothesis," and a technical companion paper is still to come.

This post summarizes what the essay claims, which numbers back it, where the author hedges, and how to read it.

TL;DR

The background: polysemanticity and superposition

Interpretability researchers try to explain what the parts of a neural network do. The obstacle has been polysemantic neurons, which fire for several seemingly unrelated things. The leading explanation was superposition: when a network runs out of neurons, it packs unrelated concepts together because they rarely appear at once, leaving a "tangled knot." If that is true, studying neurons directly tells you little, and researchers must first untangle the network by training a second model such as a sparse autoencoder to approximate it unfolded and then study that.

Cammarata's essay describes two approaches to the field: doing "rapid normal science on the network itself," or starting from hypothesized features and extracting them with increasingly sophisticated reconstruction objectives. The field started with the first, he says, and fell back on the second because networks seemed tangled.

The core argument: recursive generalization

The essay proposes that what looked like arbitrary packing was mostly a motif he calls recursive generalization: as a neuron grows, it sprouts offshoots the way a branch sprouts sub-branches. His example is a neuron that detected "iy," grew to detect "diy" and then "do it yourself," which does not contain "iy" at all. On this view, generalization is "fractal," which is "far less concerning" than superposition: there is something to learn from how things were grouped, whereas two items in superposition teach nothing.

He argues several ordinary problems compounded to make the early picture look worse than it was:

• Explainer models in 2023 could not run the loop of conjecture and test.
• Tools and metrics rewarded explanations that simulate activations over explanations that are right.
• Researchers did not know which language-neuron motifs to expect.
• Most studies used small or simplified models that really had run out of room, where neurons just get broader instead of switching principles.

The cat-and-car neuron

Mixture of experts routing diagram, a stand-in for how neurons and experts are studied in interpretability work

The most famous example is neuron 4e:55 in InceptionV1, which appeared to respond to both cat faces and the fronts of cars. Papers from Feature Visualization in 2017 to Zoom In in 2020 cited it as evidence neurons might not be the right unit of study. Cammarata, who co-authored Zoom In, writes that he "did something I'd never done": he spent a few seconds asking what one thing the images might share, and concluded it is a V-shape neuron. Its circuit connections include V-shaped flames, V-shaped water jets from a fireboat and V-shaped fox faces, and its negative pole responds to an O, round where the V is pointed. The whiskers in feature visualizations, he suggests, were a side effect of how visualization raises activation.

He applies the same reading to a circuit from 2019 that appeared to show car information spreading into unrelated dog neurons. Nothing downstream reads that combination, he says, so it was "ordinary generalization and composition," and he is not aware of a second published circuit with the right shape.

The numbers

The essay gives several figures. The numbers reflect different models and methods, so do not read them as one benchmark:

The author is open about the threshold problem. How far to look into weak activations, whether to use percentiles or fractions of the strongest activations, and what counts as "one thing" all move the number. He says a single "cleanliness" score is "a broad pointer to the regime a neural network is in, rather than something precise." He also notes that dense and mixture-of-experts models seem similar, and says similar results were found in trillion-parameter models like Kimi K3, to be covered in the companion paper.

Why the hedges matter

The essay's strongest statements sit beside its own limits:

• It is "a working hypothesis" with a 65-minute text version, not a peer-reviewed paper.
• The Qwen3-32B figure is a "preliminary audit," and the trillion-parameter results are described without full data.
• Models at that scale "aren't close to solved due to sheer size," though the author is optimistic about scaling the process.
• The fourth step of the argument, that this might be most of what is needed and interpretability could be far more tractable "possibly in the near term," is labeled "highly speculative."

We have not verified any independent reaction yet, and the essay itself argues against a widely held view in the field, so expect pushback from researchers who still favor superposition-based explanations. Because Goodfire employs the author and sells interpretability tooling, the usual advice applies: wait for replication by groups without a stake in the outcome.

Why this matters for AI safety and builders

AI safety guardrails concept for interpretability-based monitoring of model behavior

If most neurons really are legible, several things change in principle. Auditing a model for dangerous capabilities or deception could rely on studying its actual units, not on approximations. Automated agents could do the studying at scale, which the essay explicitly proposes: agents that map neurons, build explorable microscopes and track hundreds of thousands of attribution graphs. And monitoring could improve.

Goodfire's applied work, listed on its research page, already points in that direction. We covered its inside-out monitors that catch rogue agents on Kimi K3 and the way Google reports activation probes in the Gemini 4 Argon safety report. Those use internal signals as monitors. The essay's claim, if it holds, would strengthen the basis for such tools.

For readers who build with models rather than study them, the practical effect is indirect and further off. Better interpretability could eventually mean fewer surprise behaviors and better guarantees; it does not change which model you should pick this week. Compare the debate over who can measure AI pace in our post on Anthropic's R&D automation index and the proposals for outside checks in METR's embedded assessments.

How to read the essay yourself

1. Start with "The Short Version" at the top of the essay, then the "Cars and Animals" section for the concrete example.
2. Read footnote 1 on the 90% figure, which explains the threshold dependence.
3. Look for the companion paper for model-level data, especially the trillion-parameter claims.
4. Read it alongside the original superposition papers, since the disagreement turns on what superposition theory actually predicts.
5. Treat any single percentage as provisional until others reproduce the audit method on other models.

The essay does not give a single formal replication protocol. Instead it suggests checking explanations from several angles: watching neurons form across training checkpoints, since recursive generalization and superposition make opposite predictions; steering a neuron to see whether its output matches the proposed explanation; and inspecting its negative pole, upstream inputs and downstream effects. It says a correct explanation should survive all of these checks. It also notes that once a network is largely understood, failures in the tools themselves become visible, which is how the author says a double-negative regime in SwiGLU activations was found. The only named public tools so far are the explorable "microscope" pages for InceptionV1 and ConvNeXt, and the essay does not mention releasing code.

Claimed versus verified

Details are from the October 15, 2026 Goodfire essay and may change.

Related reading

• Goodfire's inside-out monitors on Kimi K3
• Gemini 4 Argon safety report: activation probes
• Anthropic R&D automation index
• METR embedded assessments for agent control
• Anthropic independent evaluators and METR