Sparse Autoencoder Features Are Often Causally Inert Despite High Recovery Scores

Jul 15, 2026

A new preprint demonstrates that up to 77% of features recovered by sparse autoencoders (SAEs) with high cosine similarity to ground-truth directions are causally inert, meaning the matched atom does not activate when the feature is present. The authors introduce sae-causal-audit, a model-agnostic tool for causal validation, and identify two types of inertness: structural (due to antipodal-pair geometry) and competitive (arising from TopK pathologies in degraded SAEs).

Why it matters: This work challenges the use of correlational metrics alone for evaluating SAE interpretability, showing that high recovery scores can mask a lack of causal relevance, which is important for mechanistic interpretability research.

Full story at: arXiv Machine Learning ↗