Targeted Parameter Decomposition Recovers Mechanistic Circuits in Neural Networks

Jul 16, 2026

Researchers introduce targeted parameter decomposition (tPD), a method for identifying interpretable computational components in neural networks that process specific inputs. tPD recovers mechanistically faithful circuits in transformer language models while using significantly less computational resources than full parameter decomposition. The approach is validated on both toy models and real transformer models, demonstrating the ability to extract targeted submodels and manipulate memorized sequences with minimal impact on unrelated inputs.

Why it matters: This work advances scalable mechanistic interpretability for large neural networks by enabling efficient, targeted analysis of model behavior.

Full story at: arXiv Machine Learning ↗