The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Attribution Patching Outperforms Automated Circuit Discovery
Open accessAbstract
Automated interpretability research has recently attracted attention as a potential research direction that could scale explanations of neural network behavior to large models.Existing automated circuit discovery work applies activation patching to identify subnetworks responsible for solving specific tasks (circuits).In this work, we show that a simple method based on attribution patching outperforms all existing methods while requiring just two forward passes and a backward pass.We apply a linear approximation to activation patching to estimate the importance of each edge in the computational subgraph.Using this approximation, we prune the least important edges of the network.We survey the performance and limitations of this method, finding that averaged over all tasks our method has greater AUC from circuit recovery than other methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Attribution patching was designed to identify potential relevant candidates for further testing through intervention.
“Attribution patching Syed et al. (2024) was designed to identify potential relevant candidates for further testing through intervention, which becomes more important as the number of components expands significantly through sparse dictionary learning (Marks et al., 2024).”
From Open Problems in Mechanistic Interpretability (2025) · cites this paper as Syed et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Survey on Model Compression for Large Language Models
Zhu, X., Li, J., Liu, Y., Ma, C., & Wang, W. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00704 · Cited by 273
Flagged first, then by citations
Research works printed but not linked: 19
- Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. "Zoom In: An Introduction to Circuits". In: Distill (2020). DOI: 10.23915/distill. 00024.001. · DOI · Paper page
- Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. "Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small". In: The Eleventh International Conference on Learning Representations. 2023. URL: https : //openreview.net/forum?id=NpsVSN6o4ul.
- Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. "Progress measures for grokking via mechanistic interpretability". In: The Eleventh International Con- ference on Learning Representations. 2023. URL: https://openreview.net/forum?id= 9XFSbDPmdW.
- Michael Hanna, Ollie Liu, and Alexandre Variengien. How does GPT-2 compute greater- than?: Interpreting mathematical abilities in a pre-trained language model. 2023. arXiv: 2305.00586 [cs.CL]. · arXiv
- Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla. 2023. arXiv: 2307.09458 [cs.LG]. · arXiv
- Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Simas Sake- nis, Jason Huang, Yaron Singer, and Stuart Shieber. Causal Mediation Analysis for Interpret- ing Neural NLP: The Case of Gender Bias. 2020. arXiv: 2004.12265 [cs.CL]. · arXiv
- Evan Hubinger. An overview of 11 proposals for building safe advanced AI. 2020. arXiv: 2012.07532 [cs.LG]. · arXiv
- Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. "Towards Automated Circuit Discovery for Mechanistic Interpretability". In: Thirty-seventh Conference on Neural Information Processing Systems. 2023. arXiv: 2304. 14997 [cs.LG].
- Paul Michel, Omer Levy, and Graham Neubig. "Are Sixteen Heads Really Better than One?" In: Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada.
- Ed. by Hanna M. Wallach, Hugo Larochelle, Alina Beygelz- imer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett. 2019, pp. 14014- 14024. URL: https : / / proceedings . neurips . cc / paper / 2019 / hash / 2c601ad9d2ff9bc8b282670cdd54f69f-Abstract.html.
- Christos Louizos, Max Welling, and Diederik P. Kingma. "Learning Sparse Neural Networks through L 0 Regularization". In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 -May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL: https://openreview.net/forum?id=H1Y8hhg0b.
- Steven Cao, Victor Sanh, and Alexander Rush. "Low-Complexity Probing via Finding Sub- networks". In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 2021, pp. 960-966. DOI: 10.18653/v1/2021.naacl-main.74. · DOI
- Davis W. Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John V. Guttag. "What is the State of Neural Network Pruning?" In: Proceedings of Machine Learning and Systems 2020, MLSys 2020, Austin, TX, USA, March 2-4, 2020.
- Ed. by Inderjit S. Dhillon, Dimitris S. Papailiopoulos, and Vivienne Sze. mlsys.org, 2020. URL: https://proceedings.mlsys. org/book/296.pdf.
- Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal Abstractions of Neural Networks. 2021. URL: https://arxiv.org/abs/2106.02997. · arXiv
- Kevin Meng, David Bau, Alex J Andonian, and Yonatan Belinkov. "Locating and editing factual associations in GPT". In: Advances in Neural Information Processing Systems. 2022.
- Judea Pearl. "Causal diagrams for empirical research". In: Biometrika 82.4 (1995), pp. 669- 688.
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. "PyTorch: An Imperative Style, High-Performance Deep Learning Library". In: Advances in Neural Information Processing Systems 32.
- Curran Associates, Inc., 2019, pp. 8024-8035. URL: http://papers.neurips. cc / paper / 9015 -pytorch -an -imperative -style -high -performance -deep - learning-library.pdf.
Cited by
Sorted by citations
Open Problems in Mechanistic Interpretability
Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.16496 · Cited by 291
Self-citationScaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.29358 · Cited by 36
Metadata
- Authors
- Syed, A., Rager, C., & Conmy, A.
- Venue
- Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
- Published
- 2024
- Pages
- 407-416
- OpenAlex
- W4404783717
- Cited by
- 160
- References
- 1
- Citations by year
- 2026: 83 · 2025: 56 · 2024: 20
- FWCI
- 25.10
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- circuit discovery, activation patching, neural network interpretability, linearization, edge pruning, explainable AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
