CoChat
Sign Up
PreprintCited by 76Explainable Artificial Intelligence (XAI)Evolutionary Algorithms and ApplicationsPhilosophy and History of Science

Towards Automated Circuit Discovery for Mechanistic Interpretability

Open access
Conmy, Arthur, Mavor-Parker, Augustine N., Lynch, Aengus, Heimersheim, Stefan, Garriga-Alonso, Adrià·2023·arXiv
DOI 10.48550/arxiv.2304.14997arXiv 2304.14997
Open this paper in CoChat — free accountOpen PDF

Abstract

Through considerable effort and intuition, several recent works have reverse-engineered nontrivial behaviors of transformer models. This paper systematizes the mechanistic interpretability process they followed. First, researchers choose a metric and dataset that elicit the desired model behavior. Then, they apply activation patching to find which abstract neural network units are involved in the behavior. By varying the dataset, metric, and units under investigation, researchers can understand the functionality of each component. We automate one of the process' steps: to identify the circuit that implements the specified behavior in the model's computational graph. We propose several algorithms and reproduce previous interpretability results to validate them. For example, the ACDC algorithm rediscovered 5/5 of the component types in a circuit in GPT-2 Small that computes the Greater-Than operation. ACDC selected 68 of the 32,000 edges in GPT-2 Small, all of which were manually found by previous work. Our code is available at https://github.com/ArthurConmy/Automatic-Circuit-Discovery.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2304.14997

Cites

0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • There is no clear understanding of how emergent properties awaken from the mathematical operations within artificial neural networks; this is currently an object of research in Mechanistic Interpretability.

    “This is still poorly understood since to date there is yet no clear understanding how these emergent properties awaken from the mathematical operations within the artificial neural networks, which is currently the object of research in a discipline called Mechanistic Interpretability”

    From Embracing the future of Artificial Intelligence in the classroom: the relevance of AI literacy, prompt engineering, and critical thinking in modern education (2024) · cites this paper as Conmy et al., 2023

  • Evaluating whether a model 'truly believes' a scenario is real or fictional is very difficult, at least without sophisticated interpretability, because models' stated reasoning may not always be faithful to the actual underlying process used to produce their outputs.

    “However, evaluating whether a model 'truly believes' a scenario is real or fictional is very difficult, at least without sophisticated interpretability (Conmy et al., 2023; Ameisen et al., 2025), as models' stated reasoning-as displayed in its chain-of-thought-may not always be faithful to the actual underlying process used to produce their outputs (Chen et al., 2025; Lindsey et al., 2025).”

    From Agentic Misalignment: How LLMs Could Be Insider Threats (2025) · cites this paper as Conmy et al., 2023

Cites

The publisher hasn’t deposited this paper’s reference list.

Cited by

Sorted by citations

Related

Authors
Conmy, A., Mavor-Parker, A. N., Lynch, A., Heimersheim, S., & Garriga-Alonso, A.
Venue
arXiv
Publisher
arXiv
Published
2023
Pages
16318-16352
ISSN
1049-5258 (ISSN-L)
DOI
10.48550/arxiv.2304.14997
arXiv
2304.14997
OpenAlex
W7133218086
Cited by
72
References
0
Citations by year
2026: 39 · 2025: 16 · 2024: 14 · 2023: 3
FWCI
12.76
Type
conference paper
Language
en
Open access
Licence
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy and History of Science
Keywords
mechanistic interpretability, neural network interpretability, circuit extraction, explainable AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review