The DOI matches its DataCite record.
Dissecting Language Models: Machine Unlearning via Selective Pruning
Open accessAbstract
Understanding and shaping the behaviour of Large Language Models (LLMs) is increasingly important as applications become more powerful and more frequently adopted. This paper introduces a machine unlearning method specifically designed for LLMs. We introduce a selective pruning method for LLMs that removes neurons based on their relative importance on a targeted capability compared to overall network performance. This approach is a compute- and data-efficient method for identifying and removing neurons that enable specific behaviours. Our findings reveal that both feed-forward and attention neurons in LLMs are specialized; that is, for specific tasks, certain neurons are more crucial than others. Code from all experiments is available at https://github.com/nickypro/selective-pruning
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Selective pruning provides a promising avenue for approximate machine unlearning [3], as it identifies and removes model components (e.g., neurons or attention heads) that are disproportionately influential on the targeted forget set , while preserving performance on the retain set .”
From Verifiable Unlearning on Edge (2025) · cites this paper as [3]
Citations
Cited by
Sorted by citations
Open Problems in Mechanistic Interpretability
Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.16496 · Cited by 291
Self-citationA Survey of Machine Unlearning
Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H. (2022) · arXiv · doi:10.48550/arxiv.2209.02299 · Cited by 193
Maheri, M. M., Davidson, A., & Haddadi, H. (2025) · arXiv · doi:10.48550/arxiv.2506.20037
“Selective pruning provides a promising avenue for approximate machine unlearning [3], as it identifies and removes model components (e.g., neurons or attention heads) that are disproportionately influential on the targeted forget set , while preserving performance on the retain set .”
Metadata
- Authors
- Pochinkov, N., & Schoots, N.
- Affiliations
- King's College London (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2403.01267
- OpenAlex
- W4392487615
- Cited by
- 1
- References
- 0
- Citations by year
- 2025: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- machine unlearning, large language models, selective pruning, neuron importance, computational efficiency, data efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
