A Survey of Machine Unlearning
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
Today, computer systems hold large amounts of personal data. Yet while such an abundance of data allows breakthroughs in AI, and especially machine learning, its existence can be a threat to user privacy, and it can weaken the bonds of trust between humans and AI. Recent regulations now require that, on request, private information about a user must be removed both from computer systems and from machine learning models—this legislation is more colloquially called “the right to be forgotten.” While removing data from back-end databases should be straightforward, it is not sufficient in the AI context as machine learning models often “remember” the old data. Contemporary adversarial attacks on trained models have proven that we can learn whether an instance or an attribute belonged to the training data. This phenomenon calls for a new paradigm, namely machine unlearning , to make machine learning models forget about particular data. It turns out that recent works on machine unlearning have not been able to completely solve the problem due to the lack of common frameworks and resources. Therefore, this article aspires to present a comprehensive examination of machine unlearning’s concepts, designs, methods, and applications. Specifically, as a category collection of cutting-edge studies, the intention behind this article is to serve as a comprehensive resource for researchers and practitioners seeking an introduction to machine unlearning and its formulations, design criteria, removal requests, algorithms, and applications. In addition, we aim to highlight the key findings, current trends, and new research areas that have not yet featured the use of machine unlearning but could benefit greatly from it. We hope that this survey serves as a valuable resource for machine learning researchers and those seeking to innovate privacy technologies. Our resources are publicly available at https://github.com/tamlhp/awesome-machine-unlearning .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · cited 5×
“In [135], a set of shallow models were trained on a different training set from that used for the target model.”
InfluentialA Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254
Differential Privacy: A Survey of Results
Dwork, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79228-4_1 · Cited by 3,542
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
The EU General Data Protection Regulation (GDPR)
Voigt, P., & von dem Bussche, A. (2017) · doi:10.1007/978-3-319-57959-7 · Cited by 3,001 · cited 1×
“However, the latter focuses only on the data perspective following the General Data Protection Regulation (GDPR) [160], while machine unlearning also addresses privacy problems from a model perspective.”
Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods
Hüllermeier, E., & Waegeman, W. (2021) · Machine Learning · doi:10.1007/s10994-021-05946-3 · Cited by 1,834 · cited 1×
“metric that measures how much we know about the optimal hypothesis in the parameter space [72] That is, are we certain that the current model parameters are optimal for a given dataset?).”
Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00019 · Cited by 810 · cited 2×
“) is the parameterized mapping function from the input space Z to the output space (i.e., the machine weak unlearning [4].”
AI can be sexist and racist — it’s time to make it fair
Zou, J., & Schiebinger, L. (2018) · Nature · doi:10.1038/d41586-018-05707-8 · Cited by 765
Explainability for Large Language Models: A Survey
Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., & Du, M. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3639372 · Cited by 681
Towards Making Systems Forget with Machine Unlearning
Cao, Y., & Yang, J. (2015) · 2015 IEEE Symposium on Security and Privacy · doi:10.1109/sp.2015.35 · Cited by 622
Bitansky, N., Canetti, R., Chiesa, A., & Tromer, E. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090263 · Cited by 493
Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks
Golatkar, A., Achille, A., & Soatto, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00932 · Cited by 394
A Survey of Learning Causality with Data: Problems and Methods
Guo, R., Cheng, L., Li, J., Hahn, P. R., & Liu, H. (2021) · ACM Computing Surveys · doi:10.1145/3397269 · Cited by 362
Algorithms that remember: model inversion attacks and data protection law
Veale, M., Binns, R., & Edwards, L. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2018.0083 · Cited by 278
Mantelero, A. (2013) · Computer Law & Security Review · doi:10.1016/j.clsr.2013.03.010 · Cited by 264
When Machine Unlearning Jeopardizes Privacy
Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3484756 · Cited by 195 · cited 9×
“ShallowAttack [23] Python Pytorch Model-agnostic https://github.”
InfluentialDeep learning-based unlearning of dataset bias for mri harmonisation and confound removal
Dinsdale, N. K., Jenkinson, M., & Namburete, A. I. L. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117689 · Cited by 178
Chen, Y., Xiong, J., Xu, W., & Zuo, J. (2019) · Cluster Computing · doi:10.1007/s10586-018-1772-4 · Cited by 166 · cited 4×
“Diabetes [12, 24, 166] is dataset of diabetic patients collected from two approaches: traditional paper documents and an automated recording system.”
InfluentialFederated Unlearning via Class-Discriminative Pruning
Wang, J., Guo, S., Xie, X., & Qi, H. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3512222 · Cited by 157
Unrolling SGD: Understanding Factors Influencing Machine Unlearning
Thudi, A., Deza, G., Chandrasekaran, V., & Papernot, N. (2022) · 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp53844.2022.00027 · Cited by 135
Mixed-Privacy Forgetting in Deep Networks
Golatkar, A., Achille, A., Ravichandran, A., Polito, M., & Soatto, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00085 · Cited by 125
Chen, C., Sun, F., Zhang, M., & Ding, B. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3511997 · Cited by 110 · cited 3×
“[18] proposed a partitionbased retraining process (smart retraining) to unlearn the model from the removed user behavior data.”
InfluentialRemember What You Want to Forget: Algorithms for Machine Unlearning
Sekhari, A., Acharya, J., Kamath, G., & Suresh, A. T. (2021) · arXiv · doi:10.48550/arxiv.2103.03279 · Cited by 99
Machine Unlearning for Random Forests
Brophy, J., & Lowd, D. (2020) · arXiv · doi:10.48550/arxiv.2009.05567 · Cited by 70
Gupta, V., Jung, C., Neel, S., Roth, A., Sharifi-Malvajerdi, S., & Waites, C. (2021) · arXiv · doi:10.48550/arxiv.2106.04378 · Cited by 67
Hard to Forget: Poisoning Attacks on Certified Machine Unlearning
Marchant, N. G., Rubinstein, B. I. P., & Alfeld, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20736 · Cited by 64 · cited 3×
“(Bounded/Certified Unlearning) Definition 1 ( 𝜖 -Approximate Unlearning). algorithm 𝐴 if ∀T ⊆ H ,𝐷 ∈ 𝑍 ∗ ,𝑧 ∈ 𝐷 :”
Unlearn What You Want to Forget: Efficient Unlearning for LLMs
Chen, J., & Yang, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.738 · Cited by 52
Athena: Probabilistic Verification of Machine Unlearning
Sommer, D. M., Song, L., Wagh, S., & Mittal, P. (2022) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2022-0072 · Cited by 51
RevFRF: Enabling Cross-Domain Random Forest Training With Revocable Federated Learning
Liu, Y., Ma, Z., Yang, Y., Liu, X., Ma, J., & Ren, K. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3104842 · Cited by 49
Learning with Selective Forgetting
Shibata, T., Irie, G., Ikami, D., & Mitsuzumi, Y. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/137 · Cited by 47 · cited 1×
“) CIFAR 60K 163MB Classification 16+ papers ([58, 70, 136, 163], .”
Digital forgetting in large language models: a survey of unlearning methods
Blanco-Justicia, A., Jebreel, N., Manzanares-Salor, B., Sánchez, D., Domingo-Ferrer, J., Collell, G., & Eeik Tan, K. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11078-6 · Cited by 31
In-Context Unlearning: Language Models as Few Shot Unlearners
Pawelczyk, M., Neel, S., & Lakkaraju, H. (2023) · arXiv · doi:10.48550/arxiv.2310.07579 · Cited by 20
Who's Harry Potter? Approximate Unlearning in LLMs
Eldan, R., & Russinovich, M. (2023) · arXiv · doi:10.48550/arxiv.2310.02238 · Cited by 19
Have You Forgotten? A Method to Assess if Machine Learning Models Have Forgotten Data
Liu, X., & Tsaftaris, S. A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-59710-8_10 · Cited by 18
Machine Unlearning for Image Retrieval: A Generative Scrubbing Approach
Zhang, P.-F., Bai, G., Huang, Z., & Xu, X.-S. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3548378 · Cited by 18
Making Machine Learning Forget
Shintre, S., Roundy, K. A., & Dhaliwal, J. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-21752-5_6 · Cited by 13 · cited 2×
“the problem completely due to the lack of common frameworks and resources [128, 134, 153, 154].”
DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks
He, Y., Meng, G., Chen, K., He, J., & Hu, X. (2021) · arXiv · doi:10.48550/arxiv.2105.06209 · Cited by 9
Dissecting Language Models: Machine Unlearning via Selective Pruning
Pochinkov, N., & Schoots, N. (2024) · arXiv · doi:10.48550/arxiv.2403.01267 · Cited by 1
Cited by
Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102301 · Cited by 626
Eight Methods to Evaluate Robust Unlearning in LLMs
Lynch, A., Guo, P., Ewart, A., Casper, S., & Hadfield-Menell, D. (2024) · arXiv · doi:10.48550/arxiv.2402.16835 · Cited by 188
Xu, H., Zhu, T., Zhang, L., Zhou, W., & Yu, P. S. (2024) · ACM Computing Surveys · doi:10.1145/3603620 · Cited by 164
Generative AI in EU law: Liability, privacy, intellectual property, and cybersecurity
Novelli, C., Casolari, F., Hacker, P., Spedicato, G., & Floridi, L. (2024) · Computer Law & Security Review · doi:10.1016/j.clsr.2024.106066 · Cited by 124
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 798
“…for specific reasoning sub-tasks (Zelikman et al., 2022; Huang et al., 2022a; Ahn et al., 2022), deleting unneeded memory items for “unlearning” (Nguyen et al., 2022c), and studying the interaction effects between multiple forms of learning (Tuyls et al., 2022; Park et al., 2023; Xie et al.,…”
Background
Metadata
- Authors
- Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H.
- Venue
- ACM Transactions on Intelligent Systems and Technology
- Published
- 2025-09-18
- Volume
- 16
- Issue
- 5
- Pages
- 1-46
- ISSN
- 2157-6904 (ISSN-L)
- 2157-6912
- OpenAlex
- W4412565294
- Cited by
- 164
- References
- 40
- FWCI
- 47.44
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.