The DOI matches its DataCite record; 245 of the 263 references it cites were found in a registry or an index.
A Survey of Machine Unlearning
Open accessAbstract
Today, computer systems hold large amounts of personal data. Yet while such an abundance of data allows breakthroughs in artificial intelligence, and especially machine learning (ML), its existence can be a threat to user privacy, and it can weaken the bonds of trust between humans and AI. Recent regulations now require that, on request, private information about a user must be removed from both computer systems and from ML models, i.e. ``the right to be forgotten''). While removing data from back-end databases should be straightforward, it is not sufficient in the AI context as ML models often `remember' the old data. Contemporary adversarial attacks on trained models have proven that we can learn whether an instance or an attribute belonged to the training data. This phenomenon calls for a new paradigm, namely machine unlearning, to make ML models forget about particular data. It turns out that recent works on machine unlearning have not been able to completely solve the problem due to the lack of common frameworks and resources. Therefore, this paper aspires to present a comprehensive examination of machine unlearning's concepts, scenarios, methods, and applications. Specifically, as a category collection of cutting-edge studies, the intention behind this article is to serve as a comprehensive resource for researchers and practitioners seeking an introduction to machine unlearning and its formulations, design criteria, removal requests, algorithms, and applications. In addition, we aim to highlight the key findings, current trends, and new research areas that have not yet featured the use of machine unlearning but could benefit greatly from it. We hope this survey serves as a valuable resource for ML researchers and those seeking to innovate privacy technologies. Our resources are publicly available at https://github.com/tamlhp/awesome-machine-unlearning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Related surveys on digital forgetting and machine unlearning, such as Qu et al (2023); Xu et al (2023); Nguyen et al (2022); Shaik et al (2023), have focused on the general ML case by considering models used to classify tabular or image data.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Nguyen et al 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Nguyen et al (2022) refer to two levels of guarantees, exact and approximate . Xu et al (2023) further expand approximate forgetting into strong and weak forgetting, where strong forgetting is equivalent to the definition of approximate forgetting by Nguyen et al ( 2022), and weak forgetting only applies to the outputs of the model.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Nguyen et al 2022
The cited source says what the claim attributes to it.
In the citing paper
“Definitions of unlearning by Nguyen et al (2022); Xu et al (2023) include D as a parameter of the unlearning mechanism (as in F ( D,D f , A ( D )) ), but access to D is not always feasible.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Nguyen et al 2022
The cited source says what the claim attributes to it.
In the citing paper
“Not surprisingly, the unlearning guarantees defined by Nguyen et al ( 2022) and Xu et al (2023 ) are heavily inspired by differential pri -vacy, and privacy attacks, such as membership inference attacks or reconstruction attacks, have been used to measure the risk of privacy leaks and to evaluate the effectiveness of unlearning.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Nguyen et al (2022)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“The MNIST dataset [83] consists of 60,000 training examples and 10,000 test samples that have been standardised to 20x20 pixels, centred by centre of mass in 28x28 pixels, and sheared by horizontally shifting so that the major axis is vertical.”
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193 · cited 1×
“5 ∗ KL(Td (x) | |m), whereM is unlearned model,Td is a competent teacher, and KL is The Kullback-Leibler divergence [84],m = M (x)+Td (x) 2 .”
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
Geurts, P., Ernst, D., & Wehenkel, L. (2006) · Machine Learning · doi:10.1007/s10994-006-6226-1 · Cited by 9,300 · cited 1×
“wrong output ( ZRF = 0) or random output ( ZRF = ( )”
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · cited 1×
“Catastrophic forgetting is a phenomenon where deep neural networks perform badly after learning too many tasks [81].”
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578 · cited 1×
“tially private models suffer a significant loss in accuracy even for large ε [1, 18].”
The MovieLens Datasets: History and Context
Harper, F., & Konstan, J. (2016) · TIIS · doi:10.1145/2827872 · Cited by 5,938 · cited 1×
“Movielens [63] comprises movie ratings from MovieLens, a website that provides movie recommendations.”
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557 · cited 1×
“( ′ | 𝑤 ) is assumed to be a As a result, a candidate set of model parameters 𝑃𝑟 ( 𝑤 | 𝐷 ) is constructed by simply maximizing the posterior probability 𝑃𝑟 ( 𝑤 | 𝐷 ) , i.e.:”
Research works printed but not linked: 19
- [15][15] Gert Cauwenberghs et al. 2000. Incremental and decremental support vector machine learning. NIPS 13 (2000). -
- [22][22] Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. 2021. Graph unlearning. arXiv preprint arXiv:2103.14991 (2021). - · arXiv
- [26][26] Jiali Cheng, George Dasoulas, Huan He, Chirag Agarwal, and Marinka Zitnik. 2023. GNNDelete: A General Strategy for Unlearning in Graph Neural Networks. In ICLR . -
- [36][36] Nicola K Dinsdale, Mark Jenkinson, et al. 2020. Unlearning scanner bias for mri harmonisation. In MICCAI . 369-378. -
- [46][46] Stefan Feuerriegel, Mateusz Dolata, and Gerhard Schwabe. 2020. Fair AI. Business & information systems engineering 62, 4 (2020), 379-384. -
- [48][48] Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. 2014. Privacy in pharmacogenetics: An { End-to-End } case study of personalized warfarin dosing. In USENIX Security . 17-32. -
- [55][55] Johannes Gasteiger, Aleksandar Bojchevski, and Stephan Günnemann. 2019. Combining Neural Networks with Personalized PageRank for Classification on Graphs. In ICLR . -
- [65][65] Ruocheng Guo, Lu Cheng, Jundong Li, P Richard Hahn, and Huan Liu. 2020. A survey of learning causality with data: Problems and methods. CSUR 53, 4 (2020), 1-37. -
- [69][69] William L Hamilton. 2020. Graph representation learning. Synthesis Lectures on Artifical Intelligence and Machine Learning 14, 3 (2020), 1-159. -
- [85][85] Masayuki Karasuyama and Ichiro Takeuchi. 2010. Multiple incremental decremental learning of support vector machines. IEEE Transactions on Neural Networks 21, 7 (2010), 1048-1059. -
- [123][123] Salvatore Mercuri, Raad Khraishi, Ramin Okhrati, Devesh Batra, Conor Hamill, Taha Ghasempour, and Andrew Nowlan. 2022. An introduction to machine unlearning. arXiv preprint arXiv:2209.00939 (2022). - · arXiv
- [125][125] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. 2020. Learning from failure: De-biasing classifier from biased classifier. NIPS 33 (2020), 20673-20684. -
- [130][130] Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. 2022. A Survey of Machine Unlearning. arXiv preprint arXiv:2209.02299 (2022). , , -
- [134][134] Stuart L Pardau. 2018. The California consumer privacy act: Towards a Europeanstyle privacy regime in the United States. J. Tech. L. & Pol'y 23 (2018), 68. -
- [157][157] S Shan, E Wenger, J Zhang, H Li, H Zheng, and BY Zhao. 2020. Protecting Personal Privacy against Una uthorized Deep Learning Models. In USENIX Security . 1-16. -
- [163][163] Abhijeet Singh and Abhineet Anand. 2017. Data leakage detection using cloud computing. IJECS 6, 4 (2017). -
- [165][165] Yash Sinha, Murari Mandal, and Mohan Kankanhalli. 2024. Multi-Modal Recommendation Unlearning. arXiv preprint arXiv:2405.15328 (2024). - · arXiv
- [189][189] Cheng-Long Wang, Mengdi Huai, and Di Wang. 2023. Inductive graph unlearning. In USENIX . 3205-3222. -
- [204][204] Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S. Yu. 2023. Machine Unlearning: A Survey. CSUR 56, 1 (2023), 36. -
Cited by
Sorted by citations
Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102301 · Cited by 626
“While the interest in the machine unlearning paradigm [280] has been on the rise [281], it is unclear how to efficiently ensure that data owned by a particular user is unlearned by a given large-scale generative model, so that it can be ensured that no instance like that of the user claiming their…”
BackgroundXu, H., Zhu, T., Zhang, L., Zhou, W., & Yu, P. S. (2024) · ACM Computing Surveys · doi:10.1145/3603620 · Cited by 164
A Survey on Federated Unlearning: Challenges, Methods, and Future Directions
Liu, Z., Jiang, Y., Shen, J., Peng, M., Lam, K., Yuan, X., & Liu, X. (2024) · ACM Computing Surveys · doi:10.1145/3679014 · Cited by 182
“A detailed comparison of related FU surveys is summarized in Table 1.”
Generative AI in EU law: Liability, privacy, intellectual property, and cybersecurity
Novelli, C., Casolari, F., Hacker, P., Spedicato, G., & Floridi, L. (2024) · Computer Law & Security Review · doi:10.1016/j.clsr.2024.106066 · Cited by 124
FedRecovery: Differentially Private Machine Unlearning for Federated Learning Frameworks
Zhang, L., Zhu, T., Zhang, H., Xiong, P., & Zhou, W. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3297905 · Cited by 118
“It stems from the increasingly urgent requirement of individuals for data protection, and is one of the centrepieces of machine learning research [1], [2].”
Background
Metadata
- Authors
- Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H.
- Affiliations
- VinUniversity (VN)
- University of Bremen (DE)
- Trường Đại học Khoa học và Công nghệ Hà Nội (VN)
- Griffith University (AU)
- The University of Queensland (AU)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- Volume
- 16
- Issue
- 5
- Pages
- 1-46
- ISSN
- 2157-6904 (ISSN-L)
- arXiv
- 2209.02299
- OpenAlex
- W4412565294
- Cited by
- 193
- References
- 263
- Citations by year
- 2026: 75 · 2025: 53 · 2024: 45 · 2023: 20
- FWCI
- 51.16
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Transactions on Intelligent Systems and Technology (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Griffith Research Online (submittedVersion)
- arXiv (Cornell University)
- Funders
- Australian Research Council
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- machine unlearning, right to be forgotten, privacy, adversarial attacks, training data reconstruction, personal data protection, privacy-preserving machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
