Machine Unlearning: A Survey
Open accessThe DOI matches its Crossref record; 132 of the 132 references it cites were found in a registry or an index; 140 of the 233 sources its claims cite were checked for support.
Abstract
This survey captures the key concepts of unlearning techniques and highlights outstanding issues with unlearning techniques, along with feasible directions for new research opportunities.
Overview
CoChat Summary
TL;DR
This survey reviews machine unlearning, proposing a taxonomy that divides existing techniques into data reorganization and model manipulation, with verification mechanisms reviewed alongside each category. It summarizes advantages, limitations, and open issues, including universality, security, verification, and applications.
Key findings
- The paper proposes a taxonomy of unlearning methods: data reorganization (data obfuscation, data pruning, data replacement) and model manipulation (model shifting, model pruning, model replacement). Verification methods are categorized as empirical (retraining-based, attack-based, relearn time-based, accuracy-based) or theoretical (theory-based, information bound-based).
- Data reorganization schemes often retain intermediate parameters and use the original training dataset; they can support complex models and strong unlearning but may affect consistency and accuracy and require additional storage. Data obfuscation seldom completely unlearns information; data pruning can reduce accuracy as sub-models train on partial data; data replacement struggles to retain all original dataset information.
- Model manipulation schemes directly adjust model parameters, reducing intermediate storage, but most focus on simple models or special assumptions; removing information from standard deep networks remains an open problem. They may affect consistency or prediction accuracy, and theory-based or information-bound verification is more convincing but complex and limited to specified models.
- Open questions include universality of unlearning solutions, security risks where unlearning can increase privacy leakage, user-friendly verification, and broader applications. Future directions include information synchronization, federated unlearning, disturbance techniques, feature-based unlearning, and game-theory-based balance.
Methods
- Design: a comprehensive survey that proposes a novel taxonomy based on unlearning rationale and strategy, summarizes state-of-the-art methods, reviews verification methods, and discusses open issues and future directions.
- Data or sample: existing machine unlearning literature; the paper does not state a systematic search protocol, inclusion criteria, or quantitative synthesis.
- Analysis: qualitative classification and comparison across dimensions such as whether original training data is required, whether intermediate data is cached, supported classes and models, unlearning targets, and verification methods.
Limitations
- The paper states limitations of existing unlearning schemes: exact unlearning can only be achieved for simple and well-structured models; many schemes lack theoretical verification; data reorganization can incur storage and accuracy costs; model manipulation is often limited to simple models or special assumptions; and many verification methods are one-sided or not user-friendly.
- The paper does not explicitly state limitations of the survey itself. Evident from the text: it does not report a systematic review protocol or quantitative meta-analysis, and its coverage reflects the literature cited up to the publication date.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
104 Valid · 10 Not valid · 55 Not checked · 91 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“However, since the summation form comes from statistical query (SQ) learning, and only a few machine learning algorithms can be implemented as SQ learning, such as naïve bayes classifiers [83], support vector machines [84], and k-means clustering [85], this scheme has low applicability.”
What the sources say
- Disputedcites [29] [29] Yinzhi Cao and Junfeng Yang. 2015. Towards making systems forget with machine unlearning. In IEEE Symposium on Security and Privacy . IEEE Computer Society, 463-480. DOI: http://dx.doi.org/10.1109/SP.2015.35 - sourceExperimental — accuracy not yet measured
The Machine Unlearning section says many algorithms—including naïve Bayes classifiers, support vector machines, and k-means clustering—can be implemented as SQ learning and describes the approach as general. The Adaptive SQ Learning section says very few practical learning algorithms cannot be converted to its adaptive SQ form. These passages contradict the claim that only a few can be implemented and that this makes the scheme minimally applicable. (LLM, conf=0.92)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“However, this solution will result in a significant reduction in accuracy when dealing with complex models.”
What the sources say
- Disputedcites [93] [93] Yi Liu, Lei Xu, Xingliang Yuan, Cong Wang, and Bo Li. 2022. The right to be forgotten in federated learning: An efficient realization with rapid retraining. In IEEE Conference on Computer Communications . IEEE, 1749-1758. DOI: http://dx.doi.org/10.1109/INFOCOM48880.2022.9796721 - sourceExperimental — accuracy not yet measured
The Evaluation (p. 9) reports that Liu et al.’s algorithm achieves accuracy comparable to retraining from scratch and says momentum compensates for the model-utility loss from its approximation. The Introduction (p. 2) describes the solution as model agnostic and reports only a small accuracy loss on CelebA, contrary to the claimed significant reduction for complex models. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The other studies [49, 50] did not provide a method for verifying the unlearning process.”
What the sources say
- Disputedcites [49] [49] [Varun Gupta, Christopher Jung, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi, and Chris Waites. 2021. Adaptive machine unlearning. In Advances in Neural Information Processing Systems: Annual Conference on Neural Information Processing Systems . 16319-16330. Retrieved from https://proceedings.neurips.cc/paper/2021/hash/ 87f7ee4fdb57bdfd52179947211b7ebb-Abstract.html.](https://proceedings.neurips.cc/paper/2021/hash/87f7ee4fdb57bdfd52179947211b7ebb-Abstract.html) - sourceExperimental — accuracy not yet measured
Section C.2 (pp. 21–22) describes a method for testing an unlearning guarantee: compare outcomes after deletion with outcomes after full retraining and use confidence intervals to assess the difference. The paper uses that method to reject a perfect-unlearning guarantee for the tested SISA algorithms. This contradicts the unqualified claim that it provides no method for verifying unlearning, although the test is not a certificate for an individual deletion request. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
- Could not verifycites [50] [50] Seth Neel, Aaron Roth, and Saeed Sharifi-Malvajerdi. 2021. Descent-to-delete: Gradient-based methods for machine unlearning. In Algorithmic Learning Theory (Proceedings of Machine Learning Research , Vol. 132). PMLR, 931-962. Retrieved from http://proceedings.mlr.press/v132/neel21a.html. - sourceExperimental — accuracy not yet measured
The paper defines an indistinguishability-based unlearning guarantee (Model and Preliminaries, pp. 6–7) and proves that its algorithms satisfy it (Theorem 3.2, p. 11). Those passages do not establish whether it provides a method for verifying that an unlearning process was carried out; a search for verification and auditing found no passage that settles that point. (LLM, conf=0.69)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“The unlearning schemes proposed in References [29, 30, 33, 47, 68, 69] are essentially based on a retraining mechanism that naturally has a verifiability property.”
What the sources say
- Disputedcites [29] [29] Yinzhi Cao and Junfeng Yang. 2015. Towards making systems forget with machine unlearning. In IEEE Symposium on Security and Privacy . IEEE Computer Society, 463-480. DOI: http://dx.doi.org/10.1109/SP.2015.35 - sourceExperimental — accuracy not yet measured
The “B. Machine Unlearning” section explicitly describes the proposed approach as unlearning “without retraining from scratch”: it subtracts the forgotten data from stored summations and updates the model. The “C. Empirical Results” section compares unlearning with retraining to evaluate completeness, but that does not make retraining the mechanism on which the scheme is based. No page numbers are available in our copy. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [30] [30] Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In 42nd IEEE Symposium on Security and Privacy . IEEE, 141-159. DOI: http://dx.doi.org/10.1109/SP40001.2021.00019 - source
- Partly supportedcites [33] [33] Chong Chen, Fei Sun, Min Zhang, and Bolin Ding. 2022. Recommendation unlearning. In ACMWebConference . ACM, 2768-2777. DOI: http://dx.doi.org/10.1145/3485447.3511997 - sourceExperimental — accuracy not yet measured
Section 4.1 (page unavailable) says RecEraser unlearns data by retraining the affected submodel and aggregation component. The discussion of exact unlearning says its aim is complete removal of the requested data, but our copy does not say that this retraining mechanism naturally has a verifiability property. (LLM, conf=0.83)
- Partly supportedcites [47] [47] [Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. 2022. Graph unlearning. In ACM SIGSAC Conference on Computer and Communications Security . ACM, 499-513. DOI: http://dx.doi.org/ 10.1145/3548606.3559352](http://dx.doi.org/10.1145/3548606.3559352) - sourceExperimental — accuracy not yet measured
Section 2.2 (pp. 2–3) describes retraining as an unlearning method, and says GraphEraser removes a requested node and retrains its shard model. The cited paper does not establish the claimed verifiability property; its description of retraining as easy to enforce is not a statement that the result can be verified. (LLM, conf=0.83)
- Not checked — no full text of the sourcecites [68] [68] Wei Qian, Chenxu Zhao, Huajie Shao, Minghan Chen, Fei Wang, and Mengdi Huai. 2022. Patient similarity learning with selective forgetting. In IEEE International Conference on Bioinformatics and Biomedicine . IEEE, 529-534. DOI: http://dx.doi.org/10.1109/BIBM55620.2022.9995016 - source
- Partly supportedcites [69] [69] Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, and Xiaodong Lin. 2022. ARCANE: An efficient architecture for exact machine unlearning. In 31st International Joint Conference on Artificial Intelligence . ijcai.org, 4006-4013. DOI: http://dx.doi.org/10.24963/ijcai.2022/556 - sourceExperimental — accuracy not yet measured
Section 1.2 describes exact unlearning as removing data and retraining, and calls that approach naturally reliable. The Discussion also says exact unlearning is based on retraining at the algorithmic level. Neither passage establishes the distinct claim that these schemes naturally have a verifiability property. (LLM, conf=0.83)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Significantly, model manipulation methods usually give a verifiability guarantee using theory-based and information bound-based methods [40, 41, 43].”
What the sources say
- Partly supportedcites [40] [40] Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition . Computer Vision Foundation/IEEE, 9301-9309. DOI: http://dx.doi.org/10.1109/CVPR42600.2020.00932 - sourceExperimental — accuracy not yet measured
Section 2 derives a KL-divergence bound that guarantees robustness to any readout function, and Section 5.2 says the authors’ proposed methods provide an upper bound on remaining information. The paper supports a theory- and information-bound-based guarantee for its own forgetting methods, but does not establish that model manipulation methods usually provide such guarantees. (LLM, conf=0.84)
- Partly supportedcites [41] [41] Chuan Guo, Tom Goldstein, Awni Y. Hannun, and Laurens van der Maaten. 2020. Certified data removal from machine learning models. In 37th International Conference on Machine Learning (Proceedings of Machine Learning Research , Vol. 119). PMLR, 3832-3842. Retrieved from http://proceedings.mlr.press/v119/guo20c.html. - sourceExperimental — accuracy not yet measured
The Introduction (p. 1) describes a theoretical guarantee for the paper’s certified-removal mechanism, and Section 3.1 (p. 3) derives it using bounds on the gradient residual and max-divergence. The paper does not establish that model-manipulation methods *usually* provide such guarantees, nor does it characterize them generally as verifiability guarantees. (LLM, conf=0.85)
- Not checked — no full text of the sourcecites [43] [43] Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020. Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In 16th European Conference on Computer Vision (Lecture Notes in Computer Science , Vol. 12374). Springer, 383-398. DOI: http://dx.doi.org/10.1007/978-3-030-58526-6\_23 - source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
86 more claims · 36 with their cited sources checked
Create a free account to see all 260 claims and to verify the 12 references still open.
- Probably not a substantial claim. 128
- It may state two separate claims as one. 68
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.1 Unlearning Taxonomy · p. 10
“| Model Pruning [54-56] | Replaces partial parameters with pre-calculated parameters”
§ 3.1 Unlearning Taxonomy · p. 9
“Model pruning: In model pruning, the model provider prunes some parameters from the trained models to unlearn the given samples”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.1 Unlearning Taxonomy · p. 9
“D i , 0 < i < m are the sub-datasets”
§ 3.1 Unlearning Taxonomy · p. 9
“D → D 1 ∪ D 2 ∪ · · · ∪ D m”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.1 Unlearning Taxonomy · p. 10
“| Model Replacement [57-60] | Prunes some parameters from already-trained models”
§ 3.1 Unlearning Taxonomy · p. 9
“Model replacement: In model replacement, the model provider directly replaces some parameters with pre-calculated parameters”
Citations
Cites
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978386 · Cited by 11,000 · cited 1×
“To analyze the rich information in these graphs, graph neural networks (GNNs) have shown unprecedented advantages [74, 75].”
BackgroundSelf-citationMachine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · cited 1×
“In recent years, machine learning has seen remarkable progress and wide exploration across every field of artificial intelligence (AI) [1].”
BackgroundFederated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3298981 · Cited by 6,322
Mosaic organization of DNA nucleotides
Peng, C.-K., Buldyrev, S. V., Havlin, S., Simons, M., Stanley, H. E., & Goldberger, A. L. (1994) · Physical Review E · doi:10.1103/physreve.49.1685 · Cited by 4,985 · cited 1×
“They irst introduce a process called detrended luctuation analysis [81], which quantiies the inluence of the unlearned data on the model parameters, termed temporal residual memory.”
BackgroundMembership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961 · cited 2×
“This is because membership inference attacks [5] and model inversion attacks [6] can reveal information about the speciic contents of a training dataset.”
BackgroundMethodFederated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565
Adaptive Federated Learning in Resource Constrained Edge Computing Systems
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904348 · Cited by 2,232
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · cited 4×
“The model manipulation methods calculate the influence of each sample and offset that influence using a range of techniques [38], while data reorganization schemes usually reorganize the training dataset to simplify the unlearning process.”
BackgroundMethodInfluentialThe 26th USENIX Security Symposium 2017
Αλεξόπουλος, Ν., Kiayias, A., Zacharias, T., & Talviste, R. (2017) · USENIX Security Symposium · Cited by 1,168
Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting
Yeom, S., Giacomelli, I., Fredrikson, M., & Jha, S. (2018) · 2018 IEEE 31st Computer Security Foundations Symposium (CSF) · doi:10.1109/csf.2018.00027 · Cited by 1,001
(2011) · PubMed · Cited by 990
Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges
Khan, L. U., Saad, W., Han, Z., Hossain, E., & Hong, C. S. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3090430 · Cited by 892 · cited 1×
“Federated unlearning: Federated learning is a special kind of distributed learning that is characterized by various unstable users distributed in diferent places, each of whom has control over their devices and data [115, 116].”
BackgroundBourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00019 · Cited by 810
A Survey on Federated Learning for Resource-Constrained IoT Devices
Imteaj, A., Thakker, U., Wang, S., Li, J., & Amini, M. H. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3095077 · Cited by 774 · cited 2×
“There are mainly three types of federated learning: horizontal, vertical, and transfer learning [95].”
BackgroundExtracting Training Data from Large Language Models
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07805 · Cited by 727 · cited 1×
“Machine learning models usually have diferent conidence levels toward the model’s outputs [101].”
BackgroundThe Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1802.08232 · Cited by 629
Towards Making Systems Forget with Machine Unlearning
Cao, Y., & Yang, J. (2015) · 2015 IEEE Symposium on Security and Privacy · doi:10.1109/sp.2015.35 · Cited by 622 · cited 13×
“Based on Data Pruning.”
BackgroundMethodInfluentialFinancial time series forecasting with multi-modality graph neural network
Cheng, D., Yang, F., Xiang, S., & Liu, J. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2021.108218 · Cited by 426 · cited 1×
“By now, numerous important real-world datasets are represented in the form of graphs, such as social networks [70], financial networks [71], biological networks [72], or transportation networks [73].”
BackgroundEternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks
Golatkar, A., Achille, A., & Soatto, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00932 · Cited by 394 · cited 10×
“[40, 43], devised a new metric for verifying the efectiveness of unlearning schemes, where they measured the upper bound of the residual information about samples that need to be unlearned.”
BackgroundMethodResultInfluentialMachine Unlearning of Features and Labels
Warnecke, A., Pirch, L., Wressnegger, C., & Rieck, K. (2023) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2023.23087 · Cited by 381
Differentially Private Data Publishing and Analysis: A Survey
Zhu, T., Li, G., Zhou, W., & Yu, P. S. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2697856 · Cited by 330 · cited 1×
“This technique ensures a subtle bound on the contribution of every sample to the final model [19, 20], but machine unlearning is targeted on the removing of user-specific training samples.”
MethodSelf-citationDynamic Spatial-Temporal Graph Convolutional Neural Networks for Traffic Forecasting
Diao, Z., Wang, X., Zhang, D., Liu, Y., Xie, K., & He, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301890 · Cited by 321 · cited 1×
“By now, numerous important real-world datasets are represented in the form of graphs, such as social networks [70], inancial networks [71], biological networks [72], or transportation networks [73].”
Background"You Might Also Like:" Privacy Risks of Collaborative Filtering
Calandrino, J. A., Kilzer, A., Narayanan, A., Felten, E. W., & Shmatikov, V. (2011) · 2011 IEEE Symposium on Security and Privacy · doi:10.1109/sp.2011.40 · Cited by 311 · cited 1×
“[87] and verify that the unlearning operations successfully prevent the attack from yielding any information.”
BackgroundNew insights and perspectives on the natural gradient method
Martens, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1193 · Cited by 283 · cited 1×
“Fisher unlearning method The second type of model shifting technique uses the Fisher information [90] of the remaining dataset to unlearn speciic samples, with noise injected to optimize the shifting efect.”
MethodAlgorithms that remember: model inversion attacks and data protection law
Veale, M., Binns, R., & Edwards, L. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2018.0083 · Cited by 278 · cited 1×
“More importantly, legislators around the world have wisely introduced laws that grant users the right to be forgotten [7, 8].”
BackgroundFedEraser: Enabling Efficient Client-Level Data Removal from Federated Learning Models
Liu, G., Ma, X., Yang, Y., Wang, C., & Liu, J. (2021) · 2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS) · doi:10.1109/iwqos52092.2021.9521274 · Cited by 248 · cited 4×
“They are mostly designed to leverage the special characteristics of a particular learning process or training scheme [24, 47, 54].”
BackgroundMethodInfluentialGraph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective
Lamb, L. C., Garcez, A. D., Gori, M., Prates, M., Avelar, P., & Vardi, M. Y. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/679 · Cited by 241 · cited 1×
“To analyze the rich information in these graphs, graph neural networks (GNNs) have shown unprecedented advantages [74, 75].”
BackgroundMore Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence
Zhu, T., Ye, D., Wang, W., Zhou, W., & Yu, P. S. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3014246 · Cited by 219 · cited 1×
“Differential privacy [17, 18] guarantees that by looking at a model output, one cannot tell whether a sample is in the training dataset or not.”
BackgroundSelf-citationThe Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining
Liu, Y., Xu, L., Yuan, X., Wang, C., & Li, B. (2022) · IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · doi:10.1109/infocom48880.2022.9796721 · Cited by 202 · cited 2×
“[93] analyzed the validity of the unlearning scheme through two aspects.”
BackgroundMethodDMCP: Differentiable Markov Channel Pruning for Neural Networks
Guo, S., Wang, Y., Li, Q., & Yan, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00161 · Cited by 198 · cited 1×
“An unlearning procedure via channel pruning [97] was also provided, followed by a fine-tuning process to recover the performance of the pruned model.”
MethodWhen Machine Unlearning Jeopardizes Privacy
Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3484756 · Cited by 195 · cited 2×
“Thus, a membership inference attack or a poisoning attack would reveal a signiicant amount of detailed information about the unlearned samples [78, 108].”
BackgroundA Survey of Machine Unlearning
Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H. (2022) · arXiv · doi:10.48550/arxiv.2209.02299 · Cited by 193
Fast Yet Effective Machine Unlearning
Tarun, A. K., Chundawat, V. S., Mandal, M., & Kankanhalli, M. (2021) · arXiv · doi:10.48550/arxiv.2111.08947 · Cited by 190
Knowledge-aware Coupled Graph Neural Network for Social Recommendation
Huang, C., Xu, H., Xu, Y., Dai, P., Xia, L., Lu, M., Bo, L., Xing, H., Lai, X., & Ye, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i5.16533 · Cited by 187 · cited 1×
“By now, numerous important real-world datasets are represented in the form of graphs, such as social networks [70], financial networks [71], biological networks [72], or transportation networks [73].”
BackgroundSelf-citationFast Yet Effective Machine Unlearning
Tarun, A. K., Chundawat, V. S., Mandal, M., & Kankanhalli, M. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3266233 · Cited by 182 · cited 7×
“[27] evaluated the verifiability through several measurements.”
BackgroundMethodInfluentialMaking AI Forget You: Data Deletion in Machine Learning
Ginart, A., Guan, M. Y., Valiant, G., & Zou, J. (2019) · arXiv · doi:10.48550/arxiv.1907.05012 · Cited by 175 · cited 1×
“For instance, it could be used to accelerate the process of leave-one-out-cross-validation, removing adversarial or poisoning samples and identifying signiicant and valuable data samples within a model [13].”
MethodA Survey of Machine Unlearning
Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3749987 · Cited by 164
Chundawat, V. S., Tarun, A. K., Mandal, M., & Kankanhalli, M. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3265506 · Cited by 162 · cited 1×
“Although it is feasible to design an appropriate unlearning scheme for every model, this is an ineicient approach that would require many manual interventions [104, 105].”
MethodCertified Data Removal from Machine Learning Models
Guo, C., Goldstein, T., Hannun, A., & van der Maaten, L. (2019) · arXiv · doi:10.48550/arxiv.1911.03030 · Cited by 160 · cited 10×
“M od el M an ip ul at io n Model Shifting [24, 45] [40, 41, 51] [42, 43, 52, 53] Directly updates model parameters to offset the impact of unlearned samples on the model Does not require too much intermediate parameter storage; can provide theoretical verification Not easy to find an appropriate offset value for compl…”
BackgroundMethodInfluentialFederated Unlearning via Class-Discriminative Pruning
Wang, J., Guo, S., Xie, X., & Qi, H. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3512222 · Cited by 157 · cited 5×
“Model pruning schemes require far-reaching modiications of the existing architecture of the model in the unlearning process [55, 56], which could afect the performance of the unlearned models.”
BackgroundInfluentialWhen is memorization of irrelevant training data necessary for high-accuracy learning?
Brown, G., Bun, M., Feldman, V., Smith, A., & Talwar, K. (2021) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3406325.3451131 · Cited by 152 · cited 1×
“They give a general reduction of the unlearning guarantees from the adaptive sequences to the non-adaptive sequences using differential privacy and max-information theory [80].”
BackgroundA Fast Adaptive k-means with No Bounds
Xia, S., Peng, D., Meng, D., Zhang, C., Wang, G., Giem, E., Wei, W., & Chen, Z. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3008694 · Cited by 149
Humans forget, machines remember: Artificial intelligence and the Right to Be Forgotten
Villaronga, E. F., Kieseberg, P., & Li, T. (2018) · Computer Law & Security Review · doi:10.1016/j.clsr.2017.08.007 · Cited by 142 · cited 1×
“More importantly, legislators around the world have wisely introduced laws that grant users the right to be forgotten [7, 8].”
BackgroundUnrolling SGD: Understanding Factors Influencing Machine Unlearning
Thudi, A., Deza, G., Chandrasekaran, V., & Papernot, N. (2022) · 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp53844.2022.00027 · Cited by 135 · cited 2×
“[37] discussed the semantics behind unlearning and reviewed existing unlearning schemes based on logits, weights, and weight distributions.”
BackgroundChen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2022) · Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3548606.3559352 · Cited by 133 · cited 15×
“Deterministic unlearning schemes, such as SISA [30] or GraphEraser [47], promise nothing about what can be learned about specific samples from the difference between a trained model and an unlearned model.”
BackgroundMethodInfluentialOvercoming Catastrophic Forgetting in Graph Neural Networks
Liu, H., Yang, Y., & Wang, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17049 · Cited by 127 · cited 1×
“Catastrophic forgetting [25, 26] refers to a significant drop in performance on previously learned tasks when a model is fine-tuned for a new task.”
BackgroundMixed-Privacy Forgetting in Deep Networks
Golatkar, A., Achille, A., Ravichandran, A., Polito, M., & Soatto, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00085 · Cited by 125 · cited 6×
“[40, 43, 52] verify the efectiveness of their unlearning schemes based on accuracy and relearning time.”
BackgroundMethodInfluentialGolatkar, A., Achille, A., & Soatto, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58526-6_23 · Cited by 122
Machine Unlearning of Features and Labels
Warnecke, A., Pirch, L., Wressnegger, C., & Rieck, K. (2021) · arXiv · doi:10.48550/arxiv.2108.11577 · Cited by 119 · cited 8×
“[53] also use the exposure metric [2] to measure the remaining information after unlearning.”
BackgroundMethodInfluentialPUMA: Performance Unchanged Model Augmentation for Training Data Removal
Wu, G., Hashemi, M., & Srinivasa, C. (2022) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2203.00846 · Cited by 111
Chen, C., Sun, F., Zhang, M., & Ding, B. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3511997 · Cited by 110 · cited 4×
“Nevertheless, the above unlearning schemes [30, 33, 69] usually need to cache a large number of intermediate results to complete the unlearning process.”
BackgroundMethodInfluentialMachine Unlearning: Linear Filtration for Logit-based Classifiers
Baumhauer, T., Schöttle, P., & Zeppelzauer, M. (2020) · arXiv · doi:10.48550/arxiv.2002.02730 · Cited by 110
Low-Latency Federated Learning Over Wireless Channels With Differential Privacy
Wei, K., Li, J., Ma, C., Ding, M., Chen, C., Jin, S., Han, Z., & Poor, H. V. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126052 · Cited by 109 · cited 1×
“Differential privacy [17, 18] guarantees that by looking at a model output, one cannot tell whether a sample is in the training dataset or not.”
BackgroundMachine unlearning: linear filtration for logit-based classifiers
Baumhauer, T., Schöttle, P., & Zeppelzauer, M. (2022) · Machine Learning · doi:10.1007/s10994-022-06178-9 · Cited by 108
Privacy-preserving PredictionPreprint
Dwork, C., & Feldman, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.10266 · Cited by 106
Second-Order Stochastic Optimization for Machine Learning in Linear TimePreprint
Agarwal, N., Bullins, B., & Hazan, E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.03943 · Cited by 104
Remember What You Want to Forget: Algorithms for Machine Unlearning
Sekhari, A., Acharya, J., Kamath, G., & Suresh, A. T. (2021) · arXiv · doi:10.48550/arxiv.2103.03279 · Cited by 99 · cited 5×
“This approximation is usually guaranteed by diferential privacy techniques, such as (ε,δ )-certiied unlearning [41, 42].”
BackgroundMethodInfluentialARCANE: An Efficient Architecture for Exact Machine Unlearning
Yan, H., Li, X., Guo, Z., Li, H., Li, F., & Lin, X. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/556 · Cited by 97 · cited 3×
“Nevertheless, the above unlearning schemes [30, 33, 69] usually need to cache a large number of intermediate results to complete the unlearning process.”
BackgroundMethodDescent-to-Delete: Gradient-Based Methods for Machine Unlearning
Neel, S., Roth, A., & Sharifi-Malvajerdi, S. (2020) · arXiv · doi:10.48550/arxiv.2007.02923 · Cited by 96
Process synchronization: design and performance evaluation of distributed algorithms
Bagrodia, R. L. (1989) · IEEE Transactions on Software Engineering · doi:10.1109/32.31364 · Cited by 95
Backdoor Defense with Machine Unlearning
Liu, Y., Fan, M., Chen, C., Liu, X., Ma, Z., Wang, L., & Ma, J. (2022) · IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · doi:10.1109/infocom48880.2022.9796974 · Cited by 95
Khosravy, M., Nakamura, K., Hirose, Y., Nitta, N., & Babaguchi, N. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3140687 · Cited by 94 · cited 1×
“Hence, some attack methods can directly and efectively verify unlearning operations ś for example, membership inference attacks [5] and model inversion attacks [4].”
BackgroundDeep Differentiable Random Forests for Age Estimation
Shen, W., Guo, Y., Wang, Y., Zhao, K., Wang, B., & Yuille, A. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2937294 · Cited by 91 · cited 1×
“Random forest is an integrated decision tree model that aims to improve prediction performance [98, 99].”
BackgroundHedgeCut: Maintaining Randomised Trees for Low-Latency Machine Unlearning
Schelter, S., Grafberger, S., & Dunning, T. (2021) · Proceedings of the 2021 International Conference on Management of Data · doi:10.1145/3448016.3457239 · Cited by 89 · cited 13×
“However, they do not consider the security of this intermediate information and whether an attack would recover any information about the unlearned samples [30, 57].”
BackgroundMethodInfluentialThakkar, A., & Chaudhari, K. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106684 · Cited by 88 · cited 1×
“They introduced the concept of term frequency-inverse document frequency (TF-IDF) [96] to quantify the class discrimination of the channels.”
BackgroundAnalyzing Information Leakage of Updates to Natural Language Models
Zanella-Béguelin, S., Wutschitz, L., Tople, S., Rühle, V., Paverd, A., Ohrimenko, O., Köpf, B., & Brockschmidt, M. (2020) · Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3372297.3417880 · Cited by 86 · cited 1×
“Recently, existing research has shown that the unlearning operation not only does not reduce the risk of user privacy leakage but actually increases this risk [106, 107].”
BackgroundFormalizing Data Deletion in the Context of the Right to Be ForgottenPreprint
Goldwasser, S., & Vasudevan, P. N. (2020) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 84
Making recommender systems forget: Learning and unlearning for erasable recommendation
Li, Y., Chen, C., Zheng, X., Liu, J., & Wang, J. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111124 · Cited by 84
PALM: Machine Learning Explanations For Iterative Debugging
Krishnan, S., & Wu, E. (2017) · Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics · doi:10.1145/3077257.3077271 · Cited by 81 · cited 1×
“In fact, removing the inluence of outlier training samples from a model will lead to higher model performance and robustness [16].”
BackgroundMultiple Kernel Clustering with Kernel k-Means Coupled Graph Tensor Learning
Ren, Z., Sun, Q., & Dong, W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17134 · Cited by 76 · cited 1×
“In the graph partitioning step, they introduced an improved balanced label propagation algorithm (LPA) [76] and a balanced embedding k -means [77] partitioning strategy to avoid highly unbalanced shard sizes.”
MethodMembership Inference Attacks Against Recommender Systems
Zhang, M., Ren, Z., Wang, Z., Ren, P., Chen, Z., Hu, P., & Zhang, Y. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3484770 · Cited by 76
Machine Unlearning for Random Forests
Brophy, J., & Lowd, D. (2020) · arXiv · doi:10.48550/arxiv.2009.05567 · Cited by 70
FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence AnalysisPreprint
Huang, B., Li, X., Song, Z., & Yang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.05001 · Cited by 68
Gupta, V., Jung, C., Neel, S., Roth, A., Sharifi-Malvajerdi, S., & Waites, C. (2021) · arXiv · doi:10.48550/arxiv.2106.04378 · Cited by 67
Neural tangent kernel: convergence and generalization in neural networks (invited paper)
Jacot, A., Gabriel, F., & Hongler, C. (2021) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3406325.3465355 · Cited by 64 · cited 1×
“Inspired by the neural tangent kernel (NTK) [91, 92], they considered that deep network activations can be approximated as a linear function of the weights.”
MethodHard to Forget: Poisoning Attacks on Certified Machine Unlearning
Marchant, N. G., Rubinstein, B. I. P., & Alfeld, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20736 · Cited by 64
Variational Bayesian Unlearning
Nguyen, Q. P., Low, B. K. H., & Jaillet, P. (2020) · arXiv · doi:10.48550/arxiv.2010.12883 · Cited by 62
Zhang, P., Wang, Y., Kumar, N., Jiang, C., & Shi, G. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3092746 · Cited by 61
PUMA: Performance Unchanged Model Augmentation for Training Data Removal
Wu, G., Hashemi, M., & Srinivasa, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20846 · Cited by 61
Federated Unlearning with Knowledge Distillation
Wu, C., Zhu, S., & Mitra, P. (2022) · arXiv · doi:10.48550/arxiv.2201.09441 · Cited by 50
RevFRF: Enabling Cross-Domain Random Forest Training With Revocable Federated Learning
Liu, Y., Ma, Z., Yang, Y., Liu, X., Ma, J., & Ren, K. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3104842 · Cited by 49
On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning
Thudi, A., Jia, H., Shumailov, I., & Papernot, N. (2021) · arXiv · doi:10.48550/arxiv.2110.11891 · Cited by 48 · cited 1×
“Those one-sided methods seldom provide strong verification of the unlearning process’s effectiveness [44, 109, 110].”
BackgroundLearning with Selective Forgetting
Shibata, T., Irie, G., Ikami, D., & Mitsuzumi, Y. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/137 · Cited by 47 · cited 2×
“[86] proposed a novel approach to lifelong learning named "Learning with Selective Forgetting", which involves updating a model for a new task by only forgetting speciic classes from previous tasks while keeping the rest.”
BackgroundKoizumi, Y., Murata, S., Harada, N., Saito, S., & Uematsu, H. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683667 · Cited by 44 · cited 1×
“[59] veriied their proposed scheme with a membership inference attack and a technique based on false negative rates (FNRs) [103], where FNR: FNR = FN T P+FN , TP means that the membership inference attack test samples were considered to be training dataset and FN means the data was deemed to be non-training data.”
MethodMaking Recommender Systems Forget: Learning and Unlearning for Erasable Recommendation
Li, Y., Zheng, X., Chen, C., & Liu, J. (2022) · arXiv · doi:10.48550/arxiv.2203.11491 · Cited by 43 · cited 1×
“Machine unlearning has played an essential role in many applications [33, 34].”
BackgroundPrivacy-preserving Feature Extraction via Adversarial Training
Ding, X., Fang, H., Zhang, Z., Choo, K. R., & Jin, H. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2997604 · Cited by 41
Privacy-Preserved Distributed Learning With Zeroth-Order Optimization
Gratton, C., Venkategowda, N. K. D., Arablouei, R., & Werner, S. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3139267 · Cited by 41 · cited 1×
“[30] proposed a łsharded, isolated, sliced, and aggregatedž (SISA) framework, similar to the current distributed training strategies [65, 66], as a method of machine unlearning.”
MethodExploration-exploitation in multi-agent learning: Catastrophe theory meets game theory
Leonardos, S., & Piliouras, G. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2021.103653 · Cited by 39 · cited 1×
“Game-theory-based balance: Game theory has been a booming field with several representative privacy-preserving techniques coming out in the past decade [122].”
BackgroundTowards Probabilistic Verification of Machine Unlearning
Sommer, D. M., Song, L., Wagh, S., & Mittal, P. (2020) · arXiv · doi:10.48550/arxiv.2003.04247 · Cited by 38
Forget-SVGD: Particle-Based Bayesian Federated Unlearning
Gong, J., Kang, J., Simeone, O., & Kassab, R. (2022) · IEEE Data Science Workshop (DSW) · doi:10.1109/dslw53931.2022.9820602 · Cited by 36
Joint Coding and Scheduling Optimization for Distributed Learning Over Wireless Edge Networks
Huynh, N. V., Hoang, D. T., Nguyen, D. N., & Dutkiewicz, E. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118432 · Cited by 34
Sample-Centric Feature Generation for Semi-Supervised Few-Shot Learning
Zhang, B., Ye, H., Yu, G., Wang, B., Wu, Y., Fan, J., & Chen, T. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3154938 · Cited by 33 · cited 1×
“Technologies that may be relevant to this question include feature extraction [119], feature generation[120], and feature selection[121], which could be integrated into unlearning operations.”
BackgroundLearning Fair Naive Bayes Classifiers by Discovering and Eliminating Discrimination Patterns
Choi, Y., Farnadi, G., Babaki, B., & den Broeck, G. V. (2019) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.1906.03843 · Cited by 33
Process synchronization in database systems
Schlageter, G. (1978) · ACM Transactions on Database Systems · doi:10.1145/320263.320279 · Cited by 32 · cited 1×
“Information synchronization: Similar to process synchronization in operating systems, machine unlearning may create information synchronization problems [113, 114].”
BackgroundImproving Data Utility Through Game Theory in Personalized Differential Privacy
Cui, L., Qu, Y., Nosouhi, M. R., Yu, S., Niu, J.-W., & Xie, G. (2019) · Journal of Computer Science and Technology · doi:10.1007/s11390-019-1910-3 · Cited by 31 · cited 1×
“There are many schemes involving privacy-preserving solutions based on game theory that trade-of data privacy and utility issues [123, 124].”
BackgroundCheng, H., Shi, Y., Wu, L., Guo, Y., & Xiong, N. (2021) · Information Sciences · doi:10.1016/j.ins.2020.12.041 · Cited by 29
More than Privacy: Adopting Differential Privacy in Game-theoretic Mechanism Design
Zhang, L., Zhu, T., Xiong, P., Zhou, W., & Yu, P. S. (2021) · ACM Computing Surveys · doi:10.1145/3460771 · Cited by 26
Self-citationMachine Unlearning via Algorithmic Stability
Ullah, E., Mai, T., Rao, A., Rossi, R., & Arora, R. (2021) · arXiv · doi:10.48550/arxiv.2102.13179 · Cited by 26 · cited 1×
“When users revoke permissions over some training data, it is not suicient to merely remove those data from the original training dataset, since the attackers can still reveal user information from the trained models [28].”
BackgroundEmpirical Risk Minimization in the Non-interactive Local Model of Differential PrivacyPreprint
Wang, D., Gaboardi, M., Smith, A., & Xu, J. (2020) · OpenBU (Boston University) · doi:10.48550/arxiv.2011.05934 · Cited by 25 · cited 2×
“Inspired by differential privacy [88], certified removal first limits the maximum difference between the unlearned and retrained models.”
BackgroundZhu, J., Liu, Y., Wen, C., & Wu, X. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2978055 · Cited by 19
MLModelCI: An Automatic Cloud Platform for Efficient MLaaS
Zhang, H., Li, Y., Huang, Y., Wen, Y., Yin, J., & Guan, K. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3414535 · Cited by 19 · cited 1×
“[44] provided a novel backdoor verification mechanism from an individual user perspective in the context of machine learning as a service (MLaaS) [61].”
BackgroundInvestigating the Adoption of Hybrid Encrypted Cloud Data Deduplication With Game Theory
Liang, X., Yan, Z., Deng, R. H., & Zheng, Q. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3028685 · Cited by 18 · cited 1×
“There are many schemes involving privacy-preserving solutions based on game theory that trade-of data privacy and utility issues [123, 124].”
BackgroundHave You Forgotten? A Method to Assess if Machine Learning Models Have Forgotten Data
Liu, X., & Tsaftaris, S. A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-59710-8_10 · Cited by 18 · cited 1×
“Those one-sided methods seldom provide strong veriication of the unlearning process’s efectiveness [44, 109, 110].”
BackgroundMachine Unlearning for Image Retrieval: A Generative Scrubbing Approach
Zhang, P.-F., Bai, G., Huang, Z., & Xu, X.-S. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3548378 · Cited by 18
Online perceptual learning and natural language acquisition for autonomous robots
Alomari, M., Li, F., Hogg, D. C., & Cohn, A. G. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2021.103637 · Cited by 17 · cited 1×
“Online learning [23] adjusts models quickly according to the data in a feedback process, such that the model can relect online changes in a timely manner.”
BackgroundA network embedding-enhanced Bayesian model for generalized community detection in complex networks
He, D., Wang, Y., Cao, J., Ding, W., Chen, S., Feng, Z., Wang, B., & Huang, Y. (2021) · Information Sciences · doi:10.1016/j.ins.2021.06.020 · Cited by 17 · cited 1×
“In the graph partitioning step, they introduced an improved balanced label propagation algorithm (LPA) [76] and a balanced embedding k -means [77] partitioning strategy to avoid highly unbalanced shard sizes.”
MethodMembership Inference Attacks against GANs by Leveraging Over-representation Regions
Hu, H., & Pang, J. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3485338 · Cited by 16 · cited 1×
“In general, the majority of model attack scenarios, such as membership inference attacks, arise from model overitting and rely on observing shifts in the output based on known input shifts [62].”
BackgroundWatanabe, O., & Yokomori, T. (1999) · Lecture notes in computer science · doi:10.1007/3-540-46769-6 · Cited by 15
Making Machine Learning Forget
Shintre, S., Roundy, K. A., & Dhaliwal, J. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-21752-5_6 · Cited by 13 · cited 2×
“[36] analyzed the problem of privacy leakage in machine learning and briely described how the łright-to-be-forgottenž can be implemented with the potential approaches.”
BackgroundUsing POMDPs for learning cost sensitive decision trees
Maliah, S., & Shani, G. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2020.103400 · Cited by 12 · cited 1×
“Random forest is an integrated decision tree model that aims to improve prediction performance [98, 99].”
BackgroundClass Clown: Data Redaction in Machine Unlearning at Enterprise Scale
Felps, D., Schwickerath, A., Williams, J., Vuong, T., Briggs, A., Hunt, M., Sakmar, E., Saranchak, D., & Shumaker, T. (2021) · Proceedings of the 10th International Conference on Operations Research and Enterprise Systems · doi:10.5220/0010419600070014 · Cited by 12 · cited 5×
“[46] intentionally poisoned the labels of the unlearning dataset and then ine-tuned the model based on the new poisoned dataset.”
BackgroundMethodInfluentialHigh-end equipment data desensitization method based on improved Stackelberg GAN
Xiang, N., Zhang, X., Dou, Y., et al. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114989 · Cited by 11 · cited 1×
“Data masking [21] is designed to hide sensitive information in the original dataset.”
BackgroundPatient Similarity Learning with Selective Forgetting
Qian, W., Zhao, C., Shao, H., Chen, M., Wang, F., & Huai, M. (2022) · 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995016 · Cited by 11 · cited 2×
“The unlearning schemes proposed in [29, 30, 33, 47, 68, 69] are essentially based on a retraining mechanism that naturally has a verifiability property.”
BackgroundXavier-Junior, J., Freitas, A., Ludermir, T. B., Feitosa-Neto, A., & Barreto, C. A. S. (2020) · Theoretical Computer Science · doi:10.1016/j.tcs.2019.12.002 · Cited by 9 · cited 1×
“, with majority voting), similar to the case of machine learning ensembles [67].”
BackgroundSinha, R., Pal, R. K., & De, R. K. (2020) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2020.3000661 · Cited by 9
DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks
He, Y., Meng, G., Chen, K., He, J., & Hu, X. (2021) · arXiv · doi:10.48550/arxiv.2105.06209 · Cited by 9 · cited 4×
“[48] use a backdoor veriication method in [44] to verify their unlearning process.”
BackgroundMethodInfluentialResearch on Productization and Development Trend of Data Desensitization Technology
Wang, Z., Wei, K., Jiang, C., Tian, J., Zhong, M., Liu, Y., & Liu, Y. (2021) · 2021 IEEE 20th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom53373.2021.00227 · Cited by 9
Formalizing Data Deletion in the Context of the Right to Be Forgotten
Garg, S., Goldwasser, S., & Vasudevan, P. N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-45724-2_13 · Cited by 8 · cited 1×
“Machine unlearning (a.k.a. selectively forgetting, data deletion, or scrubbing) requires that the samples and their influence can be completely and quickly removed from a training dataset and a trained model [13–15].”
Near-Tight Margin-Based Generalization Bounds for Support Vector MachinesPreprint
Grønlund, A., Kamma, L., & Larsen, K. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.02175 · Cited by 8
Learning Fair Naive Bayes Classifiers by Discovering and Eliminating Discrimination Patterns
Choi, Y., Farnadi, G., Babaki, B., & Van den Broeck, G. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i06.6565 · Cited by 5 · cited 1×
“However, since the summation form comes from statistical query (SQ) learning, and only a few machine learning algorithms can be implemented as SQ learning, such as naïve bayes classiiers [83], support vector machines [84], and k-means clustering [85], this scheme has low applicability.”
BackgroundGraves, L., Nagisetty, V., & Ganesh, V. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17371 · Cited by 5 · cited 7×
“For example, Graves et al. [45] relabeled D u with randomly selected incorrect labels and then fine-tuned the trained model for several iterations for unlearning data.”
BackgroundMethodInfluentialForget-SVGD: Particle-Based Bayesian Federated UnlearningPreprint
Gong, J., Simeone, O., Kassab, R., & Kang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12056 · Cited by 5
(2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-45724-2 · Cited by 5
Privacy Technologies and Policy
(2019) · Lecture notes in computer science · doi:10.1007/978-3-030-21752-5 · Cited by 5
Chen, K., Huang, Y., & Wang, Y. (2021) · arXiv · doi:10.48550/arxiv.2111.11869 · Cited by 4 · cited 2×
“[59] proposed a machine unlearning scheme called WGAN unlearning, which removes information by reducing the output conidence of unlearned samples.”
BackgroundMethodGlobal convergence of online limited memory BFGS
(2015) · Journal of Machine Learning Research · doi:10.5555/2789272.2912100 · Cited by 2 · cited 1×
“The other part uses the L-BGFS algorithm [102] and a set of updates from some prior iterations to calculate Quasi-Hessians approximating the true Hessian-vector.”
Method(2019) · arXiv (Cornell University) · Cited by 1
Kesimoglu, Z. N., & Bozdag, S. (2021) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3459930.3470853 · Cited by 1 · cited 1×
“By now, numerous important real-world datasets are represented in the form of graphs, such as social networks [70], inancial networks [71], biological networks [72], or transportation networks [73].”
BackgroundAlgorithms that Remember: Model Inversion Attacks and Data Protection Law
Veale, M., Binns, R., & Edwards, L. (2018) · arXiv · doi:10.48550/arxiv.1807.04644
Chundawat, V. S., Tarun, A. K., Mandal, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.05629
Analyzing Information Leakage of Updates to Natural Language Models
Zanella-Béguelin, S., Wutschitz, L., Tople, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.07942
Research works printed but not linked: 5
- Sebastian Schelter. 2020. “Amnesia’—Towards machine learning models that can forget user data very fast. In 10th Conference on Innovative Data Systems Research. Retrieved from http://cidrdb.org/cidr2020/papers/p32-schelter-cidr20.pdf.
- Pei-Hung Chen, Wei Wei, Cho-Jui Hsieh, and Bo Dai. 2021. Overcoming catastrophic forgetting by Bayesian generative regularization. In 38th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 139). PMLR, 1760–1770. Retrieved from http://proceedings.mlr.press/v139/chen21v.html.
- Zachary Izzo, Mary Anne Smart, Kamalika Chaudhuri, and James Zou. 2021. Approximate data deletion from machine learning models. In 24th International Conference on Artificial Intelligence and Statistics(Proceedings of Machine Learning Research, Vol. 130). PMLR, 2008–2016. Retrieved from http://proceedings.mlr.press/v130/izzo21a.html.
- Yinjun Wu, Edgar Dobriban, and Susan B. Davidson. 2020. DeltaGrad: Rapid retraining of machine learning models. In 37th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 119). PMLR, 10355–10366. Retrieved from http://proceedings.mlr.press/v119/wu20b.html.
- 10.5555/1953048.2021036 · doi:10.5555/1953048.2021036
Cited by
Vulnerability of Machine Learning Approaches Applied in IoT-Based Smart Grid: A Review
Zhang, Z., Liu, M., Sun, M., Deng, R., Cheng, P., Niyato, D. T., Chow, M., & Chen, J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3349381 · Cited by 202
A Survey on Federated Unlearning: Challenges, Methods, and Future Directions
Liu, Z., Jiang, Y., Shen, J., Peng, M., Lam, K., Yuan, X., & Liu, X. (2024) · ACM Computing Surveys · doi:10.1145/3679014 · Cited by 233
Machine Unlearning: Solutions and Challenges
Xu, J., Wu, Z., Wang, C., & Jia, X. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3379240 · Cited by 234
Toward Generalist Anomaly Detection via In-Context Residual Learning with Few-Shot Sample Prompts
Zhu, J., & Pang, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01688 · Cited by 189
A Survey on Semantic Communication Networks: Architecture, Security, and Privacy
Guo, S., Wang, Y., Zhang, N., Su, Z., Luan, T. H., Tian, Z., & Shen, X. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3516819 · Cited by 167
Metadata
- Authors
- Xu, H., Zhu, T., Zhang, L., Zhou, W., & Yu, P. S.
- Affiliations
- University of Technology Sydney (AU)
- City University of Macau (MO)
- University of Illinois Chicago (US)
- Venue
- ACM Computing Surveys
- Published
- 2023-08-28
- Volume
- 56
- Issue
- 1
- Pages
- 1-36
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4379652910
- Semantic Scholar
- 0813793c140adf6f55b5fcb95ddfb12b8a924121
- Cited by
- 164
- References
- 122
- Citations by year
- 2026: 35 · 2025: 77 · 2024: 46 · 2023: 6
- FWCI
- 14.51
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- ACM Computing Surveys (publishedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Australian Research Council Discovery (DP200100946 and DP230100246)
- NSF (III-1763325, III-1909323, III-2106758, and SaTC-1930941)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Networks and Communications › Cryptography and Data Security
- Physical Sciences › Computer Science › Information Systems › Cloud Data Security Solutions
- Keywords
- machine unlearning, right to be forgotten, privacy, model retraining, privacy-preserving machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-12, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-12 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.