Athena: Probabilistic Verification of Machine Unlearning
Open accessAbstract
The right to be forgotten, also known as the right to erasure, is the right of individuals to have their data erased from an entity storing it. The status of this long held notion was legally solidified recently by the General Data Protection Regulation (GDPR) in the European Union. As a consequence, there is a need for mechanisms whereby users can verify if service providers comply with their deletion requests. In this work, we take the first step in proposing a formal framework, called Athena, to study the design of such verification mechanisms for data deletion requests – also known as machine unlearning – in the context of systems that provide machine learning as a service (MLaaS). Athena allows the rigorous quantification of any verification mechanism based on hypothesis testing. Furthermore, we propose a novel verification mechanism that leverages backdoors and demonstrate its effectiveness in certifying data deletion with high confidence, thus providing a basis for quantitatively inferring machine unlearning. We evaluate our approach over a range of network architectures such as multi-layer perceptrons (MLP), convolutional neural networks (CNN), residual networks (ResNet), and long short-term memory (LSTM) and over 6 different datasets. We demonstrate that: (1) our approach has minimal effect on the accuracy of the ML service but provides high confidence verification of unlearning, even if multiple users employ our system to ascertain compliance with data deletion requests, and (2) our mechanism is robust against servers deploying state-of-the-art backdoor defense methods. Overall, our approach provides a foundation for a quantitative analysis of verifying machine unlearning, which can provide support for legal and regulatory frameworks pertaining to users’ data deletion requests.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
By comparing the performance of the unlearned model on clients' data with that of the original global model, backdoor effects can be well eliminated.
“By comparing the performance of the unlearned model on clients' data with that of the original global model, the backdoor effects can be well eliminated [225].”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [225]
An 'unlearning' model aims to eliminate the residual effects of the backdoor by excluding the influence of suspicious participants and has been shown to be effective.
“One such approach is the use of an 'unlearning' model, which aims to eliminate the residual effects of the backdoor by excluding the influence of suspicious participants and has been shown to be effective [225], [256], [257].”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [225]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Character-level Convolutional Networks for Text ClassificationPreprint
Zhang, X., Zhao, J., & LeCun, Y. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1509.01626 · Cited by 7,552 · see citing works
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961 · see citing works
Spectre Attacks: Exploiting Speculative Execution
Kocher, P. C., Horn, J., Fogh, A., Genkin, D., Gruss, D., Haas, W., Hamburg, M., Lipp, M., Mangard, S., Prescher, T., et al. (2019) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2019.00002 · Cited by 1,847 · see citing works
EMNIST: Extending MNIST to handwritten letters
Cohen, G., Afshar, S., Tapson, J. C., & André, V. S. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966217 · Cited by 1,782 · see citing works
Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
Wang, B., Yao, Y., Shan, S., Li, H., Viswanath, B., Zheng, H.-T., & Zhao, B. Y. (2019) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2019.00031 · Cited by 1,486 · see citing works
Trojaning Attack on Neural Networks
Liu, Y., Ma, S., Aafer, Y., Lee, W.-C., Zhai, J., Wang, W., & Zhang, X. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23291 · Cited by 1,270 · see citing works
Torch7: A Matlab-like Environment for Machine Learning
Collobert, R., Kavukcuoglu, K., & Farabet, C. (2011) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,241 · see citing works
BadNets: Evaluating Backdooring Attacks on Deep Neural Networks
Gu, T., Liu, K., Dolan-Gavitt, B., & Garg, S. (2019) · IEEE Access · doi:10.1109/access.2019.2909068 · Cited by 1,238 · see citing works
Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks
Liu, K., Dolan-Gavitt, B., & Garg, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00470-5_13 · Cited by 1,055 · see citing works
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
Chen, X., Liu, C., Li, B., Lu, K., & Song, D. (2017) · arXiv · doi:10.48550/arxiv.1712.05526 · Cited by 1,039 · see citing works
Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting
Yeom, S., Giacomelli, I., Fredrikson, M., & Jha, S. (2018) · 2018 IEEE 31st Computer Security Foundations Symposium (CSF) · doi:10.1109/csf.2018.00027 · Cited by 1,001 · see citing works
Comparative Study of CNN and RNN for Natural Language ProcessingPreprint
Yin, W., Kann, K., Yu, M., & Schütze, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.01923 · Cited by 903 · see citing works
LEAF: A Benchmark for Federated Settings
Caldas, S., Duddu, S. M. K., Wu, P., Li, T., Konečný, J., McMahan, H. B., Smith, V., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1812.01097 · Cited by 894 · see citing works
Meltdown: reading kernel memory from user space
Lipp, M., Schwarz, M., Gruss, D., Prescher, T., Haas, W., Fogh, A., Horn, J., Mangard, S., Kocher, P. C., Genkin, D., et al. (2018) · USENIX Security Symposium · Cited by 821 · see citing works
Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00019 · Cited by 810 · see citing works
Efficient noise-tolerant learning from statistical queries
Kearns, M. J. (1998) · Journal of the ACM · doi:10.1145/293347.293351 · Cited by 679 · see citing works
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1802.08232 · Cited by 629 · see citing works
Towards Making Systems Forget with Machine Unlearning
Cao, Y., & Yang, J. (2015) · 2015 IEEE Symposium on Security and Privacy · doi:10.1109/sp.2015.35 · Cited by 622 · cited 1× · see citing works
“Therefore, Cao and Yang [7] train conventional models based on intermediate summations of training data [65] and update summations upon deletion request.”
MethodSpectral Signatures in Backdoor AttacksPreprint
Tran, B., Li, J. Z., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00636 · Cited by 506 · see citing works
Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
Chen, B., Carvalho, W., Baracaldo, N., Ludwig, H., Edwards, B., Lee, T., Molloy, I., & Srivastava, B. (2018) · arXiv · doi:10.48550/arxiv.1811.03728 · Cited by 485 · see citing works
Ganju, K., Wang, Q., Yang, W., Gunter, C. A., & Borisov, N. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243834 · Cited by 475 · see citing works
ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation
Liu, Y., Lee, W., Tao, G., Ma, S., Aafer, Y., & Zhang, X. (2019) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3319535.3363216 · Cited by 419 · see citing works
Oblivious multi-party machine learning on trusted processors
Ohrimenko, O., Schuster, F., Cédric Fournet, A., Mehta, A., Nowozin, S., Vaswani, K., & Costa, M. (2016) · MPG.PuRe (Max Planck Society) · Cited by 402 · see citing works
Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by BackdooringPreprint
Adi, Y., Baum, C., Cissé, M., Benny Pinkas, B., & Keshet, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04633 · Cited by 395 · see citing works
Latent Backdoor Attacks on Deep Neural Networks
Yao, Y., Li, H., Zheng, H.-T., & Zhao, B. Y. (2019) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3319535.3354209 · Cited by 370 · see citing works
2018 REFORM OF EU DATA PROTECTION RULES
Viorescu, R. (2017) · European Journal of Law and Public Administration · doi:10.18662/eljpa/11 · Cited by 344 · see citing works
DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks
Chen, H., Fu, C., Zhao, J., & Koushanfar, F. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/647 · Cited by 314 · see citing works
Does learning require memorization? a short tale about a long tail
Feldman, V. (2020) · Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing · doi:10.1145/3357713.3384290 · Cited by 287 · see citing works
SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems
Chou, E., Tramer, F., & Pellegrino, G. (2020) · 2020 IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw50608.2020.00025 · Cited by 281 · see citing works
Detecting AI Trojans Using Meta Neural Analysis
Xu, X., Wang, Q., Li, H., Borisov, N., Gunter, C. A., & Li, B. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00034 · Cited by 278 · see citing works
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural NetworksPreprint
Li, Y., Lyu, X., Koren, N., Lyu, L., Li, B., & Ma, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05930 · Cited by 264 · see citing works
Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
Doan, B. G., Abbasnejad, E., & Ranasinghe, D. C. (2020) · Annual Computer Security Applications Conference · doi:10.1145/3427228.3427264 · Cited by 263 · see citing works
Privacy Risks of Securing Machine Learning Models against Adversarial Examples
Song, L., Shokri, R., & Mittal, P. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3354211 · Cited by 229 · see citing works
Self-citation(2024) · TIB Data Manager · doi:10.57702/h8wosv5j · Cited by 202 · see citing works
Auditing Data Provenance in Text-Generation Models
Song, C., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1811.00513 · Cited by 199 · see citing works
When Machine Unlearning Jeopardizes Privacy
Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3484756 · Cited by 195 · see citing works
Auditing Data Provenance in Text-Generation Models
Song, C., & Shmatikov, V. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330885 · Cited by 192 · see citing works
Systematic Evaluation of Privacy Risks of Machine Learning ModelsPreprint
Song, L., & Mittal, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.10595 · Cited by 185 · see citing works
Self-citationLabel-Consistent Backdoor AttacksPreprint
Turner, A., Tsipras, D., & Mądry, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02771 · Cited by 177 · see citing works
Making AI Forget You: Data Deletion in Machine Learning
Ginart, A., Guan, M. Y., Valiant, G., & Zou, J. (2019) · arXiv · doi:10.48550/arxiv.1907.05012 · Cited by 175 · see citing works
Blind Backdoors in Deep Learning ModelsPreprint
Bagdasaryan, E., & Shmatikov, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.03823 · Cited by 175 · see citing works
Strong and efficient cache side-channel protection using hardware transactional memory
Gruss, D., Lettner, J., Schuster, F., Ohrimenko, O., Haller, I., & Costa, M. (2017) · TUGraz OPEN Library (Graz University of Technology) · Cited by 160 · see citing works
Certified Data Removal from Machine Learning Models
Guo, C., Goldstein, T., Hannun, A., & van der Maaten, L. (2019) · arXiv · doi:10.48550/arxiv.1911.03030 · Cited by 160 · see citing works
Testing Statistical Hypotheses
Ziegel, E. R., & Lehmann, E. L. (1987) · Technometrics · doi:10.2307/1269474 · Cited by 157 · see citing works
STRIP: A Defence Against Trojan Attacks on Deep Neural NetworksPreprint
Gao, Y., Xu, C., Wang, D., Chen, S., Ranasinghe, D. C., & Nepal, . (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06531 · Cited by 127 · see citing works
Data Poisoning against Differentially-Private Learners: Attacks and Defenses
Ma, Y., Zhu, X., & Hsu, J. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/657 · Cited by 126 · see citing works
Machine unlearning: linear filtration for logit-based classifiers
Baumhauer, T., Schöttle, P., & Zeppelzauer, M. (2022) · Machine Learning · doi:10.1007/s10994-022-06178-9 · Cited by 123 · see citing works
Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination DetectionPreprint
Tang, D., Wang, X., Tang, H., & Zhang, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.00686 · Cited by 121 · see citing works
Differentially Private Learning Needs Better Features (or Much More Data)Preprint
Tramèr, F., & Boneh, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.11660 · Cited by 113 · see citing works
Machine Unlearning: Linear Filtration for Logit-based Classifiers
Baumhauer, T., Schöttle, P., & Zeppelzauer, M. (2020) · arXiv · doi:10.48550/arxiv.2002.02730 · Cited by 110 · see citing works
Descent-to-Delete: Gradient-Based Methods for Machine Unlearning
Neel, S., Roth, A., & Sharifi-Malvajerdi, S. (2020) · arXiv · doi:10.48550/arxiv.2007.02923 · Cited by 96 · see citing works
Being Robust (in High Dimensions) Can Be PracticalPreprint
Diakonikolas, I., Kamath, G., Kane, D. M., Li, J. Z., Moitra, A., & Stewart, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00893 · Cited by 91 · see citing works
Formalizing Data Deletion in the Context of the Right to Be ForgottenPreprint
Goldwasser, S., & Vasudevan, P. N. (2020) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 75 · see citing works
SPECTRE: Defending Against Backdoor Attacks Using Robust StatisticsPreprint
Hayase, J., Kong, W., Somani, R., & Oh, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11315 · Cited by 74 · see citing works
Quantum Entropy Scoring for Fast Robust Mean Estimation and Improved Outlier DetectionPreprint
Dong, Y., Hopkins, S. B., & Li, J. Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.11366 · Cited by 60 · see citing works
Radioactive data: tracing through trainingPreprint
Sablayrolles, A., Douze, M., Schmid, C., & Jeǵou, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.00937 · Cited by 49 · see citing works
Poisoning the Unlabeled Dataset of Semi-Supervised LearningPreprint
Carlini, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01622 · Cited by 37 · see citing works
When Machine Unlearning Jeopardizes Privacy
Chen, M., Zhang, Z., Wang, T., Michael Backes, A., Humbert, M., & Zhang, Y. (2021) · Figshare · doi:10.60882/cispa.24613773 · Cited by 33 · see citing works
When Machine Unlearning Jeopardizes Privacy
Chen, M., Zhang, Z., Wang, T., et al. (2021) · Figshare · doi:10.60882/cispa.24613773.v1 · Cited by 30 · see citing works
Data Poisoning against Differentially-Private Learners: Attacks and DefensesPreprint
Ma, Y., Zhu, X., & Hsu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.09860 · Cited by 26 · see citing works
An Algorithmic Framework For Differentially Private Data Analysis on Trusted ProcessorsPreprint
Allen, J., Ding, B., Kulkarni, J., Nori, H., Ohrimenko, O., & Yekhanin, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00736 · Cited by 13 · see citing works
Hidden Trigger Backdoor Attacks
Saha, A., Subramanya, A., & Pirsiavash, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6871 · Cited by 13 · see citing works
Fully-Dynamic Verifiable Zero-Knowledge Order Queries for Network Data.Preprint
Ghosh, E., Goodrich, M. T., Ohrimenko, O., & Tamassia, R. (2015) · IACR Cryptology ePrint Archive · Cited by 11 · see citing works
Formalizing Data Deletion in the Context of the Right to Be Forgotten
Garg, S., Goldwasser, S., & Vasudevan, P. N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-45724-2_13 · Cited by 8 · see citing works
Double-cross attacks: Subverting active learning systems
Vicarte, J. R. S., Wang, G., & Fletcher, C. W. (2021) · USENIX Security Symposium · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- G. D. P. Regulation, "Regulation (eu) 2016/679 of the euro- pean parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and re- pealing directive 95/46," Official Journal of the European Union (OJ), vol. 59, no. 1-88, p. 294, 2016.
- Z. Xi, R. Pang, S. Ji, and T. Wang, "Graph backdoor," in USENIX Security Symposium, 2021.
- J. van den Hooff, D. Lazar, M. Zaharia, and N. Zeldovich, "Vu- vuzela: Scalable private messaging resistant to traffic analysis," in ACM Symposium on Operating Systems Principles (SOSP), 2015.
- E. Ghosh, O. Ohrimenko, and R. Tamassia, "Authenticated range & closest point queries in zero-knowledge." IACR Cryptol- ogy ePrint Archive, vol. 2015, p. 1183, 2015.
- G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, "EMNIST: Extending MNIST to handwritten letters," in 2017 International Joint Conference on Neural Networks (IJCNN), 2017.
- K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in IEEE conference on computer vision and pattern recognition (CVPR), 2016, pp. 770-778.
- S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
- F. MacWilliams and N. Sloane, The Theory of Error-Correcting Codes, ser. Mathematical Studies. Elsevier Science, 1977. [Online]. Available: https://books.google.com/books?
Cited by
Sorted by citations
Li, Y., Jiang, Y., Li, Z., & Xia, S.-T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3182979 · Cited by 682 · see citing works
A Survey of Machine Unlearning
Nguyên, T. T., Huynh, T. T., Ren, Z. X., Nguyen, P. L., Liew, A. W., Yin, H., & Nguyen, Q. V. H. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3749987 · Cited by 303 · see citing works
Fast Yet Effective Machine Unlearning
Tarun, A. K., Chundawat, V. S., Mandal, M., & Kankanhalli, M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3266233 · Cited by 203 · see citing works
Fast Yet Effective Machine Unlearning
Tarun, A. K., Chundawat, V. S., Mandal, M., & Kankanhalli, M. (2021) · arXiv · doi:10.48550/arxiv.2111.08947 · Cited by 190 · see citing works
Wan, Y., Qu, Y., Ni, W., Xiang, Y., Gao, L., & Hossain, E. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361451 · Cited by 152 · see citing works
“It is of great significance to develop a defense mechanism that is capable of re-constructing the dropped transmission message and, at the same time, monitoring over-active clients to further increase the robustness of the entire network.”
BackgroundMachine Unlearning: Solutions and Challenges
Xu, J., Wu, Z., Wang, C., & Jia, X. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3379240 · Cited by 114 · see citing works
VeriFi: Towards Verifiable Federated Unlearning
Gao, X., Ma, X., Wang, J., Sun, Y., Li, B., Ji, S., Cheng, P., & Chen, J. (2024) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2024.3382321 · Cited by 111 · see citing works
A Survey of Machine Unlearning
Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H. (2022) · arXiv · doi:10.48550/arxiv.2209.02299 · Cited by 193 · see citing works
Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy
Shaik, T., Tao, X., Xie, H., Li, L., Zhu, X., & Li, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3486109 · Cited by 78 · see citing works
Verifying in the Dark: Verifiable Machine Unlearning by Using Invisible Backdoor Triggers
Guo, Y., Zhao, Y., Hou, S., Wang, C., & Jia, X. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3328269 · Cited by 44 · see citing works
Threats, Attacks, and Defenses in Machine Unlearning: A Survey
Liu, Z., Ye, H., Chen, C., Zheng, Y., & Lam, K. (2025) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2025.3543483 · Cited by 51 · see citing works
Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects
Li, N., Zhou, C., Gao, Y., Chen, H., Zhang, Z., Kuang, B., & Fu, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3530988 · Cited by 53 · see citing works
Ren, G., Wu, J., Li, G., Li, S., & Guizani, M. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3222972 · Cited by 30 · see citing works
Learn to Unlearn: Insights Into Machine Unlearning
Qu, Y., Yuan, X., Ding, M., Ni, W., Rakotoarivelo, T., & Smith, D. B. (2024) · Computer · doi:10.1109/mc.2023.3333319 · Cited by 30 · see citing works
Proof of Unlearning: Definitions and Instantiation
Weng, J., Yao, S., Du, Y., Huang, J., Weng, J., & Wang, C. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3358993 · Cited by 30 · see citing works
An overview of machine unlearning
Li, C., Jiang, H., Chen, J., Zhao, Y. L., Fu, S., Jing, F., & Guo, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100254 · Cited by 21 · see citing works
Machine Unlearning: Its Nature, Scope, and Importance for a “Delete Culture”
Floridi, L. (2023) · Philosophy & Technology · doi:10.1007/s13347-023-00644-5 · Cited by 20 · see citing works
Qiu, H., Ma, H., Zhang, Z., Gao, Y., Zheng, Y., Fu, A., Zhou, P., Abbott, D., & Al-Sarawi, S. F. (2022) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2022.3197499 · Cited by 16 · see citing works
Vertical Federated Unlearning on the Logistic Regression Model
Deng, Z., Han, Z., Ma, C., Ding, M., Yuan, L., Ge, C., & Liu, Z. (2023) · Electronics · doi:10.3390/electronics12143182 · Cited by 15 · see citing works
Class-wise federated unlearning: Harnessing active forgetting with teacher–student memory generation
Li, Y., Zhang, J., Liu, Y., & Chen, C. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113353 · Cited by 11 · see citing works
Evaluation of Machine Unlearning Through Model Difference
Wang, W., Zhang, C., Tian, Z., Yu, S., & Su, Z. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3571666 · Cited by 10 · see citing works
Poison Neural Network-Based mmWave Beam Selection and Detoxification With Machine Unlearning
Zhang, Z. J., Tian, M., Li, C., Huang, Y., & Yang, L. (2022) · IEEE Transactions on Communications · doi:10.1109/tcomm.2022.3232794 · Cited by 9 · see citing works
TAPE: Tailored Posterior Difference for Auditing of Machine Unlearning
Wang, W., Tian, Z., Liu, A., & Yu, S. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714875 · Cited by 9 · see citing works
MultiDelete for Multimodal Machine Unlearning
Cheng, J., & Amiri, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72940-9_10 · Cited by 11 · see citing works
Speed up Federated Unlearning With Temporary Local Models
Ameen, M., Wang, P., Su, W., Wei, X., & Zhang, Q. (2025) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2025.3549112 · Cited by 8 · see citing works
How to Forget Clients in Federated Online Learning to Rank?
Wang, S., Liu, B., & Zuccon, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56063-7_7 · Cited by 7 · see citing works
Yang, X., Li, G., Zhou, K., Li, J., Lin, X., & Liu, Y. (2025) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2025.3550411 · Cited by 7 · see citing works
TruVRF: Toward Triple-Granularity Verification on Machine Unlearning
Zhou, C., Gao, Y., Fu, A., Chen, K., Zhang, Z., Xue, M., Dai, Z., Ji, S., & Zhang, Y. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3565991 · Cited by 7 · see citing works
FeaUn: Feature unlearning in vertical federated learning for IIoT against feature inference attacks
Lu, Z., Zhang, X., Li, T., Wang, Y., Li, G., & Liu, Z. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131110 · Cited by 6 · see citing works
Towards Reliable Forgetting: A Survey on Machine Unlearning Verification
Xue, L., Hu, S.-S., Lu, W., Shen, Y., Li, D., Guo, P.-J., Zhou, Z.-Q., Li, M.-H., Zhang, Y.-J., & Zhang, L. (2025) · ACM Computing Surveys · doi:10.1145/3807451 · Cited by 18 · see citing works
Evaluating Machine Unlearning: Applications, Approaches, and Accuracy
Ali, Z., Muhammad, A., Adnan, R., Alkhalifah, T., & Aslam, S. (2024) · Engineering Reports · doi:10.1002/eng2.13081 · Cited by 5 · see citing works
Verification of Incomplete Graph Unlearning through Adversarial Perturbations
Wu, K., & Wang, H. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737179 · Cited by 5 · see citing works
Really Unlearned? Verifying Machine Unlearning via Influential Sample Pairs
Xu, H., Zhu, T., Zhang, L., & Zhou, W. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3620308 · Cited by 7 · see citing works
Research on Machine Unlearning Verification Based on Predictive Cross-Entropy
Yue, X., Peng, J., Liu, M., He, H., & Zhang, Y. (2024) · IEEE Smart World Congress (SWC) · doi:10.1109/swc62898.2024.00139 · Cited by 4 · see citing works
SemanAegis: Toward Credential-Aware Semantic Communication Against Knowledge Leakage Threats
Yang, X., Lai, Y., Li, G., Wu, J., Zhou, K., & Chen, M. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3627473 · Cited by 4 · see citing works
Private Data Protection With Machine Unlearning for Next-Generation Networks
Chen, K., Wang, Y., Zhao, L., Jiang, C., Mai, H., Wu, Y., Hong, H., Shen, Y., Mo, J., Huang, L.-L., et al. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3518503 · Cited by 4 · see citing works
SMS: Self-Supervised Model Seeding for Verification of Machine Unlearning
Wang, W., Zhang, C., Tian, Z., & Yu, S. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3615615 · Cited by 3 · see citing works
Continuous Verification of Catastrophic Recalling in Machine Unlearning via Adversarial Testing
Qu, Y., Ji, Z., Cui, L., Zhang, C., Liu, L., & Tian, Z. (2024) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc63484.2024.00056 · Cited by 3 · see citing works
Exploring Federated Learning Dynamics for Black-and-White-Box DNN Traitor Tracing
Rodríguez-Lois, E., & Pérez‐González, F. (2024) · International Conference on Federated Learning Technologies and Applications (FLTA) · doi:10.1109/flta63145.2024.10840113 · Cited by 3 · see citing works
A Novel Scheme for Recommendation Unlearning Verification (RUV) Using Non-Influential Trigger Data
Dang, X., Nanda, P., Xu, H., Mohanty, M., & Deng, H. (2025) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc54725.2025.10976014 · Cited by 2 · see citing works
How secure is forgetting? Linking machine unlearning to machine learning attacks
P., M. S. K., Nicolazzo, S., Nocera, A., & Vinod, P. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131971 · Cited by 6 · see citing works
A2E: Black-Box Anti-Adversarial Example Based Watermarking to Verify Federated Unlearning
Gao, X., Wang, J., Zhang, Z., Chen, J., Cheng, P., & Chen, J. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3598987 · Cited by 1 · see citing works
From Forgetting to Future: A Survey of Machine Unlearning Approaches
Sehgal, S., Verma, A., & Bansal, H. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70082 · Cited by 1 · see citing works
Sarkar, E., Doumanidis, C., & Maniatakos, M. (2023) · IFIP/IEEE International Conference on Very Large Scale Integration · doi:10.1109/vlsi-soc57769.2023.10321928 · Cited by 1 · see citing works
Collusion-resistant Black-box Watermarking in Federated Learning through Weight Relevance Analysis
Rodríguez-Lois, E., & Pérez‐González, F. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888286 · Cited by 1 · see citing works
Efficient Machine Unlearning by Model Splitting and Core Sample Selection
Egger, M., Bitar, R., & Urbanke, R. L. (2025) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itw62417.2025.11240389 · Cited by 1 · see citing works
Class-Wise Federated Unlearning: Harnessing Active Forgetting with Teacher-Student Memory GenerationPreprint
Li, Y., Zhang, J., Liu, Y., & Chen, C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5071728
A Mini-model can Make Machine Unlearning Better
Zhao, M., Yu, W., & Zhu, C. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-1525-4_8
Song, Q., Wang, R., Li, Y., Yan, Y., Zhu, X.-Z., & Li, H. (2024) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom63139.2024.00084
Vdsa: Verification Against Data Selection Abuse in Federated LearningPreprint
Zhou, M., Yu, M., Li, J., Yang, Z., Yu, H., & Yu, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5251379
A survey on bias and fairness in machine unlearning
Partohaghighi, M., Marcia, R. F., West, B. J., & Chen, Y. (2026) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2026.03.003
Chahine, K., Nashef, G. Z. E., Karim, A. K. A., Atem, M. A., Noura, H., & Arnaout, M. (2026) · Array · doi:10.1016/j.array.2026.100987
Li, A., Han, S., Li, K., Song, J., & Zheng, Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6965787
- Authors
- Sommer, D. M., Song, L., Wagh, S., & Mittal, P.
- Affiliations
- Princeton University (US)
- Venue
- Proceedings on Privacy Enhancing Technologies
- Published
- 2022-07
- Volume
- 2022
- Issue
- 3
- Pages
- 268-290
- ISSN
- 2299-0984 (ISSN-L)
- OpenAlex
- W4286256876
- Cited by
- 51
- References
- 69
- Citations by year
- 2026: 5 · 2025: 21 · 2024: 16 · 2023: 5 · 2022: 4
- FWCI
- 3.78
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings on Privacy Enhancing Technologies (publishedVersion, cc-by-nc-nd)
- Funders
- National Science Foundation (1553437, 1704105)
- Zurich Information Security and Privacy Center
- Office of Naval Research
- Army Research Office
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Information Systems › Cloud Data Security Solutions
- Keywords
- machine unlearning, right to be forgotten, machine learning as a service, backdoor attacks, backdoor defense, General Data Protection Regulation, hypothesis testing, privacy compliance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
