The DOI matches its DataCite record; 83 of the 85 references it cites were found in a registry or an index.
Making AI Forget You: Data Deletion in Machine Learning
Open accessAbstract
Intense recent discussions have focused on how to provide individuals with control over when their data can and cannot be used --- the EU's Right To Be Forgotten regulation is an example of this effort. In this paper we initiate a framework studying what to do when it is no longer permissible to deploy models derivative from specific user data. In particular, we formulate the problem of efficiently deleting individual data points from trained machine learning models. For many standard ML models, the only way to completely remove an individual's data is to retrain the whole model from scratch on the remaining data, which is often not computationally practical. We investigate algorithmic principles that enable efficient data deletion in ML. For the specific setting of k-means clustering, we propose two provably efficient deletion algorithms which achieve an average of over 100X improvement in deletion efficiency across 6 datasets, while producing clusters of comparable statistical quality to a canonical k-means++ baseline.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, it could be used to accelerate the process of leave-one-out-crossvalidation, removing adversarial or poisoning samples, and identifying significant and valuable data samples within a model [13].”
From Machine Unlearning: A Survey (2024) · cites this paper as [13]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 37,139
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Silhouettes: A graphical aid to the interpretation and validation of cluster analysis
Rousseeuw, P. J. (1987) · Journal of Computational and Applied Mathematics · doi:10.1016/0377-0427(87)90125-7 · Cited by 21,392
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
Least squares quantization in PCM
Lloyd, S. (1982) · IEEE Transactions on Information Theory · doi:10.1109/tit.1982.1056489 · Cited by 15,836
Sudlow, C., Gallacher, J., Allen, N., Beral, V., Burton, P., Danesh, J., Downey, P., Elliott, P., Green, J., Landray, M., et al. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001779 · Cited by 12,728
Least Squares Support Vector Machine Classifiers
Suykens, J., & Vandewalle, J. (1999) · Neural Processing Letters · doi:10.1023/a:1018628609742 · Cited by 9,481
Research works printed but not linked: 9
- [38][38] R. M. Gray and D. L. Neuhoff. Quantization. IEEE transactions on information theory , 44(6):2325-2383, 1998. -
- [39][39] S. Guha, R. Rastogi, and K. Shim. Cure: an efficient clustering algorithm for large databases. In ACMSigmod Record , pages 73-84. ACM, 1998. -
- [47][47] M. Kearns and D. Ron. Algorithmic stability and sanity-check bounds for leave-one-out crossvalidation. Neural computation , 11(6):1427-1453, 1999. -
- [55][55] J. H. Maindonald. Statistical Computation . John Wiley & Sons, Inc., New York, NY, USA, 1984. -
- [64][64] N. Papernot, M. Abadi, U. Erlingsson, I. Goodfellow, and K. Talwar. Semi-supervised knowledge transfer for deep learning from private training data. arXiv preprint arXiv:1610.05755 , 2016. - · arXiv
- [66][66] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research , 12:2825-2830, 2011. -
- [74][74] S. Schelter. 'amnesia'-towards machine learning models that can forget user data very fast. In 1st International Workshop on Applied AI for Database Systems and Applications (AIDB'19) , 2019. -
- [88][88] G. I. Webb. Lazy Learning , pages 571-572. Springer US, 2010. -
- [91][91] J. Zhang, A. May, T. Dao, and C. Ré. Low-precision random fourier features for memoryconstrained kernel approximation. arXiv preprint arXiv:1811.00155 , 2018. - · arXiv
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407
Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00019 · Cited by 810
“Decremental Learning: Ginart et al. [37] consider the problem from a data-protection regulation standpoint.”
BackgroundPrivacy preservation in federated learning: An insightful survey from the GDPR perspective
Truong, N., Sun, K., Wang, S., Guitton, F., & Guo, Y. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102402 · Cited by 399
Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks
Golatkar, A., Achille, A., & Soatto, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00932 · Cited by 394
Scope of machine learning in materials research—A review
Mobarak, M. H., Mimona, M. A., Islam, M. A., Hossain, N., Zohura, F. T., Imtiaz, I., & Rimon, M. I. H. (2023) · Applied Surface Science Advances · doi:10.1016/j.apsadv.2023.100523 · Cited by 253
FedEraser: Enabling Efficient Client-Level Data Removal from Federated Learning Models
Liu, G., Ma, X., Yang, Y., Wang, C., & Liu, J. (2021) · 2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS) · doi:10.1109/iwqos52092.2021.9521274 · Cited by 248
The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining
Liu, Y., Xu, L., Yuan, X., Wang, C., & Li, B. (2022) · IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · doi:10.1109/infocom48880.2022.9796721 · Cited by 202
Fast Yet Effective Machine Unlearning
Tarun, A. K., Chundawat, V. S., Mandal, M., & Kankanhalli, M. (2021) · arXiv · doi:10.48550/arxiv.2111.08947 · Cited by 190
Fast Yet Effective Machine Unlearning
Tarun, A. K., Chundawat, V. S., Mandal, M., & Kankanhalli, M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3266233 · Cited by 203
When Machine Unlearning Jeopardizes Privacy
Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3484756 · Cited by 195
Chundawat, V. S., Tarun, A. K., Mandal, M., & Kankanhalli, M. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3265506 · Cited by 162
Adversarial machine learning :
Vassilev, A., Oprea, A., Fordyce, A. J., & Anderson, H. S. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 230
Certified Data Removal from Machine Learning Models
Guo, C., Goldstein, T., Hannun, A., & van der Maaten, L. (2019) · arXiv · doi:10.48550/arxiv.1911.03030 · Cited by 160
Xu, H., Zhu, T., Zhang, L., Zhou, W., & Yu, P. S. (2024) · ACM Computing Surveys · doi:10.1145/3603620 · Cited by 164
Privacy-Preserving Techniques in Generative AI and Large Language Models: A Narrative Review
Feretzakis, G., Papaspyridis, K., Gkoulalas-Divanis, A., & Verykios, V. S. (2024) · Information · doi:10.3390/info15110697 · Cited by 186
Ablating Concepts in Text-to-Image Diffusion Models
Kumari, N., Zhang, B., Wang, S.-Y., Shechtman, E., Zhang, R., & Zhu, J.-Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02074 · Cited by 207
Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks Using an Incompetent Teacher
Chundawat, V. S., Tarun, A. K., Mandal, M., & Kankanhalli, M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i6.25879 · Cited by 129
Mixed-Privacy Forgetting in Deep Networks
Golatkar, A., Achille, A., Ravichandran, A., Polito, M., & Soatto, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00085 · Cited by 125
Machine Unlearning of Features and Labels
Warnecke, A., Pirch, L., Wressnegger, C., & Rieck, K. (2023) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2023.23087 · Cited by 278
FedRecovery: Differentially Private Machine Unlearning for Federated Learning Frameworks
Zhang, L., Zhu, T., Zhang, H., Xiong, P., & Zhou, W. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3297905 · Cited by 118
Algorithmic fairness datasets: the story so far
Fabris, A., Messina, S., Silvello, G., & Susto, G. A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00854-z · Cited by 161
Machine Unlearning: Solutions and Challenges
Xu, J., Wu, Z., Wang, C., & Jia, X. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3379240 · Cited by 139
Machine unlearning: linear filtration for logit-based classifiers
Baumhauer, T., Schöttle, P., & Zeppelzauer, M. (2022) · Machine Learning · doi:10.1007/s10994-022-06178-9 · Cited by 141
Chen, C., Sun, F., Zhang, M., & Ding, B. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3511997 · Cited by 110
VeriFi: Towards Verifiable Federated Unlearning
Gao, X., Ma, X., Wang, J., Sun, Y., Li, B., Ji, S., Cheng, P., & Chen, J. (2024) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2024.3382321 · Cited by 125
ARCANE: An Efficient Architecture for Exact Machine Unlearning
Yan, H., Li, X., Guo, Z., Li, H., Li, F., & Lin, X. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/556 · Cited by 97
Operationalizing the Legal Principle of Data Minimization for Personalization
Biega, A. J., Potash, P., Daumé, H., Díaz, F., & Finck, M. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401034 · Cited by 117
Boundary Unlearning: Rapid Forgetting of Deep Networks via Shifting the Decision Boundary
Chen, M., Gao, W., Liu, G., Peng, K., & Wang, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00750 · Cited by 95
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
Wang, Z., Yang, E., Shen, L., & Huang, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3498346 · Cited by 132
Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Jang, J., Yoon, D., Yang, S., Cha, S., Lee, M., Logeswaran, L., & Seo, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.805 · Cited by 349
Fast Machine Unlearning without Retraining through Selective Synaptic Dampening
Foster, J. D., Schoepf, S., & Brintrup, A. M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i11.29092 · Cited by 91
A Review on Machine Unlearning
Zhang, H., Nakamura, T., Isohara, T., & Sakurai, K. (2023) · SN Computer Science · doi:10.1007/s42979-023-01767-4 · Cited by 81
Formalizing Data Deletion in the Context of the Right to Be ForgottenPreprint
Goldwasser, S., & Vasudevan, P. N. (2020) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 78
Federated Unlearning: Guarantee the Right of Clients to Forget
Wu, L., Guo, S., Wang, J., Hong, Z., Zhang, J., & Ding, Y. (2022) · IEEE Network · doi:10.1109/mnet.001.2200198 · Cited by 73
Asynchronous Federated Unlearning
Su, N., & Li, B. (2023) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom53939.2023.10229075 · Cited by 75
Survey: Leakage and Privacy at Inference Time
Jegorova, M., Kaul, C., Mayor, C., O'Neil, A. Q., Weir, A., Murray-Smith, R., & Tsaftaris, S. A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3229593 · Cited by 77
Federated Unlearning With Momentum Degradation
Zhao, Y., Wang, P., Qi, H., Huang, J., Wei, Z., & Zhang, Q. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3321594 · Cited by 74
Poisoning attacks and countermeasures in intelligent networks: Status quo and prospects
Wang, C., Chen, J., Yang, Y., Ma, X., & Liu, J. (2021) · Digital Communications and Networks · doi:10.1016/j.dcan.2021.07.009 · Cited by 68
Hard to Forget: Poisoning Attacks on Certified Machine Unlearning
Marchant, N. G., Rubinstein, B. I. P., & Alfeld, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20736 · Cited by 64
“Ginart et al. (2019) were first to introduce an approximate definition of unlearning—requiring that it be statistically indistinguishable from retraining.”
InfluentialA Survey of Machine Unlearning
Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H. (2022) · arXiv · doi:10.48550/arxiv.2209.02299 · Cited by 193
“Ideas similar to knowledge-adaptation priors were also investigated in [52, 171].”
InfluentialDeep Unlearning via Randomized Conditionally Independent Hessians
Mehta, R., Pal, S., Singh, V. P., & Ravi, S. N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01017 · Cited by 62
PUMA: Performance Unchanged Model Augmentation for Training Data Removal
Wu, G., Hashemi, M., & Srinivasa, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20846 · Cited by 61
Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy
Shaik, T., Tao, X., Xie, H., Li, L., Zhu, X., & Li, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3486109 · Cited by 113
Learn to Forget: Machine Unlearning via Neuron Masking
Ma, Z., Liu, Y., Liu, X., Liu, J., Ma, J., & Ren, K. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3194884 · Cited by 90
GIF: A General Graph Unlearning Strategy via Influence Function
Wu, J., Yang, Y., Qian, Y., Sui, Y., Wang, X., & He, X. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583521 · Cited by 69
Athena: Probabilistic Verification of Machine Unlearning
Sommer, D. M., Song, L., Wagh, S., & Mittal, P. (2022) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2022-0072 · Cited by 51
On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning
Thudi, A., Jia, H., Shumailov, I., & Papernot, N. (2021) · arXiv · doi:10.48550/arxiv.2110.11891 · Cited by 48
Learning with Selective Forgetting
Shibata, T., Irie, G., Ikami, D., & Mitsuzumi, Y. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/137 · Cited by 47
Learning to Unlearn: Instance-Wise Unlearning for Pre-trained Classifiers
Cha, S., Cho, S., Hwang, D., Lee, H., Moon, T., & Lee, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.28996 · Cited by 59
Verifying in the Dark: Verifiable Machine Unlearning by Using Invisible Backdoor Triggers
Guo, Y., Zhao, Y., Hou, S., Wang, C., & Jia, X. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3328269 · Cited by 47
Making recommender systems forget: Learning and unlearning for erasable recommendation
Li, Y., Chen, C., Zheng, X., Liu, J., & Wang, J. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111124 · Cited by 55
Federated Unlearning and Its Privacy Threats
Fei, W., Li, B., & Li, B. (2023) · IEEE Network · doi:10.1109/mnet.004.2300056 · Cited by 40
Machine Unlearning via Representation Forgetting With Parameter Self-Sharing
Wang, W., Zhang, C., Tian, Z., & Yu, S. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3331239 · Cited by 36
No Matter How You Slice It: Machine Unlearning with SISA Comes at the Expense of Minority Classes
Koch, K., & Soll, M. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00047 · Cited by 34
Selective and collaborative influence function for efficient recommendation unlearning
Li, Y., Chen, C., Zheng, X., Zhang, Y., Gong, B., Wang, J., & Chen, L. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121025 · Cited by 36
Exact-Fun: An Exact and Efficient Federated Unlearning Approach
Xiong, Z., Li, W., Li, Y., & Cai, Z. (2023) · Proceedings · doi:10.1109/icdm58522.2023.00188 · Cited by 33
Aldaghri, N., Mahdavifar, H., & Beirami, A. (2021) · IEEE Access · doi:10.1109/access.2021.3090019 · Cited by 39
Markov Chain Monte Carlo-Based Machine Unlearning
Nguyen, Q. P., Oikawa, R., Divakaran, D. M., Chan, M. C., & Low, B. K. H. (2022) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3488932.3517406 · Cited by 32
Learn to Unlearn: Insights Into Machine Unlearning
Qu, Y., Yuan, X., Ding, M., Ni, W., Rakotoarivelo, T., & Smith, D. B. (2024) · Computer · doi:10.1109/mc.2023.3333319 · Cited by 30
Proof of Unlearning: Definitions and Instantiation
Weng, J., Yao, S., Du, Y., Huang, J., Weng, J., & Wang, C. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3358993 · Cited by 33
Data-related concepts for artificial intelligence education in K-12
Olari, V., & Romeike, R. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100196 · Cited by 33
KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment
Wang, L., Chen, T., Yuan, W., Zeng, X., Wong, K.-F., & Yin, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.740 · Cited by 25
Unlearning Backdoor Attacks in Federated Learning
Wu, C., Zhu, S., Mitra, P., & Wang, W. (2024) · IEEE Conference on Communications and Network Security · doi:10.1109/cns62487.2024.10735680 · Cited by 29
Deletion inference, reconstruction, and compliance in machine (un)learning
Gao, J., Garg, S., Mahmoody, M., & Vasudevan, P. N. (2022) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2022-0079 · Cited by 27
Zhou, J., Li, H., Liao, X., Zhang, B., He, W., Li, Z., Zhou, L., & Gao, X. (2023) · Nature Communications · doi:10.1038/s41467-023-41703-x · Cited by 25
An overview of machine unlearning
Li, C., Jiang, H., Chen, J., Zhao, Y. L., Fu, S., Jing, F., & Guo, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100254 · Cited by 21
Forget-SVGD: Particle-Based Bayesian Federated Unlearning
Gong, J., Kang, J., Simeone, O., & Kassab, R. (2022) · IEEE Data Science Workshop (DSW) · doi:10.1109/dslw53931.2022.9820602 · Cited by 21
Fast Model Update for IoT Traffic Anomaly Detection With Machine Unlearning
Fan, J., Wu, K., Zhou, Y., Zhao, Z., & Huang, S. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3214840 · Cited by 21
Toward Efficient and Robust Federated Unlearning in IoT Networks
Yuan, Y., Wang, B., Zhang, C., Xiong, Z., Li, C., & Zhu, L. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3378329 · Cited by 20
Certified Unlearning for Federated Recommendation
Huynh, T. T., Nguyen, T. B., Nguyen, T. T., Nguyen, P. L., Yin, H., Nguyen, Q. V. H., & Nguyên, T. T. (2024) · ACM Transactions on Information Systems · doi:10.1145/3706419 · Cited by 19
Continual Forgetting for Pre-Trained Vision Models
Zhao, H., Ni, B., Fan, J., Wang, Y., Chen, Y., Meng, G., & Zhang, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02705 · Cited by 24
DeltaBoost: Gradient Boosting Decision Trees with Efficient Machine Unlearning
Wu, Z., Zhu, J., Li, Q., & He, B. (2023) · Proceedings of the ACM on Management of Data · doi:10.1145/3589313 · Cited by 20
Generative Unlearning for Any Identity
Seo, J., Lee, S.-H., Lee, T., Moon, S., & Park, G.-M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00874 · Cited by 17
A Dataset Auditing Method for Collaboratively Trained Machine Learning Models
Huang, Y., Huang, C.-Y., Li, X., & Li, K. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3220706 · Cited by 16
Compressed Particle-Based Federated Bayesian Learning and Unlearning
Gong, J., Simeone, O., & Kang, J. (2022) · IEEE Communications Letters · doi:10.1109/lcomm.2022.3223655 · Cited by 16
MUter: Machine Unlearning on Adversarially Trained Models
Liu, J., Xue, M., Lou, J., Zhang, X., Xiong, L., & Qin, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00451 · Cited by 16
A survey on machine unlearning: Techniques and new emerged privacy risks
Liu, H., Xiong, P., Zhu, T., & Yu, P. S. (2025) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2025.104010 · Cited by 32
Vertical Federated Unlearning on the Logistic Regression Model
Deng, Z., Han, Z., Ma, C., Ding, M., Yuan, L., Ge, C., & Liu, Z. (2023) · Electronics · doi:10.3390/electronics12143182 · Cited by 15
Machine Unlearning in Gradient Boosting Decision Trees
Lin, H., Chung, J. W., Lao, Y., & Zhao, W. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599420 · Cited by 15
Machine Un-learning: An Overview of Techniques, Applications, and Future Directions
Sai, S., Mittal, U., Chamola, V., Huang, K., Spinelli, I., Scardapane, S., Tan, Z., & Hussain, A. (2023) · Cognitive Computation · doi:10.1007/s12559-023-10219-3 · Cited by 18
SAFE: Machine Unlearning With Shard Graphs
Dukler, Y., Bowman, B., Achille, A., Golatkar, A., Swaminathan, A. D., & Soatto, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01569 · Cited by 15
Texture transformer super-resolution for low-dose computed tomography
Zhou, S., Yu, L., & Jin, M. (2022) · Biomedical Physics & Engineering Express · doi:10.1088/2057-1976/ac9da7 · Cited by 15
Approximate Data Deletion from Machine Learning Models: Algorithms and Evaluations.Preprint
Izzo, Z., Smart, M. A., Chaudhuri, K., & Zou, J. (2020) · arXiv (Cornell University) · Cited by 15
Self-citationMulticlass Unlearning for Image Classification via Weight Filtering
Poppi, S., Sarto, S., Cornia, M., Baraldi, L., & Cucchiara, R. (2024) · IEEE Intelligent Systems · doi:10.1109/mis.2024.3412742 · Cited by 14
"Amnesia" - Machine Learning Models That Can Forget User Data Very Fast.
Schelter, S. (2020) · Conference on Innovative Data Systems Research · Cited by 14
Knowledge-Adaptation PriorsPreprint
Khan, M. E., & Swaroop, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.08769 · Cited by 14
Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy
Hayes, J., Shumailov, I., Triantafillou, E., Khalifa, A. M., & Papernot, N. (2025) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00034 · Cited by 75
Feature Unlearning for Pre-trained GANs and VAEs
Moon, S., Cho, S., & Kim, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30138 · Cited by 27
Goldfish: An Efficient Federated Unlearning Framework
Wang, H., Zhu, X., Chen, C., & Verı́ssimo, P. (2024) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn58291.2024.00035 · Cited by 16
Dang, Y., Liu, Y., Yang, E., Guo, G., Jiang, L., Zhao, J., & Wang, X. (2024) · ACM Transactions on Information Systems · doi:10.1145/3706633 · Cited by 12
Vertical Federated Unlearning via Backdoor Certification
Han, M., Zhu, T., Zhang, L., Huo, H., & Zhou, W. (2025) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2025.3536312 · Cited by 12
FedWiper: Federated Unlearning via Universal Adapter
Zhao, S., Zhang, J., Ma, X., Jiang, Q. G., Ma, Z., Gao, S., Ying, Z., & Ma, J. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3557671 · Cited by 14
SoK: Managing Longitudinal Privacy of Publicly Shared Personal Online Data
Schnitzler, T., Mirza, S., Dürmuth, M., & Pöpper, C. (2020) · Proceedings on Privacy Enhancing Technologies · doi:10.2478/popets-2021-0013 · Cited by 12
Hopkins, A., & Booth, S. (2021) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 12
Machine Learning Techniques for Accountability
Kim, B., & Doshi‐Velez, F. (2021) · AI Magazine · doi:10.1002/j.2371-9621.2021.tb00010.x · Cited by 18
Patient Similarity Learning with Selective Forgetting
Qian, W., Zhao, C., Shao, H., Chen, M., Wang, F., & Huai, M. (2022) · 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995016 · Cited by 11
Class-wise federated unlearning: Harnessing active forgetting with teacher–student memory generation
Li, Y., Zhang, J., Liu, Y., & Chen, C. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113353 · Cited by 12
QuickDrop: Efficient Federated Unlearning via Synthetic Data Generation
Dhasade, A., Ding, Y., Guo, S., Kermarrec, A.-M., de Vos, M., & Wu, L. (2024) · International Middleware Conference · doi:10.1145/3652892.3700764 · Cited by 10
Addressing unreliable local models in federated learning through unlearning
Ameen, M., Khan, R. U., Wang, P., Batool, S., & Alajmi, M. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106688 · Cited by 11
Metadata
- Authors
- Ginart, A., Guan, M. Y., Valiant, G., & Zou, J.
- Affiliations
- Stanford University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- Volume
- 32
- Pages
- 3513-3526
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1907.05012
- OpenAlex
- W2961073278
- Cited by
- 175
- References
- 85
- Citations by year
- 2026: 2 · 2025: 24 · 2024: 49 · 2023: 38 · 2022: 25 · 2021: 26 · 2020: 9 · 2019: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1704417, 1763191, 1813049)
- National Institutes of Health (R21 MD012867-01)
- National Institute on Aging (P30AG059307)
- Office of Naval Research (N00014-18, N00014-18-1-2295, N00014)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- data deletion, machine learning, K-means clustering, right to be forgotten, privacy, retraining costs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
