The DOI matches its DataCite record; 22 of the 23 references it cites were found in a registry or an index, and 1 are not checked yet.
On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning
Open accessAbstract
Machine unlearning, i.e. having a model forget about some of its training data, has become increasingly more important as privacy legislation promotes variants of the right-to-be-forgotten. In the context of deep learning, approaches for machine unlearning are broadly categorized into two classes: exact unlearning methods, where an entity has formally removed the data point's impact on the model by retraining the model from scratch, and approximate unlearning, where an entity approximates the model parameters one would obtain by exact unlearning to save on compute costs. In this paper, we first show that the definition that underlies approximate unlearning, which seeks to prove the approximately unlearned model is close to an exactly retrained model, is incorrect because one can obtain the same model using different datasets. Thus one could unlearn without modifying the model at all. We then turn to exact unlearning approaches and ask how to verify their claims of unlearning. Our results show that even for a given training trajectory one cannot formally prove the absence of certain data points used during training. We thus conclude that unlearning is only well-defined at the algorithmic level, where an entity's only possible auditable claim to unlearning is that they used a particular algorithm designed to allow for external scrutiny during an audit.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Those one-sided methods seldom provide strong verification of the unlearning process's effectiveness.
“Those one-sided methods seldom provide strong verification of the unlearning process's effectiveness [44, 109, 110].”
From Machine Unlearning: A Survey (2024) · cites this paper as [110]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Calibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514
Algorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687
Towards Making Systems Forget with Machine Unlearning
Cao, Y., & Yang, J. (2015) · 2015 IEEE Symposium on Security and Privacy · doi:10.1109/sp.2015.35 · Cited by 622 · cited 3×
“Because ML models are known to memorize their training data [8], this forces model owners to resort to machine unlearning [4] to remove any effect the point had on the trained model.”
Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks
Golatkar, A., Achille, A., & Soatto, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00932 · Cited by 394
Does learning require memorization? a short tale about a long tail
Feldman, V. (2020) · Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing · doi:10.1145/3357713.3384290 · Cited by 287
Mantelero, A. (2013) · Computer Law & Security Review · doi:10.1016/j.clsr.2013.03.010 · Cited by 264
Making AI Forget You: Data Deletion in Machine Learning
Ginart, A., Guan, M. Y., Valiant, G., & Zou, J. (2019) · arXiv · doi:10.48550/arxiv.1907.05012 · Cited by 175
Certified Data Removal from Machine Learning Models
Guo, C., Goldstein, T., Hannun, A., & van der Maaten, L. (2019) · arXiv · doi:10.48550/arxiv.1911.03030 · Cited by 160
Machine unlearning: linear filtration for logit-based classifiers
Baumhauer, T., Schöttle, P., & Zeppelzauer, M. (2022) · Machine Learning · doi:10.1007/s10994-022-06178-9 · Cited by 141
Unrolling SGD: Understanding Factors Influencing Machine Unlearning
Thudi, A., Deza, G., Chandrasekaran, V., & Papernot, N. (2022) · 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp53844.2022.00027 · Cited by 135
Self-citationGolatkar, A., Achille, A., & Soatto, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58526-6_23 · Cited by 122
Remember What You Want to Forget: Algorithms for Machine Unlearning
Sekhari, A., Acharya, J., Kamath, G., & Suresh, A. T. (2021) · arXiv · doi:10.48550/arxiv.2103.03279 · Cited by 99
DeltaGrad: Rapid retraining of machine learning modelsPreprint
Wu, Y., Dobriban, E., & Davidson, S. B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.14755 · Cited by 93
SoK: Machine Learning GovernancePreprint
Chandrasekaran, V., Jia, H., Thudi, A., Travers, A., Yaghini, M., & Papernot, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.10870 · Cited by 33
Self-citationMachine Unlearning via Algorithmic Stability
Ullah, E., Mai, T., Rao, A., Rossi, R., & Arora, R. (2021) · arXiv · doi:10.48550/arxiv.2102.13179 · Cited by 26
Proof-of-Learning: Definitions and PracticePreprint
Jia, H., Yaghini, M., Choquette-Choo, C. A., Dullerud, N., Thudi, A., Chandrasekaran, V., & Papernot, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.05633 · Cited by 5
Self-citationGraves, L., Nagisetty, V., & Ganesh, V. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17371 · Cited by 5
Flagged first, then by citations
Research works printed but not linked: 3
- [2][2] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. arXiv preprint arXiv:1912.03817 , 2019. - · arXiv
- [15][15] Pang Wei Koh and Percy Liang. Understanding blackbox predictions via influence functions. In International Conference on Machine Learning , pages 1885-1894. PMLR, 2017. -
- [22][22] Ilia Shumailov, Zakhar Shumaylov, Dmitry Kazhdan, Yiren Zhao, Nicolas Papernot, Murat A. Erdogdu, and Ross Anderson. Manipulating sgd with data ordering attacks. Proceedings of the 35th Conference on Neural Information Processing Systems , 2021. -
Cited by
Sorted by citations
The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining
Liu, Y., Xu, L., Yuan, X., Wang, C., & Li, B. (2022) · IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · doi:10.1109/infocom48880.2022.9796721 · Cited by 202
Fast Yet Effective Machine Unlearning
Tarun, A. K., Chundawat, V. S., Mandal, M., & Kankanhalli, M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3266233 · Cited by 203
Fast Yet Effective Machine Unlearning
Tarun, A. K., Chundawat, V. S., Mandal, M., & Kankanhalli, M. (2021) · arXiv · doi:10.48550/arxiv.2111.08947 · Cited by 190
Xu, H., Zhu, T., Zhang, L., Zhou, W., & Yu, P. S. (2024) · ACM Computing Surveys · doi:10.1145/3603620 · Cited by 164
“Those one-sided methods seldom provide strong verification of the unlearning process’s effectiveness [44, 109, 110].”
BackgroundAuditing of AI: Legal, Ethical and Technical Approaches
Mökander, J. (2023) · Digital Society · doi:10.1007/s44206-023-00074-y · Cited by 152
“However, previous research suggests that technology providers too have strong incentives to subject the AI systems they design and deploy to independent audits (Falco et al., 2021; Raji et al., 2020).”
Machine Unlearning: Solutions and Challenges
Xu, J., Wu, Z., Wang, C., & Jia, X. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3379240 · Cited by 139
ARCANE: An Efficient Architecture for Exact Machine Unlearning
Yan, H., Li, X., Guo, Z., Li, H., Li, F., & Lin, X. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/556 · Cited by 97
A Review on Machine Unlearning
Zhang, H., Nakamura, T., Isohara, T., & Sakurai, K. (2023) · SN Computer Science · doi:10.1007/s42979-023-01767-4 · Cited by 81
Federated Unlearning With Momentum Degradation
Zhao, Y., Wang, P., Qi, H., Huang, J., Wei, Z., & Zhang, Q. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3321594 · Cited by 74
A Survey of Machine Unlearning
Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H. (2022) · arXiv · doi:10.48550/arxiv.2209.02299 · Cited by 193
“Existing works define unlearning as obtaining a newmodel with an accuracy as good as if the model had retrainedwithout the data to be forgotten [55, 56, 58, 149].”
InfluentialExploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy
Shaik, T., Tao, X., Xie, H., Li, L., Zhu, X., & Li, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3486109 · Cited by 106
Learn to Forget: Machine Unlearning via Neuron Masking
Ma, Z., Liu, Y., Liu, X., Liu, J., Ma, J., & Ren, K. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3194884 · Cited by 90
Verifying in the Dark: Verifiable Machine Unlearning by Using Invisible Backdoor Triggers
Guo, Y., Zhao, Y., Hou, S., Wang, C., & Jia, X. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3328269 · Cited by 47
Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning
Hu, H., Wang, S., Dong, T., & Xue, M. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00248 · Cited by 60
ERM-KTP: Knowledge-Level Machine Unlearning via Knowledge Transfer
Lin, S., Zhang, X., Chen, C., Chen, X., & Susilo, W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01929 · Cited by 41
Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects
Li, N., Zhou, C., Gao, Y., Chen, H., Zhang, Z., Kuang, B., & Fu, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3530988 · Cited by 75
No Matter How You Slice It: Machine Unlearning with SISA Comes at the Expense of Minority Classes
Koch, K., & Soll, M. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00047 · Cited by 34
Backdoor Attacks via Machine Unlearning
Liu, Z., Wang, T., Huai, M., & Miao, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i13.29321 · Cited by 34
Feature-Based Machine Unlearning for Vertical Federated Learning in IoT Networks
Pan, Z., Ying, Z., Wang, Y., Zhang, C., Zhang, W., Zhou, W., & Zhu, L. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3530529 · Cited by 35
Proof of Unlearning: Definitions and Instantiation
Weng, J., Yao, S., Du, Y., Huang, J., Weng, J., & Wang, C. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3358993 · Cited by 33
Forget-SVGD: Particle-Based Bayesian Federated Unlearning
Gong, J., Kang, J., Simeone, O., & Kassab, R. (2022) · IEEE Data Science Workshop (DSW) · doi:10.1109/dslw53931.2022.9820602 · Cited by 21
Machine Unlearning: Its Nature, Scope, and Importance for a “Delete Culture”
Floridi, L. (2023) · Philosophy & Technology · doi:10.1007/s13347-023-00644-5 · Cited by 20
Proof-of-Learning is Currently More Broken Than You Think
Fang, C., Jia, H., Thudi, A., Yaghini, M., Choquette-Choo, C. A., Dullerud, N., Chandrasekaran, V., & Papernot, N. (2023) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp57164.2023.00052 · Cited by 20
Self-citationMUter: Machine Unlearning on Adversarially Trained Models
Liu, J., Xue, M., Lou, J., Zhang, X., Xiong, L., & Qin, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00451 · Cited by 16
Verifiable and Provably Secure Machine Unlearning
Eisenhofer, T., Riepel, D., Chandrasekaran, V., Ghosh, E., Ohrimenko, O., & Papernot, N. (2025) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00033 · Cited by 16
Self-citationImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks
Clifford, E. R., Shumailov, I., Zhao, Y., Anderson, R., & Mullins, R. (2024) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00024 · Cited by 13
Self-citationInexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy
Hayes, J., Shumailov, I., Triantafillou, E., Khalifa, A. M., & Papernot, N. (2025) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00034 · Cited by 75
Self-citationGoldfish: An Efficient Federated Unlearning Framework
Wang, H., Zhu, X., Chen, C., & Verı́ssimo, P. (2024) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn58291.2024.00035 · Cited by 16
Vertical Federated Unlearning via Backdoor Certification
Han, M., Zhu, T., Zhang, L., Huo, H., & Zhou, W. (2025) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2025.3536312 · Cited by 12
Evaluation of Machine Unlearning Through Model Difference
Wang, W., Zhang, C., Tian, Z., Yu, S., & Su, Z. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3571666 · Cited by 10
Control, Confidentiality, and the Right to be Forgotten
Cohen, A., Smith, A., Swanberg, M., & Vasudevan, P. N. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3616585 · Cited by 8
Private Data Protection with Machine Unlearning in Contrastive Learning Networks
Chen, K., Wang, Z., & Mi, B. (2024) · Mathematics · doi:10.3390/math12244001 · Cited by 7
TruVRF: Toward Triple-Granularity Verification on Machine Unlearning
Zhou, C., Gao, Y., Fu, A., Chen, K., Zhang, Z., Xue, M., Dai, Z., Ji, S., & Zhang, Y. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3565991 · Cited by 7
Machine Unlearning Through Fine-Grained Model Parameters Perturbation
Zuo, Z., Tang, Z., Li, K., & Datta, A. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3528551 · Cited by 7
Efficient federated unlearning under plausible deniability
Varshney, A. K., & Torra, V. (2025) · Machine Learning · doi:10.1007/s10994-024-06685-x · Cited by 9
ConFUSE: Confusion-based Federated Unlearning with Salience Exploration
Meerza, S. I. A., Sadovnik, A., & Liu, J. (2024) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi61997.2024.00083 · Cited by 7
A Decision-Making Process to Implement the ‘Right to Be Forgotten’ in Machine Learning
Hawkins, K., Alhuwaish, N., Belguith, S., Vranaki, A., & Charlesworth, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61089-9_2 · Cited by 5
Forgeability and Membership Inference Attacks
Kong, Z., Chowdhury, A. R., & Chaudhuri, K. (2022) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3560830.3563731 · Cited by 4
Integrally Private Model Selection for Deep Neural Networks
Varshney, A. K., & Torra, V. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-39821-6_33 · Cited by 4
Pseudo-Labeling for Enhanced User Privacy in Approximate Machine Unlearning
Watanabe, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890795 · Cited by 3
An Efficient Two-Stage Machine Unlearning Framework for Poisoned Specific Emitter Identification
Li, X., Liu, C., Sun, S., Zhou, Z., Zhang, Y., & Li, G. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3583362 · Cited by 3
Ensuring User Privacy and Model Security via Machine Unlearning: A Review
Tang, Y., Cai, Z., Liu, Q., Zhou, T., & Ni, Q. (2023) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.032307 · Cited by 3
MUSO: achieving exact machine unlearning in over-parameterized regimes
Yang, R., He, M., He, Z., Qiu, Y., & Huang, X. (2025) · Machine Learning · doi:10.1007/s10994-025-06806-0 · Cited by 5
On the Reliability of Membership Inference Attacks
Chowdhury, A. R., Kong, Z., & Chaudhuri, K. (2025) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00036 · Cited by 1
Efficient unlearning for data security in deep learning systems
Guo, E., Su, C., & Li, P. (2025) · The Computer Journal · doi:10.1093/comjnl/bxaf031 · Cited by 1
k-IPfedAvg: k-Anonymous Integrally Private Federated Averaging with Convergence GuaranteePreprint
Varshney, A. K., & Torra, V. (2023) · doi:10.36227/techrxiv.170327604.45388443/v1 · Cited by 1
Li, F., Li, H., & Niu, B. (2024) · Springer eBooks · doi:10.1007/978-981-99-4943-4_4
Learning to Unlearn in Federated Learning
Wang, Y., Taghia, J., İckin, S., Vandikas, K., & Ebrahimi, M. (2024) · International Conference on Federated Learning Technologies and Applications (FLTA) · doi:10.1109/flta63145.2024.10840121
Song, Q., Wang, R., Li, Y., Yan, Y., Zhu, X.-Z., & Li, H. (2024) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom63139.2024.00084
Song, Q., Wang, R., Li, Y., Yan, Y., Zhu, X., & Li, H. (2025) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2024.11.001
Metadata
- Authors
- Thudi, A., Jia, H., Shumailov, I., & Papernot, N.
- Affiliations
- University of Toronto (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2110.11891
- OpenAlex
- W3208439705
- Cited by
- 48
- References
- 22
- Citations by year
- 2025: 15 · 2024: 13 · 2023: 15 · 2022: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- Government of Canada
- Defense Advanced Research Projects Agency
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
- Keywords
- machine unlearning, audibility, right to be forgotten, data privacy, approximate unlearning, model retraining, deep learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
