The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Unrolling SGD: Understanding Factors Influencing Machine Unlearning
Open accessAbstract
Machine unlearning is the process through which a deployed machine learning model is made to forget about some of its training data points. While naively retraining the model from scratch is an option, it is almost always associated with large computational overheads for deep learning models. Thus, several approaches to approximately unlearn have been proposed along with corresponding metrics that formalize what it means for a model to forget about a data point. In this work, we first taxonomize approaches and metrics of approximate unlearning. As a result, we identify verification error, i.e., the$\ell_{2}$difference between the weights of an approximately unlearned and a naively retrained model, as an approximate unlearning metric that should be optimized for as it subsumes a large class of other metrics. We theoretically analyze the canonical training algorithm, stochastic gradient descent (SGD), to surface the variables which are relevant to reducing the verification error of approximate unlearning for SGD. From this analysis, we first derive an easy-to-compute proxy for verification error (termed unlearning error). The analysis also informs the design of a new training objective penalty that limits the overall change in weights during SGD and as a result facilitates approximate unlearning with lower verification error. We validate our theoretical work through an empirical evaluation on learning with CIFAR-10, CIFAR-100, and IMDB sentiment analysis.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2013) · doi:10.1561/9781601988195 · Cited by 3,856
Research works printed but not linked: 1
- ba, 2021, Csc421/2516 lecture 3: Automatic differentiation & distributed representations, Lecture Notes WS
Cited by
Sorted by citations
Items 101–136 of 136
Wang, J., Bie, H., Zhao, J., & Zhi, Y. (2026) · Scientific Reports · doi:10.1038/s41598-026-40379-9
GEIA: Generative Enhancement Inversion Attack Targeting Machine Unlearning
马尧选, Xu, X., Wang, J., Yi, W., & Liu, Y. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464727
Learn to Unlearn in Large Language Models
Saini, N., & Singh, R. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464235
Unlearning of Knowledge Graph Embedding via Preference Optimization
Liu, J., Ke, W., Wang, P. H., He, Y., Shang, Z., Li, G., Xu, Z., & Ji, K. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792397
DeepUL: Deep Unlearning via Model Sparsity
Zheng, Z., Yin, K., Kuang, Y., Wang, T., Chen, Y., Yao, S., & Huang, H. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792441
CLUE: Bringing Machine Unlearning to Mobile Devices
Sayyed, S., Bastian, N. D., Lucia, M. D., Swami, A., & Restuccia, F. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00366
Empirical Study on Adversarial Robustness Degradation in Image Classification via Unlearning
Shinno, N., Zhang, H., & Saitoh, T. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7820-7_15
Meng, X. (2026) · Big data management · doi:10.1007/978-981-95-5787-5_9
An Introduction to Machine Unlearning
Liu, S., Liu, Y., & Baracaldo, N. (2026) · Synthesis lectures on computer vision · doi:10.1007/978-3-032-17282-2_1
Optimization Foundations for Machine Unlearning
Liu, S. (2026) · Synthesis lectures on computer vision · doi:10.1007/978-3-032-17282-2_2
Machine Unlearning for AI Safety
Chen, Y., & Liu, S. (2026) · Synthesis lectures on computer vision · doi:10.1007/978-3-032-17282-2_13
Unlearning in Federated Learning Settings
Halimi, A., Kadhe, S., Rawat, A., & Baracaldo, N. (2026) · Synthesis lectures on computer vision · doi:10.1007/978-3-032-17282-2_16
Instability of Retraining as a Benchmark for Machine Unlearning Evaluation
Liu, C. Y., & Di, Z. (2026) · Synthesis lectures on computer vision · doi:10.1007/978-3-032-17282-2_12
Unlearning Through Internal Model Analysis: Sparsity, Saliency, and Attribution
Jia, J. (2026) · Synthesis lectures on computer vision · doi:10.1007/978-3-032-17282-2_9
Task-Aligned Unlearning for Multimodal Large Language Models
Zhang, T., Ding, C., Fang, J., Wu, J., & Wang, X. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3795848
Rethinking Unlearnable Examples in Machine Unlearning
Zhao, C., Qian, W., Miao, C., & Huai, M. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-1462-4_4
Federated Unlearning via Sparsity-Aware Gradient Ascent
Zhang, L., Zheng, J., & Chen, S. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582745
Yang, J., Wu, F., Si, X., Wu, L., Yao, S., Cui, L., & Qu, Y. (2026) · International Conference on Communications · doi:10.1109/icc59461.2026.11586833
Forgotten Horizons in Concept Erasure: Safeguarding Close-Proximity Concepts in Text-to-Image Models
Li, J. F., Liu, Y., Yu, R., Zhang, C., Qi, G., Bi, S., Liu, F., Geng, J., & Karray, F. O. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3714133
MUKSB: Machine unlearning via Kalai-Smorodinsky bargaining
Dubey, A., Sharma, M., & Kancharla, P. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133675
Machine unlearning through model forgetting techniques: application area and emerging methods
Büyüktanır, B., Kucur, E. N., Kuyumcu, Z., Ozdemir, N., Altınkaya, Ş., Baydoğmus, G. K., & Yıldız, K. (2026) · Cluster Computing · doi:10.1007/s10586-026-06300-9
Revisiting Privacy Leakage in Machine Unlearning: Membership Inference Beyond the Forgotten Set
Fu, J., Naderloui, N., Zhong, D., Hong, Y., & Wang, W. H. (2026) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp68448.2026.00044 · Cited by 2
Evaluating Machine Unlearning in Fingerphoto Presentation Attack Detection
Vurity, A., Yan, Z., & Albanese, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31930-2_17
Layer-Wise Diagnostic Probing to Enhance Selectivity in Machine Unlearning
Vurity, A., Yan, Z., & Albanese, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31930-2_15
Approximate Machine Unlearning
Wang, W., & Yu, S. (2026) · Digital Privacy and Security · doi:10.1007/978-981-92-1142-5_3
HyperUn: Controlling Uncertainty in Hyperbolic Space for Machine Unlearning
Jung, I., Seo, D., Baek, S., Jung, J., Lee, D., & Kim, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31438-3_29
Wang, W., Tian, Z., Zhang, C., Chen, L., & Yu, S. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817655
Background of Machine Unlearning
Xu, J., & Jia, X. (2026) · Wireless networks · doi:10.1007/978-3-032-30978-5_2
Time-Varying Data Influence Measurement
Xu, J., & Jia, X. (2026) · Wireless networks · doi:10.1007/978-3-032-30978-5_4
Xu, J., & Jia, X. (2026) · Wireless networks · doi:10.1007/978-3-032-30978-5_1
A Review on Machine Unlearning Algorithms and Privacy Protection
Aliprantis, J., & Kalloniatis, C. (2026) · Electronics · doi:10.3390/electronics15184057
Gradient Sparsity Regularization for Training Unlearning-Compatible Models
Hemati, S., Lamghari, S., Asgharian, M., Li, X., & Li, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37211-6_3
Feature Norm Normalization: A Plug-and-Play Module for Boosting Model Accuracy in Machine Unlearning
Yang, C., Yang, X. S., & Tang, X. (2026) · IEEE/CIC International Conference on Communications in China (ICCC) · doi:10.1109/iccc70295.2026.11680129
RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Models
Ranjan, R., Grover, U., Lin, X., & Polyzou, A. (2026) · doi:10.1109/cvprf71909.2026.00768
Kinalioğlu, İ. H. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01329-w
DynamicGater: Fine-Grained Recovery for Machine Unlearning via Dynamic Neuron Gating
Chen, X., Cai, H., Guo, E., Qu, S., & Shang, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-6834-4_14
Metadata
- Authors
- Thudi, A., Deza, G., Chandrasekaran, V., & Papernot, N.
- Affiliations
- University of Toronto (CA)
- University of Wisconsin–Madison (US)
- Venue
- 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 303-319
- OpenAlex
- W3202838631
- Cited by
- 135
- References
- 38
- Citations by year
- 2026: 45 · 2025: 54 · 2024: 22 · 2023: 12 · 2022: 1
- FWCI
- 12.30
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE European Symposium on Security and Privacy (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- DARPA
- NSERC
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- machine unlearning, approximate unlearning, stochastic gradient descent, weight change, deep learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
