PUMA: Performance Unchanged Model Augmentation for Training Data Removal
Open accessAbstract
Preserving the performance of a trained model while removing unique characteristics of marked training data points is challenging. Recent research usually suggests retraining a model from scratch with remaining training data or refining the model by reverting the model optimization on the marked data points. Unfortunately, aside from their computational inefficiency, those approaches inevitably hurt the resulting model's generalization ability since they remove not only unique characteristics but also discard shared (and possibly contributive) information. To address the performance degradation problem, this paper presents a novel approach called Performance Unchanged Model Augmentation (PUMA). The proposed PUMA framework explicitly models the influence of each training data point on the model's generalization ability with respect to various performance criteria. It then complements the negative impact of removing marked data by reweighting the remaining data optimally. To demonstrate the effectiveness of the PUMA framework, we compared it with multiple state-of-the-art data removal techniques in the experiments, where we show the PUMA can effectively and efficiently remove the unique characteristics of marked training data without retraining the model that can 1) fool a membership attack, and 2) resist performance degradation. In addition, as PUMA estimates the data importance during its operation, we show it could serve to debug mislabelled data points more efficiently than existing approaches.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Although it is feasible to design an appropriate unlearning scheme for every model, this is an inefficient approach that would require many manual interventions.
“Although it is feasible to design an appropriate unlearning scheme for every model, this is an inefficient approach that would require many manual interventions [104, 105].”
From Machine Unlearning: A Survey (2024) · cites this paper as [104]
The Introduction (p. 1) describes model-specific removal methods as hard to generalize and says some alternative approaches have high computational costs. It does not say that designing a scheme for every model is feasible or that doing so would require many manual interventions. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435 · see citing works
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961 · see citing works
Advances in Neural Information Processing Systems 32 (NeurIPS 2019)
Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché–Buc, F., Fox, E. B., & Garnett, R. (2020) · HAL · Cited by 2,450 · see citing works
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Jacot, A., Gabriel, F., & Hongler, C. (2018) · arXiv · doi:10.48550/arxiv.1806.07572 · Cited by 1,562 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
Homer, N., Szelinger, S., Redman, M., Duggan, D., Tembe, W., Muehling, J., Pearson, J. V., Stephan, D. A., Nelson, S. F., & Craig, D. W. (2008) · PLoS Genetics · doi:10.1371/journal.pgen.1000167 · Cited by 1,221 · see citing works
Incremental and Decremental Support Vector Machine Learning
Cauwenberghs, G., & Poggio, T. (2000) · Cited by 1,156 · see citing works
Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting
Yeom, S., Giacomelli, I., Fredrikson, M., & Jha, S. (2018) · 2018 IEEE 31st Computer Security Foundations Symposium (CSF) · doi:10.1109/csf.2018.00027 · Cited by 1,001 · see citing works
Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00019 · Cited by 810 · see citing works
Data Shapley: Equitable Valuation of Data for Machine LearningPreprint
Ghorbani, A., & Zou, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02868 · Cited by 434 · see citing works
Multiple Incremental Decremental Learning of Support Vector Machines
Karasuyama, M., & Takeuchi, I. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2048039 · Cited by 176 · see citing works
Making AI Forget You: Data Deletion in Machine Learning
Ginart, A., Guan, M. Y., Valiant, G., & Zou, J. (2019) · arXiv · doi:10.48550/arxiv.1907.05012 · Cited by 175 · see citing works
Certified Data Removal from Machine Learning Models
Guo, C., Goldstein, T., Hannun, A., & van der Maaten, L. (2019) · arXiv · doi:10.48550/arxiv.1911.03030 · Cited by 160 · see citing works
Robust Traceability from Trace Amounts
Dwork, C., Smith, A., Steinke, T., Ullman, J., & Vadhan, S. (2015) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2015.46 · Cited by 142 · see citing works
Representer Point Selection for Explaining Deep Neural NetworksPreprint
Yeh, C., Kim, J. S., Yen, I. E.-H., & Ravikumar, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.09720 · Cited by 125 · see citing works
Nasr, M., Shokri, R., & Houmansadr, A. (2018) · arXiv (Cornell University) · Cited by 104 · see citing works
Incremental and decremental training for linear classification
Tsai, C.-H., Lin, C.-Y., & Lin, C.-J. (2014) · Knowledge Discovery and Data Mining · doi:10.1145/2623330.2623661 · Cited by 91 · see citing works
Machine UnlearningPreprint
Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.03817 · Cited by 9 · see citing works
Graves, L., Nagisetty, V., & Ganesh, V. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17371 · Cited by 5 · see citing works
Nasr, M., Shokri, R., & Houmansadr, A. (2018) · arXiv · doi:10.48550/arxiv.1812.00910
Flagged first, then by citations
Cited by
Sorted by citations
Unveiling security, privacy, and ethical concerns of ChatGPT
Wu, X., Duan, R., & Ni, J. (2023) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2023.10.007 · Cited by 320 · see citing works
Xu, H., Zhu, T., Zhang, L., Zhou, W., & Yu, P. S. (2024) · ACM Computing Surveys · doi:10.1145/3603620 · Cited by 164 · see citing works
(2023) · doi:10.18653/v1/2023.emnlp-main.632 · Cited by 115 · see citing works
Machine Unlearning: Solutions and Challenges
Xu, J., Wu, Z., Wang, C., & Jia, X. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3379240 · Cited by 114 · see citing works
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
Wang, Z., Yang, E., Shen, L., & Huang, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3498346 · Cited by 101 · see citing works
Federated Unlearning With Momentum Degradation
Zhao, Y., Wang, P., Qi, H., Huang, J., Wei, Z., & Zhang, Q. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3321594 · Cited by 67 · see citing works
Making recommender systems forget: Learning and unlearning for erasable recommendation
Li, Y., Chen, C., Zheng, X., Liu, J., & Wang, J. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111124 · Cited by 55 · see citing works
Gyevnár, B., & Kasirzadeh, A. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01020-y · Cited by 40 · see citing works
Machine Unlearning via Representation Forgetting With Parameter Self-Sharing
Wang, W., Zhang, C., Tian, Z., & Yu, S. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3331239 · Cited by 36 · see citing works
Selective and collaborative influence function for efficient recommendation unlearning
Li, Y., Chen, C., Zheng, X., Zhang, Y., Gong, B., Wang, J., & Chen, L. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121025 · Cited by 36 · see citing works
An overview of machine unlearning
Li, C., Jiang, H., Chen, J., Zhao, Y. L., Fu, S., Jing, F., & Guo, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100254 · Cited by 21 · see citing works
FedU: Federated Unlearning via User-Side Influence Approximation Forgetting
Wang, W., Zhang, C., Tian, Z., & Yu, S. (2024) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2024.3520614 · Cited by 21 · see citing works
Improve Knowledge Distillation via Label Revision and Data Selection
Lan, W., Cheung, Y., Xu, Q., Liu, B., Hu, Z., Li, M., & Chen, Z. (2025) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2025.3559881 · Cited by 12 · see citing works
Patient Similarity Learning with Selective Forgetting
Qian, W., Zhao, C., Shao, H., Chen, M., Wang, F., & Huai, M. (2022) · 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995016 · Cited by 11 · see citing works
Bonato, J., Cotogni, M., & Sabetta, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_1 · Cited by 11 · see citing works
Class-wise federated unlearning: Harnessing active forgetting with teacher–student memory generation
Li, Y., Zhang, J., Liu, Y., & Chen, C. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113353 · Cited by 11 · see citing works
A Systematic Literature Review of Machine Unlearning Techniques in Neural Networks
Cevallos, I., Benalcázar, M. E., Caraguay, Á. L. V., Zea, J. A., & López, L. I. B. (2025) · Computers · doi:10.3390/computers14040150 · Cited by 10 · see citing works
Task-Aware Machine Unlearning and Its Application in Load Forecasting
Xu, W., & Teng, F. (2024) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2024.3376828 · Cited by 9 · see citing works
Implications of Privacy Regulations on Video Surveillance Systems
Kansal, K., Wong, Y., & Kankanhalli, M. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3706108 · Cited by 7 · see citing works
Private Data Protection with Machine Unlearning in Contrastive Learning Networks
Chen, K., Wang, Z., & Mi, B. (2024) · Mathematics · doi:10.3390/math12244001 · Cited by 7 · see citing works
Encoder augmentation for multi-task graph contrastive learning
Wang, X., Zhang, Q., Liu, G., Zhao, Z., & Cui, H. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129650 · Cited by 7 · see citing works
Automated Program Repair: Emerging Trends Pose and Expose Problems for Benchmarks
Renzullo, J., Reiter, P., Weimer, W. R., & Forrest, S. (2025) · ACM Computing Surveys · doi:10.1145/3704997 · Cited by 7 · see citing works
Sequence Unlearning for Sequential Recommender Systems
Ye, S., & Jie Lu, B. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8388-9_33 · Cited by 6 · see citing works
Conditional Matching GAN Guided Reconstruction Attack in Machine Unlearning
Zhang, K., Wang, W., Fan, Z., Song, X., & Yu, S. (2023) · Globecom · doi:10.1109/globecom54140.2023.10437231 · Cited by 6 · see citing works
Fuzzy rough unlearning model for feature selection
Tang, Y., Zhao, S., Chen, H., Li, C., Zhai, J., & Zhou, Q. (2023) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2023.109102 · Cited by 6 · see citing works
Towards Natural Machine Unlearning
He, Z., Li, T., Cheng, X., Huang, Z., & Huang, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3597350 · Cited by 5 · see citing works
CRFU: Compressive Representation Forgetting Against Privacy Leakage on Machine Unlearning
Wang, W., Zhang, C., Tian, Z., Liu, S., & Yu, S. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3542092 · Cited by 5 · see citing works
Graph unlearning: Efficient node removal in graph neural networks
Guan, F., Zhu, T., Wang, Z., Ren, W., & Zhou, W. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116024 · Cited by 3 · see citing works
RRL: Recommendation Reverse Learning
You, X., Xu, J., Zhang, M., Gao, Z., & Yang, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i8.28782 · Cited by 3 · see citing works
Continuous Verification of Catastrophic Recalling in Machine Unlearning via Adversarial Testing
Qu, Y., Ji, Z., Cui, L., Zhang, C., Liu, L., & Tian, Z. (2024) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc63484.2024.00056 · Cited by 3 · see citing works
ASDIA: An Adversarial Sample to Preserve Privacy Program in Federated Learning
Tian, S., Tan, Y., Wang, H., Liu, H., & Li, Z. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3545599 · Cited by 3 · see citing works
Federated Unlearning with Multiple Client Partitions
Zhu, W., Jia, J., Luo, B., & Lin, X. (2024) · International Conference on Communications · doi:10.1109/icc51166.2024.10622238 · Cited by 3 · see citing works
Hena, B., Wei, Z., Ibarra‐Castanedo, C., & Maldague, X. (2025) · NDT & E International · doi:10.1016/j.ndteint.2025.103327 · Cited by 3 · see citing works
MUSO: achieving exact machine unlearning in over-parameterized regimes
Yang, R., He, M., He, Z., Qiu, Y., & Huang, X. (2025) · Machine Learning · doi:10.1007/s10994-025-06806-0 · Cited by 1 · see citing works
Aliyeva, L., Abdullayev, N., Zarbiyeva, S., Huseynov, I., & Suleymanov, U. (2024) · IEEE International Conference on Application of Information and Communication Technologies (AICT) · doi:10.1109/aict61888.2024.10740413 · Cited by 1 · see citing works
Fed$n$nP: Federated Unlearning With Multiple Client Set Partitions
Jia, J., Zhu, W., Luo, B., Lin, X., & Ma, L. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3586441 · Cited by 1 · see citing works
FedUP: Federated Unlearning With Prototypes
Huang, Y., Li, X., Chen, S., & Chen, S. (2025) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2025.3612138 · Cited by 1 · see citing works
Rüb, M., Konegen, D., Sikora, A., & Mueller-Gritschneder, D. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228149 · Cited by 1 · see citing works
Enhancing SME Internationalization and Digitalization Through Candidate Elimination Algorithms
Babu, C. V. S., Kumar, S. B., Nithesh Kumar, C., Dhineshwaren, M., & Abbas, S. (2024) · Advances in business strategy and competitive advantage book series · doi:10.4018/979-8-3693-3518-5.ch006 · Cited by 1 · see citing works
EncoderMU: Machine Unlearning in Contrastive Learning
Wang, Z., Mi, B., & Chen, K. (2024) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-73699-5_15 · Cited by 1 · see citing works
Malnick, S., Avidan, S., & Fried, O. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00458
Zhang, N., Yao, Y., & Deng, S. (2023) · doi:10.18653/v1/2023.ijcnlp-tutorials.4
Class-Wise Federated Unlearning: Harnessing Active Forgetting with Teacher-Student Memory GenerationPreprint
Li, Y., Zhang, J., Liu, Y., & Chen, C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5071728
The Mainstays of Trustworthy Machine Learning
Yadav, C. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i28.35233
Zhang, W., Liu, T., Su, Y., & Zhu, S. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-9815-8_14
KaRe: Towards Flexible and Effective Machine Unlearning with Knowledge Alignment and Repair
Chai, Z., Liu, Q., Hu, Q., & Li, P. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-93251-9_16
EMMU: Efficient Information-Level Multimodal Machine Unlearning with High Model Fidelity
Zhang, J., Hou, J., Xiao, T., Huang, Y., & Li, X.-Y. (2025) · International Workshop on Quality of Service · doi:10.1109/iwqos65803.2025.11143267
Boundary-Anchored Functional Duplication Attacks on Machine Unlearning
Shangguan, C., Li, Y., Wang, Y., & Song, Q. (2025) · IEEE International Conference on Privacy Computing and Data Security (PCDS) · doi:10.1109/pcds65695.2025.00035
PUMA: Performance Unchanged Model Augmentation for Training Data Removal
Association for Artificial Intelligence 2022, Hashemi, M., Srinivasa, C., & Wu, G. (2022) · Underline Science Inc. · doi:10.48448/t585-6h32
Self-citationJi, Z., Qu, Y., Gao, L., & Zhang, T. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3055-7_15
Fairness-aware Graph Unlearning with Knowledge Distillation
Li, L., Meng, S., Zhou, J., Liu, N., & Li, Q. (2025) · IEEE Smart World Congress (SWC) · doi:10.1109/swc65939.2025.00109
Wang, J., Bie, H., Zhao, J., & Zhi, Y. (2026) · Scientific Reports · doi:10.1038/s41598-026-40379-9
MicroPatch : Directed Backdoor Erasing via Victim Parameter Decoupling
Ma, Y., Zhao, Y. C., Yuan, P., Zhao, L., & Wang, Q. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3705279
Efficient and High Fidelity Multimodal Machine Unlearning for Information With Deep Memory
Zhang, J., Hou, J., Zhang, H., Xiao, T., & Huang, Y. (2026) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2026.3706840
Redundancy-Aware Federated Unlearning with Reverse and Selective Distillation
Zheng, J., Li, K., Zhou, C., Zhu, D., Pan, C., & Du, X. (2026) · International Conference on Machine Intelligence Theory and Applications (MiTA) · doi:10.1109/mita69365.2026.11582666
Introduction of Machine Unlearning
Wang, W., & Yu, S. (2026) · Digital Privacy and Security · doi:10.1007/978-981-92-1142-5_1
Exact Unlearning for Large Models via Prompt Tuning
Xu, J., & Jia, X. (2026) · Wireless networks · doi:10.1007/978-3-032-30978-5_5
Background of Machine Unlearning
Xu, J., & Jia, X. (2026) · Wireless networks · doi:10.1007/978-3-032-30978-5_2
Robust Privacy Unlearning for LLM Services via Adversarially Augmented Preference Optimization
Yang, S., Guan, Y., Duan, Q., Luo, C., & Yang, H. (2026) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws72778.2026.00150
Practical Federated Unlearning: A Target Client-Driven Approach to Model Forgetting
Tian, L., Lin, F., Qin, Z., Qi, J., Li, M., & Ren, K. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3734781
Yang, S., Chen, P., Li, R., Hu, S., Guo, H., Fong, S., & Duan, Q. (2026) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2026.104020
- Authors
- Wu, G., Hashemi, M., & Srinivasa, C.
- ORCID
- Wu, Ga
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2022-06-28
- Volume
- 36
- Issue
- 8
- Pages
- 8675-8682
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W4283796010
- Cited by
- 61
- References
- 20
- Citations by year
- 2026: 11 · 2025: 21 · 2024: 16 · 2023: 11 · 2022: 2
- FWCI
- 6.91
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- membership inference attacks, data privacy, data reweighting, model generalization, performance degradation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
