CoChat
Sign Up
Cited by 162Privacy-Preserving Technologies in DataAdversarial Robustness in Machine LearningCryptography and Data Security

Zero-Shot Machine Unlearning

Open access
Chundawat, Vikram S., Tarun, Ayush K., Mandal, Murari, Kankanhalli, Mohan·2023·IEEE Transactions on Information Forensics and Security
DOI 10.1109/tifs.2023.3265506
Open this paper in CoChat — free accountOpen PDF

Abstract

Modern privacy regulations grant citizens the right to be forgotten by products, services and companies. In case of machine learning (ML) applications, this necessitates deletion of data not only from storage archives but also from ML models. Due to an increasing need for regulatory compliance required for ML applications,machine unlearningis becoming an emerging research problem. The right to be forgotten requests come in the form of removal of a certain set or class of data from the already trained ML model. Practical considerations preclude retraining of the model from scratch after discarding the deleted data. The few existing studies use either the whole training data, or a subset of training data, or some metadata stored during training to update the model weights for unlearning. However, strict regulatory compliance requires time-bound deletion of data. Thus, in many cases, no data related to the training process or training samples may be accessible even for the unlearning purpose. We therefore ask the question:is it possible to achieve unlearning with zero training samples?In this paper, we introduce the novel problem ofzero-shot machine unlearningthat caters for the extreme but practical scenario where zero original data samples are available for use. We then propose two novel solutions forzero-shot machine unlearningbased on (a) error minimizing-maximizing noise and (b) gated knowledge transfer. These methods remove the information of the forget data from the model while maintaining the model efficacy on the retain data. The zero-shot approach offers good protection against the model inversion attacks and membership inference attacks. We introduce a new evaluation metric,Anamnesis Index(AIN) to effectively measure the quality of the unlearning method. The experiments show promising results for unlearning in deep learning models on benchmark vision data-sets. The source code is available here: https://github.com/ayu987/zero-shot-unlearning.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tifs.2023.3265506

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Although it is feasible to design an appropriate unlearning scheme for every model, this is an inefficient approach that would require many manual interventions.

    “Although it is feasible to design an appropriate unlearning scheme for every model, this is an inefficient approach that would require many manual interventions [104, 105].”

    From Machine Unlearning: A Survey (2024) · cites this paper as [105]

    In “B. Deep Machine Unlearning,” the paper says that designing a mixed-linear network for every deep architecture is inefficient and requires manual intervention. The citing sentence extends that specific criticism to designing any unlearning scheme for every model, which the passage does not establish. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 46 · Cited by 162

Research works printed but not linked: 27

  1. [3][3] V. Feldman, 'Does learning require memorization? a short tale about a long tail,' in Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing , 2020, pp. 954-959. 1 -
  2. [4][4] N. Carlini, C. Liu, ´ U. Erlingsson, J. Kos, and D. Song, 'The secret sharer: Evaluating and testing unintended memorization in neural networks,' in 28th USENIX Security Symposium (USENIX Security 19) , 2019, pp. 267-284. 1 -
  3. [7][7] Y. Wu, E. Dobriban, and S. Davidson, 'Deltagrad: Rapid retraining of machine learning models,' in International Conference on Machine Learning . PMLR, 2020, pp. 10 355-10 366. 1, 2, 3 -
  4. [8][8] A. Golatkar, A. Achille, and S. Soatto, 'Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations,' in European Conference on Computer Vision . Springer, 2020, pp. 383-398. 1, 2, 3, 5, 16 -
  5. [9][9] A. Golatkar, A. Achille, A. Ravichandran, M. Polito, and S. Soatto, 'Mixed-privacy forgetting in deep networks,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 792-801. 1, 2, 3, 6 -
  6. [11][11] A. K. Tarun, V. S. Chundawat, M. Mandal, and M. Kankanhalli, 'Deep regression unlearning,' arXiv preprint arXiv:2210.08196 , 2022. 1 - · arXiv
  7. [12][12] C. Guo, T. Goldstein, A. Hannun, and L. Van Der Maaten, 'Certified data removal from machine learning models,' in International Conference on Machine Learning . PMLR, 2020, pp. 3832-3842. 1, 2, 3 -
  8. [14][14] L. Graves, V. Nagisetty, and V. Ganesh, 'Amnesiac machine learning,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 13, 2021, pp. 11 516-11 524. 2, 3, 6, 7, 8 -
  9. [15][15] J. Brophy and D. Lowd, 'Machine unlearning for random forests,' in International Conference on Machine Learning . PMLR, 2021, pp. 1092-1104. 2 -
  10. [17][17] B. Mirzasoleiman, A. Karbasi, and A. Krause, 'Deletion-robust submodular maximization: Data summarization with 'the right to be forgotten',' in International Conference on Machine Learning . PMLR, 2017, pp. 2449-2458. 2 -
  11. [18][18] Q. P. Nguyen, B. K. H. Low, and P. Jaillet, 'Variational bayesian unlearning,' Advances in Neural Information Processing Systems , vol. 33, 2020. 2 -
  12. [19][19] A. Mahadevan and M. Mathioudakis, 'Certifiable machine unlearning for linear models,' arXiv preprint arXiv:2106.15093 , 2021. 3 - · arXiv
  13. [21][21] Z. Izzo, M. A. Smart, K. Chaudhuri, and J. Zou, 'Approximate data deletion from machine learning models,' in International Conference on Artificial Intelligence and Statistics . PMLR, 2021, pp. 2008-2016. 3 -
  14. [22][22] S. Neel, A. Roth, and S. Sharifi-Malvajerdi, 'Descent-to-delete: Gradient-based methods for machine unlearning,' in Algorithmic Learning Theory . PMLR, 2021, pp. 931-962. 3 -
  15. [23][23] Y. Wu, V. Tannen, and S. B. Davidson, 'Priu: A provenance-based approach for incrementally updating regression models,' in Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data , 2020, pp. 447-462. 3 -
  16. [26][26] A. Salem, Y. Zhang, M. Humbert, M. Fritz, and M. Backes, 'Mlleaks: Model and data independent membership inference attacks and defenses on machine learning models,' in Network and Distributed Systems Security Symposium 2019 . Internet Society, 2019. 3 -
  17. [28][28] C. Dwork, A. Smith, T. Steinke, J. Ullman, and S. Vadhan, 'Robust traceability from trace amounts,' in 2015 IEEE 56th Annual Symposium on Foundations of Computer Science . IEEE, 2015, pp. 650-669. 3 -
  18. [29][29] M. Nasr, R. Shokri, and A. Houmansadr, 'Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning,' in 2019 IEEE symposium on security and privacy (SP) . IEEE, 2019, pp. 739-753. 3 -
  19. [31][31] J. Jia, A. Salem, M. Backes, Y. Zhang, and N. Z. Gong, 'Memguard: Defending against black-box membership inference attacks via adversarial examples,' in Proceedings of the 2019 ACM SIGSAC conference on computer and communications security , 2019, pp. 259-274. 3 -
  20. [32][32] M. Nasr, R. Shokri, and A. Houmansadr, 'Machine learning with membership privacy using adversarial regularization,' in Proceedings of the 2018 ACM SIGSAC conference on computer and communications security , 2018, pp. 634-646. 3 -
  21. [34][34] M. Fredrikson, S. Jha, and T. Ristenpart, 'Model inversion attacks that exploit confidence information and basic countermeasures,' in Proceedings of the 22nd ACM SIGSAC conference on computer and communications security , 2015, pp. 1322-1333. 3, 8 -
  22. [36][36] N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami, 'Practical black-box attacks against machine learning,' in Proceedings of the 2017 ACM on Asia conference on computer and communications security , 2017, pp. 506-519. 3 -
  23. [38][38] S. Kullback and R. A. Leibler, 'On information and sufficiency,' The annals of mathematical statistics , vol. 22, no. 1, pp. 79-86, 1951. 4 -
  24. [39][39] P. Micaelli and A. J. Storkey, 'Zero-shot knowledge transfer via adversarial belief matching,' Advances in Neural Information Processing Systems , vol. 32, pp. 9551-9561, 2019. 5 -
  25. [41][41] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, 'Gradient-based learning applied to document recognition,' Proceedings of the IEEE , vol. 86, no. 11, pp. 2278-2324, 1998. 6 -
  26. [45][45] V. S. Chundawat, A. K. Tarun, M. Mandal, and M. Kankanhalli, 'Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher,' in Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence , 2023. 7, 8 -
  27. [46][46] S. Rezaei and X. Liu, 'On the difficulty of membership inference attacks,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 7892-7900. 16

Authors
Chundawat, V. S., Tarun, A. K., Mandal, M., & Kankanhalli, M.
Affiliations
National University of Singapore (SG)
ORCID
Venue
IEEE Transactions on Information Forensics and Security
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2023
Volume
18
Pages
2345-2354
ISSN
  • 1556-6013 (ISSN-L)
  • 1556-6021
DOI
10.1109/tifs.2023.3265506
OpenAlex
W4362714589
Cited by
162
References
46
Citations by year
2026: 50 · 2025: 64 · 2024: 38 · 2023: 10
FWCI
14.34
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Information Forensics and Security (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Research Foundation, Singapore, under its Strategic Capability Research Centers Funding Initiative
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Cryptography and Data Security
Keywords
right to be forgotten, privacy regulation, model inversion attacks, membership inference attacks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review