Unlearn What You Want to Forget: Efficient Unlearning for LLMs
Open accessAbstract
Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data, however, this process might suffer from privacy issues and violations of data protection regulations.As a result, the ability to easily remove data related to individual users from such models while not deteriorating their predictive quality after the removal becomes increasingly important.To address these issues, in this work, we propose an efficient unlearning framework that could efficiently update LLMs without having to retrain the whole model after data removals, by introducing lightweight unlearning layers learned with a selective teacher-student objective into the transformers.In addition, we introduce a fusion mechanism to effectively combine different unlearning layers that learns to forget different sets of data to handle a sequence of forgetting operations.Experiments on classification and generation tasks demonstrate the effectiveness of our proposed methods compared to the state-of-the-art baselines 1 .
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A few authors use the success of a Membership Inference Attack (MIA) as a metric.
“Membership Inference Attack (MIA) : A few authors (Wang et al 2023; Chen and Yang 2023; Pawelczyk et al 2023) use the success of a MIA as a metric.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Chen and Yang 2023
Chen and Yang (2023), Wang et al (2023), Pawelczyk et al (2023), Ni et al (2023), and Pochinkov and Schoots (2023) evaluate retaining on remaining samples of the same dataset used for unlearning.
“Same dataset but different samples . Chen and Yang (2023); Wang et al (2023); Pawelczyk et al (2023); Ni et al (2023); Pochinkov and Schoots (2023) evaluate retaining on the remaining samples of the same dataset used for unlearning.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Chen and Yang (2023)
Reported runtime values of unlearning methods range from minutes to hours or even longer than a day.
“Reported values range from minutes (Wang et al 2023; Yao et al 2023 ; Limisiewicz and Mareček 2022; Jang et al 2022; Yu et al 2023) to hours (Chen and Yang 2023) or even longer than a day (Lu et al 2022 ).”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Chen and Yang 2023
Epoch information is only reported by a subset of works and can only be used when comparing identical models, data and forgetting tasks, which is rarely the case.
“However, this information is only reported by a subset of works (Eldan and Russinovich 2023; Chen and Yang 2023; Jang et al 2022; Yao et al 2023; Yu et al 2023; Wu et al 2023; Ni et al 2023; Ilharco et al 2022) and it can only be used when comparing identical models, data and forgetting tasks, which is rarely the case.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Chen and Yang 2023
Some researchers prefer a second, more controllable but less realistic scenario, where forgetting is applied to an LLM fine-tuned with a dataset that includes the undesired knowledge.
“Due to these limitations, some researchers, such as Eldan and Russinovich (2023); Chen and Yang (2023); Borkar (2023); Wu et al (2023); Wang et al (2023) prefer a second, more controllable but less realistic scenario, where forgetting is applied to an LLM fine-tuned with a dataset that includes the undesired knowledge.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Chen and Yang 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578 · see citing works
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
Learning Word Vectors for Sentiment Analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011) · Cited by 3,710 · see citing works
Differential Privacy: A Survey of Results
Dwork, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79228-4_1 · Cited by 3,542 · see citing works
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
Parameter-efficient transfer learning for NLP
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019) · arXiv · doi:10.48550/arxiv.1902.00751 · Cited by 2,270 · see citing works
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013 · see citing works
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.754 · Cited by 1,243 · see citing works
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101 · see citing works
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932 · see citing works
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.759 · Cited by 755 · see citing works
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · arXiv · doi:10.48550/arxiv.2202.12837 · Cited by 752 · see citing works
Extracting Training Data from Large Language Models
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07805 · Cited by 727 · see citing works
Multitask Prompted Training Enables Zero-Shot Task Generalization
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612 · see citing works
Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
Qin, C., Zhang, A., Zhang, Z., Chen, J., Yasunaga, M., & Yang, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.85 · Cited by 519 · see citing works
Self-citationK-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
Wang, R., Tang, D., Duan, N., Wei, Z., Huang, X., Ji, J., Cao, G., Jiang, D., & Zhou, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.121 · Cited by 503 · see citing works
Holistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480 · see citing works
SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization
Gliwa, B., Mochol, I., Biesek, M., & Wawer, A. (2019) · Proceedings of the 2nd Workshop on New Frontiers in Summarization · doi:10.18653/v1/d19-5409 · Cited by 424 · see citing works
Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks
Golatkar, A., Achille, A., & Soatto, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00932 · Cited by 394 · see citing works
(2022) · doi:10.52202/068431-0278 · Cited by 358 · see citing works
Solving Quantitative Reasoning Problems with Language Models
Lewkowycz, A., Andreassen, A., Dohan, D., Dyer, E., Michalewski, H., Ramasesh, V., Slone, A., Anil, C., Schlag, I., Gutman-Solo, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.14858 · Cited by 358 · see citing works
What do Neural Machine Translation Models Learn about Morphology?
Belinkov, Y., Durrani, N., Dalvi, F., Sajjad, H., & Glass, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1080 · Cited by 307 · see citing works
Deduplicating Training Data Makes Language Models Better
Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., & Carlini, N. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.577 · Cited by 304 · see citing works
A Survey of Machine Unlearning
Nguyên, T. T., Huynh, T. T., Ren, Z. X., Nguyen, P. L., Liew, A. W., Yin, H., & Nguyen, Q. V. H. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3749987 · Cited by 303 · see citing works
Smith, S., Patwary, M., Norick, B., LeGresley, P., Rajbhandari, S., Casper, J., Liu, Z., Prabhumoye, S., Zerveas, G., Korthikanti, V., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.11990 · Cited by 286 · see citing works
LIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2400 · Cited by 275 · see citing works
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · see citing works
Quantifying Memorization Across Neural Language Models
Carlini, N., Ippolito, D., Jagielski, M., Lee, K., Tramer, F., & Zhang, C. (2022) · arXiv · doi:10.48550/arxiv.2202.07646 · Cited by 204 · see citing works
Do Prompt-Based Models Really Understand the Meaning of Their Prompts?
Webson, A., & Pavlick, E. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.167 · Cited by 203 · see citing works
A Survey of Machine Unlearning
Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H. (2022) · arXiv · doi:10.48550/arxiv.2209.02299 · Cited by 193 · see citing works
Large Language Models Can Be Strong Differentially Private LearnersPreprint
Li, X., Tramèr, F., Liang, P., & Hashimoto, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05679 · Cited by 153 · see citing works
Differentially Private Fine-tuning of Language Models
Yu, D., Naik, S., Bačkurs, A., Gopi, S., Inan, H. A., Kamath, G., Kulkarni, J., Lee, Y. T., Manoel, A., Wutschitz, L., et al. (2024) · Journal of Privacy and Confidentiality · doi:10.29012/jpc.880 · Cited by 152 · see citing works
Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Jang, J., Yoon, D., Yang, S., Cha, S., Lee, M., Logeswaran, L., & Seo, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.805 · Cited by 135 · see citing works
Entailment as Few-Shot LearnerPreprint
Wang, S., Fang, H., Khabsa, M., Mao, H., & Ma, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.14690 · Cited by 130 · see citing works
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2022) · arXiv · doi:10.48550/arxiv.2212.10560 · Cited by 118 · see citing works
Differentially Private Fine-tuning of Language Models
Yu, D., Naik, S., Backurs, A., Gopi, S., Inan, H. A., Kamath, G., Kulkarni, J., Lee, Y. T., Manoel, A., Wutschitz, L., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.06500 · Cited by 101 · see citing works
Remember What You Want to Forget: Algorithms for Machine Unlearning
Sekhari, A., Acharya, J., Kamath, G., & Suresh, A. T. (2021) · arXiv · doi:10.48550/arxiv.2103.03279 · Cited by 99 · see citing works
Mitchell, E., Lin, C., Bosselut, A., Finn, C., & Manning, C. D. (2021) · arXiv · doi:10.48550/arxiv.2110.11309 · Cited by 95 · see citing works
Approximate Data Deletion from Machine Learning ModelsPreprint
Izzo, Z., Smart, M. A., Chaudhuri, K., & Zou, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10077 · Cited by 88 · see citing works
Large-Scale Differentially Private BERT
Anil, R., Ghazi, B., Gupta, V., Kumar, R., & Manurangsi, P. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.484 · Cited by 68 · see citing works
Measuring Forgetting of Memorized Training ExamplesPreprint
Jagielski, M., Thakkar, O. D., Tramèr, F., Ippolito, D., Lee, K., Carlini, N., Wallace, E., Song, S., Thakurta, A., Papernot, N., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.00099 · Cited by 41 · see citing works
Knowledge Neurons in Pretrained Transformers
Dai, D., Dong, L., Hao, Y., Sui, Z., Chang, B., & Wei, F. (2021) · arXiv · doi:10.48550/arxiv.2104.08696 · Cited by 39 · see citing works
Parameter-Efficient Fine-Tuning Design SpacesPreprint
Chen, J., Zhang, A., Shi, X., Li, M., Smola, A. J., & Yang, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.01821 · Cited by 21 · see citing works
Self-citationTowards Unbounded Machine UnlearningPreprint
Kurmanji, M., Triantafillou, P., Hayes, J., & Triantafillou, E. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.09880 · Cited by 17 · see citing works
Privacy Adhering Machine Un-learning in NLP
Kumar, V. B., Gangadharaiah, R., & Roth, D. (2023) · doi:10.18653/v1/2023.findings-ijcnlp.25 · Cited by 13 · see citing works
Continual Learning and Private UnlearningPreprint
Liu, B., Liu, Q., & Stone, P. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.12817 · Cited by 9 · see citing works
Human-in-the-loop Abstractive Dialogue Summarization
Chen, J., Dodda, M., & Yang, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.584 · Cited by 8 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Ma- chine unlearning. In 2021 IEEE Symposium on Secu- rity and Privacy (SP), pages 141-159. IEEE.
- Eli Chien, Chao Pan, and Olgica Milenkovic. 2023. Ef- ficient model updates for approximate unlearning of graph-structured data. In The Eleventh International Conference on Learning Representations.
- PMLR.
Cited by
Sorted by citations
A Survey of Machine Unlearning
Nguyên, T. T., Huynh, T. T., Ren, Z. X., Nguyen, P. L., Liew, A. W., Yin, H., & Nguyen, Q. V. H. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3749987 · Cited by 303 · see citing works
On Protecting the Data Privacy of Large Language Models (LLMs): A Survey
Yan, B., Li, K., Xu, M., Dong, Y., Zhang, Y., Ren, Z., & Cheng, X. (2024) · International Conference on Meta Computing (ICMC) · doi:10.1109/icmc60390.2024.00008 · Cited by 187 · see citing works
Rethinking machine unlearning for large language models
Liu, S., Yao, Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Yao, Y., Liu, C. Y., Xu, X., Li, H., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-00985-0 · Cited by 173 · see citing works
On protecting the data privacy of Large Language Models (LLMs) and LLM agents: A literature review
Yan, B., Li, K., Xu, M., Dong, Y., Zhang, Y., Ren, Z., & Cheng, X. (2025) · High-Confidence Computing · doi:10.1016/j.hcc.2025.100300 · Cited by 93 · see citing works
GDPR and Large Language Models: Technical and Legal Obstacles
Feretzakis, G., Vagena, E., Kalodanis, K., Peristera, P., Kalles, D., & Anastasiou, A. (2025) · Future Internet · doi:10.3390/fi17040151 · Cited by 38 · see citing works
A Primer for Evaluating Large Language Models in Social-Science Research
Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251325174 · Cited by 57 · see citing works
Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects
Li, N., Zhou, C., Gao, Y., Chen, H., Zhang, Z., Kuang, B., & Fu, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3530988 · Cited by 53 · see citing works
Digital forgetting in large language models: a survey of unlearning methods
Blanco-Justicia, A., Jebreel, N., Manzanares-Salor, B., Sánchez, D., Domingo-Ferrer, J., Collell, G., & Eeik Tan, K. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11078-6 · Cited by 31 · see citing works
Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers
Huang, C.-P., Chang, K.-P., Tsai, C.-T., Lai, Y.-H., Yang, F.-E., & Wang, Y.-C. F. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73661-2_20 · Cited by 33 · see citing works
The Frontier of Data Erasure: A Survey on Machine Unlearning for Large Language Models
Qu, Y., Ding, M., Sun, N., Thilakarathna, K., Zhu, T., & Niyato, D. T. (2025) · Computer · doi:10.1109/mc.2024.3405397 · Cited by 29 · see citing works
Privacy in Fine-Tuning Large Language Models: Attacks, Defenses, and Future Directions
Du, H., Liu, S., Zheng, L., Cao, Y., Nakamura, A., & Chen, L. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8183-9_25 · Cited by 23 · see citing works
Creative Writers' Attitudes on Writing as Training Data for Large Language Models
Gero, K. I., Desai, M., Schnitzler, C., Eom, N., Cushman, J., & Glassman, E. L. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713287 · Cited by 16 · see citing works
Towards efficient and effective unlearning of large language models for recommendation
Wang, H., Lin, J., Chen, B., Yang, Y., Tang, R., Zhang, W., & Yu, Y. H. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40044-2 · Cited by 19 · see citing works
Kumar, R., Prakash, R., Kumar, N., Chinara, S., & Mohammed, M. A. (2025) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-025-10436-z · Cited by 14 · see citing works
Dang, Y., Liu, Y., Yang, E., Guo, G., Jiang, L., Zhao, J., & Wang, X. (2024) · ACM Transactions on Information Systems · doi:10.1145/3706633 · Cited by 12 · see citing works
Generative Adversarial Networks Unlearning
Sun, H., Zhu, T., Chang, W., & Zhou, W. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3564992 · Cited by 10 · see citing works
MultiDelete for Multimodal Machine Unlearning
Cheng, J., & Amiri, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72940-9_10 · Cited by 11 · see citing works
When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?
Wang, S., Zhu, T., Ye, D., & Zhou, W. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3620832 · Cited by 6 · see citing works
A survey on large language models unlearning: taxonomy, evaluations, and future directions
Le-Khac, U. N., & Truong, V. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11376-7 · Cited by 9 · see citing works
Privacy dilemmas and opportunities in large language models: a brief review
Li, H., Ye, J., & Wu, J. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-024-40583-8 · Cited by 6 · see citing works
Unlearning in Large Language Models: We Are Not There Yet
Blanco-Justicia, A., Domingo‐Ferrer, J., Jebreel, N., Manzanares-Salor, B., & Sánchez, D. (2025) · Computer · doi:10.1109/mc.2024.3468588 · Cited by 4 · see citing works
Protecting LLMs against Privacy Attacks While Preserving Utility
Dhingra, G., Sood, S., Wase, Z. M., Bahga, A., & Madisetti, V. K. (2024) · Journal of Information Security · doi:10.4236/jis.2024.154026 · Cited by 3 · see citing works
Evaluating Large Language Models in Psychological Research: A Guide for ReviewersPreprint
Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2024) · doi:10.31234/osf.io/ag7hy · Cited by 2 · see citing works
Patient-Centered and Practical Privacy to Support AI for Healthcare
Liu, R., Lee, H. K., Bhavani, S. V., Jiang, X., Lucila Ohno-machado, A., & Xiong, L. (2024) · IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa62245.2024.00038 · Cited by 1 · see citing works
Multi-Objective Large Language Model Unlearning
Pan, Z.-B., Zhang, S.-W., Zheng, Y., Li, C.-R., Cheng, Y.-H., & Zhao, J.-H. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp49660.2025.10889776 · Cited by 15 · see citing works
Logits-Level Balanced Machine Unlearning for LLM-Based Recommendation System
Tan, C., Li, X., Qu, Y., Chen, C., Cui, S., & Gao, L. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3660137 · Cited by 1 · see citing works
Malicious Forgetting: Backdoor Injection in Active Federated Unlearning and Countermeasure Design
Zhao, W., Peng, Y., Li, X., Xu, J., Liu, Y., & Lu, Z. (2026) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom59046.2026.11571403 · Cited by 1 · see citing works
The AI Hippocampus: How Far are We From Human Memory?
Jia, Z., Li, J., Kang, Y., Wang, Y., Wu, T., Wang, Q., Wang, X., Zhang, S., Shen, J., Li, Q., et al. (2026) · arXiv · doi:10.48550/arxiv.2601.09113
Gradual Negative Matching for LLM Unlearning
Kulkarni, H., Goharian, N., & Frieder, O. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88714-7_16
Token-Level Knowledge Unlearning for Large Language Model Security
Song, X. Y., & Liao, Y. (2025) · Lecture notes in electrical engineering · doi:10.1007/978-981-96-4245-8_20
Few-Shot Unlearning for Large Language Models Through Pretrained Soft Prompts
Tang, Z. H., & Yeh, M.-Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8186-0_14
Robust Unlearning for Large Language Models
Gu, K., Rashid, M. R. U., Sultana, N., & Mehnaz, S. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8186-0_12
EMMU: Efficient Information-Level Multimodal Machine Unlearning with High Model Fidelity
Zhang, J., Hou, J., Xiao, T., Huang, Y., & Li, X.-Y. (2025) · International Workshop on Quality of Service · doi:10.1109/iwqos65803.2025.11143267
Rethinking Learning: The Role of Unlearning in Generative AI-Based Conceptual Modeling
Sakib, S. K., Liddle, S. W., Lynch, C. J., Agrawal, A., & Giabbanelli, P. J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-08623-5_2
GRAIL: Gradient-Based Adaptive Unlearning for Privacy and Copyright in LLMs
Kim, K., Park, J., Han, J.-M., & Lee, S.-W. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11229073
An Effective Approach to Class-Wise Unlearning in Pre-trained Encoders for Contrastive Learning
Yin, C., Fang, L., & Zhou, L. Q. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228327
Copyright Protection of General Information via Simulation Task Supervision
Hu, C., Hu, Y., Li, H., Hu, Y., & Zhu, N. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3638667
CTIGuardian: A Few-Shot Framework for Mitigating Privacy Leakage in Fine-Tuned LLMs
Arachchige, S. D. B., Zhao, B. Z. H., Asghar, H. J., Vatsalan, D., & Kâafar, M. A. (2025) · Annual Computer Security Applications Conference Workshops (ACSAC Workshops) · doi:10.1109/acsacw69556.2025.00060
A Survey on Continuous Unlearning in Generative AI: Approaches and Tradeoffs
Zhao, Y., Du, H., Lin, Y., Xiang, K., Niyato, D., & Poor, H. V. (2025) · IEEE Intelligent Systems · doi:10.1109/mis.2025.3616192
FADE: Progressive Unlearning for Language Models via Adversarial Disruption and Editing
Chen, Y., & Tang, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5637-3_7
Fine-Tuning Large Language Models for Vocabulary-Constrained Language Learning
Wienkamp, J., & Schilling, M. (2025) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11391123
Customized Retrieval-Augmented Generation with LLM for Debiasing Recommendation Unlearning
Zhang, H., Zhang, C., Hu, P., Qiu, S., & Wang, J. (2025) · Proceedings · doi:10.1109/icdm65498.2025.00183
Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models
Tang, H., Liu, Y., Zhao, X., Liu, X., Zhang, Y., Zhang, K., Zhou, X., & Chen, E. (2025) · Proceedings · doi:10.1109/icdm65498.2025.00082 · Cited by 1 · see citing works
Can LLMs Mimic Household Surveys?: From Representative Agents to Population DistributionsPreprint
Dalloul, A., & Pfeifer, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6703798
An LLM-based methodology for the automatic detection of bias in the DuoWikiBias corpus
Salas-jimenez, K., Ojeda-Trueba, S.-L., Bel-Enguix, G., Lee-Romero, E., & Lopez-Ponce, F. F. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1791624
Forget Data-Only Optimization for Unlearning
Wang, Y. (2026) · Synthesis lectures on computer vision · doi:10.1007/978-3-032-17282-2_4
Mitigating Security Risks in Large Language Models: A Full Lifecycle Perspective
Wang, Y. K., Bai, Z., Huo, J., Wang, B., Liu, J., Cao, H., Meng, F., Yang, X., & Gao, Y. (2026) · Machine Learning · doi:10.1007/s10994-025-06930-x
Towards refined unlearning: Utilizing the retain knowledge subspace and noise reduction
Kim, E., Choi, Y. J., & Cho, Y.-S. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116284
Efficient and High Fidelity Multimodal Machine Unlearning for Information With Deep Memory
Zhang, J., Hou, J., Zhang, H., Xiao, T., & Huang, Y. (2026) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2026.3706840
MU4Sum: Aspect Unlearning in LLM-based Document Summarization
Gallipoli, G., & Cagliero, L. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3828668
UnPII: Unlearning Personally Identifiable Information with Quantifiable Exposure Risk
Jeon, I., Kwon, Y., & Koo, H. (2026) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3786583.3786860
Forgetting Similar Samples: Can Machine Unlearning Do it Better?
Xu, H., Zhu, T., Ye, D., Zhang, L., Wang, L., & Zhou, W. (2026) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2026.3714774
From Data Mining to Copyright Infringement: Legal Challenges in Training Artificial Intelligence
Badkul, S. (2026) · Kutafin Law Review · doi:10.17803/2713-0533.2026.2.36.237-268
- Authors
- Chen, J., & Yang, D.
- Affiliations
- Georgia Institute of Technology (US)
- Stanford University (US)
- Venue
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Published
- 2023
- Pages
- 12041-12052
- OpenAlex
- W4389519496
- Cited by
- 52
- References
- 47
- Citations by year
- 2026: 13 · 2025: 31 · 2024: 8
- FWCI
- 9.21
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (2247357, 2308994)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- large language models, data deletion, privacy, data protection, fusion mechanism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
