Who's Harry Potter? Approximate Unlearning in LLMs
Open accessAbstract
Large language models (LLMs) are trained on massive internet corpora that often contain copyrighted content. This poses legal and ethical challenges for the developers and users of these models, as well as the original authors and publishers. In this paper, we propose a novel technique for unlearning a subset of the training data from a LLM, without having to retrain it from scratch. We evaluate our technique on the task of unlearning the Harry Potter books from the Llama2-7b model (a generative language model recently open-sourced by Meta). While the model took over 184K GPU-hours to pretrain, we show that in about 1 GPU hour of finetuning, we effectively erase the model's ability to generate or recall Harry Potter-related content, while its performance on common benchmarks (such as Winogrande, Hellaswag, arc, boolq and piqa) remains almost unaffected. We make our fine-tuned model publicly available on HuggingFace for community evaluation. To the best of our knowledge, this is the first paper to present an effective technique for unlearning in generative language models. Our technique consists of three main components: First, we use a reinforced model that is further trained on the target data to identify the tokens that are most related to the unlearning target, by comparing its logits with those of a baseline model. Second, we replace idiosyncratic expressions in the target data with generic counterparts, and leverage the model's own predictions to generate alternative labels for every token. These labels aim to approximate the next-token predictions of a model that has not been trained on the target data. Third, we finetune the model on these alternative labels, which effectively erases the original text from the model's memory whenever it is prompted with its context.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most surveyed papers leverage the predefined performance metric of the forgetting dataset, that is, accuracy in a classification or completion dataset.
“Most papers (Eldan and Russinovich 2023; Jang et al 2022; Yao et al 2023; Pawelczyk et al 2023; Chen and Yang 2023; Ni et al 2023; Pochinkov and Schoots 2023 ) leverage the predefined performance metric of the forgetting dataset, that is, the accuracy in a classification or completion dataset.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Eldan and Russinovich 2023
Section 3.2.1 and Additional information (pages not provided) describe evaluating forgetting through completions of 300 prompts. But the paper uses a custom, weighted familiarity score rather than a predefined accuracy metric for a forgetting dataset. Section 3.1 uses established benchmarks to assess preservation of general capabilities, not forgetting. (LLM, conf=0.86)
Experimental — accuracy not yet measuredSome researchers prefer a second, more controllable but less realistic scenario, where forgetting is applied to an LLM fine-tuned with a dataset that includes the undesired knowledge.
“Due to these limitations, some researchers, such as Eldan and Russinovich (2023); Chen and Yang (2023); Borkar (2023); Wu et al (2023); Wang et al (2023) prefer a second, more controllable but less realistic scenario, where forgetting is applied to an LLM fine-tuned with a dataset that includes the undesired knowledge.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Eldan and Russinovich 2023
The Results section (page unavailable) describes a controlled Phi-1.5 setting in which the authors added the unlearning target to the data used to obtain a baseline model. But the same section says they also tested Llama and presented its results, while noting that they lacked access to its original training data. The paper therefore does not support characterizing the controlled setting as their preferred scenario, or calling it less realistic. (LLM, conf=0.86)
Experimental — accuracy not yet measuredEldan and Russinovich (2023) demonstrated that their unlearned model no longer generates or recalls content related to the target data, while its performance on common benchmarks remains nearly unaffected.
“The authors demonstrated that the unlearned model no longer generates or recalls content related to the target data, while its performance on common benchmarks remains nearly unaffected.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Eldan and Russinovich (2023)
Gradient ascent can cause the model to lose its understanding of the language.
“However, GA can cause the model to lose its understanding of the language (Eldan and Russinovich 2023 ).”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Eldan and Russinovich 2023
Several researchers choose widely recognized benchmarks for LLMs to evaluate the preservation of overall capabilities.
“Several researchers choose widely recognized benchmarks for LLMs (Eldan and Russinovich 2023; Jang et al 2022; Yao et al 2023; Pawelczyk et al 2023; Chen and Yang 2023) to evaluate the preservation of overall capabilities.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Eldan and Russinovich 2023
Eldan and Russinovich (2023) experiment with forgetting a copyrighted corpus (Harry Potter books) as a concept.
“On the other hand, Eldan and Russinovich (2023) experiment with forgetting a copyrighted corpus (Harry Potter books) as a concept.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Eldan and Russinovich (2023)
Eldan and Russinovich (2023), Wang et al (2023), and Jang et al (2022) leverage well-established LLM benchmarks to measure retained utility.
“General benchmarks . Eldan and Russinovich (2023), Wang et al (2023), and Jang et al (2022) leverage well-established LLM benchmarks, such as those used for comparing LLMs (Sakaguchi et al 2020; Zellers et al 2019; Paperno et al 2016; Bisk et al 2020).”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Eldan and Russinovich (2023)
Epoch information is only reported by a subset of works and can only be used when comparing identical models, data and forgetting tasks, which is rarely the case.
“However, this information is only reported by a subset of works (Eldan and Russinovich 2023; Chen and Yang 2023; Jang et al 2022; Yao et al 2023; Yu et al 2023; Wu et al 2023; Ni et al 2023; Ilharco et al 2022) and it can only be used when comparing identical models, data and forgetting tasks, which is rarely the case.”
From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Eldan and Russinovich 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
Cited by 19
Research works printed but not linked: 13
- Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Y Chai, Mirella Lapata, Angeliki Lazaridou, Ryan J Maynez, Piyush Narang, et al. Piqa: Reasoning about physical commonsense in natural language. arXiv preprint arXiv:1911.11641, 2019. · arXiv
- Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2924-2936, 2019.
- Yiwen Jiang, Shenglong Liu, Tao Zhao, Wei Li, and Xianzhou Gao. Machine un- learning survey. In Fifth International Conference on Mechatronics and Computer Technology Engineering (MCTE 2022), volume 12500, pages 1596-1603. SPIE, 2022.
- Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint arXiv:2210.01504, 2022. · arXiv
- Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018. · arXiv
- Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. A survey of machine unlearning. arXiv preprint arXiv:2209.02299, 2022. · arXiv
- J.K. Rowling. The Harry Potter Series. Bloomsbury, 1997-2007. Comprising: Harry Potter and the Philosopher's Stone; Harry Potter and the Chamber of Secrets; Harry Potter and the Prisoner of Azkaban; Harry Potter and the Goblet of Fire; Harry Potter and the Order of the Phoenix; Harry Potter and the Half-Blood Prince; Harry Potter and the Deathly Hallows.
- Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Wino- grande: An adversarial winograd schema challenge at scale. arXiv preprint arXiv:1907.10641, 2019. · arXiv
- Lingzhi Wang, Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fai Wong, and Hongzhi Yin. Kga: A general machine unlearning framework based on knowledge gap align- ment. arXiv preprint arXiv:2305.06535, 2023. · arXiv
- Vikas Yadav, Steven Bethard, and Mihai Surdeanu. Quick and (not so) dirty: Unsu- pervised selection of justification sentences for multi-hop question answering. arXiv preprint arXiv:1911.07176, 2019. · arXiv
- Dawen Zhang, Pamela Finckenberg-Broman, Thong Hoang, Shidong Pan, Zhenchang Xing, Mark Staples, and Xiwei Xu. Right to be forgotten in the era of large language models: Implications, challenges, and solutions. arXiv preprint arXiv:2307.03941, 2023. · arXiv
- Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791-4800, 2019.
- Haibo Zhang, Toru Nakamura, Takamasa Isohara, and Kouichi Sakurai. A review on machine unlearning. SN Computer Science, 4(4):337, 2023.
- Authors
- Eldan, R., & Russinovich, M.
- Affiliations
- Microsoft (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2310.02238
- OpenAlex
- W4387389584
- Cited by
- 19
- References
- 0
- Citations by year
- 2026: 2 · 2025: 12 · 2024: 2 · 2023: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- unlearning, large language models, copyright, generative language models, Llama 2, fine-tuning, data deletion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
