CoChat
Sign Up
PreprintCited by 76Topic ModelingDomain Adaptation and Few-Shot LearningNatural Language Processing Techniques

Mass-Editing Memory in a Transformer

Open access
Meng, Kevin, Sharma, Arnab Sen, Andonian, Alex, Belinkov, Yonatan, Bau, David·2022·arXiv
DOI 10.48550/arxiv.2210.07229arXiv 2210.07229
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent work has shown exciting promise in updating large language models with new memories, so as to replace obsolete information or add specialized knowledge. However, this line of work is predominantly limited to updating single associations. We develop MEMIT, a method for directly updating a language model with many memories, demonstrating experimentally that it can scale up to thousands of associations for GPT-J (6B) and GPT-NeoX (20B), exceeding prior work by orders of magnitude. Our code and data are at https://memit.baulab.info.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2210.07229

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • ROME introduced causal tracing to identify key layers and applied rank-one updates to MLP weights, while MEMIT extended this to mass editing by applying multiple rank-one updates sequentially.

    “ROME [6] introduced causal tracing to identify key layers and applied rank-one updates to MLP weights, while MEMIT [7] extended this to mass editing by applying multiple rank-one updates sequentially.”

    From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [7]

    The Introduction (p. 1) says MEMIT builds on ROME and scales to thousands of edits. But Algorithm 1 (Section 4.3, p. 6) shows a batch update for all requested edits at each layer, followed by progression through the layers—not multiple rank-one updates applied sequentially. Appendix B.4 (p. 15) also contrasts MEMIT's parallel insertion with ROME's iterative updates. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • These methods compute rank-one updates that modify model behavior for specific factual associations while preserving performance on unrelated tasks.

    “These methods compute rank-one updates that modify model behavior for specific factual associations while preserving performance on unrelated tasks.”

    From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [7]

    The Introduction (p. 1) says MEMIT edits factual memories while maintaining specificity, and §5.2.1 (p. 7) evaluates whether unrelated facts remain unchanged. But §4.2 (p. 4) describes MEMIT as a batch update for many associations, contrasting it with an earlier single-association method; the paper does not characterize MEMIT's updates as rank-one or establish preservation of performance on unrelated tasks broadly. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Following prior work, mid-to-late MLP layers are targeted where factual knowledge is most directly encoded.

    “Following prior work [6,7], we target mid-to-late MLP layers where factual knowledge is most directly encoded.”

    From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [7]

    Section 4.1 (p. 3) identifies MLP layers 3–8 in GPT-J as critical to factual recall, supporting the choice to target MLP layers. But Figure 3 calls these mid-layer MLPs, and Appendix F.1 (p. 19) reports that edits to late-layer MLPs perform considerably worse. The source therefore does not support extending the target range to late layers. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Current LLMs primarily rely on parametric memory (knowledge encoded in model weights) and ephemeral activation memory (context-limited runtime states).

    “They primarily rely on parametric memory (knowledge encoded in model weights) and ephemeral activation memory (context-limited runtime states).”

    From MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models (2025) · cites this paper as [20]

    Section 3 (p. 2) explicitly describes factual associations stored in LLM parameters, and Section 4 (p. 3) describes prompt-dependent hidden states. The paper does not characterize those states as ephemeral, context-limited memory or say that current LLMs primarily rely on these two forms of memory. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • The potential difficulty of removing memories that are tightly integrated into an LLM is demonstrated by the body of work done on 'machine unlearning'.

    “The potential difficulty of removing memories that are tightly integrated into an LLM is demonstrated by the body of work done on 'machine unlearning' [114]-[116], [124][126].”

    From Episodic Memory in AI Agents Poses Risks that Should be Studied and Mitigated (2025) · cites this paper as [116]

    The Preliminaries (p. 3) say that editing factual memories while preserving generalization, specificity, and fluency is challenging, even for a few edits. The paper studies updating or replacing factual associations; it does not establish the difficulty of removing tightly integrated memories through machine unlearning. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

Cited by 76

Research works printed but not linked: 44

  1. Oshin Agarwal and Ani Nenkova. Temporal effects on pre-trained models for language processing tasks. Transactions of the Association for Computational Linguistics , 10:904-921, 2022.
  2. Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. A review on language models as knowledge bases. arXiv preprint arXiv:2204.06031 , 2022. · arXiv
  3. James A Anderson. A simple neural network generating an interactive memory. Mathematical biosciences , 14(3-4):197-220, 1972.
  4. Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. Dbpedia: A nucleus for a web of open data. In The semantic web , pp. 722-735. Springer, 2007.
  5. Kurt Bollacker, Robert Cook, and Patrick Tufts. Freebase: A shared database of structured general human knowledge. In AAAI , volume 7, pp. 1962-1963, 2007.
  6. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. Comet: Commonsense transformers for automatic knowledge graph construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pp. 4762-4779, 2019.
  7. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems , volume 33, pp. 1877-1901, 2020.
  8. Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka, and Tom M Mitchell. Toward an architecture for never-ending language learning. In Twenty-Fourth AAAI conference on artificial intelligence , 2010.
  9. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 , 2022. · arXiv
  10. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 8493-8502, 2022.
  11. Guy Dar, Mor Geva, Ankit Gupta, and Jonathan Berant. Analyzing transformers in embedding space. arXiv preprint arXiv:2209.02535 , 2022. · arXiv
  12. Randall Davis, Howard Shrobe, and Peter Szolovits. What is a knowledge representation? AI magazine , 14(1):17-17, 1993.
  13. Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pp. 6491-6506, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
  14. Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining , pp. 601-610, 2014.
  15. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics , 9:1012-1031, 2021.
  16. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pp. 5484-5495, 2021.
  17. Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. arXiv preprint arXiv:2203.14680 , 2022. · arXiv
  18. Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, and Srinivasan Iyer. Do language models have beliefs? methods for detecting, updating, and visualizing model beliefs. arXiv preprint arXiv:2111.13654 , 2021. · arXiv
  19. Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Computational Linguistics , 8:423-438, 2020.
  20. Teuvo Kohonen. Correlation matrix memories. IEEE transactions on computers , 100(4):353-359, 1972.
  21. Angeliki Lazaridou, Adhi Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tomas Kocisky, Sebastian Ruder, et al. Mind the gap: Assessing temporal generalization in neural language models. Advances in Neural Information Processing Systems , 34:29348-29363, 2021.
  22. Douglas B Lenat. Cyc: A large-scale investment in knowledge infrastructure. Communications of the ACM , 38(11):33-38, 1995.
  23. Adam Liska, Tomas Kocisky, Elena Gribovskaya, Tayfun Terzi, Eren Sezener, Devang Agrawal, D'Autume Cyprien De Masson, Tim Scholtes, Manzil Zaheer, Susannah Young, et al. StreamingQA: A benchmark for adaptation to new knowledge over time in question answering models. In International Conference on Machine Learning , pp. 13604-13622. PMLR, 2022.
  24. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems , 35, 2022.
  25. George A Miller. Wordnet: a lexical database for english. Communications of the ACM , 38(11): 39-41, 1995.
  26. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Memorybased model editing at scale. In International Conference on Machine Learning , 2022.
  27. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32, 2019.
  28. David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350 , 2021. · arXiv
  29. Judea Pearl. Direct and indirect effects. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence , pp. 411-420, 2001.
  30. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pp. 2463-2473, 2019.
  31. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog , pp. 9, 2019.
  32. Richard H Richens. Preprogramming for mechanical translation. Mechanical Translation , 3(1): 20-25, 1956.
  33. Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pp. 5418-5426, 2020.
  34. Tara Safavi and Danai Koutra. Relational world knowledge representation in contextual language models: A review. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pp. 1053-1067, 2021.
  35. Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pp. 4222-4235, 2020.
  36. Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitry Pyrkin, Sergei Popov, and Artem Babenko. Editable neural networks. In International Conference on Learning Representations , 2019.
  37. Gilbert Strang. Introduction to linear algebra . Wellesley-Cambridge Press Wellesley, MA, 1993.
  38. Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. Yago: a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web , pp. 697-706, 2007.
  39. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems , pp. 5998-6008, 2017.
  40. Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart M Shieber. Investigating gender bias in language models using causal mediation analysis. In NeurIPS , 2020.
  41. Denny Vrandeˇ ci´ c and Markus Krötzsch. Wikidata: a free collaborative knowledgebase. Communications of the ACM , 57(10):78-85, 2014.
  42. Gerhard Weikum. Knowledge graphs 2021: a data odyssey. Proceedings of the VLDB Endowment , 14(12):3233-3238, 2021.
  43. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 , 2019. · arXiv
  44. Yunzhi Yao, Shaohan Huang, Li Dong, Furu Wei, Huajun Chen, and Ningyu Zhang. Kformer: Knowledge injection in transformer feed-forward layers. In CCF International Conference on Natural Language Processing and Chinese Computing , pp. 131-143. Springer, 2022.

Authors
Meng, K., Sharma, A. S., Andonian, A., Belinkov, Y., & Bau, D.
Affiliations
  • Northeastern University (US)
  • MIT Computer Science and Artificial Intelligence Laboratory (US)
  • Technion – Israel Institute of Technology (IL)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2210.07229
arXiv
2210.07229
OpenAlex
W4306313145
Cited by
76
References
0
Citations by year
2026: 4 · 2025: 17 · 2024: 18 · 2023: 35 · 2022: 1 · 2021: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Nvidia (A6000)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
memory editing, large language models, GPT-J, knowledge injection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review