CoChat
Sign Up
PreprintCited by 51Topic ModelingDomain Adaptation and Few-Shot LearningAdversarial Robustness in Machine Learning

Memory-Based Model Editing at Scale

Open access
Mitchell, Eric, Lin, Charles, Bosselut, Antoine, Manning, Christopher D., Finn, Chelsea·2022·arXiv
DOI 10.48550/arxiv.2206.06520arXiv 2206.06520
Open this paper in CoChat — free accountOpen PDF

Abstract

Even the largest neural networks make errors, and once-correct predictions can become invalid as the world changes. Model editors make local updates to the behavior of base (pre-trained) models to inject updated knowledge or correct undesirable behaviors. Existing model editors have shown promise, but also suffer from insufficient expressiveness: they struggle to accurately model an edit's intended scope (examples affected by the edit), leading to inaccurate predictions for test inputs loosely related to the edit, and they often fail altogether after many edits. As a higher-capacity alternative, we propose Semi-Parametric Editing with a Retrieval-Augmented Counterfactual Model (SERAC), which stores edits in an explicit memory and learns to reason over them to modulate the base model's predictions as needed. To enable more rigorous evaluation of model editors, we introduce three challenging language model editing problems based on question answering, fact-checking, and dialogue generation. We find that only SERAC achieves high performance on all three problems, consistently outperforming existing approaches to model editing by a significant margin. Code, data, and additional project information will be made available at https://sites.google.com/view/serac-editing.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2206.06520

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Preserving model parameters methods usually have poor generalization, and the required additional content increases significantly with the number of editing.

    “However, these methods usually have poor generalization, and the required additional content also increases significantly with the number of editing.”

    From Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models (2024) · cites this paper as Mitchell et al. 2022

    The Discussion (p. 9) says SERAC’s edit memory may grow without bound under continuous editing, supporting the claim that preserving model parameters can require increasing additional storage. But the paper reports strong performance for SERAC, which leaves base-model parameters unchanged; its concerns about poor edit generalization chiefly concern existing parameter-updating methods (Introduction, p. 2; Model Editing Benchmarking, p. 7). It does not establish that parameter-preserving methods usually generalize poorly. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • SERAC can be used to unlearn undesirable behaviors of LLMs and replace them with desirable ones.

    “Although SERAC was evaluated on editing question answering, fact-checking, and dialogue generation, it can also be used to unlearn undesirable behaviors of LLMs and replace them with desirable ones.”

    From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Mitchell et al 2022

    Appendix A (p. 13) shows SERAC changing a dialogue model’s negative responses about vaccines into positive ones, supporting the claim that it can replace a targeted behavior. But Section 3.1 (p. 3) says SERAC overrides the base model only for inputs within an edit’s scope; it does not unlearn the behavior from the base model. The paper does not establish this ability for undesirable LLM behaviors generally. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • SERAC utilizes a memory-based model editing approach that treats an LLM as a black-box.

    “SERAC (Mitchell et al 2022) utilizes a memory-based model editing approach that treats an LLM as a black-box.”

    From Digital forgetting in large language models: a survey of unlearning methods (2025) · cites this paper as Mitchell et al 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • Zero-Shot Relation Extraction via Reading Comprehension

    Levy, O., Seo, M., Choi, E., & Zettlemoyer, L. (2017) · Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017) · doi:10.18653/v1/k17-1034 · Cited by 482 · cited 2× · see citing works

    “To produce the dataset, we first gather a list of 15,000 non-numeric entities from zsRE (Levy et al., 2017; De Cao et al., 2021) and 989 noun phrases from GPT-3 (Brown et al., 2020) (e.g., GHOST HUNTING ) for a total of 15,989 topics.”
    Method

Flagged first, then by citations

Research works printed but not linked: 48

  1. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. Neural Information Processing Systems , 2020.
  2. Buzzega, P., Boschini, M., Porrello, A., Abati, D., and Calderara, S. Dark experience for general continual learning: a strong, simple baseline. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.), Advances in Neural Information Processing Systems , volume 33, pp. 15920-15930. Curran Associates, Inc., 2020.
  3. Chen, D., Fisch, A., Weston, J., and Bordes, A. Reading wikipedia to answer open-domain questions. In ACL , 2017.
  4. Csord´ as, R., van Steenkiste, S., and Schmidhuber, J. Are neural nets modular? Inspecting functional modularity through differentiable weight masks. In International Conference on Learning Representations , 2021. URL https://openreview.net/forum? id=7uVcpu-gMD .
  5. Dai, D., Dong, L., Hao, Y., Sui, Z., and Wei, F. Knowledge neurons in pretrained transformers. CoRR , abs/2104.08696, 2021. URL https://arxiv.org/ abs/2104.08696 . · arXiv
  6. De Cao, N., Aziz, W., and Titov, I. Editing factual knowledge in language models. ArXiv , abs/2104.08164, 2021. · arXiv
  7. Demszky, D., Guu, K., and Liang, P. Transforming question answering datasets into natural language inference datasets. CoRR , abs/1809.02922, 2018. URL http: //arxiv.org/abs/1809.02922 . · arXiv
  8. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/ v1/N19-1423. URL https://www.aclweb.org/ anthology/N19-1423 .
  9. Graves, A., Liwicki, M., Fern´ andez, S., Bertolami, R., Bunke, H., and Schmidhuber, J. A novel connectionist system for unconstrained handwriting recognition. IEEE transactions on pattern analysis and machine intelligence , 31(5):855-868, 2008.
  10. Graves, A., Wayne, G., and Danihelka, I. Neural turing machines, 2014. URL http://arxiv.org/abs/ 1410.5401 . arxiv:1410.5401. · arXiv
  11. Hase, P., Diab, M., Celikyilmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., and Iyer, S. Do language models have beliefs? Methods for detecting, updating, and visualizing model beliefs, 2021. arxiv:2111.13654. · arXiv
  12. Heitmann, M., Siebert, C., Hartmann, J., and Schamp, C. More than a feeling: Benchmarks for sentiment analysis accuracy. Available at SSRN 3489963 , 2020.
  13. Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural computation , 9(8):1735-1780, 1997.
  14. Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pp. 6769-6781, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/ 2020.emnlp-main.550. URL https://www.aclweb. org/anthology/2020.emnlp-main.550 . · DOI
  15. Kassner, N., Tafjord, O., Sch¨ utze, H., and Clark, P. BeliefBank: Adding memory to a pre-trained language model for a systematic notion of belief. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pp. 8849-8861, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.697. URL https:// aclanthology.org/2021.emnlp-main.697 . · DOI
  16. Khandelwal, U., Levy, O., Jurafsky, D., Zettlemoyer, L., and Lewis, M. Generalization through memorization: Nearest neighbor language models. In ICLR , 2020.
  17. Khattab, O. and Zaharia, M. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval , pp. 39-48, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450380164. URL https://doi.org/10. 1145/3397271.3401075 .
  18. Koch, G., Zemel, R., Salakhutdinov, R., et al. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop , volume 2. Lille, 2015.
  19. Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Kelcey, M., Devlin, J., Lee, K., Toutanova, K. N., Jones, L., Chang, M.-W., Dai, A., Uszkoreit, J., Le, Q., and Petrov, S. Natural questions: a benchmark for question answering research. Transactions of the Association of Computational Linguistics , 2019.
  20. Lazaridou, A., Kuncoro, A., Gribovskaya, E., Agrawal, D., Liska, A., Terzi, T., Gimenez, M., de Masson d'Autume, C., Ruder, S., Yogatama, D., Cao, K., Kocisk´ y, T., Young, S., and Blunsom, P. Mind the gap: Assessing temporal generalization in neural language models. In NeurIPS , 2021.
  21. Lee, K., Chang, M.-W., and Toutanova, K. Latent retrieval for weakly supervised open domain question answering. In ACL , 2019.
  22. Levy, O., Seo, M., Choi, E., and Zettlemoyer, L. Zeroshot relation extraction via reading comprehension. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017) , pp. 333-342, Vancouver, Canada, August 2017. Association for Computational Linguistics. doi: 10.18653/ v1/K17-1034. URL https://www.aclweb.org/ anthology/K17-1034 .
  23. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K¨ uttler, H., Lewis, M., Yih, W.-t., Rockt¨ aschel, T., Riedel, S., and Kiela, D. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.), Advances in Neural Information Processing Systems , volume 33, pp. 9459-9474. Curran Associates, Inc., 2020. URL https://proceedings. neurips.cc/paper/2020/file/ 6b493230205f780e1bc26945df7481e5-Paper. pdf .
  24. Li, M., Roller, S., Kulikov, I., Welleck, S., Boureau, Y.-L., Cho, K., and Weston, J. Don't say that! making inconsistent dialogue unlikely with unlikelihood training. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pp. 4715-4728, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.428. URL https: //aclanthology.org/2020.acl-main.428 . · DOI
  25. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. Roberta: A robustly optimized BERT pretraining approach. CoRR , abs/1907.11692, 2019. URL http://arxiv.org/ abs/1907.11692 . · arXiv
  26. U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017. URL https://proceedings. neurips.cc/paper/2017/file/ f87522788a2be2d171666752f97ddebb-Paper. pdf .
  27. Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Locating and editing factual associations in GPT, 2022. arXiv:2202.05262. · arXiv
  28. Mitchell, E., Lin, C., Bosselut, A., Finn, C., and Manning, C. D. Fast model editing at scale. CoRR , 2021. URL https://arxiv.org/pdf/2110.11309.pdf . · arXiv
  29. Oh, J., Chockalingam, V., Lee, H., et al. Control of memory, active perception, and action in minecraft. In International Conference on Machine Learning , pp. 2790-2799. PMLR, 2016.
  30. Paranjape, A., Khattab, O., Potts, C., Zaharia, M. A., and Manning, C. D. Hindsight: Posterior-guided training of retrievers for improved open-ended generation. ArXiv , abs/2110.07752, 2021. · arXiv
  31. Pritzel, A., Uria, B., Srinivasan, S., Badia, A. P., Vinyals, O., Hassabis, D., Wierstra, D., and Blundell, C. Neural episodic control. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research , pp. 2827-2836. PMLR, 06-11 Aug 2017. URL https://proceedings.mlr.press/ v70/pritzel17a.html .
  32. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21(140):1-67, 2020. URL http://jmlr. org/papers/v21/20-074.html .
  33. Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, 11 2019. URL https://arxiv.org/ abs/1908.10084 . · arXiv
  34. Ribeiro, M. T., Guestrin, C., and Singh, S. Are red roses red? evaluating consistency of question-answering models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pp. 6174-6184, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1621. URL https://aclanthology.org/P19-1621 . · DOI
  35. Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., and Weston, J. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , pp. 300-325, Online, April 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.eacl-main.24. URL https: //aclanthology.org/2021.eacl-main.24 . · DOI · Paper page
  36. Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., and Wayne, G. Experience replay for continual learning. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alch´ e-Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings. neurips.cc/paper/2019/file/ fa7cdfad1a5aaf8370ebeda47a1ff1c3-Paper. pdf .
  37. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv , abs/1910.01108, 2019. · arXiv
  38. Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., and Zaharia, M. Colbertv2: Effective and efficient retrieval via lightweight late interaction. CoRR , abs/2112.01488, 2021. URL https://arxiv.org/ abs/2112.01488 . · arXiv
  39. Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., and Lillicrap, T. Meta-learning with memory-augmented neural networks. In Balcan, M. F. and Weinberger, K. Q. (eds.), Proceedings of The 33rd International Conference on Machine Learning , volume 48 of Proceedings of Machine Learning Research , pp. 1842-1850, New York, New York, USA, 20-22 Jun 2016. PMLR. URL https://proceedings.mlr.press/v48/ santoro16.html .
  40. Schuster, T., Fisch, A., and Barzilay, R. Get your vitamin C! robust fact verification with contrastive evidence. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pp. 624-643, Online, June 2021. Association for Computational Linguistics. URL https://www.aclweb. org/anthology/2021.naacl-main.52 .
  41. Shan, S., Li, Y., and Oliva, J. B. Meta-neighborhoods. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.), Advances in Neural Information Processing Systems , volume 33, pp. 5047-5057. Curran Associates, Inc., 2020. URL https://proceedings. neurips.cc/paper/2020/file/ 35464c848f410e55a13bb9d78e7fddd0-Paper. pdf .
  42. Sinitsin, A., Plokhotnyuk, V., Pyrkin, D., Popov, S., and Babenko, A. Editable neural networks. In ICLR , 2020. URL https://openreview.net/forum? id=HJedXaEtvS .
  43. Snell, J., Swersky, K., and Zemel, R. S. Prototypical networks for few-shot learning. arXiv preprint arXiv:1703.05175 , 2017. · arXiv
  44. Sotoudeh, M. and Thakur, A. V. Correcting deep neural networks with small, generalizing patches. In NeurIPS 2019 Workshop on Safety and Robustness in Decision Making , 2019.
  45. Thorne, J., Vlachos, A., Christodoulopoulos, C., and Mittal, A. FEVER: a large-scale dataset for fact extraction and VERification. In NAACL-HLT , 2018.
  46. Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al. Matching networks for one shot learning. Advances in neural information processing systems , 29:3630-3638, 2016.
  47. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., and Brew, J. Huggingface's transformers: State-of-the-art natural language processing. CoRR , abs/1910.03771, 2019. URL http://arxiv.org/ abs/1910.03771 . · arXiv
  48. Zhu, C., Rawat, A. S., Zaheer, M., Bhojanapalli, S., Li, D., Yu, F., and Kumar, S. Modifying memories in transformer models, 2020. URL https://arxiv.org/ abs/2012.00363 . · arXiv

Cited by

Sorted by citations

Authors
Mitchell, E., Lin, C., Bosselut, A., Manning, C. D., & Finn, C.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2206.06520
arXiv
2206.06520
OpenAlex
W4282980384
Cited by
51
References
1
Citations by year
2026: 2 · 2025: 11 · 2024: 15 · 2023: 23
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
model editing, counterfactual reasoning, seracs, language model fine-tuning, fact-checking, question answering, dialogue generation, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review