CoChat
Sign Up
Cited by 46Topic ModelingNatural Language Processing TechniquesAdversarial Robustness in Machine Learning

Large Language Models with Controllable Working Memory

Open access
Li, Daliang, Rawat, Ankit Singh, Zaheer, Manzil, Wang, Xin, Lukasik, Michal, Veit, Andreas, Yu, Felix, Kumar, Sanjiv·2023·Findings of the Association for Computational Linguistics: ACL 2023
DOI 10.18653/v1/2023.findings-acl.112
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP), partly owing to the massive amounts of world knowledge they memorize during pretraining.While many downstream applications provide the model with an informational context to aid its underlying task, how the model's world knowledge interacts with the factual information presented in the context remains under explored.As a desirable behavior, an LLM should give precedence to the context whenever it contains taskrelevant information that conflicts with the model's memorized knowledge.This enables model predictions to be grounded in the context, which then facilitates updating specific model predictions without frequently retraining the model.By contrast, when the context is irrelevant to the task, the model should ignore it and fall back on its internal knowledge.In this paper, we undertake a first joint study of the aforementioned two properties, namely controllability and robustness, in the context of LLMs.We demonstrate that state-of-theart T5 and PaLM models (both pretrained and finetuned) could exhibit low controllability and robustness that does not improve with increasing the model size.As a solution, we propose a simple yet effective method knowledge aware finetuning (KAFT) -to strengthen both controllability and robustness by injecting counterfactual and irrelevant contexts to standard supervised datasets.Our comprehensive evaluation showcases the utility of KAFT across model architectures and sizes.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2023.findings-acl.112

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The factual knowledge embedded within LLMs exhibits clear temporal boundaries and can become outdated over time.

    “The factual knowledge embedded within LLMs exhibits clear temporal boundaries and can become outdated over time [148, 166, 230].”

    From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [166]

    The Introduction says LLMs memorize factual knowledge in their parameters that could become outdated, and the “Knowledge update in language models” passage says outdated memory may lead to incorrect predictions. The paper links that knowledge to pretraining, but these passages do not establish the claim’s stronger assertion of clear temporal boundaries. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 34

  1. F. Gregory Ashby, Shawn W. Ell, Vivian V. Valentin, and Michael B. Casale. 2005. FROST: A Dis- tributed Neurocomputational Model of Working Memory Maintenance. Journal of Cognitive Neuro- science, 17(11):1728-1743. · DOI
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language under- standing. arXiv preprint arXiv:1810.04805. · arXiv
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of NAACL, pages 4171- 4186.
  4. Bhuwan Dhingra, Jeremy R Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W Cohen. 2022. Time-aware language models as temporal knowledge bases. Transactions of the Association for Computational Linguistics, 10:257-273.
  5. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. T-REx: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh Interna- tional Conference on Language Resources and Eval- uation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
  6. Keisuke Fukuda and Geoffrey F. Woodman. 2017. Vi- sual working memory buffers information retrieved from visual long-term memory. Proceedings of the National Academy of Sciences, 114/20. · DOI
  7. J M Fuster. 1973. Unit activity in prefrontal cortex dur- ing delayed-response performance: neuronal corre- lates of transient memory. Journal of Neurophysiol- ogy, 36(1):61-78. PMID: 4196203. · DOI · Paper page
  8. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, pages 5484-5495, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. · DOI · Paper page
  9. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Ming-Wei Chang. 2020. Realm: Retrieval- augmented language model pre-training. · DOI · Paper page
  10. Jonathan Ho and Tim Salimans. 2022. Classifier-free diffusion guidance. · DOI · Paper page
  11. Zhiting Hu and Li Erran Li. 2022. A causal lens for controllable text generation. CoRR, abs/2201.09119. · arXiv
  12. Mandar Joshi, Kenton Lee, Yi Luan, and Kristina Toutanova. 2020. Contextualized representations using textual encyclopedic knowledge. CoRR, abs/2004.12006. · arXiv
  13. Nora Kassner and Hinrich Schütze. 2020. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, pages 7811-7818, Online. As- sociation for Computational Linguistics. · DOI
  14. Daniel Khashabi, Yeganeh Kordi, and Hannaneh Ha- jishirzi. 2022. Unifiedqa-v2: Stronger generaliza- tion via broader cross-format training. · DOI
  15. Tushar Khot, Peter Clark, Michal Guerquin, Pe- ter Alexander Jansen, and Ashish Sabharwal. 2020. Qasc: A dataset for question answering via sentence composition. ArXiv, abs/1910.11473. · arXiv
  16. Angeliki Lazaridou, Adhi Kuncoro, Elena Gri- bovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tomas Kocisky, Sebastian Ruder, Dani Yogatama, Kris Cao, Susannah Young, and Phil Blunsom. 2021. Mind the gap: Assessing temporal generalization in neural language models. In Advances in Neural Information Processing Systems, volume 34, pages 29348-29363.
  17. Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 7052-7063, Online and Punta Cana, Do- minican Republic. Association for Computational Linguistics. · DOI · Paper page
  18. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022a. Locating and edit- ing factual knowledge in gpt. arXiv preprint arXiv:2202.05262. · arXiv
  19. Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. 2022b. Mass- editing memory in a transformer. · DOI · Paper page
  20. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2022. Fast model editing at scale. In International Conference on Learning Representations.
  21. Katsuhiko Ogata. 1996. Modern Control Engineering (3rd Ed.). Prentice-Hall, Inc., USA.
  22. Marwan Omar, Soohyeon Choi, DaeHun Nyang, and David Mohaisen. 2022. Robust natural language processing: Recent advances, challenges, and future directions. CoRR, abs/2201.00768. · arXiv
  23. Yasumasa Onoe, Michael JQ Zhang, Eunsol Choi, and Greg Durrett. 2022. Entity cloze by date: What lms know about unseen entities. arXiv preprint arXiv:2205.02832. · arXiv
  24. Liangming Pan, Wenhu Chen, Min-Yen Kan, and William Yang Wang. 2021. Contraqa: Question answering under contradicting contexts. CoRR, abs/2110.07803. · arXiv
  25. Fabio Petroni, Patrick S. H. Lewis, Aleksandra Pik- tus, Tim Rocktäschel, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. 2020. How context affects language models' factual predictions. CoRR, abs/2005.04611. · arXiv
  26. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowl- edge bases? In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP- IJCNLP), pages 2463-2473, Hong Kong, China. As- sociation for Computational Linguistics. · DOI · Paper page
  27. Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020a. Exploring the limits of transfer learning with a unified text-to- text transformer. Journal of Machine Learning Re- search, 21(140):1-67.
  28. Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020b. Exploring the limits of transfer learning with a unified text-to- text transformer. Journal of Machine Learning Re- search, 21(140):1-67.
  29. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don't know: Unanswerable ques- tions for squad. · DOI · Paper page
  30. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the pa- rameters of a language model? arXiv preprint arXiv:2002.08910. · arXiv
  31. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in NeurIPS.
  32. Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2022. A survey of controllable text generation using transformer-based pre-trained lan- guage models. CoRR, abs/2201.05337. · arXiv
  33. Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Sri- nadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. 2020. Modifying memories in transformer models. arXiv preprint arXiv:2012.00363. Model Standard QA Knowledge Probe Prompts T5 XL Q: Into what body of water does the Hudson River terminate? A: The Atlantic Ocean. Q: What method formally adds inverses to elements to any monoid? A: I don't know. Q: Supply and what else causes child labour to still exist today? A: demands. Q: Who is the prime minister of Japan in 2015? A: Shinzo Abe. Q: Who is responsible for judicial review? A: Courts. Q: what was the name of the other HD channel Virgin media could carry in the future? A: I don't know. Q: What is the term for a hyperactive immune system that attacks normal tissues? A: autoimmunity. Q: What complexity class is commonly characterized by unknown algorithms to enhance solvability? A: I don't know. Q: Which nation contains the majority of the amazon forest? A: Brazil. · arXiv
  34. T5 XXL Q: Into what body of water does the Hudson River terminate? A: The Atlantic Ocean. Q: What method formally adds inverses to elements to any monoid? A: I don't know. Q: Supply and what else causes child labour to still exist today? A: demands. Q: Who is the prime minister of Japan in 2015? A: Shinzo Abe. Q: Who is responsible for judicial review? A: Courts. Q: What religion did the French spread along with their imperialism? A: Catholicism. Q: The symbol for mercuric oxide is? A: HgO. Q: What religion did the Yuan discourage, to support Buddhism? A: Taoism. PaLM 8B Only answer the questions you know the answer to: Q: Into what body of water does the Hudson River terminate? A: The Atlantic Ocean. Q: What year was the county of Hampshire officially named? A: ?. Q: Who said the following statement? "Enlightenment is manś emergence from his self-incurred immaturity". A: Immanuel Kant. Q: What method formally adds inverses to elements to any monoid? A: ?. Q: What King and former Huguenot looked out for the welfare of the group? A: Henry IV. Q: The principle of faunal succession was developed 100 years before whose theory of evolution? A: Charles Darwin. Q: Who is the hero who killed a dragon on the Drachenfels? A: Siegfried. PaLM 62B Only answer the questions you know the answer to: Q: Into what body of water does the Hudson River terminate? A: The Atlantic Ocean. Q: What year was the county of Hampshire officially named? A: ?. Q: Who said the following statement? "Enlightenment is man's emergence from his self-incurred immaturity". A: Immanuel Kant. Q: What method formally adds inverses to elements to any monoid? A: ?. Q: Who was the US Secretary of State in 2001? A: Colin Bowell. Q: The principle of faunal succession was developed 100 years before whose theory of evolution? A: Charles Darwin. Q: Who is the hero who killed a dragon on the Drachenfels? A: Siegfried. Q: When did the European Anti-Fraud Office investigate John Dalli? A: 2012. Q: What religion did the French spread along with their imperialism? A: Catholicism. Q: When did Costa v ENEL take place? A: 1964.

Cited by

Sorted by citations

Authors
Li, D., Rawat, A. S., Zaheer, M., Wang, X., Lukasik, M., Veit, A., Yu, F., & Kumar, S.
Affiliations
Google (United States) (US)
ORCID
Venue
Findings of the Association for Computational Linguistics: ACL 2023
Publisher
Association for Computational Linguistics
Published
2023
Pages
1774-1793
DOI
10.18653/v1/2023.findings-acl.112
OpenAlex
W4385570326
Cited by
46
References
7
Citations by year
2026: 8 · 2025: 12 · 2024: 13 · 2023: 13
FWCI
8.15
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
large language models, working memory, controllability, robustness, T5, Arecaceae, model size

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review