CoChat
Sign Up
Cited by 70Topic ModelingDomain Adaptation and Few-Shot LearningNatural Language Processing Techniques

Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models

Open access
Tirumala, Kushal, Markosyan, Aram, Zettlemoyer, Luke, Aghajanyan, Armen·2022·Advances in Neural Information Processing Systems 35
DOI 10.52202/068431-2773
Open this paper in CoChat — free accountOpen PDF

Abstract

Despite their wide adoption, the underlying training and memorization dynamics of very large language models is not well understood. We empirically study exact memorization in causal and masked language modeling, across model sizes and throughout the training process. We measure the effects of dataset size, learning rate, and model size on memorization, finding that larger language models memorize training data faster across all settings. Surprisingly, we show that larger models can memorize a larger portion of the data before over-fitting and tend to forget less throughout the training process. We also analyze the memorization dynamics of different parts of speech and find that models memorize nouns and numbers first; we hypothesize and provide empirical evidence that nouns and numbers act as a unique identifier for memorizing individual training examples. Together, these findings present another piece of the broader puzzle of trying to understand what actually improves as models get bigger.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.52202/068431-2773

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Previous research has predominantly assessed memory by evaluating the extent to which models fit the training data, employing identical task formats during both training and evaluation phases.

    “Previous research has predominantly assessed memory by evaluating the extent to which models fit the training data, employing identical task formats during both the training and evaluation phases (Tirumala et al., 2022; Wang et al., 2019; Han et al., 2020).”

    From Evaluating the Long-Term Memory of Large Language Models (2025) · cites this paper as Tirumala et al., 2022

    The Experimental Setup (p. 3) measures memorization by whether a model predicts the ground-truth token in a training context, supporting the training-data-fit aspect. But its discussion of previous work (p. 3) describes several different definitions, including extraction by prompting and the influence of individual examples. It does not establish that previous research predominantly used identical task formats for training and evaluation. (LLM, conf=0.80)

    Experimental — accuracy not yet measured

  • Tirumala et al. (2022) found that models exhibit a forgetting baseline, meaning the forgetting curve has a lower bound, and this baseline increases with model size.

    “Tirumala et al. (2022) found that models exhibit a forgetting baseline, meaning the forgetting curve has a lower bound (the model retains a certain memory of the first batch of training data and does not completely forget). Moreover, this baseline increases with the model size, indicating that scaling up the model can mitigate forgetting.”

    From Evaluating the Long-Term Memory of Large Language Models (2025) · cites this paper as Tirumala et al., 2022

    The Introduction (p. 1) and §5 (p. 8) report a lower bound on the forgetting curve that increases with model size, indicating that larger models forget less. But §5 (p. 8) measures retention of a held-out validation batch introduced at a training checkpoint, not the first batch of training data as the citing sentence says. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Current knowledge injection methods often suffer from computational overhead, catastrophic forgetting, and overfitting.

    “current knowledge injection methods often suffer from computational overhead [13, 10], catastrophic forgetting [63], and overfitting [64].”

    From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [64]

    The citation [64] is attached only to overfitting, not to computational overhead or catastrophic forgetting. Section 4.2 (p. 5) discusses overfitting during language-model training, but it does not establish that current knowledge-injection methods often suffer from it. (LLM, conf=0.73)

    Experimental — accuracy not yet measured

  • Finetuning and pretraining are computation-expensive and usually suffer from catastrophic forgetting and overfitting.

    “However, finetuning and pretraining are computation-expensive [13, 10] and usually suffer from catastrophic forgetting [63] and overfitting [64].”

    From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [64]

    The citation [64] is attached to the overfitting claim, not the computation-cost or catastrophic-forgetting claims. Section 4.2 (pp. 5–6) examines when overfitting occurs in language-model training, but it does not establish that finetuning and pretraining usually suffer from it. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Tirumala et al. (2022) measures memory by evaluating the model's prediction accuracy for contexts within the training data.

    “Specifically, Tirumala et al. (2022) measures memory by evaluating the model's prediction accuracy for contexts within the training data (such as missing text segments or missing words).”

    From Evaluating the Long-Term Memory of Large Language Models (2025) · cites this paper as Tirumala et al., 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 89

  1. [1][1] Alessandro Achille, Matteo Rovere, and Stefano Soatto. Critical learning periods in deep networks. In International Conference on Learning Representations , 2018. -
  2. [2][2] Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta. Better fine-tuning by reducing representational collapse. arXiv preprint arXiv:2008.03156 , 2020. - · arXiv
  3. [3][3] Armen Aghajanyan, Bernie Huang, Candace Ross, Vladimir Karpukhin, Hu Xu, Naman Goyal, Dmytro Okhonko, Mandar Joshi, Gargi Ghosh, Mike Lewis, et al. Cm3: A causal masked multimodal model of the internet. arXiv preprint arXiv:2201.07520 , 2022. - · arXiv
  4. [4][4] Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. A review on language models as knowledge bases. arXiv preprint arXiv:2204.06031 , 2022. - · arXiv
  5. [5][5] Hadi Amiri, Timothy Miller, and Guergana Savova. Repeat before forgetting: Spaced repetition for efficient and effective training of neural networks. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , pages 2401-2410, 2017. -
  6. [7][7] Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du, Srinivasan Iyer, Ramakanth Pasunuru, et al. Efficient large scale language modeling with mixtures of experts. arXiv preprint arXiv:2112.10684 , 2021. - · arXiv
  7. [8][8] Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, and Utkarsh Sharma. Explaining neural scaling laws. arXiv preprint arXiv:2102.06701 , 2021. - · arXiv
  8. [10][10] Terra Blevins, Hila Gonen, and Luke Zettlemoyer. Analyzing the mono-and cross-lingual pretraining dynamics of multilingual language models. arXiv preprint arXiv:2205.11758 , 2022. - · arXiv
  9. [11][11] Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. arXiv preprint arXiv:2112.04426 , 2021. - · arXiv
  10. [12][12] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP) , pages 141-159. IEEE, 2021. -
  11. [13][13] Gavin Brown, Mark Bun, Vitaly Feldman, Adam Smith, and Kunal Talwar. When is memorization of irrelevant training data necessary for high-accuracy learning? In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing , pages 123-132, 2021. -
  12. [14][14] Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX Security Symposium (USENIX Security 19) , pages 267-284, 2019. -
  13. [15][15] Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21) , pages 2633-2650, 2021. -
  14. [16][16] Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. arXiv preprint arXiv:2202.07646 , 2022. - · arXiv
  15. [18][18] Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting Liu, and Xiangzhan Yu. Recall and learn: Fine-tuning deep pretrained language models with less forgetting. arXiv preprint arXiv:2004.12651 , 2020. - · arXiv
  16. [19][19] Zhiyuan Chen and Bing Liu. Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning , 12(3):1-207, 2018. -
  17. [20][20] Cheng-Han Chiang, Sung-Feng Huang, and Hung-yi Lee. Pretrained language model embryology: The birth of albert. arXiv preprint arXiv:2010.02480 , 2020. - · arXiv
  18. [21][21] Leshem Choshen, Guy Hacohen, Daphna Weinshall, and Omri Abend. The grammar-learning trajectories of neural language models. arXiv preprint arXiv:2109.06096 , 2021. - · arXiv
  19. [22][22] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 , 2022. - · arXiv
  20. [23][23] Aidan Clark, Diego de las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffmann, Bogdan Damoc, Blake Hechtman, Trevor Cai, Sebastian Borgeaud, et al. Unified scaling laws for routed language models. arXiv preprint arXiv:2202.01169 , 2022. - · arXiv
  21. [24][24] Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021. -
  22. [25][25] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference , pages 265-284. Springer, 2006. -
  23. [26][26] Vitaly Feldman. Does learning require memorization. A short tale about a long tail. CoRR, abs/1906.05271 , 2019. - · arXiv
  24. [27][27] Vitaly Feldman. Does Learning Require Memorization? A Short Tale about a Long Tail. arXiv:1906.05271 [cs, stat] , January 2021. URL http://arxiv.org/abs/1906.05271 . arXiv: 1906.05271. - · arXiv
  25. [28][28] Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems , 33: 2881-2891, 2020. -
  26. [30][30] Jonathan Frankle, David J Schwab, and Ari S Morcos. The early phase of neural network training. arXiv preprint arXiv:2002.10365 , 2020. - · arXiv
  27. [31][31] James Franklin. The elements of statistical learning: data mining, inference and prediction. The Mathematical Intelligencer , 27(2):83-85, 2005. -
  28. [32][32] Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462 , 2020. - · arXiv
  29. [33][33] Tanya Goyal, Jiacheng Xu, Junyi Jessy Li, and Greg Durrett. Training dynamics for text summarization models. arXiv preprint arXiv:2110.08370 , 2021. - · arXiv
  30. [34][34] Guy Gur-Ari, Daniel A Roberts, and Ethan Dyer. Gradient descent happens in a tiny subspace. arXiv preprint arXiv:1812.04754 , 2018. - · arXiv
  31. [35][35] Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. Realm: Retrieval-augmented language model pre-training. arXiv preprint arXiv:2002.08909 , 2020. - · arXiv
  32. [36][36] Yaru Hao, Li Dong, Furu Wei, and Ke Xu. Investigating learning dynamics of BERT finetuning. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing , pages 87-92, Suzhou, China, December 2020. Association for Computational Linguistics. URL https://aclanthology.org/2020.aacl-main.11 . -
  33. [37][37] Elizabeth Liz Harding, Jarno J Vanto, Reece Clark, L Hannah Ji, and Sara C Ainsworth. Understanding the scope and impact of the california consumer privacy act of 2018. Journal of Data Protection & Privacy , 2(3):234-253, 2019. -
  34. [38][38] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 , 2016. - · arXiv
  35. [39][39] Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701 , 2020. - · arXiv
  36. [40][40] Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish. Scaling laws for transfer. arXiv preprint arXiv:2102.01293 , 2021. - · arXiv
  37. [41][41] Sorami Hisamoto, Matt Post, and Kevin Duh. Membership inference attacks on sequence-tosequence models. arXiv preprint arXiv:1904.05506 , 2019. - · arXiv
  38. [43][43] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. arXiv preprint arXiv:2202.03629 , 2022. - · arXiv
  39. [44][44] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling Laws for Neural Language Models. arXiv:2001.08361 [cs, stat] , January 2020. URL http://arxiv.org/abs/2001. 08361 . arXiv: 2001.08361. - · arXiv
  40. [45][45] Jeffrey D Karpicke and Henry L Roediger III. Expanding retrieval practice promotes short-term retention, but equally spaced retrieval enhances long-term retention. Journal of experimental psychology: learning, memory, and cognition , 33(4):704, 2007. -
  41. [46][46] Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models. arXiv preprint arXiv:1911.00172 , 2019. - · arXiv
  42. [47][47] Eugene Kharitonov, Marco Baroni, and Dieuwke Hupkes. How bpe affects memorization in transformers. arXiv preprint arXiv:2110.02782 , 2021. - · arXiv
  43. [48][48] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014. - · arXiv
  44. [49][49] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences , 114(13):3521-3526, 2017. -
  45. [50][50] Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06499 , 2021. - · arXiv
  46. [51][51] Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. arXiv preprint arXiv:2110.05679 , 2021. - · arXiv
  47. [52][52] Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, and Joey Gonzalez. Train big, then compress: Rethinking model size for efficient training and inference of transformers. In International Conference on Machine Learning , pages 5958-5968. PMLR, 2020. -
  48. [53][53] Leo Z Liu, Yizhong Wang, Jungo Kasai, Hannaneh Hajishirzi, and Noah A Smith. Probing across time: What does roberta know and when? arXiv preprint arXiv:2104.07885 , 2021. - · arXiv
  49. [54][54] Yang Liu, Zhuo Ma, Ximeng Liu, Jian Liu, Zhongyuan Jiang, Jianfeng Ma, Philip Yu, and Kui Ren. Learn to forget: Machine unlearning via neuron masking. arXiv preprint arXiv:2003.10933 , 2020. - · arXiv
  50. [55][55] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv e-prints , July 2019. URL https://arxiv.org/abs/ 1907.11692v1 . -
  51. [56][56] Geoffrey R Loftus. Evaluating forgetting curves. Journal of Experimental Psychology: Learning, Memory, and Cognition , 11(2):397, 1985. -
  52. [57][57] Alessandro Mantelero. The eu proposal for a general data protection regulation and the roots of the 'right to be forgotten'. Computer Law & Security Review , 29(3):229-235, 2013. -
  53. [58][58] Wojciech Masarczyk, Kamil Deja, and Tomasz Trzcinski. On robustness of generative representations against catastrophic forgetting. In International Conference on Neural Information Processing , pages 325-333. Springer, 2021. -
  54. [59][59] Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation , volume 24, pages 109-165. Elsevier, 1989. -
  55. [60][60] R Thomas McCoy, Paul Smolensky, Tal Linzen, Jianfeng Gao, and Asli Celikyilmaz. How much do language models copy from their training data? evaluating linguistic novelty in text generation using raven. arXiv preprint arXiv:2111.09509 , 2021. - · arXiv
  56. [62][62] Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. ArXiv , abs/1609.07843, 2017. - · arXiv
  57. [63][63] Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. arXiv preprint arXiv:1710.03740 , 2017. - · arXiv
  58. [64][64] Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri. Quantifying privacy risks of masked language models using membership inference attacks. arXiv preprint arXiv:2203.03929 , 2022. - · arXiv
  59. [65][65] Seyed Iman Mirzadeh, Arslan Chaudhry, Huiyi Hu, Razvan Pascanu, Dilan Gorur, and Mehrdad Farajtabar. Wide neural networks forget less catastrophically. arXiv preprint arXiv:2110.11526 , 2021. - · arXiv
  60. [66][66] Ari Morcos, Maithra Raghu, and Samy Bengio. Insights on representational similarity in neural networks with canonical correlation. Advances in Neural Information Processing Systems , 31, 2018. -
  61. [67][67] Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment , 2021(12):124003, 2021. -
  62. [68][68] Shiri Oren, Charlene Willerton, and Jeff Small. Effects of spaced retrieval training on semantic memory in alzheimer's disease: A systematic review. Journal of Speech, Language and Hearing Research (Online) , 57(1):247, 2014. -
  63. [69][69] Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. arXiv preprint arXiv:1904.01038 , 2019. - · arXiv
  64. [70][70] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32:8026-8037, 2019. -
  65. [71][71] Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint arXiv:1909.01066 , 2019. - · arXiv
  66. [72][72] Vinaychandran Pondenkandath, Michele Alberti, Sammer Puran, Rolf Ingold, and Marcus Liwicki. Leveraging random label memorization for unsupervised pre-training. arXiv preprint arXiv:1811.01640 , 2018. - · arXiv
  67. [73][73] Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446 , 2021. - · arXiv
  68. [74][74] Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. Advances in neural information processing systems , 30, 2017. -
  69. [75][75] Vinay Venkatesh Ramasesh, Aitor Lewkowycz, and Ethan Dyer. Effect of scale on catastrophic forgetting in neural networks. In International Conference on Learning Representations , 2021. -
  70. [76][76] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning , pages 8821-8831. PMLR, 2021. -
  71. [77][77] Roger Ratcliff. Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psychological Review , pages 285-308, 1990. -
  72. [78][78] General Data Protection Regulation. General data protection regulation (gdpr). Intersoft Consulting, Accessed in October , 24(1), 2018. -
  73. [79][79] Jonathan S Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, and Nir Shavit. A constructive prediction of the generalization error across scales. arXiv preprint arXiv:1909.12673 , 2019. - · arXiv
  74. [80][80] Naomi Saphra and Adam Lopez. Understanding learning dynamics of language models with svcca. arXiv preprint arXiv:1811.00225 , 2018. - · arXiv
  75. [81][81] Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, and Marco Turchi. On the dynamics of gender learning in speech translation. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP) , pages 94-111, Seattle, Washington, July 2022. Association for Computational Linguistics. URL https://aclanthology.org/ 2022.gebnlp-1.12 . -
  76. [82][82] Chenze Shao and Yang Feng. Overcoming catastrophic forgetting beyond continual learning: Balanced training for neural machine translation. arXiv preprint arXiv:2203.03910 , 2022. - · arXiv
  77. [83][83] Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, et al. Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model. arXiv preprint arXiv:2201.11990 , 2022. - · arXiv
  78. [84][84] Paul Smolen, Yili Zhang, and John H Byrne. The right time to learn: mechanisms and optimization of spaced learning. Nature Reviews Neuroscience , 17(2):77-88, 2016. -
  79. [85][85] Congzheng Song and Vitaly Shmatikov. Auditing data provenance in text-generation models. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery &Data Mining , pages 196-206, 2019. -
  80. [86][86] Patrick Stadler, Vivien Macketanz, and Eleftherios Avramidis. Observing the learning curve of nmt systems with regard to linguistic phenomena. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop , pages 186-196, 2021. -
  81. [87][87] Yi Tay, Vinh Q Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, et al. Transformer memory as a differentiable search index. arXiv preprint arXiv:2202.06991 , 2022. - · arXiv
  82. [88][88] Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews, and Françoise Beaufays. Understanding unintended memorization in federated learning. arXiv preprint arXiv:2006.07490 , 2020. - · arXiv
  83. [89][89] Aleena Thomas, David Ifeoluwa Adelani, Ali Davody, Aditya Mogadala, and Dietrich Klakow. Investigating the impact of pre-trained word embeddings on memorization in neural networks. In International Conference on Text, Speech, and Dialogue , pages 273-281. Springer, 2020. -
  84. [90][90] Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159 , 2018. - · arXiv
  85. [91][91] Paul Voigt and Axel Von dem Bussche. The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing , 10(3152676):10-5555, 2017. -
  86. [93][93] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 , 2022. - · arXiv
  87. [94][94] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv:1611.03530 [cs] , February 2017. URL http://arxiv.org/abs/1611.03530 . arXiv: 1611.03530. - · arXiv
  88. [95][95] Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, and Nicholas Carlini. Counterfactual Memorization in Neural Language Models. arXiv:2112.12938 [cs] , December 2021. URL http://arxiv.org/abs/2112.12938 . arXiv: 2112.12938 version: 1. - · arXiv
  89. [96][96] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 , 2022. · arXiv

Cited by

Sorted by citations

Related

Authors
Tirumala, K., Markosyan, A., Zettlemoyer, L., & Aghajanyan, A.
ORCID
Zettlemoyer, Luke
Venue
Advances in Neural Information Processing Systems 35
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Published
2022
Pages
38274-38290
ISSN
1049-5258 (ISSN-L)
DOI
10.52202/068431-2773
OpenAlex
W7133262701
Cited by
67
References
0
Citations by year
2026: 11 · 2025: 10 · 2024: 16 · 2023: 26 · 2022: 4
FWCI
7.74
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
memorization, overfitting, training dynamics, large language models, catastrophic forgetting

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review