The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Catastrophic Forgetting, Rehearsal and Pseudorehearsal
Abstract
This paper reviews the problem of catastrophic forgetting (the loss or disruption of previously learned information when new information is learned) in neural networks, and explores rehearsal mechanisms (the retraining of some of the previously learned information as the new information is added) as a potential solution. We replicate some of the experiments described by Ratcliff (1990), including those relating to a simple 'recency' based rehearsal regime. We then develop further rehearsal regimes which are more effective than recency rehearsal. In particular, 'sweep rehearsal' is very successful at minimizing catastrophic forgetting. One possible limitation of rehearsal in general, however, is that previously learned information may not be available for retraining. We describe a solution to this problem, 'pseudorehearsal', a method which provides the advantages of rehearsal without actually requiring any access to the previously learned information (the original training population) itself. We then suggest an interpretation of these rehearsal mechanisms in the context of a function approximation based account of neural network learning. Both rehearsal and pseudorehearsal may have practical applications, allowing new information to be integrated into an existing network with minimum disruption of old information.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Inspired by replay strategies in continual learning (Robins, 1995; Rolnick et al., 2019; De Lange et al., 2021) as well as by the replay phenomena observed in humans (Smolen et al., 2016) and in neural network models (Amiri et al., 2017), we explore whether simple continual learning strategies remain effective for LLMs.”
From Evaluating the Long-Term Memory of Large Language Models (2025) · cites this paper as Robins, 1995
In the citing paper
“While replaying data from pre-training or previous training stages can alleviate forgetting (Lin, 1992; Robins, 1995; Chen et al., 2025), this strategy is data-inefficient and not scalable as we grow the amount of experience.”
From Continual Learning via Sparse Memory Finetuning (2025) · cites this paper as Robins, 1995
In the citing paper
“Before the era of deep learning, Robins [86,87] stated that catastrophic forgetting could be mitigated through rehearsal or pseudo-rehearsal.”
From An Appraisal of Incremental Learning Methods (2020) · cites this paper as [87]
Citations
Cites
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,501
Introduction to the theory of neural computation
Hertz, J., Krogh, A., & Palmer, R. G. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,410
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471
The ART of adaptive pattern recognition by a self-organizing neural network
Carpenter, G. A., & Grossberg, S. (1988) · Computer · doi:10.1109/2.33 · Cited by 1,278
Ratcliff, R. (1990) · Psychological Review · doi:10.1037/0033-295x.97.2.285 · Cited by 939
Artificial Neural Networks: Approximation and Learning Theory
White, H. (1992) · Medical Entomology and Zoology · Cited by 441
Networks of Formal Neurons and Memory Palimpsests
Nadal, J., Toulouse, G., Changeux, J., & Dehaene, S. (1986) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/1/10/008 · Cited by 215
From Statistics to Neural Networks: Theory and Pattern Recognition Applications
Cherkassky, V., Friedman, J. H., & Wechsler, H. L. (1996) · CERN Document Server (European Organization for Nuclear Research) · Cited by 177
Prediction Risk and Architecture Selection for Neural Networks
Moody, J. E. (1994) · Springer eBooks · doi:10.1007/978-3-642-79119-2_7 · Cited by 158
Winson, J. (1990) · Scientific American · doi:10.1038/scientificamerican1190-86 · Cited by 155
Cutting the REM Nerve: An Approach to the Adaptive Role of REM Sleep
Greenbérg, R., & PEARLMAN, C. A. (1974) · Perspectives in biology and medicine · doi:10.1353/pbm.1974.0032 · Cited by 120
Semi-distributed Representations and Catastrophic Forgetting in Connectionist Networks
French, R. M. (1992) · Connection Science · doi:10.1080/09540099208946624 · Cited by 119
French, R. M. (2019) · Routledge eBooks · doi:10.4324/9781315789354-58 · Cited by 99
Catastrophic forgetting in neural networks: the role of rehearsal mechanisms
Robins, A. V. (2002) · doi:10.1109/annes.1993.323080 · Cited by 77
Self-citationCatastrophic interference in neural networks
Lewandowsky, S., & Li, S. (1995) · Elsevier eBooks · doi:10.1016/b978-012208930-5/50011-8 · Cited by 55
Is there 'catastrophic interference' in connectionist networks?
Hetherington, P. A. (1989) · Conference Cognitive Science · Cited by 55
Neural network models of list learning
Burgess, N., Shapiro, J., & Moore, M. A. (1991) · Network Computation in Neural Systems · doi:10.1088/0954-898x_2_4_005 · Cited by 26
Cited by
Sorted by citations
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Rasch, B., & Born, J. (2013) · Physiological Reviews · doi:10.1152/physrev.00032.2012 · Cited by 2,783
Catastrophic forgetting in connectionist networks
French, R. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01294-2 · Cited by 2,311
A Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647
Metadata
- Authors
- ROBINS, A.
- Affiliations
- University of Otago (NZ)
- Corresponding author
- ROBINS, ANTHONY
- ORCID
- ROBINS, ANTHONY
- Venue
- Connection Science
- Publisher
- Informa UK Limited
- Published
- 1995-06
- Volume
- 7
- Issue
- 2
- Pages
- 123-146
- ISSN
- 0954-0091 (ISSN-L)
- 1360-0494
- OpenAlex
- W2116522068
- Cited by
- 833
- References
- 18
- Citations by year
- 2026: 52 · 2025: 79 · 2024: 92 · 2023: 109 · 2022: 94 · 2021: 100 · 2020: 77 · 2019: 68 · 2018: 23 · 2017: 15 · 2016: 4 · 2015: 6 · 2014: 3 · 2013: 8 · 2012: 1
- FWCI
- 6.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Connection Science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- catastrophic forgetting, rehearsal, neural network learning, function approximation, incremental learning, knowledge retention
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
