The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Catastrophic Forgetting, Rehearsal and Pseudorehearsal
Abstract
This paper reviews the problem of catastrophic forgetting (the loss or disruption of previously learned information when new information is learned) in neural networks, and explores rehearsal mechanisms (the retraining of some of the previously learned information as the new information is added) as a potential solution. We replicate some of the experiments described by Ratcliff (1990), including those relating to a simple 'recency' based rehearsal regime. We then develop further rehearsal regimes which are more effective than recency rehearsal. In particular, 'sweep rehearsal' is very successful at minimizing catastrophic forgetting. One possible limitation of rehearsal in general, however, is that previously learned information may not be available for retraining. We describe a solution to this problem, 'pseudorehearsal', a method which provides the advantages of rehearsal without actually requiring any access to the previously learned information (the original training population) itself. We then suggest an interpretation of these rehearsal mechanisms in the context of a function approximation based account of neural network learning. Both rehearsal and pseudorehearsal may have practical applications, allowing new information to be integrated into an existing network with minimum disruption of old information.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Inspired by replay strategies in continual learning (Robins, 1995; Rolnick et al., 2019; De Lange et al., 2021) as well as by the replay phenomena observed in humans (Smolen et al., 2016) and in neural network models (Amiri et al., 2017), we explore whether simple continual learning strategies remain effective for LLMs.”
From Evaluating the Long-Term Memory of Large Language Models (2025) · cites this paper as Robins, 1995
In the citing paper
“While replaying data from pre-training or previous training stages can alleviate forgetting (Lin, 1992; Robins, 1995; Chen et al., 2025), this strategy is data-inefficient and not scalable as we grow the amount of experience.”
From Continual Learning via Sparse Memory Finetuning (2025) · cites this paper as Robins, 1995
In the citing paper
“Before the era of deep learning, Robins [86,87] stated that catastrophic forgetting could be mitigated through rehearsal or pseudo-rehearsal.”
From An Appraisal of Incremental Learning Methods (2020) · cites this paper as [87]
Citations
Cites
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,501
Introduction to the theory of neural computation
Hertz, J., Krogh, A., & Palmer, R. G. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,410
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471
The ART of adaptive pattern recognition by a self-organizing neural network
Carpenter, G. A., & Grossberg, S. (1988) · Computer · doi:10.1109/2.33 · Cited by 1,278
Ratcliff, R. (1990) · Psychological Review · doi:10.1037/0033-295x.97.2.285 · Cited by 939
Artificial Neural Networks: Approximation and Learning Theory
White, H. (1992) · Medical Entomology and Zoology · Cited by 441
Networks of Formal Neurons and Memory Palimpsests
Nadal, J., Toulouse, G., Changeux, J., & Dehaene, S. (1986) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/1/10/008 · Cited by 215
From Statistics to Neural Networks: Theory and Pattern Recognition Applications
Cherkassky, V., Friedman, J. H., & Wechsler, H. L. (1996) · CERN Document Server (European Organization for Nuclear Research) · Cited by 177
Prediction Risk and Architecture Selection for Neural Networks
Moody, J. E. (1994) · Springer eBooks · doi:10.1007/978-3-642-79119-2_7 · Cited by 158
Cited by
Sorted by citations
Items 801–830 of 830
Dynamic grey box modeling of decanter centrifuges via continual learning
Zhai, O., Reppich, H. R., Nirschl, H., & Gleiß, M. (2026) · Separation and Purification Technology · doi:10.1016/j.seppur.2026.137951
Su, S., Liu, S., Hu, M., Wei, L., & Yan, Z. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.110365
Kaur, R., Guha, D., & Singh, G. (2025) · Artificial Intelligence for Business (AIxB) · doi:10.1109/aixb65684.2025.00018
A Contemporary Analysis of Zero-Shot Learning in Machine Vision Tasks
Arun, V. S., & Simon, P. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-18135-0_15
Prompting Rain Off: Evolving Compact Dual Prompts for Continual De-Raining
Liu, M., Yang, W., & Liu, J. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3689428
Adaptive Prompt Evolution for Continual Learning in Diabetic Retinopathy Detection
Bravo-Rocca, G., Liu, P., Guitart, J., Dholakia, A., Ellison, D. D., & Carrillo-Larco, R. M. (2026) · SN Computer Science · doi:10.1007/s42979-026-05036-y
SketchOGD: Memory-Efficient Continual Learning
Min, Y., Wright, B., Bernstein, J., & Azizan, N. (2026) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2026.3695343
Efficient dataset condensation with learnable color representation and subset matching
Tran, L.-T., Vo, Q. H., Qamar, M., Zhang, C., Kim, H. Y., & Bae, S.-H. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116216
Random Dense Knowledge Distillation for Continual Learning
Chu, J., Liu, P., Su, T., Wu, Y. D., Song, Y., & Shi, Z. (2026) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3816731
A Comprehensive Survey of Continual Learning Techniques in NER: Challenges and Opportunities
Cutler, C., Taylor, S., Tang, C., & McInnes, B. T. (2026) · ACM Computing Surveys · doi:10.1145/3820036 · Cited by 1
Prompt-Based Hypernetwork in Rehearsal-Free Class Incremental Learning
Liu, Y., & Zeng, Z. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-1599-7_29
SGM-Net: 3D Point Cloud Class-Incremental Segmentation via Semantic-Aware Global Modeling
Liu, J., Ma, B., Zhao, Z., Yan, C. C., & Wang, S. (2026) · IEEE Signal Processing Letters · doi:10.1109/lsp.2026.3706433
Physics prior adapter tuning for thermal infrared tracking
Guo, J., Liu, Q., Tan, K., Yuan, D., & Liu, Y. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133472
Selective Distillation for Continual Named Entity Recognition with Memory Replay
Wang, W., Cao, Y., & Bao, F. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809972
ASBR-CL: Stage-Aware Balanced Replay for Memory-Limited Continual Fault Diagnosis
Li, J., & Zhu, G. (2026) · Sensors · doi:10.3390/s26144416
STAER: Temporal Aligned Rehearsal for Continual Spiking Neural Network
Gianferrari, M., Moussadek, O., Salami, R., Fiorini, C., Tartarini, L., Gandolfi, D., & Calderara, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31927-2_26
Domain-Incremental Learning for UAV Traffic Video Anomaly Detection
Doan, D., & Nguyen, K.-D. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-2590-3_46
Dream2Learn: Structured Generative Dreaming for Continual Learning
Calcagno, S., Pennisi, M., Salanitri, F. P., Sorrenti, A., Palazzo, S., Spampinato, C., & Bellitto, G. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02958-1
Bhooanusas, N., Srisakulmekee, A., Iamsudha, O., & Pipatanagovit, P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7305688
Ye, F., & Borş, A. G. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.134227
Orlov, A. A., & Abramova, E. S. (2026) · Pattern Recognition and Image Analysis · doi:10.1134/s1054661826700525
shen, G., li, L., & Zhao, E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7513272
COLA: Continual Orthogonal Low-Rank Adaptation for Class-Incremental Learning
Nagar, M., & Das, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37138-6_26
Manohar, M. S., & Zhang, Y. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7593546
Putting Large Language Models into Environmental Context
Jiang, J., Zhu, J., & Ren, Z. J. (2026) · Environmental Science & Technology · doi:10.1021/acs.est.6c06045 · Cited by 1
Manohar, M. S., & Zhang, Y. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7616060
Kumar, N., & Srivastava, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7615680
Lin, Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7629592
An acetylcholine-inhibited sleep-active hippocampal interneuron orchestrates memory consolidation
Feng, Y., Dai, Q., Fei, X., Yang, H., Jiang, T., Liu, Z., Ye, L., Lin, Z., Gao, S., Hu, R., et al. (2026) · Neuron · doi:10.1016/j.neuron.2026.09.031 · Cited by 1
Bhooanusas, N., Srisakulmekee, A., Iamsudha, O., & Pipatanagovit, P. (2026) · Array · doi:10.1016/j.array.2026.101375
Metadata
- Authors
- ROBINS, A.
- Affiliations
- University of Otago (NZ)
- Corresponding author
- ROBINS, ANTHONY
- ORCID
- ROBINS, ANTHONY
- Venue
- Connection Science
- Publisher
- Informa UK Limited
- Published
- 1995-06
- Volume
- 7
- Issue
- 2
- Pages
- 123-146
- ISSN
- 0954-0091 (ISSN-L)
- 1360-0494
- OpenAlex
- W2116522068
- Cited by
- 833
- References
- 18
- Citations by year
- 2026: 52 · 2025: 79 · 2024: 92 · 2023: 109 · 2022: 94 · 2021: 100 · 2020: 77 · 2019: 68 · 2018: 23 · 2017: 15 · 2016: 4 · 2015: 6 · 2014: 3 · 2013: 8 · 2012: 1
- FWCI
- 6.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Connection Science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- catastrophic forgetting, rehearsal, neural network learning, function approximation, incremental learning, knowledge retention
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
