Catastrophic forgetting, rehearsal and pseudorehearsal
Abstract
This paper reviews the problem of catastrophic forgetting (the loss or disruption of previously learned information when new information is learned) in neural networks, and explores rehearsal mechanisms (the retraining of some of the previously learned information as the new information is added) as a potential solution. We replicate some of the experiments described by Ratcliff (1990), including those relating to a simple 'recency' based rehearsal regime. We then develop further rehearsal regimes which are more effective than recency rehearsal. In particular, 'sweep rehearsal' is very successful at minimizing catastrophic forgetting. One possible limitation of rehearsal in general, however, is that previously learned information may not be available for retraining. We describe a solution to this problem, 'pseudorehearsal', a method which provides the advantages of rehearsal without actually requiring any access to the previously learned information (the original training population) itself. We then suggest an interpretation of these rehearsal mechanisms in the context of a function approximation based account of neural network learning. Both rehearsal and pseudorehearsal may have practical applications, allowing new information to be integrated into an existing network with minimum disruption of old information.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Replay strategies in continual learning and human/neural replay phenomena inspired the authors' exploration of whether simple continual learning strategies remain effective for LLMs.
“Inspired by replay strategies in continual learning (Robins, 1995; Rolnick et al., 2019; De Lange et al., 2021) as well as by the replay phenomena observed in humans (Smolen et al., 2016) and in neural network models (Amiri et al., 2017), we explore whether simple continual learning strategies remain effective for LLMs.”
From Evaluating the Long-Term Memory of Large Language Models (2025) · cites this paper as Robins, 1995
Replaying data from pre-training or previous training stages can alleviate forgetting, but is data-inefficient and not scalable as experience grows.
“While replaying data from pre-training or previous training stages can alleviate forgetting (Lin, 1992; Robins, 1995; Chen et al., 2025), this strategy is data-inefficient and not scalable as we grow the amount of experience.”
From Continual Learning via Sparse Memory Finetuning (2025) · cites this paper as Robins, 1995
Before deep learning, Robins stated that catastrophic forgetting could be mitigated through rehearsal or pseudo-rehearsal.
“Before the era of deep learning, Robins [86,87] stated that catastrophic forgetting could be mitigated through rehearsal or pseudo-rehearsal.”
From An Appraisal of Incremental Learning Methods (2020) · cites this paper as [87]
Cites
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,501 · see citing works
Introduction to the theory of neural computation
Hertz, J., Krogh, A., & Palmer, R. G. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,431 · see citing works
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770 · see citing works
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471 · see citing works
The ART of adaptive pattern recognition by a self-organizing neural network
Carpenter, G. A., & Grossberg, S. (1988) · Computer · doi:10.1109/2.33 · Cited by 1,278 · see citing works
Ratcliff, R. (1990) · Psychological Review · doi:10.1037/0033-295x.97.2.285 · Cited by 892 · see citing works
Artificial Neural Networks: Approximation and Learning Theory
White, H. (1992) · Medical Entomology and Zoology · Cited by 441 · see citing works
Networks of Formal Neurons and Memory Palimpsests
Nadal, J., Toulouse, G., Changeux, J., & Dehaene, S. (1986) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/1/10/008 · Cited by 213 · see citing works
From Statistics to Neural Networks: Theory and Pattern Recognition Applications
Cherkassky, V., Friedman, J. H., & Wechsler, H. L. (1996) · CERN Document Server (European Organization for Nuclear Research) · Cited by 177 · see citing works
Prediction Risk and Architecture Selection for Neural Networks
Moody, J. E. (1994) · Springer eBooks · doi:10.1007/978-3-642-79119-2_7 · Cited by 158 · see citing works
Winson, J. (1990) · Scientific American · doi:10.1038/scientificamerican1190-86 · Cited by 155 · see citing works
Cutting the REM Nerve: An Approach to the Adaptive Role of REM Sleep
Greenbérg, R., & PEARLMAN, C. A. (1974) · Perspectives in biology and medicine · doi:10.1353/pbm.1974.0032 · Cited by 119 · see citing works
Semi-distributed Representations and Catastrophic Forgetting in Connectionist Networks
French, R. M. (1992) · Connection Science · doi:10.1080/09540099208946624 · Cited by 119 · see citing works
French, R. M. (2019) · Routledge eBooks · doi:10.4324/9781315789354-58 · Cited by 99 · see citing works
Catastrophic forgetting in neural networks: the role of rehearsal mechanisms
Robins, A. V. (2002) · doi:10.1109/annes.1993.323080 · Cited by 77 · see citing works
Self-citationCatastrophic interference in neural networks
Lewandowsky, S., & Li, S. (1995) · Elsevier eBooks · doi:10.1016/b978-012208930-5/50011-8 · Cited by 55 · see citing works
Is there 'catastrophic interference' in connectionist networks?
Hetherington, P. A. (1989) · Conference Cognitive Science · Cited by 55 · see citing works
Neural network models of list learning
Burgess, N., Shapiro, J., & Moore, M. A. (1991) · Network Computation in Neural Systems · doi:10.1088/0954-898x_2_4_005 · Cited by 25 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · see citing works
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117 · see citing works
Rasch, B., & Born, J. (2013) · Physiological Reviews · doi:10.1152/physrev.00032.2012 · Cited by 2,783 · see citing works
Catastrophic forgetting in connectionist networks
French, R. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01294-2 · Cited by 2,311 · see citing works
A Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647 · see citing works
Continual Learning with Deep Generative Replay
Shin, H., Lee, J. K., Kim, J., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1705.08690 · Cited by 1,385 · see citing works
PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning
Douillard, A., Cord, M., Ollion, C., Robert, T., & Junior, E. A. D. V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_6 · Cited by 720 · see citing works
DER: Dynamically Expandable Representation for Class Incremental Learning
Yan, S., Xie, J., & He, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00303 · Cited by 752 · see citing works
Measuring Catastrophic Forgetting in Neural Networks
Kemker, R., McClure, M., Abitino, A., Hayes, T., & Kanan, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11651 · Cited by 672 · see citing works
A dynamic ensemble learning algorithm for neural networks
Alam, K. M. R., Siddique, N. H., & Adeli, H. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04359-7 · Cited by 538 · see citing works
Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., & Díaz-Rodríguez, N. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.12.004 · Cited by 531 · see citing works
Brain-inspired replay for continual learning with artificial neural networks
van de Ven, G. M., Siegelmann, H. T., & Tolias, A. S. (2020) · Nature Communications · doi:10.1038/s41467-020-17866-2 · Cited by 474 · see citing works
Embracing Change: Continual Learning in Deep Neural Networks
Hadsell, R., Rao, D., Rusu, A. A., & Pascanu, R. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.09.004 · Cited by 467 · see citing works
Learning to Learn without Forgetting by Maximizing Transfer and Minimizing InterferencePreprint
Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., & Tesauro, G. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11910 · Cited by 534 · see citing works
Dark Experience for General Continual Learning: a Strong, Simple Baseline
Buzzega, P., Boschini, M., Porrello, A., Abati, D., & Calderara, S. (2020) · arXiv · doi:10.48550/arxiv.2004.07211 · Cited by 473 · see citing works
The Natural Language Decathlon: Multitask Learning as Question Answering
McCann, B., Keskar, N. S., Xiong, C., & Socher, R. (2018) · arXiv · doi:10.48550/arxiv.1806.08730 · Cited by 448 · see citing works
Mermillod, M., Bugaïska, A., & Bonin, P. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00504 · Cited by 425 · see citing works
Gradient based sample selection for online continual learningPreprint
Aljundi, R., Lin, M., Goujaud, B., & Bengio, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08671 · Cited by 397 · see citing works
DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion
Douillard, A., Ramé, A., Couairon, G., & Cord, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00907 · Cited by 379 · see citing works
Memory retention – the synaptic stability versus plasticity dilemma
Abraham, W. C., & Robins, A. (2005) · Trends in Neurosciences · doi:10.1016/j.tins.2004.12.003 · Cited by 353 · see citing works
Self-citationClass-incremental Learning via Deep Model Consolidation
Zhang, J., Zhang, J., Ghosh, S., Li, D., Taşcı, Ş., Heck, L. P., Zhang, H., & Kuo, C. ‐. J. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093365 · Cited by 350 · see citing works
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning
Smith, J. S., Karlinsky, L., Gutta, V., Cascante-Bonilla, P., Kim, D., Arbelle, A., Panda, R., Feris, R., & Kira, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01146 · Cited by 358 · see citing works
Aljundi, R., Kelchtermans, K., & Tuytelaars, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01151 · Cited by 335 · see citing works
(2013) · Cited by 290 · see citing works
PLOP: Learning without Forgetting for Continual Semantic Segmentation
Douillard, A., Chen, Y., Dapogny, A., & Cord, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00403 · Cited by 274 · see citing works
Connectionist modelling in psychology: A localist manifesto
Page, M. P. A. (2000) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00003356 · Cited by 268 · see citing works
Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic ForgettingPreprint
Li, X., Zhou, Y., Wu, T., Socher, R., & Xiong, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.00310 · Cited by 255 · see citing works
Incremental learning algorithms and applications
Gepperth, A., & Hammer, B. (2016) · HAL · Cited by 253 · see citing works
Biological underpinnings for lifelong learning machines
Kudithipudi, D., Aguilar-Simon, M., Babb, J., Bazhenov, M., Blackiston, D., Bongard, J., Brna, A. P., Chakravarthi Raja, S., Cheney, N., Clune, J., et al. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00452-0 · Cited by 236 · see citing works
Towards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224 · see citing works
Co 2 L: Contrastive Continual Learning
Cha, H., Lee, J., & Shin, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00938 · Cited by 225 · see citing works
Overcoming catastrophic forgetting with hard attention to the task
Serrà, J., Surís, D., Miron, M., & Karatzoglou, A. (2018) · arXiv · doi:10.48550/arxiv.1801.01423 · Cited by 213 · see citing works
NEUZZ: Efficient Fuzzing with Neural Program Smoothing
She, D., Pei, K., Epstein, D., Yang, J., Ray, B., & Jana, S. (2019) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2019.00052 · Cited by 238 · see citing works
A comparison of techniques for developing predictive models of software metrics
Gray, A. R., & MacDonell, S. G. (1997) · Information and Software Technology · doi:10.1016/s0950-5849(96)00006-7 · Cited by 188 · see citing works
Distilling Causal Effect of Data in Class-Incremental Learning
Hu, X., Tang, K., Miao, C., Hua, X., & Zhang, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00395 · Cited by 199 · see citing works
Towards Robust Evaluations of Continual LearningPreprint
Farquhar, S., & Gal, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09733 · Cited by 202 · see citing works
Adversarial Continual Learning
Ebrahimi, S., Meier, F., Calandra, R., Darrell, T., & Rohrbach, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58621-8_23 · Cited by 178 · see citing works
Memory-Efficient Incremental Learning Through Feature Adaptation
İşcen, A., Zhang, J., Lazebnik, S., & Schmid, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_41 · Cited by 170 · see citing works
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning
Smith, J., Hsu, Y.-C., Balloch, J., Shen, Y., Jin, H., & Kira, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00924 · Cited by 169 · see citing works
Generative replay with feedback connections as a general strategy for continual learningPreprint
van de Ven, G. M., & Tolias, A. S. (2018) · Lirias · doi:10.48550/arxiv.1809.10635 · Cited by 194 · see citing works
Latent Replay for Real-Time Continual Learning
Pellegrini, L., Graffieti, G., Lomonaco, V., & Maltoni, D. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341460 · Cited by 19 · see citing works
Self-Supervised Models are Continual Learners
Fini, E., da Costa, V. G. T., Alameda-Pineda, X., Ricci, E., Alahari, K., & Mairal, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00940 · Cited by 155 · see citing works
Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills
Ellefsen, K. O., Mouret, J.-B., & Clune, J. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004128 · Cited by 148 · see citing works
Continual learning with hypernetworks
von Oswald, J., Henning, C., Grewe, B. F., & Sacramento, J. (2019) · Zurich Open Repository and Archive (University of Zurich) · doi:10.3929/ethz-b-000465911 · Cited by 189 · see citing works
Consolidation of Long-Term Memory: Evidence and Alternatives.
Meeter, M., & Murre, J. M. J. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.6.843 · Cited by 148 · see citing works
Linear hidden transformations for adaptation of hybrid ANN/HMM models
Gemello, R., Mana, F., Scanzio, S., Laface, P., & Mori, R. D. (2007) · Speech Communication · doi:10.1016/j.specom.2006.11.005 · Cited by 147 · see citing works
RECALL: Replay-based Continual Learning in Semantic Segmentation
Maracani, A., Michieli, U., Toldo, M., & Zanuttigh, P. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00694 · Cited by 141 · see citing works
Mundt, M., Hong, Y., Pliushch, I., & Ramesh, V. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.01.014 · Cited by 148 · see citing works
Generative Feature Replay For Class-Incremental Learning
Liu, X., Wu, C., Menta, M., Herranz, L., Raducanu, B., Bagdanov, A. D., Jui, S., & van de Weijer, J. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00121 · Cited by 135 · see citing works
Class-Incremental Continual Learning Into the eXtended DER-Verse
Boschini, M., Bonicelli, L., Buzzega, P., Porrello, A., & Calderara, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3206549 · Cited by 133 · see citing works
Gradient Projection Memory for Continual Learning
Saha, G., Garg, I., & Roy, K. (2021) · arXiv · doi:10.48550/arxiv.2103.09762 · Cited by 129 · see citing works
The Task Rehearsal Method of Life-Long Learning: Overcoming Impoverished Data
Silver, D. L., & Mercer, R. E. (2002) · Lecture notes in computer science · doi:10.1007/3-540-47922-8_8 · Cited by 127 · see citing works
Overcoming Catastrophic Forgetting in Graph Neural Networks
Liu, H., Yang, Y., & Wang, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17049 · Cited by 127 · see citing works
Continual learning for recurrent neural networks: An empirical evaluation
Cossu, A., Carta, A., Lomonaco, V., & Bacciu, D. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.07.021 · Cited by 126 · see citing works
Learning in situ: a randomized experiment in video streamingPreprint
Yan, F. Y., Ayers, H. R., Zhu, C., Fouladi, S., Hong, J. W., Zhang, K., Levis, P. A., & Winstein, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.01113 · Cited by 124 · see citing works
Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks
Soltoggio, A., Stanley, K. O., & Risi, S. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.013 · Cited by 124 · see citing works
An Appraisal of Incremental Learning Methods
Luo, Y., Yin, L., Bai, W., & Mao, K. (2020) · Entropy · doi:10.3390/e22111190 · Cited by 120 · see citing works
GCR: Gradient Coreset based Replay Buffer Selection for Continual Learning
Tiwari, R., Killamsetty, K., Iyer, R., & Shenoy, P. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00020 · Cited by 122 · see citing works
Deep Generative Dual Memory Network for Continual LearningPreprint
Kamra, N., Gupta, U., & Liu, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10368 · Cited by 129 · see citing works
Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization
Parisi, G. I., Tani, J., Weber, C., & Wermter, S. (2018) · Institutional Repositories DataBase (IRDB) · Cited by 120 · see citing works
Rethinking Experience Replay: a Bag of Tricks for Continual Learning
Buzzega, P., Boschini, M., Porrello, A., & Calderara, S. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412614 · Cited by 118 · see citing works
Pseudo-recurrent Connectionist Networks: An Approach to the 'Sensitivity-Stability' Dilemma
French, R. M. (1997) · Connection Science · doi:10.1080/095400997116595 · Cited by 116 · see citing works
Uncertainty-guided Continual Learning with Bayesian Neural NetworksPreprint
Ebrahimi, S., Elhoseiny, M., Darrell, T., & Rohrbach, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02425 · Cited by 123 · see citing works
Replay in Deep Learning: Current Approaches and Missing Biological Elements
Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski, T. J., & Kanan, C. (2021) · Neural Computation · doi:10.1162/neco_a_01433 · Cited by 106 · see citing works
A Unified Continual Learning Framework with General Parameter-Efficient Tuning
Gao, Q., Zhao, C., Sun, Y., Xi, T., Zhang, G., Ghanem, B., & Zhang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01055 · Cited by 104 · see citing works
Online Continual Learning with Maximally Interfered RetrievalPreprint
Aljundi, R., Caccia, L., Belilovsky, E., Caccia, M., Lin, M., Charlin, L., & Tuytelaars, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04742 · Cited by 107 · see citing works
Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization
Parisi, G. I., Tani, J., Weber, C., & Wermter, S. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00078 · Cited by 126 · see citing works
Functional Regularisation for Continual Learning with Gaussian ProcessesPreprint
Titsias, M. K., Schwarz, J., Matthews, A., Pascanu, R., & Teh, Y. W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.11356 · Cited by 92 · see citing works
Pseudo-rehearsal: Achieving deep reinforcement learning without catastrophic forgetting
Atkinson, C., Brendan Mccane, A., Szymanski, L., & Robins, A. V. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.11.050 · Cited by 91 · see citing works
Self-citationKinder, A., & Shanks, D. R. (2003) · Psychological Review · doi:10.1037/0033-295x.110.4.728 · Cited by 90 · see citing works
TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation
Zhang, J., Chen, C., Zhuang, W., & Lyu, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00441 · Cited by 90 · see citing works
Avoiding catastrophic forgetting by coupling two reverberating neural networks
Ans, B., & Rousset, S. (1997) · Comptes Rendus de l Académie des Sciences - Series III - Sciences de la Vie · doi:10.1016/s0764-4469(97)82472-9 · Cited by 87 · see citing works
Selfless Sequential LearningPreprint
Aljundi, R., Rohrbach, M., & Tuytelaars, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05421 · Cited by 99 · see citing works
Consolidation in Neural Networks and in the Sleeping Brain
Robins, A. V. (1996) · Connection Science · doi:10.1080/095400996116910 · Cited by 83 · see citing works
Self-citationR-DFCIL: Relation-Guided Representation Learning for Data-Free Class Incremental Learning
Gao, Q., Zhao, C., Ghanem, B., & Zhang, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20050-2_25 · Cited by 81 · see citing works
Memory Replay GANs: learning to generate images from new categories without forgettingPreprint
Wu, C., Herranz, L., Liu, X., Wang, Y., van de Weijer, J., & Raducanu, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02058 · Cited by 80 · see citing works
Continual Object Detection: A review of definitions, strategies, and challenges
Menezes, A. G., de Moura, G., Alves, C., & de Carvalho, A. C. P. L. F. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.01.041 · Cited by 77 · see citing works
Resistance to interference in human associative learning: Evidence of configural processing.
Shanks, D. R., Darby, R. J., & Charles, D. (1998) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.24.2.136 · Cited by 72 · see citing works
Continual Learning: A Review of Techniques, Challenges, and Future Directions
Wickramasinghe, B., Saha, G., & Roy, K. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3339091 · Cited by 73 · see citing works
Sentiment analysis for customer relationship management: an incremental learning approach
Capuano, N., Greco, L., Ritrovato, P., & Vento, M. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01984-x · Cited by 69 · see citing works
Continual Learning with Lifelong Vision Transformer
Wang, Z., Liu, L., Duan, Y., Kong, Y., & Tao, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00027 · Cited by 70 · see citing works
A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials
Li, C., Huang, Z., Paudel, D. P., Wang, Y., Shahbazi, M., Hong, X., & Gool, L. V. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00139 · Cited by 69 · see citing works
Knowledge-based cascade-correlation: Using knowledge to speed learning
Shultz, T. R., & Rivest, F. (2001) · Connection Science · doi:10.1080/09540090110047767 · Cited by 67 · see citing works
Graph Lifelong Learning: A Survey
Febrinanto, F. G., Xia, F., Moore, K., Thapa, C., & Aggarwal, C. C. (2023) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2022.3222049 · Cited by 67 · see citing works
Recent Advances of Continual Learning in Computer Vision: An Overview
Qu, H., Rahmani, H., Xu, L., Williams, B. M., & Liu, J. (2025) · IET Computer Vision · doi:10.1049/cvi2.70013 · Cited by 80 · see citing works
Ans, B., & Rousset, S. (2000) · Connection Science · doi:10.1080/095400900116177 · Cited by 66 · see citing works
Lessons from infant learning for unsupervised machine learning
Zaadnoordijk, L., Besold, T. R., & Cusack, R. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00488-2 · Cited by 65 · see citing works
French, R. M., & Nick Chater, A. (2002) · Neural Computation · doi:10.1162/08997660260028700 · Cited by 64 · see citing works
CIRCLE: continual repair across programming languages
Yuan, W., Zhang, Q., He, T., Fang, C., Nguyen, Q. V. H., Hao, X., & Yin, H. (2022) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3533767.3534219 · Cited by 65 · see citing works
Continuous transfer of neural network representational similarity for incremental learning
Tian, S., Li, W., Ning, X., Ran, H., Qin, H., & Tiwari, P. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126300 · Cited by 63 · see citing works
Efficient Architecture Search for Continual Learning
Gao, Q., Luo, Z., Klabjan, D., & Zhang, F. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3151511 · Cited by 62 · see citing works
Continual Learning for Affective Robotics: Why, What and How?
Churamani, N., Kalkan, S., & Güneş, H. (2020) · IEEE RO-MAN · doi:10.1109/ro-man47096.2020.9223564 · Cited by 62 · see citing works
Churamani, N., Kara, Ö., & Güneş, H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3181033 · Cited by 62 · see citing works
Ensemble Learning in Fixed Expansion Layer Networks for Mitigating Catastrophic Forgetting
Coop, R., Mishtal, A., & Arel, I. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2264952 · Cited by 61 · see citing works
Sarmas, E., Strompolas, S., Marinakis, V., Santori, F., Bucarelli, M. A., & Doukas, H. (2022) · Electronics · doi:10.3390/electronics11233962 · Cited by 60 · see citing works
A Closer Look at Rehearsal-Free Continual Learning *
Smith, J. S., Tian, J., Halbe, S., Hsu, Y.-C., & Kira, Z. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00239 · Cited by 60 · see citing works
Feng, Y., & Tu, Y. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2015617118 · Cited by 59 · see citing works
Task Agnostic Continual Learning via Meta LearningPreprint
He, X., Sygnowski, J., Galashov, A., Rusu, A. A., Teh, Y. W., & Pascanu, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05201 · Cited by 59 · see citing works
Sleep-like unsupervised replay reduces catastrophic forgetting in artificial neural networks
Tadros, T., Krishnan, G. P., Ramyaa, R., & Bazhenov, M. (2022) · Nature Communications · doi:10.1038/s41467-022-34938-7 · Cited by 59 · see citing works
Tracelink: A model of consolidation and amnesia
Meeter, M., & Murre, J. M. J. (2005) · Cognitive Neuropsychology · doi:10.1080/02643290442000194 · Cited by 57 · see citing works
- Authors
- ROBINS, A.
- Affiliations
- University of Otago (NZ)
- Corresponding author
- ROBINS, ANTHONY
- ORCID
- ROBINS, ANTHONY
- Venue
- Connection Science
- Publisher
- Informa UK Limited
- Published
- 1995-06
- Volume
- 7
- Issue
- 2
- Pages
- 123-146
- ISSN
- 0954-0091 (ISSN-L)
- 1360-0494
- OpenAlex
- W2116522068
- Cited by
- 833
- References
- 18
- Citations by year
- 2026: 52 · 2025: 79 · 2024: 92 · 2023: 109 · 2022: 94 · 2021: 100 · 2020: 77 · 2019: 68 · 2018: 23 · 2017: 15 · 2016: 4 · 2015: 6 · 2014: 3 · 2013: 8 · 2012: 1
- FWCI
- 6.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Connection Science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- catastrophic forgetting, rehearsal, neural network learning, function approximation, incremental learning, knowledge retention
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
