Cited by 833Domain Adaptation and Few-Shot LearningMemory Processes and InfluencesNeural Networks and Applications

The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.

Catastrophic Forgetting, Rehearsal and Pseudorehearsal

ROBINS, ANTHONY·1995·Connection Science

Open this paper in CoChat — free account

Abstract

This paper reviews the problem of catastrophic forgetting (the loss or disruption of previously learned information when new information is learned) in neural networks, and explores rehearsal mechanisms (the retraining of some of the previously learned information as the new information is added) as a potential solution. We replicate some of the experiments described by Ratcliff (1990), including those relating to a simple 'recency' based rehearsal regime. We then develop further rehearsal regimes which are more effective than recency rehearsal. In particular, 'sweep rehearsal' is very successful at minimizing catastrophic forgetting. One possible limitation of rehearsal in general, however, is that previously learned information may not be available for retraining. We describe a solution to this problem, 'pseudorehearsal', a method which provides the advantages of rehearsal without actually requiring any access to the previously learned information (the original training population) itself. We then suggest an interpretation of these rehearsal mechanisms in the context of a function approximation based account of neural network learning. Both rehearsal and pseudorehearsal may have practical applications, allowing new information to be integrated into an existing network with minimum disruption of old information.

Integrity signals

Registry

Crossref · 10.1080/09540099550039318

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,501
Introduction to the theory of neural computation
Hertz, J., Krogh, A., & Palmer, R. G. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,410
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471
The ART of adaptive pattern recognition by a self-organizing neural network
Carpenter, G. A., & Grossberg, S. (1988) · Computer · doi:10.1109/2.33 · Cited by 1,278
Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions.
Ratcliff, R. (1990) · Psychological Review · doi:10.1037/0033-295x.97.2.285 · Cited by 939
Artificial Neural Networks: Approximation and Learning Theory
White, H. (1992) · Medical Entomology and Zoology · Cited by 441
Networks of Formal Neurons and Memory Palimpsests
Nadal, J., Toulouse, G., Changeux, J., & Dehaene, S. (1986) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/1/10/008 · Cited by 215
From Statistics to Neural Networks: Theory and Pattern Recognition Applications
Cherkassky, V., Friedman, J. H., & Wechsler, H. L. (1996) · CERN Document Server (European Organization for Nuclear Research) · Cited by 177
Prediction Risk and Architecture Selection for Neural Networks
Moody, J. E. (1994) · Springer eBooks · doi:10.1007/978-3-642-79119-2_7 · Cited by 158

Show all 18 references

Sorted by citations

Items 101–200 of 830

A Dynamic Role of the Medial Temporal Lobe during Retrieval of Declarative Memory in Man
Petersson, K. M., Elfgren, C., & Ingvar, M. (1997) · NeuroImage · doi:10.1006/nimg.1997.0276 · Cited by 58
Rehearsal-Free Continual Learning over Small Non-I.I.D. Batches
Lomonaco, V., Maltoni, D., & Pellegrini, L. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00131 · Cited by 57
Continual learning and catastrophic forgetting
van de Ven, G. M., Soures, N., & Kudithipudi, D. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-15754-7.00073-0 · Cited by 165
Toward Understanding Catastrophic Forgetting in Continual LearningPreprint
Nguyen, C. V., Achille, A., Lam, M., Hassner, T., Mahadevan, V., & Soatto, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01091 · Cited by 73
Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2024) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2024.3966 · Cited by 54
Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2023) · arXiv · doi:10.48550/arxiv.2312.10549 · Cited by 54
Class-Incremental Learning with Generative Classifiers
van de Ven, G. M., Li, Z., & Tolias, A. S. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00400 · Cited by 56
Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural NetworksPreprint
Atkinson, C., Brendan Mccane, A., Szymanski, L., & Robins, A. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03875 · Cited by 54
Self-citation
Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay
Rostami, M. G., Kolouri, S., & Pilly, P. K. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/463 · Cited by 65
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
Ramasesh, V., Dyer, E., & Raghu, M. (2021) · International Conference on Learning Representations · Cited by 53
FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity
Li, G., Hu, Y., Zhang, M., Liu, J., Yin, Q., Peng, Y., & Dou, D. (2022) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3545008.3545065 · Cited by 52
Learning, Memory, and the Role of Neural Network Architecture
Hermundstad, A. M., Brown, K., Bassett, D. S., & Carlson, J. M. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002063 · Cited by 52
Continual Learning Using Bayesian Neural Networks
Li, H., Barnaghi, P. M., Enshaeifar, S., & Ganz, F. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3017292 · Cited by 58
Continual Learning with Adaptive Weights (CLAW)
Adel, T., Zhao, H., & Turner, R. E. (2019) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · doi:10.48550/arxiv.1911.09514 · Cited by 66
A Unifying Bayesian View of Continual LearningPreprint
Farquhar, S., & Gal, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06494 · Cited by 80
Enhancing textual textbook question answering with large language models and retrieval augmented generation
Alawwad, H. A., Alhothali, A., Naseem, U., Alkhathlan, A., & Jamal, A. T. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2024.111332 · Cited by 96
Lifelong Graph Learning
Wang, C., Qiu, Y., Gao, D., & Scherer, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01335 · Cited by 57
Editable Neural Networks
Sinitsin, A., Plokhotnyuk, V., Pyrkin, D., Popov, S., & Babenko, A. (2020) · arXiv · doi:10.48550/arxiv.2004.00345
A neural network account of memory replay and knowledge consolidation
Barry, D. N., & Love, B. C. (2022) · Cerebral Cortex · doi:10.1093/cercor/bhac054 · Cited by 52
Self-refreshing memory in artificial neural networks: learning temporal sequences without catastrophic forgetting
Ans, B., Rousset, S., French, R. M., & Musca, S. C. (2004) · Connection Science · doi:10.1080/09540090412331271199 · Cited by 48
Continual Learning with Bayesian Neural Networks for Non-Stationary Data
Kurle, R., Cseke, B., Klushyn, A., van der Smagt, P., & Günnemann, S. (2020) · International Conference on Learning Representations · Cited by 49
SDDGR: Stable Diffusion-Based Deep Generative Replay for Class Incremental Object Detection
Kim, J., Cho, H., Kim, J., Tiruneh, Y. Y., & Baek, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02718 · Cited by 55
Deep imitation learning for 3D navigation tasks
Hussein, A. M., Elyan, E., Gaber, M. M., & Jayne, C. (2017) · Neural Computing and Applications · doi:10.1007/s00521-017-3241-z · Cited by 57
Efficient continual learning in neural networks with embedding regularization
Pomponi, J., Scardapane, S., Lomonaco, V., & Uncini, A. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.093 · Cited by 47
Dendritic Computing: Branching Deeper into Machine Learning
Acharya, J., Basu, A., Legenstein, R. A., Limbacher, T., Poirazi, P., & Wu, X. (2021) · Neuroscience · doi:10.1016/j.neuroscience.2021.10.001 · Cited by 54
SOINN+, a Self-Organizing Incremental Neural Network for Unsupervised Learning from Noisy Data Streams
Wiwatcharakoses, C., & Berrar, D. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.113069 · Cited by 46
Incremental Zero-Shot Learning
Wei, K., Deng, C. D., Yang, X., & Tao, D. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3110369 · Cited by 46
Catastrophic Forgetting and the Pseudorehearsal Solution in Hopfield-type Networks
Robins, A. V., & McCallum, S. (1998) · Connection Science · doi:10.1080/095400998116530 · Cited by 49
Self-citation
Incremental learning with neural networks for computer vision: a survey
Liu, H., Zhou, Y. Z., Liu, B., Zhao, J., Yao, R., & Shao, Z. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10294-2 · Cited by 49
Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective
Sun, H., Pu, W., Fu, X., Chang, T., & Hong, M. (2022) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2022.3143372 · Cited by 46
Class-Incremental Unsupervised Domain Adaptation via Pseudo-Label Distillation
Wei, K., Yang, X., Xu, Z., & Deng, C. D. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3357258 · Cited by 44
Detection and Continual Learning of Novel Face Presentation Attacks
Rostami, M. G., Spinoulas, L., Hussein, M. E., Mathai, J., & AbdAlmageed, W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01458 · Cited by 53
Remembering for the right reasons: Explanations reduce catastrophic forgetting
Ebrahimi, S., Petryk, S., Gokul, A., Gan, W., Gonzalez, J. E., Rohrbach, M., & Darrell, T. (2021) · Applied AI Letters · doi:10.1002/ail2.44 · Cited by 43
Linear Mode Connectivity in Multitask and Continual LearningPreprint
Mirzadeh, S. I., Farajtabar, M., Görür, D., Pascanu, R., & Ghasemzadeh, H. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04495 · Cited by 55
Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks
Velez, R., & Clune, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0187736 · Cited by 49
Transfer Without Forgetting
Boschini, M., Bonicelli, L., Porrello, A., Bellitto, G., Pennisi, M., Palazzo, S., Spampinato, C., & Calderara, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20050-2_40 · Cited by 49
Boosting Soft Actor-Critic: Emphasizing Recent Experience without Forgetting the PastPreprint
Wang, C., & Ross, K. W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04009 · Cited by 48
DER: Dynamically Expandable Representation for Class Incremental LearningPreprint
Yan, S., Xie, J., & He, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.16788 · Cited by 41
Curiosity-Driven Class-Incremental Learning via Adaptive Sample Selection
Hu, Q., Gao, Y., & Cao, B. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3196092 · Cited by 40
Adaptation of Hybrid ANN/HMM Models Using Linear Hidden Transformations and Conservative Training
Gemello, R., Mana, F., Scanzio, S., Laface, P., & Mori, R. D. (2006) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2006.1660239 · Cited by 40
Unsupervised Learning to Overcome Catastrophic Forgetting in Neural Networks
Muñoz-Martín, I., Bianchi, S., Pedretti, G., Melnic, O., Ambrogio, S., & Ielmini, D. (2019) · IEEE Journal on Exploratory Solid-State Computational Devices and Circuits · doi:10.1109/jxcdc.2019.2911135 · Cited by 40
Analyzing the Forgetting Problem in Pretrain-Finetuning of Open-domain Dialogue Response Models
He, T., Liu, J., Cho, K., Ott, M., Liu, B., Glass, J. R., & Peng, F. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.95 · Cited by 43
The consolidation of learning during sleep: comparing the pseudorehearsal and unlearning accounts
Robins, A. V., & McCallum, S. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00056-8 · Cited by 40
Self-citation
Catastrophic Forgetting in Connectionist Networks
French, R. M. (2005) · Encyclopedia of Cognitive Science · doi:10.1002/0470018860.s00096 · Cited by 40
Continual Reinforcement Learning in 3D Non-stationary Environments
Lomonaco, V., Desai, K., Culurciello, E., & Maltoni, D. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00132 · Cited by 41
Memory Recall: A Simple Neural Network Training Framework Against Catastrophic Forgetting
Zhang, B., Guo, Y., Li, Y., He, Y., Wang, H., & Dai, Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3099700 · Cited by 42
When Prompt-based Incremental Learning Does Not Meet Strong Pretraining
Tang, Y.-M., Peng, Y.-X., & Zheng, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00164 · Cited by 40
Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
Mundt, M., Pliushch, I., Majumder, S., Hong, Y., & Ramesh, V. (2022) · Journal of Imaging · doi:10.3390/jimaging8040093 · Cited by 41
Towards Continual Egocentric Activity Recognition: A Multi-Modal Egocentric Activity Dataset for Continual Learning
Xu, L., Wu, Q., Pan, L., Meng, F., Li, H., He, C., Wang, H., Cheng, S., & Dai, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3295899 · Cited by 47
Sequential Consolidation of Learned Task Knowledge
Silver, D. L., & Poirier, R. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24840-8_16 · Cited by 37
A Strategy for an Uncompromising Incremental Learner
Venkatesan, R., Venkateswara, H., Panchanathan, S., & Li, B. (2017) · arXiv · doi:10.48550/arxiv.1705.00744 · Cited by 37
LLM-Commentator: Novel fine-tuning strategies of large language models for automatic commentary generation using football event data
Cook, A., & Karakuş, O. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112219 · Cited by 42
Augmented Geometric Distillation for Data-Free Incremental Person ReID
Lu, Y., Wang, M., & Deng, W. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00718 · Cited by 36
Critical periods and catastrophic interference effects in the development of self‐organizing feature maps
Richardson, F. M., & Thomas, M. S. C. (2008) · Developmental Science · doi:10.1111/j.1467-7687.2008.00682.x · Cited by 36
Psycholinguistics Meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering
Greco, C., Plank, B., Fernández, R., & Raffaella Bernardi, B. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1350 · Cited by 38
Contributions by metaplasticity to solving the Catastrophic Forgetting Problem
Jedlička, P., Tomko, M., Robins, A. V., & Abraham, W. C. (2022) · Trends in Neurosciences · doi:10.1016/j.tins.2022.06.002 · Cited by 36
Self-citation
EEC: Learning to Encode and Regenerate Images for Continual LearningPreprint
Ayub, A., & Wagner, A. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.04904 · Cited by 41
FedKNOW: Federated Continual Learning with Signature Task Knowledge Integration at Edge
Luopan, Y., Han, R., Zhang, Q., Liu, C. H., Wang, G., & Chen, L. Y. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00033 · Cited by 37
Online Continual Learning on Sequences
Parisi, G. I., & Lomonaco, V. (2020) · Studies in computational intelligence · doi:10.1007/978-3-030-43883-8_8 · Cited by 39
Class Incremental Learning with Multi-Teacher Distillation
Wen, H., Pan, L., Dai, Y., Qiu, H., Wang, L., Wu, Q., & Li, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02687 · Cited by 36
Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning
Goswami, D., Soutif--Cormerais, A., Liu, Y., Kamath, S., Twardowski, B., & van de Weijer, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02695 · Cited by 34
Incremental learning using generative-rehearsal strategy for fault detection and classification
Lee, S., Chang, K., & Baek, J. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115477 · Cited by 32
Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic Samples
Xia, Y., Wang, Q., Lyu, Y., Yong, Z., Wu, W., Li, S., & Dai, D. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.179 · Cited by 35
Sequential learning in neural networks: A review and a discussion of pseudorehearsal based methods
Robins, A. V. (2004) · Intelligent Data Analysis · doi:10.3233/ida-2004-8306 · Cited by 32
Self-citation
CLIFER: Continual Learning with Imagination for Facial Expression Recognition
Churamani, N., & Güneş, H. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00110 · Cited by 34
Sample Condensation in Online Continual Learning
Sangermano, M., Carta, A., Cossu, A., & Bacciu, D. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892299 · Cited by 33
The recommendation architecture: lessons from large-scale electronic systems applied to cognition
Coward, L. A. (2001) · Cognitive Systems Research · doi:10.1016/s1389-0417(01)00024-9 · Cited by 31
A Connectionist Account of Interference Effects in Early Infant Memory and Categorization
Denis Mareschal, B., & French, R. M. (1997) · BIROn (Birkbeck, University of London) · Cited by 31
Continual Learning for Image Segmentation With Dynamic Query
Wu, W., Zhao, Y., Li, Z., Shan, L., Zhou, H., & Shou, M. Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3337884 · Cited by 30
Incremental Concept Learning via Online Generative Memory Recall
Li, H., Dong, W., & Hu, B.-G. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3010581 · Cited by 32
CoMFormer: Continual Learning in Semantic and Panoptic Segmentation
Cermelli, F., Cord, M., & Douillard, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00294 · Cited by 30
Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning
Jin, X., Lin, B. Y., Rostami, M. G., & Ren, X. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.62 · Cited by 39
Geometry and Uncertainty-Aware 3D Point Cloud Class-Incremental Semantic Segmentation
Yang, Y., Hayat, M., Jin, Z., Ren, C., & Lei, Y.-J. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.02084 · Cited by 33
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora
Jin, X., Zhang, D., Zhu, H., Xiao, W., Li, S.-W. D., Wei, X., Arnold, A., & Ren, X. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.351 · Cited by 38
Continual Learning with Gated Incremental Memories for sequential data processing
Cossu, A., Carta, A., & Bacciu, D. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207550 · Cited by 28
Preventing Catastrophic Interference in Multiple-Sequence Learning Using Coupled Reverberating Elman Networks
Ans, B., Rousset, S., French, R. M., & Musca, S. C. (2019) · Routledge eBooks · doi:10.4324/9781315782379-50 · Cited by 28
Lifelong Object DetectionPreprint
Zhou, W., Chang, S., Sosa, N. E., Hamann, H. F., & Cox, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01129 · Cited by 42
Convolutional Prompting meets Language Models for Continual Learning
Roy, A., Moulick, R., Verma, V. K., Ghosh, S., & Das, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02229 · Cited by 31
Lifelong Learning of Graph Neural Networks for Open-World Node Classification
Galke, L. P. A., Franke, B., Zielke, T., & Scherp, A. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533412 · Cited by 32
Learning an Adaptive Meta Model-Generator for Incrementally Updating Recommender Systems
Peng, D., Pan, S. J., Zhang, J., & Zeng, A. (2021) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3460231.3474239 · Cited by 27
Continual Learning with Scaled Gradient Projection
Saha, G., & Roy, K. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26157 · Cited by 28
Continuous Domain Adaptation with Variational Domain-Agnostic Feature ReplayPreprint
Lao, Q., Jiang, X., Havaei, M., & Bengio, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04382 · Cited by 27
Improved broad learning system for machinery intelligent fault diagnosis with increasing fault samples, fault modes, and running conditions
Fu, Y., Cao, H., Chen, X., & Ding, J. (2022) · ISA Transactions · doi:10.1016/j.isatra.2022.10.014 · Cited by 26
Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey
Li, Y., Wang, H., Xu, W., Xiao, T., Liu, H., Tu, M., Wang, Y., Yang, X., Zhang, R., Yu, S., et al. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3568279 · Cited by 36
Cyberattack Prediction Through Public Text Analysis and Mini-Theories
Perera, I., Hwang, J. D., Bayas, K., Dorr, B. J., & Wilks, Y. A. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622106 · Cited by 26
Generative Memory for Lifelong Learning
Su, X., Guo, S., Tan, T., & Chen, F. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2927369 · Cited by 26
Dynamic Memory to Alleviate Catastrophic Forgetting in Continuous Learning Settings
Hofmanninger, J., Perkonigg, M., Brink, J. A., Pianykh, O. S., Herold, C., & Langs, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59713-9_35 · Cited by 26
A Two-Stream Continual Learning System With Variational Domain-Agnostic Feature Replay
Lao, Q., Jiang, X., Havaei, M., & Bengio, Y. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3057453 · Cited by 56
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora
Jin, X., Zhang, D., Zhu, H., Xiao, W., Li, S.-W. D., Wei, X., Arnold, A., & Ren, X. (2022) · doi:10.18653/v1/2022.bigscience-1.1 · Cited by 29
Deep Bayesian Unsupervised Lifelong Learning
Zhao, T., Wang, Z., Masoomi, A., & Dy, J. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.02.001 · Cited by 28
A Dirichlet Process Mixture of Robust Task Models for Scalable Lifelong Reinforcement Learning
Wang, Z., Chen, C., & Dong, D. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3170485 · Cited by 25
Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition
Zheng, J., Liang, Z., Chen, H., & Ma, Q. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.236 · Cited by 27
Reinforcement Learning in Continuous Time and Space: Interference and Not Ill Conditioning Is the Main Problem When Using Distributed Function Approximators
Baddeley, B. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.921000 · Cited by 26
Pseudopatterns and dual-network memory models: Advantages and shortcomings
French, R. M., Ans, B., & Rousset, S. (2001) · Perspectives in Neural Computing · doi:10.1007/978-1-4471-0281-6_2 · Cited by 25
Practical Recommendations for Replay-Based Continual Learning Methods
Merlin, G., Lomonaco, V., Cossu, A., Carta, A., & Bacciu, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13324-4_47 · Cited by 25
Lifelong Learning With Cycle Memory Networks
Peng, J., Ye, D., Tang, B., Lei, Y., Liu, Y., & Li, H. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3294495 · Cited by 25
Subspace distillation for continual learning
Roy, K., Simon, C., Moghadam, P., & Harandi, M. T. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.07.047 · Cited by 31
Incremental Generalized Category Discovery
Zhao, B., & Aodha, O. M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01753 · Cited by 31
A neural network based retrainable framework for robust object recognition with application to mobile robotics
An, S.-Y., Kang, J.-G., Choi, W., & Oh, S. (2010) · Applied Intelligence · doi:10.1007/s10489-010-0212-9 · Cited by 25
Connectionism and Classical Conditioning
Dawson, M. R. W. (2008) · Comparative Cognition & Behavior Reviews · doi:10.3819/ccbr.2008.30008 · Cited by 25

Authors: ROBINS, A.
Affiliations: University of Otago (NZ)
Corresponding author: ROBINS, ANTHONY
ORCID: ROBINS, ANTHONY
Venue: Connection Science
Publisher: Informa UK Limited
Published: 1995-06
Volume: 7
Issue: 2
Pages: 123-146
ISSN: 0954-0091 (ISSN-L)
1360-0494
DOI: 10.1080/09540099550039318
OpenAlex: W2116522068
Cited by: 833
References: 18
Citations by year: 2026: 52 · 2025: 79 · 2024: 92 · 2023: 109 · 2022: 94 · 2021: 100 · 2020: 77 · 2019: 68 · 2018: 23 · 2017: 15 · 2016: 4 · 2015: 6 · 2014: 3 · 2013: 8 · 2012: 1
FWCI: 6.68
Type: journal article
Language: en
Open access: closed
Locations: Connection Science (publishedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords: catastrophic forgetting, rehearsal, neural network learning, function approximation, incremental learning, knowledge retention

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Catastrophic Forgetting, Rehearsal and Pseudorehearsal

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks