The DOI matches its DataCite record; 32 of the 34 references it cites were found in a registry or an index.
Overcoming catastrophic forgetting with hard attention to the task
Open accessAbstract
Catastrophic forgetting occurs when a neural network loses the information learned in a previous task after training on subsequent tasks. This problem remains a hurdle for artificial intelligence systems with sequential learning capabilities. In this paper, we propose a task-based hard attention mechanism that preserves previous tasks' information without affecting the current task's learning. A hard attention mask is learned concurrently to every task, through stochastic gradient descent, and previous masks are exploited to condition such learning. We show that the proposed mechanism is effective for reducing catastrophic forgetting, cutting current rates by 45 to 80%. We also show that it is robust to different hyperparameter choices, and that it offers a number of monitoring capabilities. The approach features the possibility to control both the stability and compactness of the learned knowledge, which we believe makes it also attractive for online learning or network compression applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Methods like regularizing changes to prevent overwriting important weights and applying soft masks to selectively update parameters can largely preserve the previous domain knowledge while recognizing new information (Serra et al., 2018).”
From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Serra et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
A logical calculus of the ideas immanent in nervous activity
McCulloch, W. S., & Pitts, W. (1943) · The Bulletin of Mathematical Biophysics · doi:10.1007/bf02478259 · Cited by 18,566
(2017) · Cited by 11,625
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Xiao, H., Rasul, K., & Vollgraf, R. (2017) · arXiv · doi:10.48550/arxiv.1708.07747 · Cited by 6,108
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
Regularized multi--task learning
Evgeniou, T., & Pontil, M. (2004) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1014052.1014067 · Cited by 1,473
Research works printed but not linked: 13
- Bakker, B. and Heskes, T. Task clustering and gating for bayesian multitask learning. Journal of Machine Learn- ing Research, 4:83-99, 2003.
- Clegg, B. A., DiGirolamo, G. J., and Keele, S. W. Sequence learning. Trends in Cognitive Sciences, 2(8):275-281, 1998.
- Glorot, X. and Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. of the Int. Conf. on Artificial Intelligence and Statistics (AISTATS), pp. 249-256, 2010.
- Krizhevsky, A., Sutskever, I., and Hinton, G. ImageNet classification with deep convolutional neural networks. In Pereira, F., Burges, C. J. C., Bottou, L., and Weinberger, K. Q. (eds.), Advances in Neural Information Processing Systems (NIPS), volume 25, pp. 1097-1105. 2012.
- LeCun, Y., Denker, J. S., and Solla, S. A. Optimal brain damage. In Touretzky, D. S. (ed.), Advances in Neural Information Processing Systems (NIPS), volume 2, pp. 598-605.
- Morgan Kaufmann, 1990.
- Legg, S. and Hutter, M. Universal intelligence: a definition of machine intelligence. Minds and Machines, 17(4): 391-444, 2007.
- Lopez-Paz, D. and Ranzato, M. A. Gradient episodic mem- ory for continuum learning. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems (NIPS), volume 30, pp. 6449-6458.
- Ratcliff, R. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological Review, 97:285-308, 1990.
- Rebuffi, S., Kolesnikov, A., Sperl, G., and Lampert, C. iCaRL: incremental classifier and representation learn- ing. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2001-2010, 2017.
- Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., and Had- sell, R. Progressive neural networks. ArXiv, 1606.04671, 2016. · arXiv
- Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. The German traffic sign recognition benchmark: a multi-class classification competition. In Proc. of the Int. Joint Conf. on Neural Networks (IJCNN), pp. 1453-1460, 2011.
- Venkatesan, R., Venkateswara, H., Panchanathan, S., and Li, B. A strategy for an uncompromising incremental learner. ArXiv, 1705.00744, 2017. · arXiv
Cited by
Sorted by citations
Adding Conditional Control to Text-to-Image Diffusion Models
Zhang, L., Rao, A., & Agrawala, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00355 · Cited by 4,547
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113
A Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647
Towards Open World Object Detection
Joseph, K. J., Khan, S. H., Khan, F. S., & Balasubramanian, V. N. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00577 · Cited by 606
Three scenarios for continual learningPreprint
van de Ven, G. M., & Tolias, A. S. (2019) · Lirias · doi:10.48550/arxiv.1904.07734 · Cited by 1,042
Learning to Learn without Forgetting by Maximizing Transfer and Minimizing InterferencePreprint
Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., & Tesauro, G. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11910 · Cited by 534
Embracing Change: Continual Learning in Deep Neural Networks
Hadsell, R., Rao, D., Rusu, A. A., & Pascanu, R. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.09.004 · Cited by 467
Class-incremental Learning via Deep Model Consolidation
Zhang, J., Zhang, J., Ghosh, S., Li, D., Taşcı, Ş., Heck, L. P., Zhang, H., & Kuo, C. ‐. J. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093365 · Cited by 361
Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning
Ostapenko, O., Puscas, M., Klein, T., Jähnichen, P., & Nabi, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01158 · Cited by 294
REMIND Your Neural Network to Prevent Catastrophic Forgetting
Hayes, T. L., Kafle, K., Shrestha, R., Acharya, M., & Kanan, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_28 · Cited by 286
Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
Masse, N. Y., Grant, G. D., & Freedman, D. J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1803839115 · Cited by 278
Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic ForgettingPreprint
Li, X., Zhou, Y., Wu, T., Socher, R., & Xiong, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.00310 · Cited by 255
Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
Hsu, Y.-C., Liu, Y.-C., Ramasamy, A., & Kira, Z. (2018) · arXiv · doi:10.48550/arxiv.1810.12488 · Cited by 211
Conditional Channel Gated Networks for Task-Aware Continual Learning
Abati, D., Tomczak, J. M., Blankevoort, T., Calderara, S., Cucchiara, R., & Bejnordi, B. E. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00399 · Cited by 182
Generative replay with feedback connections as a general strategy for continual learningPreprint
van de Ven, G. M., & Tolias, A. S. (2018) · Lirias · doi:10.48550/arxiv.1809.10635 · Cited by 194
Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion
Yang, S., Tan, J., & Chen, B. (2022) · Entropy · doi:10.3390/e24040455 · Cited by 157
Mundt, M., Hong, Y., Pliushch, I., & Ramesh, V. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.01.014 · Cited by 167
An Appraisal of Incremental Learning Methods
Luo, Y., Yin, L., Bai, W., & Mao, K. (2020) · Entropy · doi:10.3390/e22111190 · Cited by 120
Incremental Object Detection via Meta-Learning
Joseph, K. J., Rajasegaran, J., Khan, S. H., Khan, F. S., & Balasubramanian, V. N. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3124133 · Cited by 139
Overcoming Catastrophic Forgetting in Graph Neural Networks
Liu, H., Yang, Y., & Wang, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17049 · Cited by 127
MgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning
Zhao, H., Fu, Y., Kang, M., Tian, Q., Wu, F., & Li, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3133897 · Cited by 115
Memory-Efficient Class-Incremental Learning for Image Classification
Zhao, H., Wang, H., Fu, Y., Wu, F. Q., & Li, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3072041 · Cited by 120
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
Räuker, T., Ho, A., Casper, S., & Hadfield-Menell, D. (2023) · 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00039 · Cited by 122
Dynamic Memory to Alleviate Catastrophic Forgetting in Continual Learning with Medical Imaging
Perkonigg, M., Hofmanninger, J., Herold, C. J., Brink, J. A., Pianykh, O., Prosch, H., & Langs, G. (2021) · Nature Communications · doi:10.1038/s41467-021-25858-z · Cited by 106
Replay in Deep Learning: Current Approaches and Missing Biological Elements
Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski, T. J., & Kanan, C. (2021) · Neural Computation · doi:10.1162/neco_a_01433 · Cited by 106
Don't forget, there is more than forgetting: new metrics for Continual LearningPreprint
Díaz-Rodríguez, N., Lomonaco, V., Filliat, D., & Maltoni, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.13166 · Cited by 102
More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning
Liu, Y., Parisot, S., Slabaugh, G., Jia, X., Leonardis, A., & Tuytelaars, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58574-7_42 · Cited by 96
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
Wang, Z., Yang, E., Shen, L., & Huang, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3498346 · Cited by 132
Selfless Sequential LearningPreprint
Aljundi, R., Rohrbach, M., & Tuytelaars, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05421 · Cited by 114
Superposition of many models into onePreprint
Cheung, B., Terekhov, A. L., Chen, Y., Agrawal, P., & Olshausen, B. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.05522 · Cited by 96
The limits of machine intelligence
Shevlin, H., Vold, K., Crosby, M., & Halina, M. (2019) · EMBO Reports · doi:10.15252/embr.201949177 · Cited by 101
Continual Learning with Node-Importance based Adaptive Group Sparse RegularizationPreprint
Jung, S., Ahn, H., Cha, S., & Moon, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13726 · Cited by 86
Dynamic Support Network for Few-Shot Class Incremental Learning
Yang, B., Lin, M., Zhang, Y., Liu, B., Liang, X., Ji, R., & Ye, Q. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3175849 · Cited by 80
Anomaly Detection in the Open World: Normality Shift Detection, Explanation, and Adaptation
Han, D., Wang, Z., Chen, W., Wang, K., Yu, R., Wang, S., Zhang, H., Wang, Z., Jin, M., Yang, J., et al. (2023) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2023.24830 · Cited by 83
Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector
Lin, G., Chu, H., & Lai, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00019 · Cited by 67
Continual Learning With Knowledge Distillation: A Survey
Li, S., Su, T., Zhang, X.-Y., & Wang, Z. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3476068 · Cited by 71
Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interference
Wang, Y., Ma, Z., Huang, Z., Wang, Y., Su, Z., & Hong, X. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26216 · Cited by 55
Triple-Memory Networks: A Brain-Inspired Method for Continual Learning
Wang, L., Lei, B., Li, Q., Su, H., Zhu, J., & Zhong, Y. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3111019 · Cited by 57
A self-adaptive fuzzy learning system for streaming data prediction
Gu, X., & Shen, Q. (2021) · Information Sciences · doi:10.1016/j.ins.2021.08.023 · Cited by 52
Lifelong Person Re-identification by Pseudo Task Knowledge Preservation
Ge, W., Du, J., Wu, A., Xian, Y., Yan, K., Huang, F., & Zheng, W. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19949 · Cited by 52
Flesch, T., Nagy, D., Saxe, A., & Summerfield, C. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010808 · Cited by 53
Aspect-Based Fashion Recommendation With Attention Mechanism
Li, W., & Xu, B. (2020) · IEEE Access · doi:10.1109/access.2020.3013639 · Cited by 52
Lifelong Person Re-identification via Knowledge Refreshing and Consolidation
Yu, C., Ye, S., Liu, Z., Gao, S., & Wang, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25436 · Cited by 44
Continual Learning by Using Information of Each Class Holistically
Hu, W., Qin, Q., Wang, M., Ma, J., & Liu, B. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16952 · Cited by 45
Continual Learning by Asymmetric Loss Approximation With Single-Side Overestimation
Park, D.-M., Hong, S., Han, B., & Lee, K. M. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00343 · Cited by 40
Memory Recall: A Simple Neural Network Training Framework Against Catastrophic Forgetting
Zhang, B., Guo, Y., Li, Y., He, Y., Wang, H., & Dai, Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3099700 · Cited by 42
Incremental Predictive Process Monitoring: The Next Activity Case
Pauwels, S., & Calders, T. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85469-0_10 · Cited by 38
Learnable Distribution Calibration for Few-Shot Class-Incremental Learning
Liu, B., Yang, B., Xie, L., Wang, R., Tian, Q. C., & Ye, Q. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3273291 · Cited by 38
CFA: Constraint-based Finetuning Approach for Generalized Few-Shot Object Detection
Guirguis, K., Hendawy, A., Eskandar, G., Abdelsamad, M., Kayser, M., & Beyerer, J. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00449 · Cited by 36
AI Autonomy: Self‐initiated Open‐world Continual Learning and Adaptation
Liu, B., Mazumder, S., Robertson, E., & Grigsby, S. (2023) · AI Magazine · doi:10.1002/aaai.12087 · Cited by 50
An Investigation of Replay-based Approaches for Continual Learning
Bagus, B., & Gepperth, A. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533862 · Cited by 41
CRNet: A Fast Continual Learning Framework With Random Theory
Li, D., & Zeng, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3262853 · Cited by 39
Efficient Continual Learning with Modular Networks and Task-Driven PriorsPreprint
Véniat, T., Denoyer, L., & Ranzato, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.12631 · Cited by 66
A Spike Time-Dependent Online Learning Algorithm Derived From Biological Olfaction
Borthakur, A., & Cleland, T. A. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00656 · Cited by 31
Continual Learning through Retrieval and Imagination
Wang, Z., Liu, L., Duan, Y., & Tao, D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20837 · Cited by 34
Symbolic Replay: Scene Graph as Prompt for Continual Learning on VQA Task
Lei, S. W., Gao, D., Wu, J. Z., Wang, Y., Liu, W., Zhang, M., & Shou, M. Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25208 · Cited by 39
Adaptive Progressive Continual Learning
Xu, J., Ma, J., Gao, X., & Zhu, Z. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095064 · Cited by 35
Adaptive Orthogonal Projection for Batch and Online Continual Learning
Guo, Y., Hu, W., Zhao, D., & Liu, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i6.20634 · Cited by 34
MUSE-RNN: A Multilayer Self-Evolving Recurrent Neural Network for Data Stream Classification
Das, M., Pratama, M., Savitri, S., et al. (2019) · Proceedings · doi:10.1109/icdm.2019.00021 · Cited by 32
Optimizing Attention for Sequence Modeling via Reinforcement Learning
Fei, H., Zhang, Y., Ren, Y., & Ji, D. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3053633 · Cited by 30
In Defense of the Learning Without Forgetting for Task Incremental Learning
Oren, G., & Wolf, L. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00250 · Cited by 31
Analogical Learning-Based Few-Shot Class-Incremental Learning
Li, J., Dong, S., Gong, Y., He, Y., & Wei, X. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3350913 · Cited by 32
Overcoming Negative Transfer: A SurveyPreprint
Zhang, W., Deng, L., & Wu, D. (2020) · arXiv (Cornell University) · Cited by 31
Marginal Replay vs Conditional Replay for Continual Learning
Lesort, T., Gepperth, A., Stoian, A., & Filliat, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30484-3_38 · Cited by 31
Unsupervised Model Personalization While Preserving Privacy and Scalability: An Open Problem
Lange, M. D., Jia, X., Parisot, S., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01447 · Cited by 30
Overcoming Catastrophic Forgetting for Multi-Label Class-Incremental Learning
Song, X. R., Shu, K., Dong, S., Cheng, J., Wei, X., & Gong, Y. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00238 · Cited by 30
SpaRCe: Improved Learning of Reservoir Computing Systems Through Sparse Representations
Manneschi, L., Lin, A. C., & Vasilaki, E. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3102378 · Cited by 30
Residual Tuning: Toward Novel Category Discovery Without Labels
Liu, Y., & Tuytelaars, T. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3140235 · Cited by 27
Continual Learning with Scaled Gradient Projection
Saha, G., & Roy, K. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26157 · Cited by 28
Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generation
Zhai, M., Chen, L., He, J., Nawhal, M., Tung, F., & Mori, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_24 · Cited by 30
Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Network
Kim, J.-Y., & Choi, D.-W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16991 · Cited by 25
Saliency Guided Experience Packing for Replay in Continual Learning
Saha, G., & Roy, K. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00524 · Cited by 24
Federated Semi-Supervised Learning with Inter-Client Consistency.Preprint
Jeong, W., Yoon, J., Yang, E., & Hwang, S. J. (2020) · arXiv (Cornell University) · Cited by 24
Using Adapters to Overcome Catastrophic Forgetting in End-to-End Automatic Speech Recognition
Eeckt, S. V., & hamme, H. V. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095837 · Cited by 22
Generalisation Guarantees for Continual Learning with Orthogonal Gradient DescentPreprint
Bennani, M., Doan, T., & Sugiyama, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11942 · Cited by 76
IIRC: Incremental Implicitly-Refined Classification
Abdelsalam, M. F. A., Faramarzi, M., Sodhani, S., & Chandar, S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01089 · Cited by 22
Dealing with Cross-Task Class Discrimination in Online Continual Learning
Guo, Y., Liu, B., & Zhao, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01143 · Cited by 21
Continual learning with attentive recurrent neural networks for temporal data classification
Yin, S.-Y., Huang, Y., Chang, T.-Y., Chang, S.-F., & Tseng, V. S. M. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.10.031 · Cited by 20
A Survey of Incremental Deep Learning for Defect Detection in Manufacturing
Mohandas, R., Southern, M., O’Connell, E., & Hayes, M. J. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8010007 · Cited by 25
Generative feature-driven image replay for continual learning
Thandiackal, K., Portenier, T., Giovannini, A. A., Gabrani, M., & Göksel, O. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105187 · Cited by 23
Open-world continual learning: Unifying novelty detection and continual learning
Kim, G., Xiao, C., Konishi, T., Ke, Z., & Liu, B. (2025) · Artificial Intelligence · doi:10.1016/j.artint.2024.104237 · Cited by 19
Coordinating Experience Replay: A Harmonious Experience Retention approach for Continual Learning
Ji, Z., Liu, J., Wang, Q., & Zhang, Z. M. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107589 · Cited by 21
Continual Learning Based on OOD Detection and Task Masking
Kim, G., Esmaeilpour, S., Xiao, C., & Liu, B. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00431 · Cited by 19
Overcoming Forgetting in Fine-Grained Urban Flow Inference via Adaptive Knowledge Replay
Yu, H., Xu, X., Zhong, T., et al. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25671 · Cited by 19
Generative appearance replay for continual unsupervised domain adaptation
Chen, B., Thandiackal, K., Pati, P., & Göksel, O. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102924 · Cited by 20
Multi-view class incremental learning
Li, D., Wang, T., Chen, J., Kawaguchi, K., Lian, C., & Zeng, Z. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102021 · Cited by 20
Continual learning in cross-modal retrieval
Wang, K., Herranz, L., & van de Weijer, J. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00402 · Cited by 19
Gradient Based Memory Editing for Task-Free Continual LearningPreprint
Jin, X., Du, J., & Ren, X. (2021) · arXiv (Cornell University) · Cited by 19
Energy-Based Models for Continual LearningPreprint
Li, S., Du, Y., van de Ven, G. M., & Mordatch, I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.12216 · Cited by 19
Data Augmented Flatness-aware Gradient Projection for Continual Learning
Yang, E., Shen, L., Wang, Z., Liu, S., Guo, G., & Wang, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00518 · Cited by 18
Real-world Cross-modal Retrieval via Sequential Learning
Song, G., & Tan, X. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3002177 · Cited by 19
Drinking from a Firehose: Continual Learning with Web-scale Natural Language
Hu, H., Şener, O., Sha, F., & Koltun, V. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3218265 · Cited by 23
Sequential Domain Adaptation through Elastic Weight Consolidation for Sentiment Analysis
Madasu, A., & Vijjini, A. R. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412617 · Cited by 20
Das, M., Pratama, M., Zhang, J., et al. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5781 · Cited by 17
A Neural Span-Based Continual Named Entity Recognition Model
Zhang, Y., & Chen, Q. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26638 · Cited by 16
Learning to Prompt Knowledge Transfer for Open-World Continual Learning
Li, Y., Yang, X. S., Wang, H., Wang, X., & Li, T. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29275 · Cited by 17
Evolving Fully Automated Machine Learning via Life-Long Knowledge Anchors
Zheng, X., Zhang, Y., Hong, S., Li, H., Tang, L., Xiong, Y., Zhou, J., Wang, Y., Sun, X., Zhu, P., et al. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3069250 · Cited by 16
Fully Decoupled End-to-End Person Search: An Approach without Conflicting Objectives
Zhang, P., Yu, X., Xiao, B., Zheng, J., Ning, X., & Hancock, E. R. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02407-5 · Cited by 17
Memory-Latency-Accuracy Trade-Offs for Continual Learning on a RISC-V Extreme-Edge Node
Ravaglia, L., Rusci, M., Capotondi, A., Conti, F., Pellegrini, L., Lomonaco, V., Maltoni, D., & Luca Benini, C. (2020) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips50750.2020.9195220 · Cited by 15
Metadata
- Authors
- Serrà, J., Surís, D., Miron, M., & Karatzoglou, A.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- arXiv
- 1801.01423
- OpenAlex
- W2782276075
- Semantic Scholar
- 3aa673abd49f0837ed2fbf5cffcada9ba6f48693
- Cited by
- 213
- References
- 34
- Citations by year
- 2025: 20 · 2024: 30 · 2023: 45 · 2022: 22 · 2021: 40 · 2020: 27 · 2019: 21 · 2018: 8
- FWCI
- 9.76
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- Locations
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (submittedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Keywords
- catastrophic forgetting, hard attention, continual learning, sequential learning, attention masks, knowledge preservation, online learning, network compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
