Encoder Based Lifelong Learning
Open accessAbstract
This paper introduces a new lifelong learning solution where a single model is trained for a sequence of tasks. The main challenge that vision systems face in this context is catastrophic forgetting: as they tend to adapt to the most recently seen task, they lose performance on the tasks that were learned previously. Our method aims at preserving the knowledge of the previous tasks while learning a new one by using autoencoders. For each task, an under-complete autoencoder is learned, capturing the features that are crucial for its achievement. When a new task is presented to the system, we prevent the reconstructions of the features with these autoencoders from changing, which has the effect of preserving the information on which the previous tasks are mainly relying. At the same time, the features are given space to adjust to the most recent environment as only their projection into a low dimension submanifold is controlled. The proposed system is evaluated on image classification tasks and shows a reduction of forgetting over the state-ofthe-art.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LwF suffers from a buildup of errors when facing a long sequence while EBLL reduces slightly this effect.
“LwF [17] suffers from a buildup of errors when facing a long sequence while EBLL [28] reduces slightly this effect.”
From Memory Aware Synapses: Learning What (not) to Forget (2018) · cites this paper as [28]
The standard LLL setup consists of a sequence of disjoint tasks learned one after the other, where tasks may correspond to different datasets or different splits of a dataset without overlap in category labels.
“It focuses on image classification and consists of a sequence of disjoint tasks , which are learned one after the other. Tasks may correspond to different datasets, or different splits of a dataset, without overlap in category labels.”
From Memory Aware Synapses: Learning What (not) to Forget (2018) · cites this paper as [28]
Data-based approaches use data from the new task to approximate the performance of the previous tasks, and this works best if the data distribution mismatch between tasks is limited.
“Data-based approaches [1,17,28,34] use data from the new task to approximate the performance of the previous tasks. This works best if the data distribution mismatch between tasks is limited.”
From Memory Aware Synapses: Learning What (not) to Forget (2018) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · see citing works
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795 · see citing works
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322 · see citing works
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256 · see citing works
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · see citing works
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · see citing works
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770 · see citing works
Automated Flower Classification over a Large Number of Classes
Nilsback, M.-E., & Zisserman, A. (2008) · 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing · doi:10.1109/icvgip.2008.47 · Cited by 3,442 · see citing works
Catastrophic forgetting in connectionist networks
French, R. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01294-2 · Cited by 2,311 · see citing works
Quattoni, A., & Torralba, A. B. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206537 · Cited by 1,533 · see citing works
Auto-association by multilayer perceptrons and singular value decomposition
Bourlard, H., & Kamp, Y. (1988) · Biological Cybernetics · doi:10.1007/bf00332918 · Cited by 1,332 · see citing works
Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., & Hadsell, R. (2016) · arXiv · doi:10.48550/arxiv.1606.04671 · Cited by 1,307 · see citing works
An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A., & Bengio, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6211 · Cited by 1,089 · see citing works
Ratcliff, R. (1990) · Psychological Review · doi:10.1037/0033-295x.97.2.285 · Cited by 892 · see citing works
Li, Z., & Hoiem, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_37 · Cited by 659 · see citing works
Expert Gate: Lifelong Learning with a Network of Experts
Aljundi, R., Chakravarty, P., & Tuytelaars, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.753 · Cited by 619 · see citing works
Self-citationCaruana, R. (1998) · Learning to Learn · doi:10.1007/978-1-4615-5529-2_5 · Cited by 577 · see citing works
(2013) · Cited by 290 · see citing works
The Task Rehearsal Method of Life-Long Learning: Overcoming Impoverished Data
Silver, D. L., & Mercer, R. E. (2002) · Lecture notes in computer science · doi:10.1007/3-540-47922-8_8 · Cited by 127 · see citing works
Lifelong learning with non-i.i.d. tasks
Pentina, A., & Lampert, C. H. (2015) · Cited by 37 · see citing works
Expert Gate: Lifelong Learning with a Network of ExpertsPreprint
Aljundi, R., Chakravarty, P., & Tuytelaars, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06194 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- [8][8] I. Goodfellow, Y. Bengio, and A. Courville. Deep learning. Book in preparation for MIT Press, 2016. -
- [18][18] A. Pentina and C. H. Lampert. Lifelong learning with noniid tasks. In Advances in Neural Information Processing Systems , pages 1540-1548, 2015. -
Cited by
Sorted by citations
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · see citing works
A Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647 · see citing works
Self-citationMemory Aware Synapses: Learning What (not) to Forget
Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., & Tuytelaars, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_9 · Cited by 1,590 · see citing works
Self-citationLarge Scale Incremental Learning
Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., & Fu, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00046 · Cited by 1,313 · see citing works
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
Mallya, A., & Lazebnik, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00810 · Cited by 1,258 · see citing works
End-to-End Incremental Learning
Castro, F. M., Marín-Jiménez, M. J., Guil, N., Schmid, C., & Alahari, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01258-8_15 · Cited by 1,184 · see citing works
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113 · see citing works
“LwM [119] exploits the attention maps of new training samples for KD. EBLL [121] learns task-specific autoencoders and prevents changes in feature reconstruction.”
BackgroundLearning a Unified Classifier Incrementally via Rebalancing
Hou, S., Pan, X., Loy, C. C., Wang, Z., & Lin, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00092 · Cited by 1,195 · see citing works
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification
Masana, M., Liu, X., Twardowski, B., Menta, M., Bagdanov, A. D., & van de Weijer, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3213473 · Cited by 740 · see citing works
Transferable Joint Attribute-Identity Deep Learning for Unsupervised Person Re-identification
Wang, J., Zhu, X., Gong, S., & Li, W. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00242 · Cited by 671 · see citing works
Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights
Mallya, A., Davis, D., & Lazebnik, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01225-0_5 · Cited by 644 · see citing works
Measuring Catastrophic Forgetting in Neural Networks
Kemker, R., McClure, M., Abitino, A., Hayes, T., & Kanan, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11651 · Cited by 672 · see citing works
Dreaming to Distill: Data-Free Knowledge Transfer via DeepInversion
Yin, H., Molchanov, P., Alvarez, J. M., Li, Z., Mallya, A., Hoiem, D., Jha, N. K., & Kautz, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00874 · Cited by 540 · see citing works
Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., & Díaz-Rodríguez, N. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.12.004 · Cited by 531 · see citing works
GDumb: A Simple Approach that Questions Our Progress in Continual Learning
Prabhu, A., Torr, P. H. S., & Dokania, P. K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_31 · Cited by 497 · see citing works
Embracing Change: Continual Learning in Deep Neural Networks
Hadsell, R., Rao, D., Rusu, A. A., & Pascanu, R. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.09.004 · Cited by 467 · see citing works
Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems
Wu, C.-S., Madotto, A., Hosseini-Asl, E., Xiong, C., Socher, R., & Fung, P. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1078 · Cited by 423 · see citing works
Online continual learning in image classification: An empirical survey
Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., & Sanner, S. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.10.021 · Cited by 422 · see citing works
Class-Incremental Learning: A Survey
Zhou, D.-W., Wang, Q.-W., Qi, Z.-H., Ye, H.-J., Zhan, D.-C., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429383 · Cited by 376 · see citing works
Aljundi, R., Kelchtermans, K., & Tuytelaars, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01151 · Cited by 335 · see citing works
Self-citationRotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting
Liu, X., Masana, M., Herranz, L., van de Weijer, J., López, A. M., & Bagdanov, A. D. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545895 · Cited by 273 · see citing works
Overcoming Catastrophic Forgetting With Unlabeled Data in the Wild
Lee, K., Lee, K., Shin, J., & Lee, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00040 · Cited by 214 · see citing works
Rethinking the Distribution Gap of Person Re-identification with Camera-Based Batch Normalization
Zhuang, Z., Wei, L., Xie, L., Zhang, T., Zhang, H., Wu, H., Ai, H., & Tian, Q. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58610-2_9 · Cited by 204 · see citing works
Online Class-Incremental Continual Learning with Adversarial Shapley Value
Shim, D., Mai, Z., Jeong, J., Sanner, S., Kim, H., & Jang, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17159 · Cited by 203 · see citing works
Lifelong Learning via Progressive Distillation and Retrospection
Hou, S., Pan, X., Loy, C. C., Wang, Z., & Lin, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_27 · Cited by 190 · see citing works
Mai, Z., Li, R., Kim, H., & Sanner, S. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00398 · Cited by 191 · see citing works
Incremental Learning in Online Scenario
He, J., Mao, R., Shao, Z., & Zhu, F. M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01394 · Cited by 180 · see citing works
Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting
Chen, S., Hou, Y., Cui, Y., Che, W., Liu, T., & Yu, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.634 · Cited by 181 · see citing works
Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
Taşar, O., Tarabalka, Y., & Alliez, P. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2925416 · Cited by 144 · see citing works
Mundt, M., Hong, Y., Pliushch, I., & Ramesh, V. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.01.014 · Cited by 148 · see citing works
An Appraisal of Incremental Learning Methods
Luo, Y., Yin, L., Bai, W., & Mao, K. (2020) · Entropy · doi:10.3390/e22111190 · Cited by 120 · see citing works
Memory-Efficient Class-Incremental Learning for Image Classification
Zhao, H., Wang, H., Fu, Y., Wu, F. Q., & Li, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3072041 · Cited by 114 · see citing works
What and How: Generalized Lifelong Spectral Clustering via Dual Memory
Sun, G., Cong, Y., Dong, J., Liu, Y., Ding, Z., & Yu, H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3058852 · Cited by 106 · see citing works
Continual Learning for Blind Image Quality Assessment
Zhang, W., Li, D., Ma, C., Zhai, G., Yang, X., & Ma, K. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3178874 · Cited by 108 · see citing works
More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning
Liu, Y., Parisot, S., Slabaugh, G., Jia, X., Leonardis, A., & Tuytelaars, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58574-7_42 · Cited by 93 · see citing works
Self-citationSelfless Sequential LearningPreprint
Aljundi, R., Rohrbach, M., & Tuytelaars, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05421 · Cited by 99 · see citing works
Self-citationLearning Latent Representations Across Multiple Data Domains Using Lifelong VAEGAN
Ye, F., & Borş, A. G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_46 · Cited by 76 · see citing works
Continual Learning for Text Classification with Information Disentanglement Based Regularization
Huang, Y., Zhang, Y., Chen, J., Wang, X., & Yang, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.218 · Cited by 74 · see citing works
Dense Network Expansion for Class Incremental Learning
Hu, Z., Li, Y., Lyu, J., Gao, D., & Vasconcelos, N. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01141 · Cited by 74 · see citing works
Continual Learning for Named Entity Recognition
Monaikul, N., Castellucci, G., Filice, S., & Rokhlenko, O. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17600 · Cited by 71 · see citing works
Multi-View correlation distillation for incremental object detection
Yang, D., Zhou, Y., Zhang, A., Sun, X., Wu, D., Wang, W., & Ye, Q. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108863 · Cited by 70 · see citing works
Continual learning for predictive maintenance: Overview and challenges
Hurtado, J., Salvati, D., Semola, R., Bosio, M., & Lomonaco, V. (2023) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200251 · Cited by 70 · see citing works
Continual Learning: A Review of Techniques, Challenges, and Future Directions
Wickramasinghe, B., Saha, G., & Roy, K. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3339091 · Cited by 73 · see citing works
Representative Task Self-Selection for Flexible Clustered Lifelong Learning
Sun, G., Cong, Y., Wang, Q., Zhong, B., & Fu, Y. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3042500 · Cited by 68 · see citing works
Continual Learning with Lifelong Vision Transformer
Wang, Z., Liu, L., Duan, Y., Kong, Y., & Tao, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00027 · Cited by 70 · see citing works
Lessons from infant learning for unsupervised machine learning
Zaadnoordijk, L., Besold, T. R., & Cusack, R. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00488-2 · Cited by 65 · see citing works
An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditions
Mirza, M. J., Masana, M., Possegger, H., & Bischof, H. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00339 · Cited by 67 · see citing works
Efficient Parametrization of Multi-domain Deep Neural Networks
Rebuffi, S.-A., Vedaldi, A., & Bilen, H. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00847 · Cited by 75 · see citing works
Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector
Lin, G., Chu, H., & Lai, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00019 · Cited by 67 · see citing works
Continuous transfer of neural network representational similarity for incremental learning
Tian, S., Li, W., Ning, X., Ran, H., Qin, H., & Tiwari, P. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126300 · Cited by 63 · see citing works
Traj-MAE: Masked Autoencoders for Trajectory Prediction
Chen, H., Wang, J., Shao, K., Liu, F., Hao, J., Guan, C., Chen, G., & Heng, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00767 · Cited by 59 · see citing works
Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2023) · arXiv · doi:10.48550/arxiv.2312.10549 · Cited by 54
Lü, X., Sun, X., Diao, W., Feng, Y., Wang, P., & Fu, K. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3102629 · Cited by 54 · see citing works
A review of machine learning with small and limited data
Naser, M. Z. (2026) · Journal Of Big Data · doi:10.1186/s40537-025-01346-9 · Cited by 80 · see citing works
Revisiting Distillation and Incremental Classifier Learning
Javed, K., & Shafait, F. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-20876-9_1 · Cited by 53 · see citing works
Graph Few-shot Class-incremental Learning
Tan, Z., Ding, K., Guo, R., & Liu, H. (2022) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3488560.3498455 · Cited by 52 · see citing works
Pattern Expansion and Consolidation on Evolving Graphs for Continual Traffic Prediction
Wang, B., Zhang, Y., Wang, X., Wang, P., Zhou, Z., Bai, L., & Wang, Y. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599463 · Cited by 50 · see citing works
Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2024) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2024.3966 · Cited by 54 · see citing works
Scarcity of Labels in Non-Stationary Data Streams: A Survey
Fahy, C., Yang, S., & Góngora, M. (2022) · ACM Computing Surveys · doi:10.1145/3494832 · Cited by 45 · see citing works
Lifelong Visual-Tactile Cross-Modal Learning for Robotic Material Perception
Zheng, W., Liu, H., & Sun, F. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2980892 · Cited by 45 · see citing works
Curiosity-Driven Class-Incremental Learning via Adaptive Sample Selection
Hu, Q., Gao, Y., & Cao, B. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3196092 · Cited by 40 · see citing works
Dynamically Expandable Graph Convolution for Streaming Recommendation
He, B., He, X. D., Zhang, Y., Tang, R., & Ma, C. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583237 · Cited by 40 · see citing works
Memory Recall: A Simple Neural Network Training Framework Against Catastrophic Forgetting
Zhang, B., Guo, Y., Li, Y., He, Y., Wang, H., & Dai, Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3099700 · Cited by 41 · see citing works
The Challenges of Continuous Self-Supervised Learning
Purushwalkam, S., Morgado, P. M., & Gupta, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_40 · Cited by 40 · see citing works
Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
Mundt, M., Pliushch, I., Majumder, S., Hong, Y., & Ramesh, V. (2022) · Journal of Imaging · doi:10.3390/jimaging8040093 · Cited by 38 · see citing works
Lifelong robotic visual-tactile perception learning
Dong, J., Cong, Y., Sun, G., & Zhang, T. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108176 · Cited by 38 · see citing works
Continual Neural Mapping: Learning An Implicit Scene Representation from Sequential Observations
Yan, Z., Tian, Y., Shi, X., Guo, P. Y., Wang, P., & Zha, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01549 · Cited by 36 · see citing works
Open World Object Detection: A Survey
Li, Y., Wang, Y., Wang, W., Lin, D., Li, B., & Yap, K. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3480691 · Cited by 39 · see citing works
CFA: Constraint-based Finetuning Approach for Generalized Few-Shot Object Detection
Guirguis, K., Hendawy, A., Eskandar, G., Abdelsamad, M., Kayser, M., & Beyerer, J. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00449 · Cited by 36 · see citing works
Online Hyperparameter Optimization for Class-Incremental Learning
Liu, Y., Li, Y., Schiele, B., & Sun, Q. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26070 · Cited by 36 · see citing works
Survey of continuous deep learning methods and techniques used for incremental learning
Leo, J., & Kalita, J. K. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127545 · Cited by 37 · see citing works
Hung, S. C. Y., Lee, J., Wan, T. S. T., Chen, C., Chan, Y.-M., & Chen, C. (2019) · International Conference on Multimedia Retrieval · doi:10.1145/3323873.3325053 · Cited by 35 · see citing works
Online Continual Learning with Contrastive Vision Transformer
Wang, Z., Liu, L., Kong, Y., Guo, J., & Tao, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20044-1_36 · Cited by 36 · see citing works
EETC: An extended encrypted traffic classification algorithm based on variant resnet network
Ma, X., Zhu, W., Wei, J., Jin, Y., Gu, D., & Wang, R. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103175 · Cited by 33 · see citing works
NetTailor: Tuning the Architecture, Not Just the Weights
Morgado, P. M., & Vasconcelos, N. M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00316 · Cited by 33 · see citing works
Convolutional Neural Network With Developmental Memory for Continual Learning
Park, G.-M., Yoo, S.-M., & Kim, J.-H. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3007548 · Cited by 33 · see citing works
Continual Learning through Retrieval and Imagination
Wang, Z., Liu, L., Duan, Y., & Tao, D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20837 · Cited by 33 · see citing works
Continual Learning for Visual Search with Backward Consistent Feature Embedding
Wan, T. S. T., Chen, J.-C., Wu, T.-Y., & Chen, C.-S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01620 · Cited by 32 · see citing works
Contrastive Correlation Preserving Replay for Online Continual Learning
Yu, D.-P., Zhang, M., Li, M., Zha, F., Zhang, J., Sun, L., & Huang, K. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3285221 · Cited by 33 · see citing works
Overcoming Catastrophic Forgetting in Continual Learning by Exploring Eigenvalues of Hessian Matrix
Kong, Y., Liu, L., Chen, H., Kacprzyk, J., & Tao, D. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3292359 · Cited by 32 · see citing works
Zhuang, Z., Wei, L., Xie, L., Ai, H., & Tian, Q. C. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3058111 · Cited by 32 · see citing works
Open-world Semantic Segmentation for LIDAR Point Clouds
Cen, J., Peng, Y., Zhang, S., Cai, J., Luan, D., Tang, M., Liu, M., & Wang, M. Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_19 · Cited by 32 · see citing works
Balancing Stability and Plasticity Through Advanced Null Space in Continual Learning
Kong, Y., Liu, L., Wang, Z., & Tao, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_13 · Cited by 32 · see citing works
Efficient Multi-Domain Learning by Covariance Normalization
Li, Y., & Vasconcelos, N. M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00557 · Cited by 31 · see citing works
In Defense of the Learning Without Forgetting for Task Incremental Learning
Oren, G., & Wolf, L. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00250 · Cited by 31 · see citing works
Self-Paced Weight Consolidation for Continual Learning
Cong, W. W., Cong, Y., Sun, G., Liu, Y., & Dong, J. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3304567 · Cited by 31 · see citing works
Analogical Learning-Based Few-Shot Class-Incremental Learning
Li, J., Dong, S., Gong, Y., He, Y., & Wei, X. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3350913 · Cited by 30 · see citing works
Unsupervised Model Personalization While Preserving Privacy and Scalability: An Open Problem
Lange, M. D., Jia, X., Parisot, S., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01447 · Cited by 29 · see citing works
Self-citationContinuous Learning Method of Radar HRRP Based on CVAE-GAN
Li, X., Ouyang, W., Pan, M., Lv, S., & Ma, Q. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3268219 · Cited by 29 · see citing works
Lifelong Object DetectionPreprint
Zhou, W., Chang, S., Sosa, N. E., Hamann, H. F., & Cox, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01129 · Cited by 32 · see citing works
Shi, M., Ding, C., Shen, C., Huang, W., & Zhu, Z. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102832 · Cited by 28 · see citing works
Do not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting
Kurmi, V. K., Patro, B. N., Subramanian, V. K., & Namboodiri, V. P. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00078 · Cited by 30 · see citing works
Ternary Feature Masks: zero-forgetting for task-incremental learning
Masana, M., Tuytelaars, T., & van de Weijer, J. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00396 · Cited by 37 · see citing works
Self-citationRehearsal-free Continual Language Learning via Efficient Parameter Isolation
Wang, Z., Liu, Y., Ji, T., Wang, X., Wu, Y., Jiang, C., Ye, C., Han, Z., Wang, L., Shao, X., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.612 · Cited by 28 · see citing works
End-to-End Incremental LearningPreprint
Castro, F. M., Marín-Jiménez, M. J., Guil, N., Schmid, C., & Alahari, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09536 · Cited by 28 · see citing works
CoinSeg: Contrast Inter- and Intra- Class Representations for Incremental Segmentation
Zhang, Z., Gao, G., Jiao, J., Liu, C. H., & Wei, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00084 · Cited by 27 · see citing works
Kumari, P., Chauhan, J., Bozorgpour, A., Huang, B., Azad, R., & Merhof, D. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103730 · Cited by 33 · see citing works
Sun, G., Cong, Y., Wang, Q., Li, J., & Fu, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6045 · Cited by 26 · see citing works
RD-IOD: Two-Level Residual-Distillation-Based Triple-Network for Incremental Object Detection
Yang, D., Zhou, Y., Shi, W., Wu, D., & Wang, W. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3472393 · Cited by 26 · see citing works
Hyperspectral Image Classification Based on Class-Incremental Learning with Knowledge Distillation
Xu, M., Zhao, Y., Yajun, L., & Ma, X. (2022) · Remote Sensing · doi:10.3390/rs14112556 · Cited by 26 · see citing works
- Authors
- Rannen, A., Aljundi, R., Blaschko, M. B., & Tuytelaars, T.
- Affiliations
- IMEC (BE)
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 1329-1337
- OpenAlex
- W2605911906
- Cited by
- 324
- References
- 23
- Citations by year
- 2026: 10 · 2025: 23 · 2024: 58 · 2023: 53 · 2022: 47 · 2021: 46 · 2020: 41 · 2019: 23 · 2018: 22 · 2017: 1
- FWCI
- 10.85
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Lirias (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Fonds Wetenschappelijk Onderzoek
- KU Leuven (FP7-MC-CIG 334380)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- lifelong learning, catastrophic forgetting, autoencoder, feature preservation, image classification, vision system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
