Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
RWalk improves the regularization methods of EWC and SI and further improves performance, but it still retains some old data for performance reasons, which shows the effectiveness of rehearsal strategies for mitigating catastrophic forgetting.
“Rwalk improves the regularization methods of EWC and SI, and further improves the performance based on them. However, for performance reasons, some old data are still retained. This shows the effectiveness of rehearsal strategies for mitigating catastrophic forgetting.”
From An Appraisal of Incremental Learning Methods (2020) · cites this paper as [47]
Many incremental learning methods based on regularization save a small amount of old data to enhance model performance.
“Many incremental learning methods based on the regularization strategy save a small amount of old data to enhance model performance [47,70,76].”
From An Appraisal of Incremental Learning Methods (2020) · cites this paper as [47]
Experimental results prove that combining regularization strategy with rehearsal or pseudo-rehearsal strategy greatly helps improve model performance.
“The experimental results of [47,70,76] prove that the algorithm based on the regularization strategy is appropriately combined with a rehearsal or pseudo-rehearsal strategy, which is of great help to improve the performance of the model.”
From An Appraisal of Incremental Learning Methods (2020) · cites this paper as [47]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · see citing works
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · see citing works
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · see citing works
Natural Gradient Works Efficiently in Learning
Амари, Ш.-И. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,745 · see citing works
Theory of the backpropagation neural network
Hecht-Nielsen (1989) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1989.118638 · Cited by 2,221 · see citing works
Continual Learning Through Synaptic Intelligence
Zenke, F., Poole, B., & Ganguli, S. (2017) · arXiv · doi:10.48550/arxiv.1703.04200 · Cited by 1,784 · see citing works
Gradient Episodic Memory for Continual Learning
Lopez-Paz, D., & Ranzato, M. (2017) · arXiv · doi:10.48550/arxiv.1706.08840 · Cited by 1,403 · see citing works
Continual Learning with Deep Generative Replay
Shin, H., Lee, J. K., Kim, J., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1705.08690 · Cited by 1,385 · see citing works
Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., & Hadsell, R. (2016) · arXiv · doi:10.48550/arxiv.1606.04671 · Cited by 1,307 · see citing works
Lee, J. M. (1997) · Graduate texts in mathematics · doi:10.1007/b98852 · Cited by 1,033 · see citing works
Lifelong Learning with Dynamically Expandable NetworksPreprint
Yoon, J., Yang, E., Lee, J., & Hwang, S. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.01547 · Cited by 786 · see citing works
Li, Z., & Hoiem, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_37 · Cited by 659 · see citing works
Learning multiple visual domains with residual adapters
Rebuffi, S.-A., Bilen, H., & Vedaldi, A. (2017) · arXiv · doi:10.48550/arxiv.1705.08045 · Cited by 605 · see citing works
Variational Continual Learning
Nguyen, C. V., Li, Y., Bui, T. D., & Turner, R. E. (2017) · arXiv · doi:10.48550/arxiv.1710.10628 · Cited by 491 · see citing works
Theory of the backpropagation neural network
Hecht-Nielsen, R. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90469-8 · Cited by 481 · see citing works
Optimizing Neural Networks with Kronecker-factored Approximate CurvaturePreprint
Martens, J., & Grosse, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.05671 · Cited by 476 · see citing works
Overcoming Catastrophic Forgetting by Incremental Moment MatchingPreprint
Lee, S.-W., Kim, J.-H., Jun, J., Ha, J.-W., & Zhang, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08475 · Cited by 472 · see citing works
Variational Continual Learning
Turner, R., Bui, T., Li, Y., & Cuong, N. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.35471 · Cited by 438 · see citing works
Progress & Compress: A scalable framework for continual learningPreprint
Schwarz, J., Luketina, J., Czarnecki, W. M., Grabska‐Barwińska, A., Teh, Y. W., Pascanu, R., & Hadsell, R. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06370 · Cited by 422 · see citing works
Revisiting Natural Gradient for Deep NetworksPreprint
Pascanu, R., & Bengio, Y. (2014) · Cited by 159 · see citing works
Natural Gradient Works Efficiently in Learning
Амари, Ш.-И. (1999) · The MIT Press eBooks · doi:10.7551/mitpress/7011.003.0010 · Cited by 126 · see citing works
Revisiting Natural Gradient for Deep NetworksPreprint
Pascanu, R., & Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3584 · Cited by 116 · see citing works
A Kronecker-factored approximate Fisher matrix for convolution layersPreprint
Grosse, R., & Martens, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.01407 · Cited by 79 · see citing works
Knowledge Transfer in Deep Block-Modular Neural Networks
Terekhov, A. V., Montone, G., & O'Regan, J. K. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22979-9_27 · Cited by 70 · see citing works
Flagged first, then by citations
4 of the 33 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [10]10. Le Roux, N., Pierre-Antoine, M., Bengio, Y.: Topmoumoute online natural gradient algorithm. In: NIPS (2007) 4
Cited by
Sorted by citations
A Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647 · see citing works
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113 · see citing works
“Besides, weight regularization can also be combined with experience replay for better performance and generality [44], [52].”
BackgroundMethodInfluentialEfficient Lifelong Learning with A-GEM
Chaudhry, A., Ranzato, M., Rohrbach, M., & Elhoseiny, M. (2018) · arXiv · doi:10.48550/arxiv.1812.00420 · Cited by 852 · see citing works
Self-citationLearning to Prompt for Continual Learning
Wang, Z., Zhang, Z., Lee, C., Zhang, H., Sun, R., Ren, X., Su, G., Perot, V., Dy, J., & Pfister, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00024 · Cited by 846 · see citing works
DER: Dynamically Expandable Representation for Class Incremental Learning
Yan, S., Xie, J., & He, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00303 · Cited by 752 · see citing works
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification
Masana, M., Liu, X., Twardowski, B., Menta, M., Bagdanov, A. D., & van de Weijer, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3213473 · Cited by 740 · see citing works
Three scenarios for continual learningPreprint
van de Ven, G. M., & Tolias, A. S. (2019) · Lirias · doi:10.48550/arxiv.1904.07734 · Cited by 714 · see citing works
Dhar, P., Singh, R., Peng, K., Wu, Z., & Chellappa, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00528 · Cited by 548 · see citing works
Learning to Learn without Forgetting by Maximizing Transfer and Minimizing InterferencePreprint
Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., & Tesauro, G. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11910 · Cited by 534 · see citing works
Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., & Díaz-Rodríguez, N. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.12.004 · Cited by 531 · see citing works
On Tiny Episodic Memories in Continual LearningPreprint
Chaudhry, A., Rohrbach, M., Elhoseiny, M., Ajanthan, T., Dokania, P. K., Torr, P. H. S., & Ranzato, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10486 · Cited by 506 · see citing works
Self-citationGDumb: A Simple Approach that Questions Our Progress in Continual Learning
Prabhu, A., Torr, P. H. S., & Dokania, P. K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_31 · Cited by 497 · see citing works
Self-citationPrototype Augmentation and Self-Supervision for Incremental Learning
Zhu, F., Zhang, X.-Y., Wang, C., Yin, F., & Liu, C.-L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00581 · Cited by 427 · see citing works
Online continual learning in image classification: An empirical survey
Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., & Sanner, S. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.10.021 · Cited by 422 · see citing works
DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion
Douillard, A., Ramé, A., Couairon, G., & Cord, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00907 · Cited by 379 · see citing works
Class-Incremental Learning: A Survey
Zhou, D.-W., Wang, Q.-W., Qi, Z.-H., Ye, H.-J., Zhan, D.-C., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429383 · Cited by 376 · see citing works
Rainbow Memory: Continual Learning with a Memory of Diverse Samples
Bang, J., Kim, H., Yoo, Y., Ha, J.-W., & Choi, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00812 · Cited by 376 · see citing works
Modeling the Background for Incremental Learning in Semantic Segmentation
Cermelli, F., Mancini, M., Bulò, S. R., Ricci, E., & Barbara Caputo, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00925 · Cited by 350 · see citing works
Class-incremental Learning via Deep Model Consolidation
Zhang, J., Zhang, J., Ghosh, S., Li, D., Taşcı, Ş., Heck, L. P., Zhang, H., & Kuo, C. ‐. J. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093365 · Cited by 350 · see citing works
Semantic Drift Compensation for Class-Incremental Learning
Yu, L., Twardowski, B., Liu, X., Herranz, L., Wang, K., Cheng, Y., Jui, S., & van de Weijer, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00701 · Cited by 340 · see citing works
Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning
Ostapenko, O., Puscas, M., Klein, T., Jähnichen, P., & Nabi, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01158 · Cited by 294 · see citing works
PLOP: Learning without Forgetting for Continual Semantic Segmentation
Douillard, A., Chen, Y., Dapogny, A., & Cord, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00403 · Cited by 274 · see citing works
Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting
Liu, X., Masana, M., Herranz, L., van de Weijer, J., López, A. M., & Bagdanov, A. D. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545895 · Cited by 273 · see citing works
REMIND Your Neural Network to Prevent Catastrophic Forgetting
Hayes, T. L., Kafle, K., Shrestha, R., Acharya, M., & Kanan, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_28 · Cited by 269 · see citing works
Incremental Learning Techniques for Semantic Segmentation
Michieli, U., & Zanuttigh, P. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00400 · Cited by 262 · see citing works
Adaptive Aggregation Networks for Class-Incremental Learning
Liu, Y., Schiele, B., & Sun, Q. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00257 · Cited by 245 · see citing works
Co 2 L: Contrastive Continual Learning
Cha, H., Lee, J., & Shin, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00938 · Cited by 225 · see citing works
(2019) · arXiv (Cornell University) · Cited by 214 · see citing works
Self-citationOvercoming Catastrophic Forgetting With Unlabeled Data in the Wild
Lee, K., Lee, K., Shin, J., & Lee, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00040 · Cited by 214 · see citing works
Using Hindsight to Anchor Past Knowledge in Continual Learning
Chaudhry, A., Gordo, A., Dokania, P. K., Torr, P. H. S., & López-Paz, D. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16861 · Cited by 208 · see citing works
Self-citationCompacting, Picking and Growing for Unforgetting Continual LearningPreprint
Hung, S. C. Y., Tu, C.-H., Wu, C.-E. R., Chen, C., Chan, Y.-M., & Chen, C.-S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.06562 · Cited by 204 · see citing works
Online Class-Incremental Continual Learning with Adversarial Shapley Value
Shim, D., Mai, Z., Jeong, J., Sanner, S., Kim, H., & Jang, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17159 · Cited by 203 · see citing works
Towards Robust Evaluations of Continual LearningPreprint
Farquhar, S., & Gal, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09733 · Cited by 202 · see citing works
Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning
Cheraghian, A., Rahman, S., Fang, P., Roy, S. K., Petersson, L., & Harandi, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00256 · Cited by 201 · see citing works
Distilling Causal Effect of Data in Class-Incremental Learning
Hu, X., Tang, K., Miao, C., Hua, X., & Zhang, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00395 · Cited by 199 · see citing works
SS-IL: Separated Softmax for Incremental Learning
Ahn, H., Kwak, J., Lim, S., Bang, H., Kim, H., & Moon, T. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00088 · Cited by 196 · see citing works
Mai, Z., Li, R., Kim, H., & Sanner, S. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00398 · Cited by 191 · see citing works
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks
Zhang, J. O., (Sasha), A. S., Zamir, A., Guibas, L. J., & Malik, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_41 · Cited by 190 · see citing works
Continual Lifelong Learning in Natural Language Processing: A Survey
Biesialska, M., Biesialska, K., & Costa-jussà, M. R. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.574 · Cited by 190 · see citing works
Lifelong GAN: Continual Learning for Conditional Image Generation
Zhai, M., Chen, L., Tung, F., He, J., Nawhal, M., & Mori, G. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00285 · Cited by 186 · see citing works
Incremental Learning Using Conditional Adversarial Networks
Xiang, Y., Fu, Y., Ji, P., & Huang, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00672 · Cited by 185 · see citing works
Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting
Chen, S., Hou, Y., Cui, Y., Che, W., Liu, T., & Yu, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.634 · Cited by 181 · see citing works
Adversarial Continual Learning
Ebrahimi, S., Meier, F., Calandra, R., Darrell, T., & Rohrbach, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58621-8_23 · Cited by 178 · see citing works
Loss of plasticity in deep continual learning
Dohare, S., Hernandez-Garcia, J. F., Lan, Q., Rahman, P., Mahmood, A. R., & Sutton, R. S. (2024) · Nature · doi:10.1038/s41586-024-07711-7 · Cited by 172 · see citing works
“[20] observed loss of plasticity in continual image classification problems.”
Memory-Efficient Incremental Learning Through Feature Adaptation
İşcen, A., Zhang, J., Lazebnik, S., & Schmid, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_41 · Cited by 170 · see citing works
Michieli, U., & Zanuttigh, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00117 · Cited by 158 · see citing works
Self-Supervised Models are Continual Learners
Fini, E., da Costa, V. G. T., Alameda-Pineda, X., Ricci, E., Alahari, K., & Mairal, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00940 · Cited by 155 · see citing works
Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
Hayes, T. L., & Kanan, C. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00118 · Cited by 148 · see citing works
Mundt, M., Hong, Y., Pliushch, I., & Ramesh, V. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.01.014 · Cited by 148 · see citing works
iTAML: An Incremental Task-Agnostic Meta-learning Approach
Rajasegaran, J., Khan, S. H., Hayat, M., Khan, F. S., & Shah, M. A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01360 · Cited by 147 · see citing works
Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay
Zhou, F., & Cao, C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i5.16602 · Cited by 144 · see citing works
Generative Feature Replay For Class-Incremental Learning
Liu, X., Wu, C., Menta, M., Herranz, L., Raducanu, B., Bagdanov, A. D., Jui, S., & van de Weijer, J. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00121 · Cited by 135 · see citing works
Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive ProcessesPreprint
Requeima, J., Gordon, J. A. R., Bronskill, J. F., Nowozin, S., & Turner, R. E. (2019) · arXiv (Cornell University) · doi:10.17863/cam.47574 · Cited by 134 · see citing works
Class-Incremental Continual Learning Into the eXtended DER-Verse
Boschini, M., Bonicelli, L., Buzzega, P., Porrello, A., & Calderara, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3206549 · Cited by 133 · see citing works
On Learning the Geodesic Path for Incremental Learning
Simon, C., Koniusz, P., & Harandi, M. T. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00164 · Cited by 132 · see citing works
Continual learning for recurrent neural networks: An empirical evaluation
Cossu, A., Carta, A., Lomonaco, V., & Bacciu, D. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.07.021 · Cited by 126 · see citing works
Training Networks in Null Space of Feature Covariance for Continual Learning
Wang, S., Li, X., Sun, J., & Xu, Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00025 · Cited by 126 · see citing works
Representation Compensation Networks for Continual Semantic Segmentation
Zhang, C., Xiao, J.-W., Liu, X., Chen, Y.-C., & Cheng, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00692 · Cited by 126 · see citing works
Continual Learning, Fast and Slow
Pham, Q., Liu, C., & Hoi, S. C. H. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3324203 · Cited by 125 · see citing works
Understanding the Role of Training Regimes in Continual LearningPreprint
Mirzadeh, S. I., Farajtabar, M., Pascanu, R., & Ghasemzadeh, H. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06958 · Cited by 123 · see citing works
GCR: Gradient Coreset based Replay Buffer Selection for Continual Learning
Tiwari, R., Killamsetty, K., Iyer, R., & Shenoy, P. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00020 · Cited by 122 · see citing works
An Appraisal of Incremental Learning Methods
Luo, Y., Yin, L., Bai, W., & Mao, K. (2020) · Entropy · doi:10.3390/e22111190 · Cited by 120 · see citing works
IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing
Xie, G., Wang, J., Liu, J., Lyu, J., Liu, Y., Wang, C., Zheng, F., & Jin, Y. (2024) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3357213 · Cited by 119 · see citing works
Rethinking Experience Replay: a Bag of Tricks for Continual Learning
Buzzega, P., Boschini, M., Porrello, A., & Calderara, S. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412614 · Cited by 118 · see citing works
MgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning
Zhao, H., Fu, Y., Kang, M., Tian, Q., Wu, F., & Li, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3133897 · Cited by 115 · see citing works
Memory-Efficient Class-Incremental Learning for Image Classification
Zhao, H., Wang, H., Fu, Y., Wu, F. Q., & Li, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3072041 · Cited by 114 · see citing works
Jha, A. K., Bradshaw, T., Buvat, I., Hatt, M., Prabhat, K. C., Liu, C., Obuchowski, N., Saboury, B., Slomka, P. J., Sunderland, J. J., et al. (2022) · Journal of Nuclear Medicine · doi:10.2967/jnumed.121.263239 · Cited by 113 · see citing works
Online Continual Learning with Maximally Interfered RetrievalPreprint
Aljundi, R., Caccia, L., Belilovsky, E., Caccia, M., Lin, M., Charlin, L., & Tuytelaars, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04742 · Cited by 107 · see citing works
Replay in Deep Learning: Current Approaches and Missing Biological Elements
Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski, T. J., & Kanan, C. (2021) · Neural Computation · doi:10.1162/neco_a_01433 · Cited by 106 · see citing works
Knowledge-Preserving Incremental Social Event Detection via Heterogeneous GNNs
Cao, Y., Peng, H. M., Wu, J. X., Dou, Y., Li, J., & Yu, P. S. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449834 · Cited by 101 · see citing works
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
Wang, Z., Yang, E., Shen, L., & Huang, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3498346 · Cited by 101 · see citing works
Selfless Sequential LearningPreprint
Aljundi, R., Rohrbach, M., & Tuytelaars, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05421 · Cited by 99 · see citing works
Knowledge distillation for incremental learning in semantic segmentation
Michieli, U., & Zanuttigh, P. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103167 · Cited by 98 · see citing works
Task-Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates
Zeno, C., Golan, I., Hoffer, E., & Soudry, D. (2021) · Neural Computation · doi:10.1162/neco_a_01430 · Cited by 98 · see citing works
Imbalanced Continual Learning with Partitioning Reservoir Sampling
Kim, C. D., Jeong, J., & Kim, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_25 · Cited by 97 · see citing works
Episodic Memory in Lifelong Language Learning
d'Autume, C. D. M., Ruder, S., Kong, L., & Yogatama, D. (2019) · arXiv · doi:10.48550/arxiv.1906.01076 · Cited by 95 · see citing works
Toward Training Recurrent Neural Networks for Lifelong Learning
Sodhani, S., Chandar, S., & Bengio, Y. (2019) · Neural Computation · doi:10.1162/neco_a_01246 · Cited by 93 · see citing works
More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning
Liu, Y., Parisot, S., Slabaugh, G., Jia, X., Leonardis, A., & Tuytelaars, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58574-7_42 · Cited by 93 · see citing works
Incorporating neuro-inspired adaptability for continual learning in artificial intelligence
Wang, L., Zhang, X., Li, Q., Zhang, M., Su, H., Zhu, J., & Zhong, Y. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00747-w · Cited by 91 · see citing works
IncDet: In Defense of Elastic Weight Consolidation for Incremental Object Detection
Liu, L., Kuang, Z., Chen, Y., Xue, J.-H., Yang, W., & Zhang, W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3002583 · Cited by 90 · see citing works
TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation
Zhang, J., Chen, C., Zhuang, W., & Lyu, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00441 · Cited by 90 · see citing works
Uncertainty-Aware Contrastive Distillation for Incremental Semantic Segmentation
Yang, G., Fini, E., Xu, D., Rota, P., Ding, M., Nabi, M., Alameda-Pineda, X., & Ricci, E. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3163806 · Cited by 88 · see citing works
LIQA: Lifelong Blind Image Quality Assessment
Liu, J., Zhou, W., Li, X., Xu, J., & Chen, Z. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3190700 · Cited by 85 · see citing works
Adaptive Deep Models for Incremental Learning: Considering Capacity Scalability and Sustainability
Yang, Y., Zhou, D.-W., Zhan, D.-C., Xiong, H., & Jiang, Y. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330865 · Cited by 83 · see citing works
Gu, Y., Yang, X., Wei, K., & Deng, C. D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00729 · Cited by 82 · see citing works
Continual Learning with Node-Importance based Adaptive Group Sparse RegularizationPreprint
Jung, S., Ahn, H., Cha, S., & Moon, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13726 · Cited by 82 · see citing works
Wang, L., Yang, K., Li, C., Hong, L., Li, Z., & Zhu, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00534 · Cited by 81 · see citing works
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual LearningPreprint
Caccia, M., Rodríguez, P., Ostapenko, O., Normandin, F., Lin, M., Caccia, L., Laradji, I., Rish, I., Lacoste, A., Vázquez, D., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.05856 · Cited by 81 · see citing works
Continual Adaptation of Visual Representations via Domain Randomization and Meta-learning
Volpi, R., Larlus, D., & Rogez, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00442 · Cited by 80 · see citing works
Dynamic Support Network for Few-Shot Class Incremental Learning
Yang, B., Lin, M., Zhang, Y., Liu, B., Liang, X., Ji, R., & Ye, Q. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3175849 · Cited by 80 · see citing works
Incremental Learning Meets Transfer Learning: Application to Multi-site Prostate MRI Segmentation
You, C., Xiang, J., Su, K., Zhang, X., Dong, S., Onofrey, J. A., Staib, L. H., & Duncan, J. S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18523-6_1 · Cited by 80 · see citing works
Recent Advances of Continual Learning in Computer Vision: An Overview
Qu, H., Rahmani, H., Xu, L., Williams, B. M., & Liu, J. (2025) · IET Computer Vision · doi:10.1049/cvi2.70013 · Cited by 80 · see citing works
Continual Learning for Text Classification with Information Disentanglement Based Regularization
Huang, Y., Zhang, Y., Chen, J., Wang, X., & Yang, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.218 · Cited by 74 · see citing works
MetaSearch: Incremental Product Search via Deep Meta-Learning
Wang, Q., Liu, X., Liu, W., Liu, A.-A., Liu, W., & Mei, T. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3004249 · Cited by 74 · see citing works
Continual Deep Learning by Functional Regularisation of Memorable PastPreprint
Pan, P., Swaroop, S., Immer, A., Eschenhagen, R., Turner, R. E., & Khan, M. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.14070 · Cited by 74 · see citing works
Continual Learning: A Review of Techniques, Challenges, and Future Directions
Wickramasinghe, B., Saha, G., & Roy, K. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3339091 · Cited by 73 · see citing works
Toward Understanding Catastrophic Forgetting in Continual LearningPreprint
Nguyen, C. V., Achille, A., Lam, M., Hassner, T., Mahadevan, V., & Soatto, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01091 · Cited by 73 · see citing works
Learning Without Forgetting for Vision-Language Models
Zhou, D.-W., Zhang, Y., Wang, Y., Ning, J., Ye, H.-J., Zhan, D., & Liu, Z. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3540889 · Cited by 71 · see citing works
Self-Training for Class-Incremental Semantic Segmentation
Yu, L., Liu, X., & van de Weijer, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3155746 · Cited by 70 · see citing works
Continual Learning with Lifelong Vision Transformer
Wang, Z., Liu, L., Duan, Y., Kong, Y., & Tao, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00027 · Cited by 70 · see citing works
- Authors
- Chaudhry, A., Dokania, P. K., Ajanthan, T., & Torr, P. H. S.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Ajanthan, Thalaiyasingam
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2018
- Pages
- 556-572
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W2786446225
- Cited by
- 1,091
- References
- 33
- Citations by year
- 2026: 78 · 2025: 182 · 2024: 217 · 2023: 189 · 2022: 135 · 2021: 154 · 2020: 84 · 2019: 45 · 2018: 7
- FWCI
- 29.59
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Lecture notes in computer science (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/N019474/1, EP/M013774/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- incremental learning, catastrophic forgetting, intransigence, path integral, Kullback-Leibler divergence, knowledge preservation, classifier updating
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
