The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Three types of incremental learning
Open accessAbstract
Incrementally learning new information from a non-stationary stream of data, referred to as ‘continual learning’, is a key feature of natural intelligence, but a challenging problem for deep neural networks. In recent years, numerous deep learning methods for continual learning have been proposed, but comparing their performances is difficult due to the lack of a common framework. To help address this, we describe three fundamental types, or ‘scenarios’, of continual learning: task-incremental, domain-incremental and class-incremental learning. Each of these scenarios has its own set of challenges. To illustrate this, we provide a comprehensive empirical comparison of currently used continual learning strategies, by performing the Split MNIST and Split CIFAR-100 protocols according to each scenario. We demonstrate substantial differences between the three scenarios in terms of difficulty and in terms of the effectiveness of different strategies. The proposed categorization aims to structure the continual learning field, by forming a key foundation for clearly defining benchmark problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“To avert catastrophic forgetting, various algorithms[11, 48, 49] employ task-specific components or design separate networks for each task.”
From Few-Shot Class-Incremental Learning for Classification and Object Detection: A Survey (2025) · cites this paper as [11]
The cited source says what the claim attributes to it.
In the citing paper
“DIL is an ML problem designed to continuously adapt to data distribution from different domains while the structure of the problem is always the same [11].”
From Few-Shot Class-Incremental Learning for Classification and Object Detection: A Survey (2025) · cites this paper as [11]
The cited source says what the claim attributes to it.
In the citing paper
“TIL's primary challenge lies in identifying shared features across tasks to balance performance and computational complexity, using knowledge from one task to enhance performance in others [11].”
From Few-Shot Class-Incremental Learning for Classification and Object Detection: A Survey (2025) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
(2017) · Cited by 11,625
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
Catastrophic forgetting in connectionist networks
French, R. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01294-2 · Cited by 2,311
Research works printed but not linked: 19
- Chen, Z. & Liu, B. Lifelong machine learning. Synth. Lect. Artif. Intell. Mach. Learn. 12, 1–207 (2018).
- Mundt, M., Lang, S., Delfosse, Q. & Kersting, K. CLEVA-compass: A continual learning evaluation assessment compass to promote research transparency and comparability. In International Conference on Learning Representations (2022).
- Ruvolo, P. & Eaton, E. ELLA: An efficient lifelong learning algorithm. In International Conference on Machine Learning 507–515 (PMLR, 2013).
- Ramesh, R. & Chaudhari, P. Model Zoo: A growing brain that learns continually. In International Conference on Learning Representations (2022).
- Lopez-Paz, D. & Ranzato, M. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems Vol. 30, 6470–6479 (2017).
- Mirza, M. J., Masana, M., Possegger, H. & Bischof, H. An efficient domain-incremental learning approach to drive in all weather conditions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 3001–3011 (2022).
- von Oswald, J., Henning, C., Sacramento, J. & Grewe, B. F. Continual learning with hypernetworks. In International Conference on Learning Representations (2020).
- Lomonaco, V. & Maltoni, D. Core50: a new dataset and benchmark for continuous object recognition. In Conference on Robot Learning 17–26 (PMLR, 2017).
- Shin, H., Lee, J. K., Kim, J. & Kim, J. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems Vol. 30, 2994–3003 (2017).
- Zeno, C., Golan, I., Hoffer, E. & Soudry, D. Task agnostic continual learning using online variational bayes. Preprint at https://arxiv.org/abs/1803.10123v3 (2019). · arXiv
- Caccia, L., Belilovsky, E., Caccia, M. & Pineau, J. Online learned continual compression with adaptive quantization modules. In International Conference on Machine Learning 1240–1250 (PMLR, 2020).
- Lee, S., Ha, J., Zhang, D. & Kim, G.A neural dirichlet process mixture model for task-free continual learning. In International Conference on Learning Representations (2020).
- Jin, X., Sadhu, A., Du, J. & Ren, X. Gradient-based editing of memory examples for online task-free continual learning. In Advances in Neural Information Processing Systems Vol. 34, 29193–29205 (2021).
- Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T. & Wayne, G. Experience replay for continual learning. In Advances in Neural Information Processing Systems Vol. 32 (2019).
- Chaudhry, A., Ranzato, M., Rohrbach, M. & Elhoseiny, M. Efficient lifelong learning with a-gem. In International Conference on Learning Representations (2019).
- Aljundi, R. et al. Online continual learning with maximally interfered retrieval. In Advances in Neural Information Processing Systems Vol. 32 (2019). · DOI · Paper page
- Hess, T., Mundt, M., Pliushch, I. & Ramesh, V. A procedural world generation framework for systematic evaluation of continual learning. In Thirty-fifth Conference on Neural Information Processing Systems, Datasets and Benchmarks Track (2021).
- Ioffe, S. & Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning 448–456 (PMLR, 2015).
- Rezende, D. & Mohamed, S. Variational inference with normalizing flows. In International Conference on Machine Learning 1530–1538 (PMLR, 2015).
Cited by
Class-Incremental Learning: A Survey
Zhou, D.-W., Wang, Q.-W., Qi, Z.-H., Ye, H.-J., Zhan, D.-C., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429383 · Cited by 376
Advances and Challenges in Meta-Learning: A Technical Review
Vettoruzzo, A., Bouguelia, M.-R., Vanschoren, J., Rögnvaldsson, T., & Santosh, K. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3357847 · Cited by 314
Continual Learning of Large Language Models: A Comprehensive Survey
Shi, H., Xu, Z., Wang, H., Qin, W., Wang, W., Wang, Y., Wang, Z., Ebrahimi, S., & Wang, H. (2025) · ACM Computing Surveys · doi:10.1145/3735633 · Cited by 293
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts
Liang, J., He, R., & Tan, T. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-024-02181-w · Cited by 271
Challenges in data-driven geospatial modeling for environmental research and practice
Koldasbayeva, D., Tregubova, P., Gasanov, M., Zaytsev, A., Petrovskaia, A., & Burnaev, E. (2024) · Nature Communications · doi:10.1038/s41467-024-55240-8 · Cited by 184
Metadata
- Authors
- van de Ven, G. M., Tuytelaars, T., & Tolias, A. S.
- Affiliations
- Baylor College of Medicine (US)
- KU Leuven (BE)
- Corresponding author
- van de Ven, Gido M.
- Venue
- Nature Machine Intelligence
- Published
- 2022-12-05
- Volume
- 4
- Issue
- 12
- Pages
- 1185-1197
- ISSN
- 2522-5839 (ISSN-L)
- PMC
- PMC9771807
- OpenAlex
- W4312210066
- Cited by
- 665
- References
- 43
- Citations by year
- 2026: 173 · 2025: 258 · 2024: 172 · 2023: 60 · 2022: 2
- FWCI
- 40.48
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature Machine Intelligence (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Lirias (KU Leuven) (submittedVersion, cc-by)
- Rice Research Repository (Rice University) (submittedVersion, cc-by)
- Apollo (University of Cambridge) (submittedVersion, cc-by)
- Apollo
- Funders
- International Brain Research Organization
- National Institutes of Health
- Defense Advanced Research Projects Agency (HR0011, HR0011-18-2-0025)
- Advanced Research Projects Agency
- Intelligence Advanced Research Projects Activity (D16PC00003)
- Interior Business Center
- National Institute of Mental Health (R01MH109556)
- National Eye Institute (P30EY002520)
- H2020 European Research Council (101021347)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- continual learning, incremental learning, task-incremental learning, domain-incremental learning, class-incremental learning, catastrophic forgetting, non-stationary data streams, deep neural network
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Three types of incremental learning. https://app.cochat.ai/papers/10.1038/s42256-022-00568-3
- The paper
- van de Ven, G. M., Tuytelaars, T., & Tolias, A. S. (2022). Three types of incremental learning. Nature Machine Intelligence. https://doi.org/10.1038/s42256-022-00568-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.