A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
Open accessThe DOI matches its DataCite record; 102 of the 102 references it cites were found in a registry or an index; 43 of the 89 sources its claims cite were checked for support.
Abstract
The ability of artificial agents to increment their capabilities when\nconfronted with new data is an open challenge in artificial intelligence. The\nmain challenge faced in such cases is catastrophic forgetting, i.e., the\ntendency of neural networks to underfit past data when new ones are ingested. A\nfirst group of approaches tackles forgetting by increasing deep model capacity\nto accommodate new knowledge. A second type of approaches fix the deep model\nsize and introduce a mechanism whose objective is to ensure a good compromise\nbetween stability and plasticity of the model. While the first type of\nalgorithms were compared thoroughly, this is not the case for methods which\nexploit a fixed size model. Here, we focus on the latter, place them in a\ncommon conceptual and experimental framework and propose the following\ncontributions: (1) define six desirable properties of incremental learning\nalgorithms and analyze them according to these properties, (2) introduce a\nunified formalization of the class-incremental learning problem, (3) propose a\ncommon evaluation framework which is more thorough than existing ones in terms\nof number of datasets, size of datasets, size of bounded memory and number of\nincremental states, (4) investigate the usefulness of herding for past\nexemplars selection, (5) provide experimental evidence that it is possible to\nobtain competitive performance without the use of knowledge distillation to\ntackle catastrophic forgetting and (6) facilitate reproducibility by\nintegrating all tested methods in a common open-source repository. The main\nexperimental finding is that none of the existing algorithms achieves the best\nresults in all evaluated settings. Important differences arise notably if a\nbounded memory of past classes is allowed or not.\n
Overview
CoChat Summary
TL;DR
This paper provides a comparative study of class incremental learning (IL) methods for visual tasks under a unified formalization and evaluation protocol. It finds that no single algorithm is best in all settings, and that the best choice depends strongly on whether a bounded memory of past classes is available.
Key findings
- With memory allowed, the strongest overall results came from methods that treat class IL as an imbalanced learning problem: the paper reports FT th as best overall with GIL = -3.62, followed by ScaIL (-3.70), BiC (-4.03), and LUCIR CNN (-4.13).
- In the memory setting, the authors report that vanilla FT can outperform iCaRL globally: FT has GIL = -5.19 versus iCaRL’s -7.36; they also state that FT is globally better than iCaRL by over 2 GIL points.
- Without memory, fixed-representation methods are reported to be much more competitive than fine-tuning methods; the best global score is said to come from DeeSIL, followed by Deep-SLDA, FR, and REMIND.
- Herding-based exemplar selection outperformed random selection for all methods tested in the reported experiments, though the gain varied by method.
Methods
- The paper compares class incremental learning algorithms in a common conceptual and experimental framework, focusing on methods for constant model complexity with bounded or no memory of past classes.
- Experiments use four datasets: ILSVRC (1,231,167 train / 50,000 test), VGGFACE2 (491,746 / 50,000), LANDMARKS (374,367 / 20,000), and CIFAR100 (50,000 / 10,000).
- Evaluation uses a ResNet-18 backbone with SGD, top-5 accuracy averaged over incremental states, and the Incremental Learning Gap (GIL) across configurations with varying numbers of states (T = 10, 20, 50) and memory sizes (including K = 2%, 1%, 0.5%, and K = 0 in the no-memory case).
Limitations
- The paper states that the evaluation scenario is still simplified and recommends future work on more realistic settings: no assumption of readily annotated incoming data, variable numbers of classes per incremental state, and imbalanced datasets.
- The authors note that their conclusions are limited to the tested setting of fixed-complexity visual class incremental learning; methods that grow model capacity, generative replay methods, and some online approaches are discussed but not evaluated under this protocol.
- Evident from the text: the study is constrained to four datasets and a specific set of algorithm families, so the findings may not generalize beyond the tested visual tasks and configurations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
28 Valid · 3 Not valid · 29 Not checked · 127 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The method does not need a bounded memory for the update.”
What the sources say
- Disputedcites [56] [56] Rebuffi, S., Kolesnikov, A., Sperl, G., Lampert, C.H., 2017b. icarl: Incremental classifier and representation learning, in: Conference on Computer Vision and Pattern Recognition. - sourceExperimental — accuracy not yet measured
If the cited method is iCaRL, the paper says the opposite. Section 2.4 (p. 4) states that its stored exemplars never exceed a memory budget of K, and the representation-learning discussion (p. 4) says those exemplars are used during updates. The copy does not identify a method called FR. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“However, fixed-representations work well only if the task does not change over time, as it is the case in the evaluated scenarios presented in [2].”
What the sources say
- Disputedcites [2] [2] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T., 2018. Memory aware synapses: Learning what (not) to forget, in: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (Eds.), Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part III, Springer. pp. 144-161. - sourceExperimental — accuracy not yet measured
The evaluated scenarios do change tasks over time. Section 5.1 (p. 10) describes sequences such as Scenes → Birds, and the Longer Sequence results (p. 11) evaluate performance after successive tasks. The paper therefore contradicts the claim that its evaluated scenarios involve an unchanging task; it does not establish the broader rule about when fixed representations work well. (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“FT BAL is also tributary to the fixed memory K because past exemplars are needed for the balancing step.”
What the sources say
- Supportedcites [7] [7] Belouadah, E., Popescu, A., 2019. Il2m: Class incremental learning with dual memory, in: Proceedings of the IEEE International Conference on Computer Vision, pp. 583-592. - source
Section 5.3.1 says FT BAL uses fine-tuning with memory and that, as memory decreases, more data must be dropped during its balancing step. Section 5.1 identifies that step as balanced fine-tuning after the initial imbalanced training. Together, these passages support the claim that FT BAL's balancing depends on stored past exemplars. (LLM, conf=0.88)
- Not matchedcites [8] [8] Belouadah, E., Popescu, A., 2020. Scail: Classifier weights scaling for class incremental learning, in: The IEEE Winter Conference on Applications of Computer Vision (WACV). -
More than one paper fits this citation.
- Partly supportedcites [14] [14] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K., 2018. End-to-end incremental learning, in: Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 814, 2018, Proceedings, Part XII, pp. 241-257. - sourceExperimental — accuracy not yet measured
Section 4 (Incremental Learning; page not provided) says balanced fine-tuning uses equal numbers of samples from old and new classes, supporting the need for stored past exemplars. But Section 3.1 (Representative memory; page not provided) describes both a fixed-capacity memory K and a memory that grows with the number of classes. The balancing step therefore depends on past exemplars, not specifically on fixed memory K. (LLM, conf=0.87)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“one early example of such work [56] is shown to outperform continual learning approaches when tested in a common experimental setting [35].”
What the sources say
- Supportedcites [56] [56] Rebuffi, S., Kolesnikov, A., Sperl, G., Lampert, C.H., 2017b. icarl: Incremental classifier and representation learning, in: Conference on Computer Vision and Pattern Recognition. - source
The Abstract (p. 1) identifies iCaRL as a class-incremental learning strategy. The Results (p. 7) report that iCaRL outperformed the other methods tested. The separate citation [35] is attached to the common experimental setting, so that setting is not attributed to this source. (LLM, conf=0.91)
- Supportedcites [35] [35] Lange, M.D., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G.G., Tuytelaars, T., 2019. Continual learning: A comparative study on how to defy forgetting in classification tasks. CoRR abs/1909.08383. - source
Section 5.1 (p. 4) identifies iCaRL as an early class-incremental method and says the authors adapted it to their task-incremental, multi-head setting for a fair comparison. Section 6.3.1 (p. 10) reports that iCaRL outperformed all compared continual-learning methods for the DEEP model on Tiny Imagenet. The reported result is specific to that configuration. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“FT NEM can only be deployed if a bounded memory is allowed to alleviate catastrophic forgetting.”
What the sources say
- Supportedcites [7] [7] Belouadah, E., Popescu, A., 2019. Il2m: Class incremental learning with dual memory, in: Proceedings of the IEEE International Conference on Computer Vision, pp. 583-592. - source
Section 5.1 defines FT NEM by its nearest-exemplars-mean classifier. Section 5.3 reports that, with no memory (K = 0), all methods derived from fine tuning perform identically because their score-rectification methods rely on exemplars. Thus FT NEM’s distinctive classifier requires exemplar memory, even though the paper still reports a K = 0 result under the FT NEM label. (LLM, conf=0.86)
- Not matchedcites [8] [8] Belouadah, E., Popescu, A., 2020. Scail: Classifier weights scaling for class incremental learning, in: The IEEE Winter Conference on Applications of Computer Vision (WACV). -
More than one paper fits this citation.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
106 more claims · 6 with their cited sources checked
Create a free account to see all 187 claims and to verify the 12 references still open.
- It may state two separate claims as one. 52
- Probably not a substantial claim. 27
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Although some surveys briefly mention FSCIL [32, 35], they only provide a short introduction to the concept and a few studies without offering a thorough or systematic analysis.”
From Few-Shot Class-Incremental Learning for Classification and Object Detection: A Survey (2025) · cites this paper as [32]
Experimental — accuracy not yet measured
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“In an end-to-end DNN [17], a softmax function is applied to transform raw scores 𝒐 𝑡 = ( 1 𝑡 , 𝑜 2 𝑡 , ..., ) to probabilities 𝒑 𝑡 = ( 1 𝑡 , 𝑝 2 𝑡 , ..., ) .”
BackgroundMethodInfluentialA Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 2×
“• ILSVRC [47]”
MethodA mathematical theory of communication
Hockett, C. F., Shannon, C., & Weaver, W. (1948) · MOCO · doi:10.1145/584091.584093 · Cited by 30,543 · cited 1×
“We also tried exemplar selection techniques inspired from active learning approaches such as: entropy [52], min margin [51], core-set [50], k-means [62].”
MethodLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992 · cited 1×
“batch of classes and a battery of Support Vector Machines (SVMs) [7] to incrementally learn new classes.”
Background(2017) · Cited by 11,625
The Mathematical Theory of Communication
Shannon, C. E., Weaver, W., & Wiener, N. (1950) · Physics Today · doi:10.1063/1.3067010 · Cited by 9,801 · cited 1×
“We also tried exemplar selection techniques inspired from active learning approaches such as: entropy [65], min margin [64], coreset [63], k-means [81].”
MethodScikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539 · cited 1×
“since it provides better performance than the original implementation. LwF[36] is run using the code from [56], The SVMs in DeeSIL[6] are implemented using LinearSVC solver from Scikit-Learn toolbox [51]. The SVMs were optimized using classical grid search as described in the original paper. FTand its derivatives are…”
MethodJoint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2603342 · Cited by 6,214 · cited 1×
“Face cropping is done with MTCNN [64] before further processing.”
MethodBasic objects in natural categories
Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M., & Boyes-Braem, P. (1976) · Cognitive Psychology · doi:10.1016/0010-0285(76)90013-x · Cited by 5,643 · cited 1×
“• CIFAR100 [25] is designed for object recognition and includes 100 basic level classes [44].”
Background(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451 · cited 4×
“Performance drops significantly if the incremental tasks change a lot and the initial representation is not transferable anymore [40].”
BackgroundMethodInfluentialLi, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · cited 10×
“Incremental Classifier and Representation Learning (iCaRL) [43] is a popular IL method that combines the use of LwF and of a memory for past class exemplars storage.”
BackgroundMethodInfluentialCatastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770 · cited 4×
“The challenge faced in all cases is catastrophic forgetting [31], i.e., the tendency of a neural network to underfit past data when new ones are ingested.”
BackgroundMethodInfluentialContinual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
A systematic study of the class imbalance problem in convolutional neural networks
Buda, M., Maki, A., & Mazurowski, M. A. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.011 · Cited by 3,088 · cited 3×
“The well-known thresholding method [8] is used to reduce the classification bias between past and new classes.”
MethodInfluentialVGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032 · cited 8×
“5% while the corresponding drop for VGGFACE2 is only of 1 .”
BackgroundInfluentialProduct Quantization for Nearest Neighbor Search
Jégou, H., Douze, M., & Schmid, C. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.57 · Cited by 2,956 · cited 1×
“REMIND relies on Product Quantizer (PQ) [21] algorithm to store intermediate representations of images as compressed vectors for fast learning.”
MethodA Growing Neural Gas Network Learns Topologies
Fritzke, B. (1994) · Cited by 1,684
Relational Knowledge Distillation
Park, W., Kim, D., Lu, Y., & Cho, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00409 · Cited by 1,681 · cited 4×
“Relational Knowledge Distillation ( ) [36] encourages the model to mimic the mutual relations between different samples.”
BackgroundMethodInfluentialA continual learning survey: Defying forgetting in classification tasks
De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2019) · arXiv · doi:10.48550/arxiv.1909.08383 · Cited by 1,647
Memory Aware Synapses: Learning What (not) to Forget
Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., & Tuytelaars, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_9 · Cited by 1,590 · cited 4×
“MG-based methods are specifically designed to cope with different visual tasks [1, 29].”
BackgroundMethodInfluentialA Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning
Yim, J., Joo, D., Bae, J., & Kim, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.754 · Cited by 1,570 · cited 1×
“The use of knowledge distillation in an IL context is similar to self-distillation [12, 61] in that it operates with the same network architecture for the teacher and the student.”
'Neural-gas' network for vector quantization and its application to time-series prediction
Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238311 · Cited by 1,431 · cited 1×
“n. Self-Organizing Maps (SOMs) are online unsupervised learning algorithms that rely on approximate stochastic gradient technique, and can be adapted to Incremental Learning. Neural Gas (NG) networks [43] and its growing NG variant [18] are related to SOMs which are often exploited for incremental learning. PROjection…”
Continual Learning with Deep Generative Replay
Shin, H., Lee, J. K., Kim, J., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1705.08690 · Cited by 1,385 · cited 1×
“Depending on the work hypotheses made, names such as continual learning [49, 53], lifelong learning [2, 35] or incremental learning (IL) [4, 10, 16, 43] are used to describe associated works.”
BackgroundDo Better ImageNet Models Transfer Better?
Kornblith, S., Shlens, J., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00277 · Cited by 1,325 · cited 3×
“DeeSIL is a straightforward application of a transfer learning [24, 40] scheme in an incremental context.”
BackgroundMethodLarge Scale Incremental Learning
Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., & Fu, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00046 · Cited by 1,313 · cited 10×
“Recent approaches report strong performance gain compared to previous work such as [10, 20, 43] either through more sophisticated definitions of knowledge distillation [19] or through the casting of IL as an imbalanced learning problem [5, 6] or a combination of both [58].”
BackgroundMethodInfluentialRusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., & Hadsell, R. (2016) · arXiv · doi:10.48550/arxiv.1606.04671 · Cited by 1,307 · cited 2×
“The model evolves by adding parameters and weights to interconnect them [42, 48, 56] or small networks [2] to include new knowledge.”
BackgroundPackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
Mallya, A., & Lazebnik, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00810 · Cited by 1,258 · cited 1×
“[30] is based on a pruning technique that iden-tifies redundant parameters and uses them to train the network on new tasks.”
MethodLearning a Unified Classifier Incrementally via Rebalancing
Hou, S., Pan, X., Loy, C. C., Wang, Z., & Lin, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00092 · Cited by 1,195 · cited 10×
“[19] present Learning a Unified Classifier Incrementally via Rebalancing (LUCIR), a method which gains a lot of traction.”
BackgroundMethodInfluentialEnd-to-End Incremental Learning
Castro, F. M., Marín-Jiménez, M. J., Guil, N., Schmid, C., & Alahari, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01258-8_15 · Cited by 1,184 · cited 10×
“The authors of [10] present an IL algorithm which differs from mainly through the way prediction bias is reduced.”
BackgroundMethodInfluentialActive Learning for Convolutional Neural Networks: A Core-Set Approach
Sener, O., & Savarese, S. (2017) · arXiv · doi:10.48550/arxiv.1708.00489 · Cited by 922
Large-Scale Image Retrieval with Attentive Deep Local Features
Noh, H., Araujo, A., Sim, J., Weyand, T., & Han, B. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.374 · Cited by 847 · cited 1×
“• Google Landmarks [33] ( LANDMARKS below) is built for landmark recognition, and we selected 1000 classes having the largest number of associated images.”
BackgroundLi, Z., & Hoiem, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_37 · Cited by 659 · cited 6×
“Learning without Forgetting ( ) [27] is a pioneering work that does not require a memory of past classes.”
BackgroundMethodInfluentialPODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning
Douillard, A., Cord, M., Ollion, C., Robert, T., & Valle, E. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58565-5_6 · Cited by 638 · cited 1×
“Alternately, PODNet [17] relies on a spacial-based distillation loss that constrains the evolution of the model’s representation, and multiple proxy vectors to flexibly represent learned classes.”
MethodExpert Gate: Lifelong Learning with a Network of Experts
Aljundi, R., Chakravarty, P., & Tuytelaars, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.753 · Cited by 619 · cited 4×
“C om pl ex ity The model evolves by adding parameters and weights to interconnect them [42, 48, 56] or small networks [2] to include new knowledge.”
BackgroundInfluentialPiggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights
Mallya, A., Davis, D., & Lazebnik, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_5 · Cited by 613 · cited 2×
“MG-based methods are specifically designed to cope with different visual tasks [1, 29].”
BackgroundMethodCNN Features off-the-shelf: an Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · arXiv · doi:10.48550/arxiv.1403.6382 · Cited by 607
Learning multiple visual domains with residual adapters
Rebuffi, S.-A., Bilen, H., & Vedaldi, A. (2017) · arXiv · doi:10.48550/arxiv.1705.08045 · Cited by 605 · cited 6×
“In spite of this observation, knowledge distillation is commonly used to tackle catastrophic forgetting [10, 16, 20, 23, 35, 41, 65].”
BackgroundMethodResultInfluentialDhar, P., Singh, R. V., Peng, K.-C., Wu, Z., & Chellappa, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00528 · Cited by 516 · cited 1×
“impact on performance and leads to a strong improvement compared to iCaRL. Sophisticated data augmentation is also used and has a small positive influence on results. Learning without Memorizing (LwM) [16] is a distillation based approach that does not need memory for past classes. Instead, the authors propose an infor…”
BackgroundFew-Shot Class-Incremental Learning
Tao, X., Hong, X., Chang, X., Dong, S., Wei, X., & Gong, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01220 · Cited by 499
Maintaining Discrimination and Fairness in Class Incremental Learning
Zhao, B., Xiao, X., Gan, G., Zhang, B., & Xia, S.-T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01322 · Cited by 494
Active Learning for Convolutional Neural Networks: A Core-Set Approach
Şener, O., & Savarese, S. (2018) · International Conference on Learning Representations · Cited by 456
Efficient Parametrization of Multi-domain Deep Neural Networks
Rebuffi, S.-A., Vedaldi, A., & Bilen, H. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00847 · Cited by 445 · cited 3×
“C om pl ex ity The model evolves by adding parameters and weights to interconnect them [42, 48, 56] or small networks [2] to include new knowledge.”
BackgroundDistance-Based Image Classification: Generalizing to New Classes at Near-Zero Cost
Mensink, T., Verbeek, J., Perronnin, F., & Csurka, G. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.83 · Cited by 431 · cited 1×
“Class-Mean [32] to counter imbalance between past and new classes.”
BackgroundFurlanello, T., Lipton, Z. C., Tschannen, M., Itti, L., & Anandkumar, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04770 · Cited by 407 · cited 1×
“The use of knowledge distillation in an IL context is similar to self-distillation [12, 61] in that it operates with the same network architecture for the teacher and the student.”
Herding dynamical weights to learn
Welling, M. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553517 · Cited by 401 · cited 2×
“Introduced in [74] and first used in an IL context by [56], its usefulness was questioned in [14, 29, 37] where it was reported to provide only marginal improvement compared to random selection.”
BackgroundMethodIncremental Linear Discriminant Analysis for Classification of Data Streams
Pang, S., Ozawa, S., & Kasabov, N. (2005) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.847744 · Cited by 343 · cited 3×
“D-SLDA [15] defines the model as M 𝑡 ≡ 𝐹 ( 𝐺 ( ⋅ )) where is the fixed upper part of the network and 𝐹 ( ⋅ ) is the output layer.”
MethodInfluentialSemantic Drift Compensation for Class-Incremental Learning
Yu, L., Twardowski, B., Liu, X., Herranz, L., Wang, K., Cheng, Y., Jui, S., & van de Weijer, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00701 · Cited by 340
Class-incremental Learning via Deep Model Consolidation
Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., & Jay Kuo, C.-C. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093365 · Cited by 328 · cited 1×
“In [63], another distillation based system is proposed, it trains two separate networks, one for new classes and one for past classes, and then combines them via a double distillation loss.”
BackgroundAljundi, R., Kelchtermans, K., & Tuytelaars, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01151 · Cited by 324 · cited 1×
“The method was later adapted for usage with unlabeled datasets [3].”
MethodMnemonics Training: Multi-Class Incremental Learning Without Forgetting
Liu, Y., Su, Y., Liu, A.-A., Schiele, B., & Sun, Q. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01226 · Cited by 309 · cited 8×
“Further work in this direction could followup on [28] and investigate in more depth which exemplar distribution is optimal for replay.”
BackgroundResultInfluentialIL2M: Class Incremental Learning With Dual Memory
Belouadah, E., & Popescu, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00067 · Cited by 302 · cited 10×
“This is the case for approaches such as [7, 14, 28].”
BackgroundMethodInfluentialSelf-citationFearNet: Brain-Inspired Model for Incremental Learning
Kemker, R., & Kanan, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10563 · Cited by 296 · cited 2×
“In a more advanced setting [22], additional parameters are needed to improve past classes performance.”
BackgroundMethodMulti-agent Diverse Generative Adversarial Networks
Ghosh, A., Kulharia, V., Namboodiri, V., Torr, P. H. S., & Dokania, P. K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00888 · Cited by 295 · cited 1×
“However, despite recent progress [13], generated images are still suboptimal for IL.”
BackgroundMnemonics Training: Multi-Class Incremental Learning Without Forgetting
LIU, Y., SU, Y., LIU, A.-A., Schiele, B., & SUN, Q. (2020) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 278
REMIND Your Neural Network to Prevent Catastrophic Forgetting
Hayes, T. L., Kafle, K., Shrestha, R., Acharya, M., & Kanan, C. (2019) · arXiv · doi:10.48550/arxiv.1910.02509 · Cited by 268
Tree-CNN: A hierarchical Deep Convolutional Neural Network for incremental learning
Roy, D., Panda, P., & Roy, K. (2020) · Neural Networks · doi:10.1016/j.neunet.2019.09.010 · Cited by 257 · cited 3×
“Recently, [46] propose an adaptive network that enables self-growth in a tree-like manner.”
BackgroundCompetitive Hebbian Learning Rule Forms Perfectly Topology Preserving Maps
Martinetz, T. (1993) · ICANN ’93 · doi:10.1007/978-1-4471-2063-6_104 · Cited by 256
REMIND Your Neural Network to Prevent Catastrophic Forgetting
Hayes, T. L., Kafle, K., Shrestha, R., Acharya, M., & Kanan, C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58598-3_28 · Cited by 248 · cited 5×
“te new knowledge [7,8,14,28,75]. The performance of these algorithms depends heavily on the existence of a bounded memory of the past. We also present results with fixedrepresentation based algorithms [6,23,24,56], which do not update the model but are also less dependent on memory. LwF [36] uses distillation loss to e…”
BackgroundMethodInfluentialIncremental Learning Through Deep Adaptation
Rosenfeld, A., & Tsotsos, J. K. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2884462 · Cited by 244 · cited 1×
“sent a lifelong learning architecture based on a network of experts. A gating mechanism uses training samples to decide which expert to transfer knowledge from for each task. Deep Adaptation Networks [58] is another model-growth based approach which adds additional parameters for each new task. The architecture is imp…”
BackgroundCOMPETITIVE HEBBIAN LEARNING RULE FORMS PERFECTLY TOPOLOGY PRESERVING MAPS
Martinez, T. M. B. (1993) · Cited by 228
Topology-Preserving Class-Incremental Learning
Tao, X., Chang, X., Hong, X., Wei, X., & Gong, Y. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58529-7_16 · Cited by 162
Extensions of vector quantization for incremental clustering
Lughofer, E. D. (2007) · Pattern Recognition · doi:10.1016/j.patcog.2007.07.019 · Cited by 160 · cited 1×
“ions of images, which are more compact than images. The stored vectors are reconstructed and replayed for memory consolidation. Vector quantization is widely used in unsupervised incremental learning [39]. Here, the authors combine the Adaptive Resonance Theory (ART) with variant of vector quantization to balance the…”
MethodGrowing a Brain: Fine-Tuning by Increasing Model Capacity
Wang, Y.-X., Ramanan, D., & Hebert, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.323 · Cited by 154 · cited 2×
“C om pl ex ity The model evolves by adding parameters and weights to interconnect them [42, 48, 56] or small networks [2] to include new knowledge.”
BackgroundMethodLifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
Hayes, T. L., & Kanan, C. (2019) · arXiv · doi:10.48550/arxiv.1909.01520 · Cited by 150
Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
Hayes, T. L., & Kanan, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00118 · Cited by 145 · cited 2×
“L without memory [56]. We note that these methods work better than distillation based IL algorithms even when initial representations are learned with a few dozens of classes. Online learning methods [23,24] are useful when the stream of data arrives image per image. The model capacity to learn individual classes is i…”
MethodInfluentialIncremental Classifier Learning with Generative Adversarial NetworksPreprint
Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., Zhang, Z., & Fu, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.00853 · Cited by 142
Towards Understanding Knowledge Distillation
Phuong, M., & Lampert, C. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.13093 · Cited by 115 · cited 2×
“LUCIR is based on three main components: (1) cosine normalization balances the magnitudes of past and new class probabilities, (2) less-forget constraint modifies the usual distillation loss to handle feature vectors instead of raw scores and (3) inter-class separation encourages the network to separate past and new c…”
BackgroundMethodGAN Memory with No ForgettingPreprint
Cong, Y., Zhao, M., Li, J., Wang, S., & Carin, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07543 · Cited by 93 · cited 1×
“in slightly better performance than that of iCaRL[56]. However, the performance significantly drops when relying exclusively on artificially generated images. Alternately, GAN Memory with No Forgetting [15] is based on sequential style modulations to represent the past memory by forming a sequential targeted generative…”
BackgroundContinual Learning in Generative Adversarial Nets
Seff, A., Beatson, A., Suo, D., & Liu, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08395 · Cited by 86 · cited 1×
“Depending on the work hypotheses made, names such as continual learning [62, 67], lifelong learning [3, 48] or incremental learning (IL) [6, 14, 25, 56] are used to describe associated works.”
BackgroundButalia, T. S. (1964) · Encyclopedia of Food and Agricultural Ethics · doi:10.2307/j.ctt1trkj1b.22 · Cited by 84
ScaIL: Classifier Weights Scaling for Class Incremental Learning
Belouadah, E., & Popescu, A. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093562 · Cited by 72 · cited 10×
“The dominant approach is based on model updating via fine-tuning to integrate new knowledge [5, 6, 10, 19, 58].”
BackgroundMethodInfluentialSelf-citationLearning More Universal Representations for Transfer-Learning
Tamaazousti, Y., Borgne, H. L., Hudelot, C., Seddik, M. E. A., & Tamaazousti, M. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913857 · Cited by 69 · cited 3×
“currence of new data and their integration in the incremental models. 5. Plasticity (P) - capacity to deal with new classes that are significantly different from the ones that were learned in the past [69]. E. Belouadah et al.: Preprint submitted to Elsevier Page 1 of 18 arXiv:2011.01844v4 [cs.LG] 15 Dec 2020 A Compreh…”
BackgroundInfluentialM2KD: Multi-model and Multi-level Knowledge Distillation for Incremental LearningPreprint
Zhou, P., Mai, L., Zhang, J., Xu, N., Wu, Z., & Davis, L. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01769 · Cited by 62 · cited 2×
“In spite of this observation, knowledge distillation is commonly used to tackle catastrophic forgetting [10, 16, 20, 23, 35, 41, 65].”
BackgroundMethodDeeSIL: Deep-Shallow Incremental Learning
Belouadah, E., & Popescu, A. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11012-3_11 · Cited by 60 · cited 10×
“Yet another choice [4] is to exploit a fixed deep representation to extract features for all incremental states and a battery of linear SVMs to implement C .”
BackgroundMethodResultInfluentialSelf-citationExemplar-Supported Generative Reproduction for Class Incremental Learning.
(2018) · British Machine Vision Conference (BMVC) · Cited by 60
Revisiting Distillation and Incremental Classifier Learning
Javed, K., & Shafait, F. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-20876-9_1 · Cited by 53 · cited 10×
“Different flavors of distillation were later proposed in [10, 19, 20, 58].”
BackgroundMethodInfluentialCalibrating CNNs for Lifelong Learning
Singh, P. P., Verma, V. K., Mazumder, P., Carin, L., & Rai, P. (2020) · neural information processing systems · Cited by 53
A Strategy for an Uncompromising Incremental Learner
Venkatesan, R., Venkateswara, H., Panchanathan, S., & Li, B. (2017) · arXiv · doi:10.48550/arxiv.1705.00744 · Cited by 37 · cited 1×
“me as random selection, with higher computational time. 7.3. Incremental learning without memory In many applications, no memory of past classes is available. For instance, in medical data processing [72], this is often due to privacy issues. We study the behavior of the main algorithms which can be deployed in the ab…”
BackgroundInfluentialInitial Classifier Weights Replay for Memoryless Class Incremental Learning
Belouadah, E., Popescu, A., & Kanellos, I. (2020) · British Machine Vision Conference (BMVC) · doi:10.5244/c.34.163 · Cited by 32
Self-citationActive Learning with Clustering
Bodó, Z., Minier, Z., & Csató, L. (2011) · Cited by 32
Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and IntransigencePreprint
Kim, D., Bae, J.-H., Jo, Y., & Choi, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00829 · Cited by 28 · cited 1×
“In spite of this observation, knowledge distillation is commonly used to tackle catastrophic forgetting [10, 16, 20, 23, 35, 41, 65].”
BackgroundOnline semi-supervised learning with learning vector quantization
Shen, Y., Zhang, Y.-M., Zhang, X.-Y., & Liu, C.-L. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.03.025 · Cited by 27
Efficient Incremental Learning Using Dynamic Correction Vector
Xiang, Y., Miao, Y., Chen, J., & Xuan, Q. (2020) · IEEE Access · doi:10.1109/access.2019.2963461 · Cited by 16 · cited 1×
“In [60], knowledge distillation is also combined with a fixed-size memory of the past.”
BackgroundIncremental learning with self-organizing maps
Gepperth, A., & Karaoguz, C. (2017) · Workshop on Self-Organizing Maps · doi:10.1109/wsom.2017.8020021 · Cited by 12
Do Better ImageNet Models Transfer Better?Preprint
Kornblith, S., Shlens, J., & Le, Q. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08974 · Cited by 12
An Online Incremental Learning Vector Quantization
Xu, Y., Furao, S., Hasegawa, O., & Zhao, J. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01307-2_112 · Cited by 10 · cited 1×
“Incremental Learning Vector Quantization (ILV Q) [78] is a prototype-based classifier that does not require prior knowledge of the number of prototypes or their initial value.”
BackgroundOnline incremental supervised growing neural gas
de Oliveira, F. D. B., Bassani, H. F., & Araujo, A. F. R. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7965966 · Cited by 8
DYNG: Dynamic Online Growing Neural Gas for stream data classification.
Beyer, O., & Cimiano, P. (2013) · The European Symposium on Artificial Neural Networks · Cited by 5
Incremental learning for visual classification using Neural Gas
Aleo, I., Arena, P., & Patané, L. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596594 · Cited by 2 · cited 2×
“∈ [0, 1] is a hyper-parameter that provides the weight of each loss term.”
BackgroundMethodSelf-supervised Monocular Depth Estimation with Multiple Resolution Images
Ryu, G., & Kim, W. (2024) · Journal of Broadcast Engineering · doi:10.5909/jbe.2024.29.4.399 · cited 1×
“The SVMs were optimized using classical grid search as described in the original paper. and its derivatives are based on the same fine-tuning backbone and are implemented in Pytorch [37].”
MethodIncremental Learning Through Deep Adaptation
Rosenfeld, A., & Tsotsos, J. K. (2017) · arXiv · doi:10.48550/arxiv.1705.04228
Learning More Universal Representations for Transfer-Learning
Tamaazousti, Y., Borgne, H. L., Hudelot, C., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.09708
Dhar, P., Singh, R. V., Peng, K.-C., et al. (2018) · arXiv · doi:10.48550/arxiv.1811.08051
Revisiting Distillation and Incremental Classifier Learning
Javed, K., & Shafait, F. (2018) · arXiv · doi:10.48550/arxiv.1807.02802
Continual Lifelong Learning with Neural Networks: A Review
Parisi, G. I., Kemker, R., Part, J. L., et al. (2018) · arXiv · doi:10.48550/arxiv.1802.07569
Research works printed but not linked: 4
- [19][19] Fritzke, B., 1994b. A growing neural gas network learns topologies, in: Tesauro, G., Touretzky, D.S., Leen, T.K. (Eds.), Advances in Neural Information Processing Systems 7, [NIPS Conference, Denver, Colorado, USA, 1994], MIT Press. pp. 625-632. -
- [70][70] Tao, X., Chang, X., Hong, X., Wei, X., Gong, Y., 2020a. Topologypreserving class-incremental learning, in: Vedaldi, A., Bischof, H., Brox, T., Frahm, J. (Eds.), Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIX, Springer. pp. 254-270. -
- [73][73] Wang, Y., Ramanan, D., Hebert, M., 2017. Growing a brain: Finetuning by increasing model capacity, in: Conference on Computer Vision and Pattern Recognition. -
- [80][80] Yu, L., Twardowski, B., Liu, X., Herranz, L., Wang, K., Cheng, Y., Jui, S., van de Weijer, J., 2020. Semantic drift compensation for classincremental learning, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, IEEE. pp. 6980-6989. -
Cited by
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification
Masana, M., Liu, X., Twardowski, B., Menta, M., Bagdanov, A. D., & van de Weijer, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3213473 · Cited by 740
“[63] perform a comparison of several classIL methods.”
MethodThree types of incremental learning
van de Ven, G. M., Tuytelaars, T., & Tolias, A. S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00568-3 · Cited by 665
“An important challenge in this scenario is learning to discriminate between classes that are not observed together, which has turned out to be very challenging for deep neural networks, especially when storing examples of previously seen classes is not allowed 29,30 .”
Class-Incremental Learning: A Survey
Zhou, D.-W., Wang, Q.-W., Qi, Z.-H., Ye, H.-J., Zhan, D.-C., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429383 · Cited by 376
“Other surveys either focus on the specific field [20], [21] or lack the performance evolution among state-of-the-arts [8], [22], [23].”
BackgroundDing, A., Qin, Y., Wang, B., Guo, L., Jia, L., & Cheng, X. (2024) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2024.111175 · Cited by 254
SS-IL: Separated Softmax for Incremental Learning
Ahn, H., Kwak, J., Lim, S., Bang, H., Kim, H., & Moon, T. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00088 · Cited by 177
“Also, results in [4] and [6] show that CIL models are sensitive to incremental conditions such as the number of incremental tasks (T ) and exemplar-memory size (|M|).”
BackgroundMethodResult
Metadata
- Authors
- Belouadah, E., Popescu, A., & Kanellos, I.
- Affiliations
- Commissariat à l'Énergie Atomique et aux Énergies Alternatives (FR)
- IMT Atlantique (FR)
- Corresponding author
- Popescu, Adrian
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- Volume
- 135
- Pages
- 38-54
- ISSN
- 0893-6080 (ISSN-L)
- arXiv
- 2011.01844
- OpenAlex
- W3096398238
- Cited by
- 235
- References
- 86
- Citations by year
- 2026: 28 · 2025: 40 · 2024: 61 · 2023: 58 · 2022: 32 · 2021: 16
- FWCI
- 0.91
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Neural Networks (acceptedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion, cc-by-nc)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- European Commission (951911, HORIZON2020)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Keywords
- class-incremental learning, catastrophic forgetting, stability-plasticity balance, herding behavior, knowledge distillation, evaluation framework, reproducibility, open-source repositories, visual task
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Memory, Psychomotor Performance, Visual Perception, Reproducibility of Results, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.