CoChat
Sign Up
Cited by 340Domain Adaptation and Few-Shot LearningAdvanced Neural Network ApplicationsData Stream Mining Techniques

Semantic Drift Compensation for Class-Incremental Learning

Open access
Yu, Lu, Twardowski, Bartlomiej, Liu, Xialei, Herranz, Luis, Wang, Kai, Cheng, Yongmei, Jui, Shangling, van de Weijer, Joost·2020·2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr42600.2020.00701
Open this paper in CoChat — free accountOpen PDF

Abstract

Class-incremental learning of deep networks sequentially increases the number of classes to be classified. During training, the network has only access to data of one task at a time, where each task contains several classes. In this setting, networks suffer from catastrophic forgetting which refers to the drastic drop in performance on previous tasks. The vast majority of methods have studied this scenario for classification networks, where for each new task the classification layer of the network must be augmented with additional weights to make room for the newly added classes. Embedding networks have the advantage that new classes can be naturally included into the network without adding new weights. Therefore, we study incremental learning for embedding networks. In addition, we propose a new method to estimate the drift, called semantic drift, of features and compensate for it without the need of any exemplars. We approximate the drift of previous tasks based on the drift that is experienced by current task data. We perform experiments on fine-grained datasets, CIFAR100 and ImageNet-Subset. We demonstrate that embedding networks suffer significantly less from catastrophic forgetting. We outperform existing methods which do not require exemplars and obtain competitive results compared to methods which store exemplars. Furthermore, we show that our proposed SDC when combined with existing methods to prevent forgetting consistently improves results.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr42600.2020.00701

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • FSCIL has two main frameworks: one uses a feature extractor with a softmax classifier, and the other involves a feature embedding network and the nearest class mean classifier.

    “FSCIL has two main frameworks, as shown in Fig. 6. The first uses a feature extractor with a softmax classifier, while the second involves a feature embedding network and the nearest class mean classifier [22, 42, 43].”

    From Few-Shot Class-Incremental Learning for Classification and Object Detection: A Survey (2025) · cites this paper as [42]

    Section 3.2 (page not specified) compares softmax classification with an embedding network that uses nearest-class-mean classification, supporting the two architectures described. It discusses class-incremental learning, however, and does not establish that these are the two main frameworks for few-shot class-incremental learning (FSCIL). (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 36

  1. R. Aljundi, P. Chakravarty, and T. Tuytelaars. Expert gate: Lifelong learning with a network of experts. In CVPR, pages 3366-3375, 2017. 2, 4, 6, 8
  2. A. Chaudhry, P. K. Dokania, T. Ajanthan, and P. H. Torr. Rie- mannian walk for incremental learning: Understanding for- getting and intransigence. In ECCV, pages 532-547, 2018. 2, 4, 6, 8
  3. W. Chen, X. Chen, J. Zhang, and K. Huang. Beyond triplet loss: a deep quadruplet network for person re-identification. In CVPR, pages 403-412, 2017. 3
  4. S. Chopra, R. Hadsell, and Y. LeCun. Learning a similarity metric discriminatively, with application to face verification. In CVPR, volume 1, pages 539-546. IEEE, 2005. 2, 3
  5. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei- Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255. Ieee, 2009. 6
  6. P. Dhar, R. V. Singh, K.-C. Peng, Z. Wu, and R. Chellappa. Learning without memorizing. In CVPR, pages 5138-5146, 2019. 2, 8
  7. L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In CVPR workshop, pages 178-178. IEEE, 2004. 6
  8. E. Hoffer and N. Ailon. Deep metric learning using triplet network. In International Workshop on Similarity-Based Pattern Recognition, pages 84-92. Springer, 2015. 3, 4, 7
  9. S. Horiguchi, D. Ikami, and K. Aizawa. Significance of softmax-based features in comparison to distance metric learning-based features. IEEE Trans. on PAMI, 2019. 4
  10. S. Hou, X. Pan, C. C. Loy, Z. Wang, and D. Lin. Learning a unified classifier incrementally via rebalancing. In CVPR, pages 831-839, 2019. 2, 4, 7, 8, 11
  11. J. Krause, M. Stark, J. Deng, and L. Fei-Fei. 3d object rep- resentations for fine-grained categorization. In IEEE Inter- national Conference on Computer Vision Workshops, pages 554-561, 2013. 11
  12. Z. Li and D. Hoiem. Learning without forgetting. IEEE Trans. on PAMI, 40(12):2935-2947, 2018. 1, 2, 4, 6
  13. X. Liu, M. Masana, L. Herranz, J. Van de Weijer, A. M. Lopez, and A. D. Bagdanov. Rotate your networks: Bet- ter weight consolidation and less catastrophic forgetting. In Proc. ICPR, 2018. 2, 4, 6, 11
  14. D. Lopez-Paz and M. Ranzato. Gradient episodic memory for continual learning. In NIPS, pages 6467-6476, 2017. 2
  15. A. Mallya, D. Davis, and S. Lazebnik. Piggyback: Adapt- ing a single network to multiple tasks by learning to mask weights. In ECCV, pages 67-82, 2018. 2
  16. A. Mallya and S. Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In CVPR, pages 7765-7773, 2018. 2
  17. M. Masana, I. Ruiz, J. Serrat, J. van de Weijer, and A. M. Lopez. Metric learning for novelty and anomaly detection. In BMVC, 2018. 3
  18. M. Masana, T. Tuytelaars, and J. van de Weijer. Ternary fea- ture masks: continual learning without any forgetting. arXiv preprint arXiv:2001.08714, 2020. 2 · arXiv
  19. T. Mensink, J. Verbeek, F. Perronnin, and G. Csurka. Distance-based image classification: Generalizing to new classes at near-zero cost. IEEE Trans. on PAMI, 35(11):2624-2637, 2013.
  20. J. Rajasegaran, M. Hayat, S. H. Khan, F. S. Khan, and L. Shao. Random path selection for continual learning. In Advances in Neural Information Processing Systems, pages 12648-12658, 2019. 3
  21. O. Rippel, M. Paluri, P. Dollar, and L. Bourdev. Metric learn- ing with adaptive density discrimination. In ICLR, 2016. 3
  22. A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Had- sell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016. 2 · arXiv
  23. T. Scott, K. Ridgeway, and M. C. Mozer. Adapted deep em- beddings: A synthesis of methods for k-shot inductive trans- fer learning. In NIPS, pages 76-85, 2018. 3
  24. J. Serra, D. Suris, M. Miron, and A. Karatzoglou. Overcom- ing catastrophic forgetting with hard attention to the task. In ICML, pages 4555-4564, 2018. 2
  25. H. Shin, J. K. Lee, J. Kim, and J. Kim. Continual learn- ing with deep generative replay. In NIPS, pages 2990-2999, 2017. 2, 4
  26. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 11 · arXiv
  27. J. Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. In NIPS, pages 4077-4087, 2017. 3
  28. G. M. van de Ven and A. S. Tolias. Three scenarios for con- tinual learning. arXiv preprint arXiv:1904.07734, 2019. 2 · arXiv
  29. J. Wang, Y. Song, T. Leung, C. Rosenberg, J. Wang, J. Philbin, B. Chen, and Y. Wu. Learning fine-grained image similarity with deep ranking. In CVPR, pages 1386-1393, 2014. 3
  30. J. Wang, F. Zhou, S. Wen, X. Liu, and Y. Lin. Deep met- ric learning with angular loss. In ICCV, pages 2612-2620. IEEE, 2017. 2, 3, 7, 11
  31. X. Wang, X. Han, W. Huang, D. Dong, and M. R. Scott. Multi-similarity loss with general pair weighting for deep metric learning. In CVPR, pages 5022-5030, 2019. 3, 7, 11
  32. C. Wu, L. Herranz, X. Liu, Y. Wang, J. van de Weijer, and B. Raducanu. Memory replay gans: learning to generate im- ages from new categories without forgetting. In NIPS, 2018. 2, 4
  33. Y. Wu, Y. Chen, L. Wang, Y. Ye, Z. Liu, Y. Guo, and Y. Fu. Large scale incremental learning. In CVPR, 2019. 2, 4
  34. H.-M. Yang, X.-Y. Zhang, F. Yin, and C.-L. Liu. Robust clas- sification with convolutional prototype learning. In CVPR, pages 3474-3482, 2018. 3
  35. L. Yu, V. O. Yazici, X. Liu, J. Van de Weijer, Y. Cheng, and A. Ramisa. Learning metrics from teachers: Compact net- works for image embedding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2907-2916, 2019. 5
  36. F. Zenke, B. Poole, and S. Ganguli. Continual learning through synaptic intelligence. In ICML, pages 3987-3995. JMLR. org, 2017.

Cited by

Sorted by citations

Authors
Yu, L., Twardowski, B., Liu, X., Herranz, L., Wang, K., Cheng, Y., Jui, S., & van de Weijer, J.
Affiliations
  • Universitat Autònoma de Barcelona (ES)
  • Northwestern Polytechnical University (CN)
  • Huawei Technologies (China) (CN)
ORCID
Venue
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2020-06
Pages
6980-6989
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr42600.2020.00701
OpenAlex
W3035501943
Cited by
340
References
11
Citations by year
2026: 30 · 2025: 61 · 2024: 78 · 2023: 63 · 2022: 53 · 2021: 47 · 2020: 8
FWCI
16.33
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
Generalitat de Catalunya
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
Keywords
class-incremental learning, catastrophic forgetting, semantic drift, drift correction, embedding network, fine-grained classification, continual learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review