CoChat
Sign Up
Cited by 309Domain Adaptation and Few-Shot LearningAdvanced Neural Network ApplicationsData Stream Mining Techniques

Mnemonics Training: Multi-Class Incremental Learning Without Forgetting

Open access
Liu, Yaoyao, Su, Yuting, Liu, An-An, Schiele, Bernt, Sun, Qianru·2020·2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr42600.2020.01226
Open this paper in CoChat — free accountOpen PDF

Abstract

Multi-Class Incremental Learning (MCIL) aims to learn new concepts by incrementally updating a model trained on previous concepts. However, there is an inherent trade-off to effectively learning new concepts without catastrophic forgetting of previous ones. To alleviate this issue, it has been proposed to keep around a few examples of the previous concepts but the effectiveness of this approach heavily depends on the representativeness of these examples. This paper proposes a novel and automatic framework we call mnemonics, where we parameterize exemplars and make them optimizable in an end-to-end manner. We train the framework through bilevel optimizations, i.e., model-level and exemplar-level. We conduct extensive experiments on three MCIL benchmarks, CIFAR-100, ImageNet-Subset and ImageNet, and show that using mnemonics exemplars can surpass the state-of-the-art by a large margin. Interestingly and quite intriguingly, the mnemonics exemplars tend to be on the boundaries between different classes.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr42600.2020.01226

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 32

  1. [1][1] Mart´ ın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Gregory S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian J. Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal J´ ozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Man´ e, Rajat Monga, Sherry Moore, Derek Gordon Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul A. Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda B. Vi´ egas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Largescale machine learning on heterogeneous distributed systems. arXiv , 1603.04467, 2016. 5 - · arXiv
  2. [2][2] Francisco M. Castro, Manuel J. Mar´ ın-Jim´ enez, Nicol´ as Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremental learning. In ECCV , pages 241-257, 2018. 1, 2, 3, 4, 6, 7 -
  3. [3][3] Gert Cauwenberghs and Tomaso Poggio. Incremental and decremental support vector machine learning. In NIPS , pages 409-415, 2000. 2 -
  4. [4][4] Arslan Chaudhry, Marc'Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with agem. In ICLR , 2019. 2 -
  5. [5][5] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Modelagnostic meta-learning for fast adaptation of deep networks. In ICML , pages 1126-1135, 2017. 2, 5 -
  6. [6][6] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS , pages 2672-2680, 2014. 2 -
  7. [8][8] Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. arXiv , 1503.02531, 2015. 2, 3 - · arXiv
  8. [9][9] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In CVPR , pages 831-839, 2019. 1, 2, 3, 4, 6, 7, 8, 12, 13 -
  9. [10][10] Wenpeng Hu, Zhou Lin, Bing Liu, Chongyang Tao, Zhengwei Tao, Jinwen Ma, Dongyan Zhao, and Rui Yan. Overcoming catastrophic forgetting for continual learning via model adaptation. In ICLR , 2019. 2 -
  10. [11][11] Nitin Kamra, Umang Gupta, and Yan Liu. Deep generative dual memory network for continual learning. arXiv , 1710.10368, 2017. 2 - · arXiv
  11. [14][14] Ilja Kuzborskij, Francesco Orabona, and Barbara Caputo. From N to N+1: multiclass transfer incremental learning. In CVPR , pages 3358-3365, 2013. 2 -
  12. [15][15] Xinzhe Li, Qianru Sun, Yaoyao Liu, Qin Zhou, Shibao Zheng, Tat-Seng Chua, and Bernt Schiele. Learning to self-train for semi-supervised few-shot classification. In NeurIPS , pages 10276-10286, 2019. 2 -
  13. [16][16] Yingying Li, Xin Chen, and Na Li. Online optimal control with linear dynamics and predictions: Algorithms and regret analysis. In NeurIPS , pages 14858-14870, 2019. 1 -
  14. [17][17] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence , 40(12):2935-2947, 2018. 1, 2, 3, 5, 6, 7, 8, 12, 13 -
  15. [18][18] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research , 9(Nov):2579-2605, 2008. 1, 9 -
  16. [19][19] Matthew MacKay, Paul Vicol, Jon Lorraine, David Duvenaud, and Roger Grosse. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. In ICLR , 2019. 2, 3 -
  17. [22][22] Thomas Mensink, Jakob J. Verbeek, Florent Perronnin, and Gabriela Csurka. Distance-based image classification: Generalizing to new classes at near-zero cost. IEEE Transactions on Pattern Analysis and Machine Intelligence , 35(11):26242637, 2013. 2 -
  18. [24][24] R. Ratcliff. Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psychological Review , 97:285-308, 1990. 1 -
  19. [26][26] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In ICLR , 2019. 2 -
  20. [27][27] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision , 115(3):211-252, 2015. 6 -
  21. [28][28] Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. In NIPS , pages 2990-2999, 2017. 1, 2, 12 -
  22. [29][29] Ankur Sinha, Pekka Malo, and Kalyanmoy Deb. A review on bilevel optimization: From classical to evolutionary approaches and applications. IEEE Transactions on Evolutionary Computation , 22(2):276-295, 2018. 2 -
  23. [30][30] Benoit Steiner, Zachary DeVito, Soumith Chintala, Sam Gross, Adam Paszke, Francisco Massa, Adam Lerer, Gregory Chanan, Zeming Lin, Edward Yang, et al. PyTorch: An imperative style, high-performance deep learning library. In NeurIPS , pages 8024-8035, 2019. 5 -
  24. [31][31] Qianru Sun, Yaoyao Liu, Zhaozheng Chen, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning through hard tasks. arXiv , 1910.03648, 2019. 2 - · arXiv
  25. [32][32] Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning for few-shot learning. In CVPR , pages 403-412, 2019. 2, 5, 6 -
  26. [33][33] Ragav Venkatesan, Hemanth Venkateswara, Sethuraman Panchanathan, and Baoxin Li. A strategy for an uncompromising incremental learner. arXiv , 1705.00744, 2017. 2 - · arXiv
  27. [34][34] Heinrich Von Stackelberg and Stackelberg Heinrich Von. The theory of the market economy . Oxford University Press, 1952. 2 -
  28. [35][35] Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A. Efros. Dataset distillation. arXiv , 1811.10959, 2018. 2 - · arXiv
  29. [36][36] Max Welling. Herding dynamical weights to learn. In ICML , pages 1121-1128, 2009. 2 -
  30. [37][37] Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In CVPR , pages 374-382, 2019. 1, 2, 3, 4, 6, 7, 8, 12, 13 -
  31. [38][38] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classification with differentiable earth mover's distance and structured classifiers. arXiv , 2003.06777, 2020. 2 - · arXiv
  32. [39][39] Chi Zhang, Guosheng Lin, Fayao Liu, Rui Yao, and Chunhua Shen. Canet: Class-agnostic segmentation networks with iterative refinement and attentive few-shot learning. In CVPR , pages 5217-5226, 2019. 2

Cited by

Sorted by citations

Authors
Liu, Y., Su, Y., Liu, A.-A., Schiele, B., & Sun, Q.
Affiliations
  • Tianjin University (CN)
  • Max Planck Institute for Informatics (DE)
  • Singapore Management University (SG)
ORCID
Schiele, Bernt
Venue
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2020-06
Pages
12242-12251
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr42600.2020.01226
OpenAlex
W3007041883
Cited by
309
References
5
Citations by year
2026: 23 · 2025: 52 · 2024: 63 · 2023: 73 · 2022: 54 · 2021: 36 · 2020: 8
FWCI
16.33
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
Keywords
catastrophic forgetting, mnemonics, exemplar-based learning, bi-level optimization, end-to-end optimization, class boundaries, continual learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review