The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Distilling Causal Effect of Data in Class-Incremental Learning
Abstract
We propose a causal framework to explain the catastrophic forgetting in Class-Incremental Learning (CIL) and then derive a novel distillation method that is orthogonal to the existing anti-forgetting techniques, such as data replay and feature/label distillation. We first 1) place CIL into the framework, 2) answer why the forgetting happens: the causal effect of the old data is lost in new training, and then 3) explain how the existing techniques mitigate it: they bring the causal effect back. Based on the causal framework, we propose to distill the Colliding Effect between the old and the new data, which is fundamentally equivalent to the causal effect of data replay, but without any cost of replay storage. Thanks to the causal effect analysis, we can further capture the Incremental Momentum Effect of the data stream, removing which can help to retain the old effect overwhelmed by the new data effect, and thus alleviate the forgetting of the old class in testing. Extensive experiments on three CIL benchmarks: CIFAR-100, ImageNet-Sub&Full, show that the proposed causal effect distillation can improve various state-of-the-art CIL methods by a large margin (0.72%–9.06%).1
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Learning a Unified Classifier Incrementally via Rebalancing
Hou, S., Pan, X., Loy, C. C., Wang, Z., & Lin, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00092 · Cited by 1,195
Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
Chaudhry, A., Dokania, P. K., Ajanthan, T., & Torr, P. H. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_33 · Cited by 1,091
Catastrophic forgetting, rehearsal and pseudorehearsal
ROBINS, A. (1995) · Connection Science · doi:10.1080/09540099550039318 · Cited by 833
Interpretation and identification of causal mediation
Pearl, J. (2014) · Psychological Methods · doi:10.1037/a0036434 · Cited by 554 · cited 1×
“Based on the graph, we can formulate the causal effect between variables by causal intervention, which is denoted as the do(·) operation [30, 31].”
MethodFew-Shot Class-Incremental Learning
Tao, X., Hong, X., Chang, X., Dong, S., Wei, X., & Gong, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01220 · Cited by 499
Cited by
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113
“In this regard, replay of the old training samples is able to impose an end-to-end effect, while feature and label distillations of the new training samples are limited to the output side with a biased distribution [171].”
BackgroundMethodInfluentialCounterfactual VQA: A Cause-Effect Look at Language Bias
Niu, Y., Tang, K., Zhang, H., Lu, Z., Hua, X.-S., & Wen, J.-R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01251 · Cited by 418
Self-citationFew-Shot Incremental Learning with Continually Evolved Classifiers
Zhang, C., Song, N., Lin, G., Zheng, Y., Pan, P., & Xu, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01227 · Cited by 382
Class-Incremental Learning: A Survey
Zhou, D.-W., Wang, Q.-W., Qi, Z.-H., Ye, H.-J., Zhan, D.-C., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429383 · Cited by 376
Prompt-aligned Gradient for Prompt Tuning
Zhu, B., Niu, Y., Han, Y., Wu, Y., & Zhang, H. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01435 · Cited by 298
Self-citation
Metadata
- Authors
- Hu, X., Tang, K., Miao, C., Hua, X.-S., & Zhang, H.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 3956-3965
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3175294706
- Cited by
- 167
- References
- 15
- FWCI
- 14.93
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Distilling Causal Effect of Data in Class-Incremental Learning. https://app.cochat.ai/papers/10.1109/cvpr46437.2021.00395
- The paper
- Hu, X., Tang, K., Miao, C., Hua, X.-S., & Zhang, H. (2021). Distilling Causal Effect of Data in Class-Incremental Learning. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr46437.2021.00395
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.