Learning Without Forgetting
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LwF was designed to work without a memory of past classes.
“LwF was designed to work without a memory of past classes.”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [36]
Even without memory, the reuse of the weights of past classes from their initial states in FT init is better than the sole use of knowledge distillation in LwF [36].
“Even without memory, the reuse of the weights of past classes from their initial states in FT init is better than the sole use of knowledge distillation in LwF [36].”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [36]
Data-based approaches use data from the new task to approximate the performance of the previous tasks, and this works best if the data distribution mismatch between tasks is limited.
“Data-based approaches [1,17,28,34] use data from the new task to approximate the performance of the previous tasks. This works best if the data distribution mismatch between tasks is limited.”
From Memory Aware Synapses: Learning What (not) to Forget (2018) · cites this paper as [17]
LwF suffers from a buildup of errors when facing a long sequence while EBLL reduces slightly this effect.
“LwF [17] suffers from a buildup of errors when facing a long sequence while EBLL [28] reduces slightly this effect.”
From Memory Aware Synapses: Learning What (not) to Forget (2018) · cites this paper as [17]
The standard LLL setup consists of a sequence of disjoint tasks learned one after the other, where tasks may correspond to different datasets or different splits of a dataset without overlap in category labels.
“It focuses on image classification and consists of a sequence of disjoint tasks , which are learned one after the other. Tasks may correspond to different datasets, or different splits of a dataset, without overlap in category labels.”
From Memory Aware Synapses: Learning What (not) to Forget (2018) · cites this paper as [17]
LwF, while training a new task, needs to load the last task model and all previous heads to get target predictions, but has the same offline storage requirements between tasks as Finetuning.
“LwF [17], while training a new task, needs to load the last task model and all the previous heads to get the target predictions. However, LwF has the same offline storage requirements between tasks as Finetuning.”
From Memory Aware Synapses: Learning What (not) to Forget (2018) · cites this paper as [17]
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139 · see citing works
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · see citing works
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451 · see citing works
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · see citing works
Caltech-UCSD Birds-200-2011 Dataset
Wah, C. (2024) · CaltechAUTHORS (California Institute of Technology) · doi:10.57702/i9oeewf6 · Cited by 3,875 · see citing works
MatConvNet: Convolutional Neural Networks for MATLAB
Vedaldi, A., & Lenc, K. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2807412 · Cited by 2,444 · see citing works
Quattoni, A., & Torralba, A. B. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206537 · Cited by 1,533 · see citing works
Simultaneous Deep Transfer Across Domains and Tasks
Tzeng, E., Hoffman, J., Darrell, T., & Saenko, K. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.463 · Cited by 1,203 · see citing works
International Conference on Machine Learning (ICML-99).
Scheffer, T. (1999) · Künstliche Intell. · Cited by 1,011 · see citing works
Thrun, S. (1998) · Springer eBooks · doi:10.1007/978-1-4615-5529-2_8 · Cited by 434 · see citing works
Analyzing the Performance of Multilayer Neural Networks for Object Recognition
Agrawal, P., Girshick, R., & Malik, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_22 · Cited by 427 · see citing works
Factors of Transferability for a Generic ConvNet Representation
Azizpour, H., Razavian, A. S., Sullivan, J., Maki, A., & Carlsson, S. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2500224 · Cited by 301 · see citing works
Mitchell, T. M., Cohen, W. W., Junior, E. R. H., Talukdar, P. P., Betteridge, J., Carlson, A., Dalvi Mishra, B., Gardner, M., Kisiel, B., Krishnamurthy, J., et al. (2015) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v29i1.9498 · Cited by 105 · see citing works
Chapelle, O., Shivaswamy, P. K., Vadrevu, S., Weinberger, K. Q., Zhang, Y., & Tseng, B. L. (2010) · Machine Learning · doi:10.1007/s10994-010-5231-6 · Cited by 73 · see citing works
Knowledge Transfer in Deep Block-Modular Neural Networks
Terekhov, A. V., Montone, G., & O'Regan, J. K. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22979-9_27 · Cited by 70 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · see citing works
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117 · see citing works
Memory Aware Synapses: Learning What (not) to Forget
Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., & Tuytelaars, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_9 · Cited by 1,590 · see citing works
Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
Chaudhry, A., Dokania, P. K., Ajanthan, T., & Torr, P. H. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_33 · Cited by 1,091 · see citing works
Expert Gate: Lifelong Learning with a Network of Experts
Aljundi, R., Chakravarty, P., & Tuytelaars, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.753 · Cited by 619 · see citing works
Incremental Learning of Object Detectors without Catastrophic Forgetting
Shmelkov, K., Schmid, C., & Alahari, K. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.368 · Cited by 559 · see citing works
SpotTune: Transfer Learning Through Adaptive Fine-Tuning
Guo, Y., Shi, H., Kumar, A., Grauman, K., Rosing, T., & Feris, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00494 · Cited by 506 · see citing works
IL2M: Class Incremental Learning With Dual Memory
Belouadah, E., & Popescu, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00067 · Cited by 302 · see citing works
Towards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224 · see citing works
Encoder Based Lifelong Learning
Rannen, A., Aljundi, R., Blaschko, M. B., & Tuytelaars, T. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.148 · Cited by 324 · see citing works
Overcoming catastrophic forgetting with hard attention to the task
Serrà, J., Surís, D., Miron, M., & Karatzoglou, A. (2018) · arXiv · doi:10.48550/arxiv.1801.01423 · Cited by 213 · see citing works
Growing a Brain: Fine-Tuning by Increasing Model Capacity
Wang, Y.-X., Ramanan, D., & Hebert, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.323 · Cited by 154 · see citing works
“Importance of reconciling the learning pace of new and old units: The previous work showed that network expansion did not introduce additional benefits [22].”
BackgroundMethodResultInfluentialSamant, R. M., Bachute, M. R., Gite, S., & Kotecha, K. (2022) · IEEE Access · doi:10.1109/access.2022.3149798 · Cited by 123 · see citing works
An Appraisal of Incremental Learning Methods
Luo, Y., Yin, L., Bai, W., & Mao, K. (2020) · Entropy · doi:10.3390/e22111190 · Cited by 120 · see citing works
Continual Relation Learning via Episodic Memory Activation and Reconsolidation
Han, X., Dai, Y., Gao, T., Lin, Y., Liu, Z., Li, P., Sun, M., & Zhou, J. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.573 · Cited by 104 · see citing works
Toward an artificial intelligence physicist for unsupervised learning
Wu, T., & Tegmark, M. (2019) · Physical Review E · doi:10.1103/physreve.100.033311 · Cited by 99 · see citing works
Less-Forgetful Learning for Domain Expansion in Deep Neural Networks
Jung, H., Ju, J., Jung, M., & Kim, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11769 · Cited by 75 · see citing works
A Strategy for an Uncompromising Incremental Learner
Venkatesan, R., Venkateswara, H., Panchanathan, S., & Li, B. (2017) · arXiv · doi:10.48550/arxiv.1705.00744 · Cited by 37 · see citing works
Open-world continual learning: Unifying novelty detection and continual learning
Kim, G., Xiao, C., Konishi, T., Ke, Z., & Liu, B. (2025) · Artificial Intelligence · doi:10.1016/j.artint.2024.104237 · Cited by 19 · see citing works
Self-Net: Lifelong Learning via Continual Self-Modeling
Mandivarapu, J. K., Camp, B., & Estrada, R. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00019 · Cited by 14 · see citing works
End-to-End Test-Time Training for Long Context
Tandon, A., Dalal, K., Li, X., Koceja, D., Rød, M., Buchanan, S., Wang, X., Leskovec, J., Koyejo, S., Hashimoto, T., et al. (2025) · arXiv
“Under this setting, most algorithms focus on not forgetting the past when learning from the present [75, 64, 57, 30].”
When Continual Learning Moves to Memory: A Study of Experience Reuse in LLM Agents
Hu, Q., Long, Q., & Wang, W. (2026) · arXiv · doi:10.48550/arxiv.2604.27003
“…and plasticity under non-stationary data through regularization, replay, and architectural isolation or expansion (Kirkpatrick et al., 2017; Li and Hoiem, 2017; Lopez-Paz and Ranzato, 2017; Mallya and Lazebnik, 2018), with recent surveys extending this perspective to foundation models (Wang et…”
Huang, H., Ye, X., Mustafa, M. B., Dong, Q., Li, Y., Asemi, A., & Asemi, A. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-02086-2
A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
Belouadah, E., Popescu, A., & Kanellos, I. (2020) · arXiv · doi:10.48550/arxiv.2011.01844 · Cited by 235 · see citing works
“Learning without Forgetting ( ) [27] is a pioneering work that does not require a memory of past classes.”
BackgroundMethodInfluential
- Authors
- Li, Z., & Hoiem, D.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Li, Zhizhong
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2016
- Pages
- 614-629
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W3021931813
- Cited by
- 659
- References
- 21
- Citations by year
- 2026: 54 · 2025: 34 · 2024: 71 · 2023: 93 · 2022: 84 · 2021: 103 · 2020: 64 · 2019: 68 · 2018: 60 · 2017: 23 · 2016: 5
- FWCI
- 28.02
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Lecture notes in computer science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Keywords
- learning without forgetting, catastrophic forgetting, convolutional neural networks, continual learning, fine-tuning, multi-task learning, feature extraction, vision system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
