Lifelong Learning with Dynamically Expandable Networks
Open accessThe DOI matches its OpenAlex record; 4 of the 4 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Learning to detect unseen object classes by between-class attribute transfer
Lampert, C. H., Nickisch, H., & Harmeling, S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206594 · Cited by 1,849 · cited 2×
“This dataset consists of 30, 475 images of 50 animals (Lampert et al., 2009).”
Cited by
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Learning a Unified Classifier Incrementally via Rebalancing
Hou, S., Pan, X., Loy, C. C., Wang, Z., & Lin, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00092 · Cited by 1,195
“It is noteworthy that the previous works have also explored other ideas for incremental learning, such as adopting dynamic network structures [31, 36] or using a generative model to produce samples for old classes [35, 20].”
BackgroundRiemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
Chaudhry, A., Dokania, P. K., Ajanthan, T., & Torr, P. H. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_33 · Cited by 1,091
“One way to address catastrophic forgetting is by dynamically expanding the network for each new task [12,20,22,24].”
BackgroundA Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113
“To alleviate this dilemma, the network architecture can be dynamically expanded if its capacity is not sufficient to learn a new task well, such as DEN [481], CPG [174] and DGMa/DGMw [317].”
BackgroundFew-Shot Class-Incremental Learning
Tao, X., Hong, X., Chang, X., Dong, S., Wei, X., & Gong, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01220 · Cited by 499
Class-Incremental Learning: A Survey
Zhou, D.-W., Wang, Q.-W., Qi, Z.-H., Ye, H.-J., Zhan, D.-C., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429383 · Cited by 376
Aljundi, R., Kelchtermans, K., & Tuytelaars, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01151 · Cited by 324
Online Deep Learning: Learning Deep Neural Networks on the Fly
Sahoo, D., Pham, Q., Lu, J., & Hoi, S. C. H. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/369 · Cited by 323
On Tiny Episodic Memories in Continual Learning
Chaudhry, A., Rohrbach, M., Elhoseiny, M., Ajanthan, T., Dokania, P. K., Torr, P. H. S., & Ranzato, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10486 · Cited by 298
REMIND Your Neural Network to Prevent Catastrophic Forgetting
Hayes, T. L., Kafle, K., Shrestha, R., Acharya, M., & Kanan, C. (2019) · arXiv · doi:10.48550/arxiv.1910.02509 · Cited by 268
Overcoming catastrophic forgetting with hard attention to the task
Serrà, J., Surís, D., Miron, M., & Karatzoglou, A. (2018) · arXiv · doi:10.48550/arxiv.1801.01423 · Cited by 213
“Dynamically expandable networks (DEN; Yoon et al., 2018) also assign network capacity depending on the task at hand.”
BackgroundFew-Shot Class-Incremental Learning via Relation Knowledge Distillation
Dong, S., Hong, X., Tao, X., Chang, X., Wei, X., & Gong, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16213 · Cited by 161
“…from the following three perspectives: (1) architectural approaches: improving the network architecture, such as dynamic expansion, network pruning, and parameter masking methods to reduce forgetting (Mallya and Lazebnik 2018; Mallya, Davis, and Lazebnik 2018; Serr`a et al. 2018; Yoon et al. 2017).”
BackgroundWortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., & Farhadi, A. (2020) · arXiv · doi:10.48550/arxiv.2006.14769 · Cited by 144
Gradient Projection Memory for Continual Learning
Saha, G., Garg, I., & Roy, K. (2021) · arXiv · doi:10.48550/arxiv.2103.09762 · Cited by 129
Overcoming Catastrophic Forgetting in Graph Neural Networks
Liu, H., Yang, Y., & Wang, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17049 · Cited by 127
An Appraisal of Incremental Learning Methods
Luo, Y., Yin, L., Bai, W., & Mao, K. (2020) · Entropy · doi:10.3390/e22111190 · Cited by 120
“Overlapping knowledge between stages was learned, and the network structure can be dynamically determined [29].”
BackgroundReplay in Deep Learning: Current Approaches and Missing Biological Elements
Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski, T. J., & Kanan, C. (2021) · Neural Computation · doi:10.1162/neco_a_01433 · Cited by 106
“, 2019; Lopez-Paz and Ranzato, 2017), 2) network expansion techniques for adding new parameters to a network to learn new information (Rusu et al., 2016; Yoon et al., 2018; Ostapenko et al., 2019; Hou et al., 2018), and 3) replay mechanisms for storing a representation of previous data to mix with new data when updati…”
Adaptive Deep Models for Incremental Learning: Considering Capacity Scalability and Sustainability
Yang, Y., Zhou, D.-W., Zhan, D.-C., Xiong, H., & Jiang, Y. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330865 · Cited by 83
Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2023) · arXiv · doi:10.48550/arxiv.2312.10549 · Cited by 54
“One method in this regard is the Dynamically Expandable Network (DEN) introduced by Yoon et al .”
MethodCatastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2024) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2024.3966 · Cited by 54
Towards Continual Knowledge Learning of Language Models
Jang, J., Ye, S., Yang, S., Shin, J., Han, J., Kim, G., Choi, S. J., & Seo, M. (2021) · arXiv · doi:10.48550/arxiv.2110.03215 · Cited by 49
“(3) Lastly, prior parameter-expansion methods (Rusu et al., 2016; Yoon et al., 2018) focus on learning a stream of different tasks via strong supervision, while in CKL, the focus is constantly updating world knowledge from a stream of corpora via self-supervision.”
BackgroundLearning on the Job: Online Lifelong and Continual Learning
Liu, B. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i09.7079 · Cited by 47
TRGP: Trust Region Gradient Projection for Continual Learning
Lin, S., Yang, L., Fan, D., & Zhang, J. (2022) · arXiv · doi:10.48550/arxiv.2202.02931 · Cited by 43
“We further compare with Progressive Neural Network (PNN) (Rusu et al., 2016), Learning Without Forgetting (LWF) (Li & Hoiem, 2017), Dynamic-Expansion Net (DEN) (Yoon et al., 2017), and APD (Yoon et al., 2020), by using CIFAR-100 Sup dataset.”
BackgroundMethodMultimodal Parameter-Efficient Few-Shot Class Incremental Learning
D’Alessandro, M., Alonso, A., Calabrés, E., & Galar, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw60793.2023.00364 · Cited by 40
Using machine learning for handover optimization in vehicular fog computing
Memon, S., & Maheswaran, M. (2019) · Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing · doi:10.1145/3297280.3297300 · Cited by 47
Learning to Continuously Optimize Wireless Resource in Episodically Dynamic Environment
Sun, H., Pu, W., Zhu, M., Fu, X., Chang, T.-H., & Hong, M. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9413503 · Cited by 21
Self-Net: Lifelong Learning via Continual Self-Modeling
Mandivarapu, J. K., Camp, B., & Estrada, R. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00019 · Cited by 14
Verse: Virtual-gradient aware streaming lifelong learning with anytime inference
Banerjee, S., Verma, V. K., Mukherjee, A., Gupta, D., Namboodiri, V. P., & Rai, P. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610702 · Cited by 3
A biologically inspired architecture with switching units can learn to generalize across backgrounds
Voina, D., Shea-Brown, E., & Mihalas, S. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.09.014 · Cited by 2
Huang, H., Ye, X., Mustafa, M. B., Dong, Q., Li, Y., Asemi, A., & Asemi, A. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-02086-2
Metadata
- Authors
- Yoon, J., Yang, E., Lee, J., & Hwang, S. J.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2017-08-04
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1708.01547
- OpenAlex
- W2744654301
- Cited by
- 168
- References
- 4
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.