Variational Continual Learning
Open accessAbstract
This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks. The framework can suc- cessfully train both deep discriminative models and deep generative models in complex continual learning settings where existing tasks evolve over time and en- tirely new tasks emerge. Experimental results show that VCL outperforms state- of-the-art continual learning methods on a variety of tasks, avoiding catastrophic forgetting in a fully automatic way.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Non-parametric episodic memory can serve as rehearsal samples to alleviate catastrophic forgetting in parametric memory.
“Additionally, non-parametric episodic memory can serve as rehearsal samples to alleviate catastrophic forgetting in parametric memory [138, 139, 141].”
From Human-inspired Perspectives: A Survey on AI Long-term Memory (2024) · cites this paper as [139]
Cited by
Sorted by citations
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495 · see citing works
Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
Chaudhry, A., Dokania, P. K., Ajanthan, T., & Torr, P. H. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_33 · Cited by 1,091 · see citing works
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113 · see citing works
Overcoming catastrophic forgetting with hard attention to the task
Serrà, J., Surís, D., Miron, M., & Karatzoglou, A. (2018) · arXiv · doi:10.48550/arxiv.1801.01423 · Cited by 213 · see citing works
Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
Hsu, Y.-C., Liu, Y.-C., Ramasamy, A., & Kira, Z. (2018) · arXiv · doi:10.48550/arxiv.1810.12488 · Cited by 211 · see citing works
Replay in Deep Learning: Current Approaches and Missing Biological Elements
Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski, T. J., & Kanan, C. (2021) · Neural Computation · doi:10.1162/neco_a_01433 · Cited by 106 · see citing works
Physical Security and Safety of IoT Equipment: A Survey of Recent Advances and Opportunities
Yang, X., Shu, L., Liu, Y., Hancke, G. P., Ferrag, M. A., & Huang, K. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3141408 · Cited by 93 · see citing works
Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2023) · arXiv · doi:10.48550/arxiv.2312.10549 · Cited by 54
Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2024) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2024.3966 · Cited by 54 · see citing works
Getting a CLUE: A Method for Explaining Uncertainty Estimates
Antorán, J., Bhatt, U., Adel, T., Weller, A., & Hernández-Lobato, J. M. (2021) · Apollo - University of Cambridge Repository · doi:10.17863/cam.69929 · Cited by 49 · see citing works
Decentralized Bayesian Learning over Graphs
Lalitha, A., Wang, X., Kilinc, O., Lu, Y., Javidi, T., & Koushanfar, F. (2019) · arXiv · doi:10.48550/arxiv.1905.10466 · Cited by 20 · see citing works
Self-Net: Lifelong Learning via Continual Self-Modeling
Mandivarapu, J. K., Camp, B., & Estrada, R. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00019 · Cited by 14 · see citing works
Human-inspired Perspectives: A Survey on AI Long-term Memory
He, Z., Lin, W., Zheng, H., Zhang, F., Jones, M. W., Aitchison, L., Xu, X., Liu, M., Kristensson, P. O., & Shen, J. (2024) · arXiv · doi:10.48550/arxiv.2411.00489 · Cited by 3 · see citing works
“For instance, Nguyen et al. [133] employed episodic memory as a mechanism for storing key data points from previous tasks, allowing the model to periodically revisit these data points when learning new tasks, thus ensuring the retention of knowledge from old tasks.”
Influential
- Authors
- Turner, R., Bui, T., Li, Y., & Cuong, N.
- Affiliations
- University of Cambridge (GB)
- Venue
- Apollo - University of Cambridge Repository
- Publisher
- Apollo - University of Cambridge Repository
- Published
- 2019-01-18
- ISSN
- 2331-8422 (ISSN-L)
- OpenAlex
- W2963577698
- Cited by
- 438
- References
- 0
- Citations by year
- 2026: 2 · 2025: 19 · 2024: 38 · 2023: 65 · 2022: 47 · 2021: 103 · 2020: 83 · 2019: 60 · 2018: 21
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- arXiv (Cornell University)
- Apollo
- Funders
- Engineering and Physical Sciences Research Council (EP/L000776/1, EP/M026957/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- continual learning, variational inference, catastrophic forgetting, deep generative models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
