Variational Continual Learning
Open accessAbstract
This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks. The framework can successfully train both deep discriminative models and deep generative models in complex continual learning settings where existing tasks evolve over time and entirely new tasks emerge. Experimental results show that VCL outperforms state-of-the-art continual learning methods on a variety of tasks, avoiding catastrophic forgetting in a fully automatic way.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Epistemic uncertainty is used to guide methods that require exploration like active learning, continual learning, Bayesian optimisation, and reinforcement learning.
“It is also used to guide methods that require exploration like active learning [53], continual learning [92], Bayesian optimisation [47], and reinforcement learning [58].”
From Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2021) · cites this paper as [92]
The Related Work section (p. 6) says better uncertainty estimates are critical for continual learning, supporting the citation’s link between uncertainty and that method. The paper does not describe continual learning as requiring exploration or say that uncertainty guides exploration in it. The other methods in the citing sentence have separate citations and are not assessed here. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 41
- Olivier Bachem, Mario Lucic, and Andreas Krause. Coresets for nonparametric estimation - the case of DP-means. In International Conference on Machine Learning , 2015.
- Bart Bakker and Tom Heskes. Task clustering and gating for Bayesian multitask learning. Journal of Machine Learning Research , 2003.
- Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International Conference on Machine Learning , 2015.
- Tamara Broderick, Nicholas Boyd, Andre Wibisono, Ashia C. Wilson, and Michael I. Jordan. Streaming variational Bayes. In Advances in Neural Information Processing Systems , 2013.
- Thang Bui, Daniel Hern´ andez-Lobato, Jose Hernandez-Lobato, Yingzhen Li, and Richard Turner. Deep Gaussian processes for regression using approximate expectation propagation. In International Conference on Machine Learning , 2016.
- Thang D. Bui, Cuong V. Nguyen, and Richard E. Turner. Streaming sparse Gaussian process approximations. In Advances in Neural Information Processing Systems , 2017.
- Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. In International Conference on Learning Representations , 2016.
- Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C. Courville, and Yoshua Bengio. A recurrent latent variable model for sequential data. In Advances in Neural Information Processing Systems , 2015.
- Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning , 2016.
- Zoubin Ghahramani and H. Attias. Online variational Bayesian learning. In NIPS Workshop on Online Learning , 2000.
- Ian J. Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks. In International Conference on Learning Representations , 2014a.
- Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems , 2014b.
- Alex Graves. Practical variational inference for neural networks. In Advances in Neural Information Processing Systems , 2011.
- Jos´ e Miguel Hern´ andez-Lobato and Ryan P. Adams. Probabilistic backpropagation for scalable learning of Bayesian neural networks. In International Conference on Machine Learning , 2015.
- Jos´ e Miguel Hern´ andez-Lobato, Yingzhen Li, Mark Rowland, Daniel Hern´ andez-Lobato, Thang D. Bui, and Richard E. Turner. Black-box α -divergence minimization. In International Conference on Machine Learning , 2016.
- Geoffrey E. Hinton and Drew Van Camp. Keeping the neural networks simple by minimizing the description length of the weights. In Conference on Computational Learning Theory , 1993.
- Jonathan Huggins, Trevor Campbell, and Tamara Broderick. Coresets for scalable Bayesian logistic regression. In Advances in Neural Information Processing Systems , 2016.
- Ferenc Husz´ ar. Note on the quadratic penalties in elastic weight consolidation. Proceedings of the National Academy of Sciences , 2018.
- Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations , 2015.
- Diederik P. Kingma and Max Welling. Stochastic gradient VB and the variational auto-encoder. In International Conference on Learning Representations , 2014.
- Diederik P. Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameterization trick. In Advances in Neural Information Processing Systems , 2015.
- Diederik P. Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improved variational inference with inverse autoregressive flow. In Advances in Neural Information Processing Systems , 2016.
- James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences , 2017.
- James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Reply to Husz´ ar: The elastic weight consolidation penalty is empirically valid. Proceedings of the National Academy of Sciences , 2018.
- Sang-Woo Lee, Jin-Hwa Kim, Jung-Woo Ha, and Byoung-Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In Advances in Neural Information Processing Systems , 2017.
- Zhizhong Li and Derek Hoiem. Learning without forgetting. In European Conference on Computer Vision , 2016.
- Jun S. Liu and Rong Chen. Sequential Monte Carlo methods for dynamic systems. Journal of the American Statistical Association , 1998.
- David Lopez-Paz and Marc'Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems , 2017.
- Peter S. Maybeck. Stochastic models, estimation, and control . Academic Press, 1982.
- Roger Ratcliff. Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psychological Review , 1990.
- Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. Learning multiple visual domains with residual adapters. In Advances in Neural Information Processing Systems , 2017.
- Danilo J. Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International Conference on Machine Learning , 2014.
- Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv:1606.04671 , 2016. · arXiv
- Jeffrey C. Schlimmer and Douglas Fisher. A case study of incremental concept induction. In The National Conference on Artificial Intelligence , 1986.
- Ari Seff, Alex Beatson, Daniel Suo, and Han Liu. Continual learning in generative adversarial nets. arXiv:1705.08395 , 2017. · arXiv
- Sharad Singhal and Lance Wu. Training multilayer perceptrons with the extended Kalman algorithm. In Advances in Neural Information Processing Systems , 1989.
- Alex J. Smola, S.V.N. Vishwanathan, and Eleazar Eskin. Laplace propagation. In Advances in Neural Information Processing Systems , 2004.
- Richard S. Sutton and Steven D. Whitehead. Online learning with random representations. In International Conference on Machine Learning , 1993.
- Pawel Swietojanski and Steve Renals. Learning hidden unit contributions for unsupervised speaker adaptation of neural network acoustic models. In Spoken Language Technology Workshop , 2014.
- Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dynamics. In Advances In Neural Information Processing Systems , 2016.
- Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning , 2017.
Cited by
Sorted by citations
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 2,720 · see citing works
“[505], [506] proposed variational CL (VCL) to deal with uncertainty and showed the effectiveness of such a approach.”
BackgroundRiemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
Chaudhry, A., Dokania, P. K., Ajanthan, T., & Torr, P. H. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_33 · Cited by 1,091 · see citing works
Physical Security and Safety of IoT Equipment: A Survey of Recent Advances and Opportunities
Yang, X., Shu, L., Liu, Y., Hancke, G. P., Ferrag, M. A., & Huang, K. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3141408 · Cited by 93 · see citing works
Challenges in Task Incremental Learning for Assistive Robotics
Feng, F., Chan, R. H. M., Shi, X., Zhang, Y., & She, Q. (2020) · IEEE Access · doi:10.1109/access.2019.2955480 · Cited by 39 · see citing works
Federated Gradient Averaging for Multi-Site Training with Momentum-Based Optimizers
Remedios, S. W., Butman, J. A., Landman, B. A., & Pham, D. L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60548-3_17 · Cited by 19 · see citing works
“Related research in the field of continual learning [11, 20, 20] investigates methods to mitigate catastrophic forgetting [6], the phenomenon by which neural networks have no guarantee to retain useful weight values when training on new data.”
Background
- Authors
- Nguyen, C. V., Li, Y., Bui, T. D., & Turner, R. E.
- Affiliations
- University of Cambridge (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1710.10628
- OpenAlex
- W2963577698
- Cited by
- 491
- References
- 0
- Citations by year
- 2026: 12 · 2025: 36 · 2024: 44 · 2023: 70 · 2022: 50 · 2021: 104 · 2020: 88 · 2019: 66 · 2018: 21
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- arXiv (Cornell University)
- Apollo
- Funders
- Engineering and Physical Sciences Research Council (EP/M026957/1, EP/L000776/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- continual learning, variational inference, catastrophic forgetting, deep generative models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
