Generative replay with feedback connections as a general strategy for continual learning
Open accessThe DOI matches its OpenAlex record; 34 of the 34 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“The approach of matching predicted (and typically temperature-raised) probabilities of one network to those of another network had previously been used to compress (or “distill”) information from one (large) network to another (smaller) network [13].”
MethodOvercoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · cited 1×
“- EWC: The regularization term proposed in [8] was added to the loss, see Appendix A.”
BackgroundLi, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · cited 3×
“However, due to the wide variety of experimental protocols used to evaluate them, many of these methods claim “state-of-the-art” performance [ e.g. , 1, 2, 3, 4, 5, 6].”
BackgroundMethodCatastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770 · cited 1×
“When trained on a new task, standard neural networks tend to quickly and completely forget previously learned tasks, a phenomenon referred to as “catastrophic forgetting” [1, 2, 3].”
BackgroundContinual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Catastrophic forgetting in connectionist networks
French, R. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01294-2 · Cited by 2,311 · cited 1×
“When trained on a new task, standard neural networks tend to quickly and completely forget previously learned tasks, a phenomenon referred to as “catastrophic forgetting” [1, 2, 3].”
BackgroundContinual Learning Through Synaptic Intelligence
Zenke, F., Poole, B., & Ganguli, S. (2017) · arXiv · doi:10.48550/arxiv.1703.04200 · Cited by 1,784 · cited 5×
“To obscure things further, some methods shown to perform well in some experimental settings are reported to dramatically fail in others ( e.g. , compare the performance of elastic weight consolidation in [1] and [7] with that in [8] and [9]).”
BackgroundMethodInfluentialTutorial on Variational Autoencoders
Doersch, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05908 · Cited by 1,470 · cited 1×
“Following [31], the output layer of the decoder network p ψ has a sigmoid non-linearity and the “reconstruction term” is given by the binary cross entropy between the original and decoded pixel values: whereby x i is the value of the i th pixel of the original input image x and ˜ x i is the value…”
BackgroundPixel Recurrent Neural Networks
van den Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.06759 · Cited by 1,437 · cited 1×
“Finally, of course, the capabilities of generative models are improving at a rapid pace [ e.g. , 27, 28, 29].”
BackgroundContinual Learning with Deep Generative Replay
Shin, H., Lee, J. K., Kim, J., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1705.08690 · Cited by 1,385 · cited 1×
“Following [14], the replayed images were labeled with the most likely category predicted by a copy of the main model stored after training on the previous task (i.”
MethodAn Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A., & Bengio, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6211 · Cited by 1,089 · cited 1×
“The second task protocol was permuted MNIST ([21], see Figure 2).”
BackgroundRatcliff, R. (1990) · Psychological Review · doi:10.1037//0033-295x.97.2.285 · Cited by 941
Catastrophic forgetting, rehearsal and pseudorehearsal
ROBINS, A. (1995) · Connection Science · doi:10.1080/09540099550039318 · Cited by 833 · cited 1×
“An early implementation of this strategy, called pseudo-rehearsal, generated completely random inputs as pseudo-data and labeled them based on the predictions of a copy of the model stored after finishing training on the previous task [10].”
MethodWhat Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated
Kumaran, D., Hassabis, D., & McClelland, J. L. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.05.004 · Cited by 759 · cited 1×
“One of the main challenges currently facing deep learning is that artificial neural networks are extremely bad at retaining old information when trained on something new, and enabling these networks to sequentially learn multiple tasks without catastrophic forgetting has become a topic of intense research [28, 29].”
BackgroundMeasuring Catastrophic Forgetting in Neural Networks
Kemker, R., McClure, M., Abitino, A., Hayes, T., & Kanan, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11651 · Cited by 672
PathNet: Evolution Channels Gradient Descent in Super Neural Networks
Fernando, C., Banarse, D., Blundell, C., Zwólš, Y., Ha, D., Rusu, A. A., Pritzel, A., & Wierstra, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.08734 · Cited by 591 · cited 1×
“Other approaches use evolutionary algorithms [6] or gradient descent [7] to learn which sets of units to employ for each task.”
BackgroundThree scenarios for continual learning
van de Ven, G. M., & Tolias, A. S. (2019) · Lirias · doi:10.48550/arxiv.1904.07734 · Cited by 523 · cited 2×
“In April 2019, we released a new report with a more in-depth treatment of these three scenarios, including a more extensive comparison of recently proposed methods (see [4]).”
BackgroundMethodSelf-citationVariational Continual Learning
Nguyen, C. V., Li, Y., Bui, T. D., & Turner, R. E. (2017) · arXiv · doi:10.48550/arxiv.1710.10628 · Cited by 491 · cited 1×
“Such “exact replay” can substantially boost performance [17, 18, 19], but due to for example privacy concerns or memory constraints, it is not always possible to do so.”
BackgroundVariational Continual Learning
Turner, R., Bui, T., Li, Y., & Cuong, N. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.35471 · Cited by 438
Principled Hybrids of Generative and Discriminative Models
Lasserre, J. A., Bishop, C. M., & Minka, T. P. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.227 · Cited by 335 · cited 1×
“Besides removing the need for a separate generative model, it is possible that regularization provided by the added generative objective helps to train a more robust classifier [23, 24].”
BackgroundFearNet: Brain-Inspired Model for Incremental Learning
Kemker, R., & Kanan, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10563 · Cited by 296 · cited 1×
“Such “exact replay” can substantially boost performance [17, 18, 19], but due to for example privacy concerns or memory constraints, it is not always possible to do so.”
BackgroundNew insights and perspectives on the natural gradient method
Martens, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1193 · Cited by 283 · cited 1×
“Following the definitions and notation in [32], the i th diagonal element of F ( k ) is defined as: the previous task, with the strength of each parameter’s penalty depending on how important that parameter is thought to be for the tasks learned so far.”
MethodAlleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
Masse, N. Y., Grant, G. D., & Freedman, D. J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1803839115 · Cited by 255 · cited 4×
“A simple approach is to randomly and a priori assign which nodes will participate in each task ( Context-dependent Gating [XdG; 4]).”
BackgroundMethodInfluentialOvercoming catastrophic forgetting with hard attention to the task
Serrà, J., Surís, D., Miron, M., & Karatzoglou, A. (2018) · arXiv · doi:10.48550/arxiv.1801.01423 · Cited by 213 · cited 1×
“Other approaches use evolutionary algorithms [6] or gradient descent [7] to learn which sets of units to employ for each task.”
BackgroundDeep Generative Dual Memory Network for Continual LearningPreprint
Kamra, N., Gupta, U., & Liu, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10368 · Cited by 178 · cited 1×
“To obscure things further, some methods shown to perform well in some experimental settings are reported to dramatically fail in others ( e.g. , compare the performance of elastic weight consolidation in [1] and [7] with that in [8] and [9]).”
MethodTowards Robust Evaluations of Continual Learning
Farquhar, S., & Gal, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09733 · Cited by 169 · cited 1×
“We therefore emphasize that continual learning methods should not only be evaluated in terms of their performance, but also in terms of for example their training time ( e.g. , Figures 4 and 5; see also [30]).”
BackgroundIncremental Classifier Learning with Generative Adversarial NetworksPreprint
Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., Zhang, Z., & Fu, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.00853 · Cited by 141 · cited 1×
“However, due to the wide variety of experimental protocols used to evaluate them, many of these methods claim “state-of-the-art” performance (e.g., Kirkpatrick et al., 2017; Rebuffi et al., 2017; Nguyen et al., 2017; Masse et al., 2018; Kemker & Kanan, 2018; Wu et al., 2018).”
Note on the quadratic penalties in elastic weight consolidation
Huszár, F. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1717042115 · Cited by 105 · cited 1×
“- Online EWC: This is a modification of EWC proposed by [22], with inspiration from [23], that improves EWC’s scalability by ensuring the computational cost of the regularization term does not grow with number of tasks (see Appendix A.”
BackgroundPseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural NetworksPreprint
Atkinson, C., McCane, B., Szymanski, L., & Robins, A. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03875 · Cited by 85 · cited 1×
“This approach had some success with very simple, artificial inputs, but does not work with more complicated inputs [11].”
BackgroundScalable Recollections for Continual Lifelong Learning
Riemer, M., Klinger, T., Bouneffouf, D., & Franceschini, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011352 · Cited by 48 · cited 1×
“The promising idea of storing hidden representations, which besides reducing memory storage demands could also address the privacy issue, is further worked out in [33].”
BackgroundA Strategy for an Uncompromising Incremental Learner
Venkatesan, R., Venkateswara, H., Panchanathan, S., & Li, B. (2017) · arXiv · doi:10.48550/arxiv.1705.00744 · Cited by 37 · cited 1×
“We note that it is possible to combine LwF and DGR by replaying input samples from a generative model and pairing them with soft targets [see also 6, 18].”
Method
Cited by
Three types of incremental learning
van de Ven, G. M., Tuytelaars, T., & Tolias, A. S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00568-3 · Cited by 665
“At least in part because of two preprints of this article 58,59 , the terms ‘task-incremental learning’, ‘domain-incremental learning’ and ‘class-incremental learning’ are sometimes being used in the recent literature in a way that restricts them to this academic setting.”
Self-citationThree scenarios for continual learning
van de Ven, G. M., & Tolias, A. S. (2019) · Lirias · doi:10.48550/arxiv.1904.07734 · Cited by 523
Self-citationBrain-inspired replay for continual learning with artificial neural networks
van de Ven, G. M., Siegelmann, H. T., & Tolias, A. S. (2020) · Nature Communications · doi:10.1038/s41467-020-17866-2 · Cited by 474
Self-citationGDumb: A Simple Approach that Questions Our Progress in Continual Learning
Prabhu, A., Torr, P. H. S., & Dokania, P. K. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_31 · Cited by 465
Online continual learning in image classification: An empirical survey
Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., & Sanner, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.10.021 · Cited by 410
Metadata
- Authors
- van de Ven, G. M., & Tolias, A. S.
- Venue
- Lirias
- Published
- 2018-09-27
- arXiv
- 1809.10635
- OpenAlex
- W2894094671
- Cited by
- 161
- References
- 34
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.