The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Memory Efficient Experience Replay for Streaming Learning
Open accessAbstract
In supervised machine learning, an agent is typically trained once and then deployed. While this works well for static settings, robots often operate in changing environments and must quickly learn new things from data streams. In this paradigm, known as streaming learning, a learner is trained online, in a single pass, from a data stream that cannot be assumed to be independent and identically distributed (iid). Streaming learning will cause conventional deep neural networks (DNNs) to fail for two reasons: 1) they need multiple passes through the entire dataset; and 2) non-iid data will cause catastrophic forgetting. An old fix to both of these issues is rehearsal. To learn a new example, rehearsal mixes it with previous examples, and then this mixture is used to update the DNN. Full rehearsal is slow and memory intensive because it stores all previously observed examples, and its effectiveness for preventing catastrophic forgetting has not been studied in modern DNNs. Here, we describe the ExStream algorithm for memory efficient rehearsal and compare it to alternatives. We find that full rehearsal can eliminate catastrophic forgetting in a variety of streaming learning settings, with ExStream performing well using far less memory and computation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Wolpert, D. H. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80023-1 · Cited by 7,951
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256 · cited 1×
“catastrophic forgetting of previously learned information [58]. Rehearsal is one of the earliest methods for preventing this phenomenon [50]. Rehearsal, also known as replay and interleaved learning [57], limits catastrophic forgetting by mixing past experience with new data [50]. Originally, it was implemented by mix…”
MethodiCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · cited 3×
“Streaming classification is distinct from incremental batch learning, which has recently received much attention [41], [42], [49], [54], [65], [74], and it has unique challenges.”
BackgroundMethodCatastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
Catastrophic forgetting in connectionist networks
French, R. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01294-2 · Cited by 2,311
Memory consolidation, retrograde amnesia and the hippocampal complex
Nadel, L., & Moscovitch, M. (1997) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(97)80010-4 · Cited by 1,924
Continual Learning Through Synaptic Intelligence
Zenke, F., Poole, B., & Ganguli, S. (2017) · arXiv · doi:10.48550/arxiv.1703.04200 · Cited by 1,784
Self-improving reactive agents based on reinforcement learning, planning and teaching
Lin, L.-J. (1992) · Machine Learning · doi:10.1007/bf00992699 · Cited by 1,564 · cited 1×
“s realized updating a DNN with new information often results in catastrophic forgetting of previously learned information [58]. Rehearsal is one of the earliest methods for preventing this phenomenon [50]. Rehearsal, also known as replay and interleaved learning [57], limits catastrophic forgetting by mixing past expe…”
Method
Cited by
Learning to Prompt for Continual Learning
Wang, Z., Zhang, Z., Lee, C.-Y., Zhang, H., Sun, R., Ren, X., Su, G., Perot, V., Dy, J., & Pfister, T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00024 · Cited by 778
Experience Replay for Continual Learning
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T. P., & Wayne, G. (2018) · arXiv · doi:10.48550/arxiv.1811.11682 · Cited by 529
Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., & Díaz-Rodríguez, N. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.004 · Cited by 506
DualPrompt: Complementary Prompting for Rehearsal-Free Continual Learning
Wang, Z., Zhang, Z., Ebrahimi, S., Sun, R., Zhang, H., Lee, C.-Y., Ren, X., Su, G., Perot, V., Dy, J., et al. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19809-0_36 · Cited by 472
Online continual learning in image classification: An empirical survey
Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., & Sanner, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.10.021 · Cited by 410
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning
Smith, J. S., Karlinsky, L., Gutta, V., Cascante-Bonilla, P., Kim, D., Arbelle, A., Panda, R., Feris, R., & Kira, Z. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01146 · Cited by 334
Continuous learning in single-incremental-task scenarios
Maltoni, D., & Lomonaco, V. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.010 · Cited by 309
On Tiny Episodic Memories in Continual Learning
Chaudhry, A., Rohrbach, M., Elhoseiny, M., Ajanthan, T., Dokania, P. K., Torr, P. H. S., & Ranzato, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10486 · Cited by 298
REMIND Your Neural Network to Prevent Catastrophic Forgetting
Hayes, T. L., Kafle, K., Shrestha, R., Acharya, M., & Kanan, C. (2019) · arXiv · doi:10.48550/arxiv.1910.02509 · Cited by 268
Self-citationBiological underpinnings for lifelong learning machines
Kudithipudi, D., Aguilar-Simon, M., Babb, J., Bazhenov, M., Blackiston, D., Bongard, J., Brna, A. P., Chakravarthi Raja, S., Cheney, N., Clune, J., et al. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00452-0 · Cited by 236
Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
Hayes, T. L., & Kanan, C. (2019) · arXiv · doi:10.48550/arxiv.1909.01520 · Cited by 150
Self-citationAn Appraisal of Incremental Learning Methods
Luo, Y., Yin, L., Bai, W., & Mao, K. (2020) · Entropy · doi:10.3390/e22111190 · Cited by 120
“In the context of reinforcement learning, in order to learn an approximately stable data distribution, techniques similar to those proposed in incremental learning are often used, such as the rehearsal method [139].”
MethodReplay in Deep Learning: Current Approaches and Missing Biological Elements
Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski, T. J., & Kanan, C. (2021) · Neural Computation · doi:10.1162/neco_a_01433 · Cited by 106
“…2017; Castro, Marín-Jiménez, Guil, Schmid, & Ala-hari, 2018; Chaudhry et al., 2018; Nguyen, Li, Bui, & Turner, 2018; Hou et al., 2018, 2019; Hayes et al., 2019; Wu, Chen et al., 2019; Lee, Lee, Shin, & Lee, 2019; Belouadah & Popescu, 2019, 2020; Chaudhry et al., 2019; Riemer et al., 2019;…”
InfluentialSelf-citationCatastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2023) · arXiv · doi:10.48550/arxiv.2312.10549 · Cited by 54
Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2024) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2024.3966 · Cited by 54
Learning to Continuously Optimize Wireless Resource in Episodically Dynamic Environment
Sun, H., Pu, W., Zhu, M., Fu, X., Chang, T.-H., & Hong, M. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9413503 · Cited by 21
Verse: Virtual-gradient aware streaming lifelong learning with anytime inference
Banerjee, S., Verma, V. K., Mukherjee, A., Gupta, D., Namboodiri, V. P., & Rai, P. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610702 · Cited by 3
Online continual streaming learning for embedded space applications
Mazouz, A. E., & Nguyen, V.-T. (2024) · Journal of Real-Time Image Processing · doi:10.1007/s11554-024-01438-4 · Cited by 3
Metadata
- Authors
- Hayes, T. L., Cahill, N. D., & Kanan, C.
- Venue
- 2019 International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 9769-9776
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2968596670
- Cited by
- 200
- References
- 15
- FWCI
- 10.32
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.