REMIND Your Neural Network to Prevent Catastrophic Forgetting
Open accessAbstract
People learn throughout life. However, incrementally updating conventional neural networks leads to catastrophic forgetting. A common remedy is replay, which is inspired by how the brain consolidates memory. Replay involves fine-tuning a network on a mixture of new and old instances. While there is neuroscientific evidence that the brain replays compressed memories, existing methods for convolutional networks replay raw images. Here, we propose REMIND, a brain-inspired approach that enables efficient replay with compressed representations. REMIND is trained in an online manner, meaning it learns one example at a time, which is closer to how humans learn. Under the same constraints, REMIND outperforms other methods for incremental class learning on the ImageNet ILSVRC-2012 dataset. We probe REMIND's robustness to data ordering schemes known to induce catastrophic forgetting. We demonstrate REMIND's generality by pioneering online learning for Visual Question Answering (VQA).
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Class IL algorithms provide the most interesting results in recent works from literature.
“We focus on class IL algorithms which provide the most interesting results in recent works from literature [7, 8, 14, 23, 24, 28, 56, 75].”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [23]
Fixed-representation based IL algorithms do not update the model but are less dependent on memory than fine-tuning based algorithms.
“We also present results with fixedrepresentation based algorithms [6, 23, 24, 56], which do not update the model but are also less dependent on memory.”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [23]
Compact vectors allow saving much more past data than raw images; for instance, all ILSVRC can fit in the memory when K=20000.
“Note that compact vectors allow us to save much more past data than with raw images (for instance, all ILSVRC can fit in the memory when K = 20000 ).”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [23]
Online learning methods are useful when the stream of data arrives image per image.
“Online learning methods [23, 24] are useful when the stream of data arrives image per image.”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [23]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1× · see citing works
“We hypothesize that the q-type ordering may be acting as a natural curriculum [11], allowing our streaming model to train more efficiently.”
BackgroundProduct Quantization for Nearest Neighbor Search
Jégou, H., Douze, M., & Schmid, C. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.57 · Cited by 2,956 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
Belouadah, E., Popescu, A., & Kanellos, I. (2020) · arXiv · doi:10.48550/arxiv.2011.01844 · Cited by 235 · see citing works
- Authors
- Hayes, T. L., Kafle, K., Shrestha, R., Acharya, M., & Kanan, C.
- Affiliations
- Rochester Institute of Technology (US)
- Adobe Systems (United States) (US)
- Corresponding author
- Hayes, Tyler L.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- Pages
- 466-483
- ISSN
- 0302-9743 (ISSN-L)
- arXiv
- 1910.02509
- OpenAlex
- W3108280718
- Cited by
- 268
- References
- 3
- Citations by year
- 2026: 14 · 2025: 42 · 2024: 59 · 2023: 53 · 2022: 46 · 2021: 48 · 2020: 6
- FWCI
- 14.59
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Lecture notes in computer science (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1909696)
- Nvidia
- Defense Advanced Research Projects Agency (W911NF-18-2-0263)
- Air Force Office of Scientific Research (FA9550-, FA9550-18-1-0121)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Keywords
- catastrophic forgetting, incremental learning, memory replay, compressed representations, online learning, brain-inspired learning, visual question answering, convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
