CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay
The DOI matches its DataCite record.
Abstract
This paper introduces a novel perspective to significantly mitigate catastrophic forgetting in continuous learning (CL), which emphasizes models' capacity to preserve existing knowledge and assimilate new information. Current replay-based methods treat every task and data sample equally and thus can not fully exploit the potential of the replay buffer. In response, we propose COgnitive REplay (CORE), which draws inspiration from human cognitive review processes. CORE includes two key strategies: Adaptive Quantity Allocation and Quality-Focused Data Selection. The former adaptively modulates the replay buffer allocation for each task based on its forgetting rate, while the latter guarantees the inclusion of representative data that best encapsulates the characteristics of each task within the buffer. Our approach achieves an average accuracy of 37.95% on split-CIFAR10, surpassing the best baseline method by 6.52%. Additionally, it significantly enhances the accuracy of the poorest-performing task by 6.30% compared to the top baseline. Code is available at https://github.com/sterzhang/CORE.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although methods like Elastic Weight Consolidation (EWC) [16] and COgnitive REplay (CORE) [32] have been proposed to mitigate this issue, they still struggle to effectively balance retaining past knowledge with integrating new data.”
From GPT-based lifelong learning and ANFIS-driven reply memory ratio prediction for aspect-based sentiment analysis (2025) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 31
- Carpenter, S. K., Cepeda, N. J., Rohrer, D., Kang, S. H. K., & Pashler, H. (2012, Sep). Using spacing to enhance di- verse forms of learning: Review of recent research and im- plications for instruction. Educational Psychology Review, 369-378. doi: 10.1007/s10648-012-9205-z · DOI · Paper page
- Chen, Z., & Liu, B. (2020, Jan). Lifelong machine learn- ing. In Transfer learning (p. 196-208). doi: 10.1017/ 9781139061773.016 · DOI
- Craik, F. I., & Lockhart, R. S. (1972, Dec). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 671-684. doi: 10.1016/s0022-5371(72)80001-x · DOI · Paper page
- De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., . . . Tuytelaars, T. (2021). A continual learn- ing survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7), 3366-3385.
- De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., . . . Tuytelaars, T. (2022). A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine In- telligence, 44(7), 3366-3385. doi: 10.1109/TPAMI.2021 .3057446 · DOI · Paper page
- Flesch, T., Juechems, K., Dumbalska, T., Saxe, A., & Sum- merfield, C. (2022). Orthogonal representations for robust context-dependent task performance in brains and neural networks. Neuron, 110(7), 1258-1270.
- French, R. (1999, Apr). Catastrophic forgetting in connec- tionist networks. Trends in Cognitive Sciences, 128-135. doi: 10.1016/s1364-6613(99)01294-2 · DOI · Paper page
- French, R. M. (1999). Catastrophic forgetting in connection- ist networks. Trends in cognitive sciences, 3(4), 128-135.
- Huszár, F. (2018). Note on the quadratic penalties in elas- tic weight consolidation. Proceedings of the National Academy of Sciences, 115(11), E2496-E2497.
- Li, Z., & Hoiem, D. (2017). Learning without forgetting. IEEE transactions on pattern analysis and machine intelli- gence, 40(12), 2935-2947.
- Lockhart, R. S., & Craik, F. I. (1990). Levels of processing: A retrospective commentary on a framework for memory research. Canadian Journal of Psychology/Revue canadi- enne de psychologie, 44(1), 87.
- Masse, N. Y., Grant, G. D., & Freedman, D. J. (2018). Allevi- ating catastrophic forgetting using context-dependent gat- ing and synaptic stabilization. Proceedings of the National Academy of Sciences, 115(44), E10467-E10475.
- Mayo, D., Scott, T. R., Ren, M., Elsayed, G., Hermann, K., Jones, M., & Mozer, M. (2023). Multitask learning via interleaving: A neural network investigation. In Proceed- ings of the annual meeting of the cognitive science society (Vol. 45).
- McCloskey, M., & Cohen, N. J. (1989). Catastrophic inter- ference in connectionist networks: The sequential learn- ing problem. In Psychology of learning and motivation (Vol. 24, pp. 109-165). Elsevier.
- McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5, 115-133.
- Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013, Dec). Playing atari with deep reinforcement learning. arXiv: Learning,arXiv: Learning.
- Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Ve- ness, J., Bellemare, M. G., . . . others (2015). Human- level control through deep reinforcement learning. nature, 518(7540), 529-533.
- Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017). icarl: Incremental classifier and representation learning. In Proceedings of the ieee conference on com- puter vision and pattern recognition (pp. 2001-2010).
- Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., & Wayne, G. (2019). Experience replay for continual learning. Ad- vances in Neural Information Processing Systems, 32.
- Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., . . . Hadsell, R. (2016). Progressive neural networks. arXiv preprint arXiv:1606.04671. · arXiv
- Sadeh, T., Ozubko, J. D., Winocur, G., & Moscovitch, M. (2016). Forgetting patterns differentiate between two forms of memory representation. Psychological science, 27(6), 810-820.
- Sanford, A. J., & Graesser, A. C. (2006). Shallow processing and underspecification. Discourse Processes, 42(2), 99- 108.
- Schmidgall, S., Achterberg, J., Miconi, T., Kirsch, L., Ziaei, R., Hajiseyedrazi, S., & Eshraghian, J. (2023). Brain- inspired learning in artificial neural networks: a review. arXiv preprint arXiv:2305.11252. · arXiv
- Shin, H., Lee, J. K., Kim, J., & Kim, J. (2017). Continual learning with deep generative replay. Advances in neural information processing systems, 30.
- Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017). Ef- ficient processing of deep neural networks: A tutorial and survey. Proceedings of the IEEE, 105(12), 2295-2329.
- Tiwari, R., Killamsetty, K., Iyer, R., & Shenoy, P. (2022). Gcr: Gradient coreset based replay buffer selection for con- tinual learning. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition (pp. 99-108).
- Van de Ven, G. M., & Tolias, A. S. (2019). Three scenarios for continual learning. arXiv preprint arXiv:1904.07734. · arXiv
- Wang, L., Zhang, X., Su, H., & Zhu, J. (2023). A compre- hensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:2302.00487. · arXiv
- Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., & Lipson, H. (2015, Jun). Understanding neural networks through deep visualization. arXiv: Computer Vision and Pattern Recog- nition,arXiv: Computer Vision and Pattern Recognition.
- Zeiler, M. D., & Fergus, R. (2014). Visualizing and un- derstanding convolutional networks. In Computer vision- eccv 2014: 13th european conference, zurich, switzerland, september 6-12, 2014, proceedings, part i 13 (pp. 818- 833).
- Zenke, F., Poole, B., & Ganguli, S. (2017). Continual learn- ing through synaptic intelligence. In International confer- ence on machine learning (pp. 3987-3995).
Cited by
Huang, H., Ye, X., Mustafa, M. B., Dong, Q., Li, Y., Asemi, A., & Asemi, A. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-02086-2
Metadata
- Authors
- Zhang, J., Fu, Y., Peng, Z., Yao, D., & He, K.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- arXiv
- 2402.01348
- References
- 0
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay. https://app.cochat.ai/papers/10.48550/arxiv.2402.01348
- The paper
- Zhang, J., Fu, Y., Peng, Z., Yao, D., & He, K. (2024). CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay. arXiv. https://doi.org/10.48550/arxiv.2402.01348
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.