CoChat
Sign Up
PreprintCited by 1,403Domain Adaptation and Few-Shot LearningReinforcement Learning in RoboticsMemory Processes and Influences

Gradient Episodic Memory for Continual Learning

Open access
Lopez-Paz, David, Ranzato, Marc'Aurelio·2017·arXiv
DOI 10.48550/arxiv.1706.08840arXiv 1706.08840
Open this paper in CoChat — free accountOpen PDF

Abstract

One major obstacle towards AI is the poor ability of models to solve new problems quicker, and without forgetting previously acquired knowledge. To better understand this issue, we study the problem of continual learning, where the model observes, once and one by one, examples concerning a sequence of tasks. First, we propose a set of metrics to evaluate models learning over a continuum of data. These metrics characterize models not only by their test accuracy, but also in terms of their ability to transfer knowledge across tasks. Second, we propose a model for continual learning, called Gradient Episodic Memory (GEM) that alleviates forgetting, while allowing beneficial transfer of knowledge to previous tasks. Our experiments on variants of the MNIST and CIFAR-100 datasets demonstrate the strong performance of GEM when compared to the state-of-the-art.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1706.08840

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Rehearsal-based approaches allow access to a memory buffer with examples from prior tasks and train the model jointly with the current task.

    “Rehearsalbased approaches (Lopez-Paz and Ranzato, 2017; de Masson D'Autume et al., 2019) allow access to a memory buffer with examples from prior tasks and train the model jointly with the current task.”

    From Orthogonal Subspace Learning for Language Model Continual Learning (2023) · cites this paper as Lopez-Paz and Ranzato, 2017

    Section 3 (p. 3) says GEM stores examples from prior tasks in episodic memory. But it explicitly uses losses on those examples as constraints while optimizing the current example, rather than jointly minimizing losses on old and current examples. Section 5 (p. 8) also describes other episodic-memory approaches that store and replay prior examples. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • In parametric continual learning, the central difficulty is interference in weight space, where learning new tasks can overwrite knowledge useful for old ones, leading to the stability-plasticity dilemma.

    “In parametric CL, the central difficulty is interference in weight space: learning new tasks can overwrite knowledge useful for old ones, leading to the well-known stability-plasticity dilemma (Kirkpatrick et al., 2017; Lopez-Paz and Ranzato, 2017; Wang et al., 2024a).”

    From When Continual Learning Moves to Memory: A Study of Experience Reuse in LLM Agents (2026) · cites this paper as Lopez-Paz and Ranzato, 2017

  • Non-parametric episodic memory can serve as rehearsal samples to alleviate catastrophic forgetting in parametric memory.

    “Additionally, non-parametric episodic memory can serve as rehearsal samples to alleviate catastrophic forgetting in parametric memory [138, 139, 141].”

    From Human-inspired Perspectives: A Survey on AI Long-term Memory (2024) · cites this paper as [138]

  • Classical continual learning methods typically treat memory as a static mechanism for knowledge retention.

    “but they typically treat memory as a static mechanism for knowledge retention.”

    From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Lopez-Paz & Ranzato, 2017

  • Lopez-Paz and Ranzato employ episodic memory and loss constraints to prevent the degradation of performance on previous tasks while learning new ones.

    “Lopez-Paz and Ranzato [2] employ episodic memory and loss constraints to prevent the degradation of performance on previous tasks while learning new ones.”

    From GPT-based lifelong learning and ANFIS-driven reply memory ratio prediction for aspect-based sentiment analysis (2025) · cites this paper as [2]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Part of this paper’s reference list could not be split into single entries.

Flagged first, then by citations

Research works printed but not linked: 1

  1. [13]13. S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert. iCaRL: Incremental classifier and representation learning. CVPR , 2017. - M. B. Ring. Continual Learning in Reinforcement Environments . PhD thesis, University of Texas at Austin, Austin, Texas 78712, 1994. - M. B. Ring. CHILD: A first step towards continual learning. Machine Learning , 1997. - A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell. Progressive neural networks. NIPS , 2016. - P. Ruvolo and E. Eaton. ELLA: An Efficient Lifelong Learning Algorithm. ICML , 2013. - A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap. One-shot learning with memory-augmented neural networks. arXiv , 2016. - T. Schaul, D. Horgan, K. Gregor, and D. Silver. Universal value function approximators. ICML , 2015. - B. Scholkopf and A. J. Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond . MIT press, 2001. - B. Schölkopf, D. Janzing, and D. Lopez-Paz. Causal and statistical learning. In Learning Theory and Approximation . Oberwolfach Research Institute for Mathematics, 2016. - R. S. Sutton, J. Modayil, M. Delp, T. Degris, P. M. Pilarski, A. White, and D. Precup. Horde: A scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction. The 10th International Conference on Autonomous Agents and Multiagent Systems , 2011. - S. Thrun. A lifelong learning perspective for mobile robot control. Proceedings of the IEEE/RSJ/GI Conference on Intelligent Robots and Systems , 1994. - S. Thrun. Is learning the n-th thing any easier than learning the first? NIPS , 1996. - S. Thrun. Lifelong learning algorithms. In Learning to learn . Springer, 1998. - S. Thrun and L. Pratt. Learning to learn . Springer Science & Business Media, 2012. - V. Vapnik. Statistical learning theory . Wiley New York, 1998. - O. Vinyals, C. Blundell, T. Lillicrap, and D. Wierstra. Matching networks for one shot learning. NIPS , 2016. - F. Zenke, B. Poole, and S. Ganguli. Improved multitask learning through synaptic intelligence. arXiv , 2017.

Cited by

Sorted by citations

Authors
Lopez-Paz, D., & Ranzato, M.
Affiliations
Meta (Israel) (IL)
Venue
arXiv
Publisher
arXiv
Published
2017
Volume
30
Pages
6467-6476
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1706.08840
arXiv
1706.08840
OpenAlex
W2734314755
Cited by
1,403
References
1
Citations by year
2026: 2 · 2025: 174 · 2024: 283 · 2023: 336 · 2022: 215 · 2021: 213 · 2020: 110 · 2019: 47 · 2018: 22 · 2017: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • neural information processing systems
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
Keywords
continual learning, catastrophic forgetting, episodic memory, knowledge transfer, sequence-to-sequence learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review