CoChat
Sign Up
Cited by 6Memory Processes and InfluencesMemory and Neural MechanismsCognitive Functions and Memory

Elements of episodic memory: insights from artificial agents

Open access
Boyle, Alexandria, Blomkvist, Andrea·2024·Philosophical Transactions of the Royal Society B: Biological Sciences
DOI 10.1098/rstb.2023.0416PMC PMC11449156
Open this paper in CoChat — free accountOpen PDF

Abstract

Many recent artificial intelligence (AI) systems take inspiration from biological episodic memory. Here, we ask how these ‘episodic-inspired’ AI systems might inform our understanding of biological episodic memory. We discuss work showing that these systems implement some key features of episodic memory while differing in important respects and appear to enjoy behavioural advantages in the domains of strategic decision-making, fast learning, navigation, exploration and acting over temporal distance. We propose that these systems could be used to evaluate competing theories of episodic memory’s operations and function. However, further work is needed to validate them as models of episodic memory and isolate the contributions of their memory systems to their behaviour. More immediately, we propose that these systems have a role to play in directing episodic memory research by highlighting novel or neglected hypotheses as pursuit-worthy. In this vein, we propose that the evidence reviewed here highlights two pursuit-worthy hypotheses about episodic memory’s function: that it plays a role in planning that is independent of future-oriented simulation, and that it is adaptive in virtue of its contributions to fast learning in novel, sparse-reward environments. This article is part of the theme issue ‘Elements of episodic memory: lessons from 40 years of research’.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1098/rstb.2023.0416

Cites

69 of 69 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Memory architectures address AI Agents' limitations in long-horizon planning and session continuity by persisting information across tasks.

    “Memory architectures address this by persisting information across tasks [223].”

    From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [223]

    Section 3.5 (page unavailable) describes AI systems that store and retrieve past events to guide decisions and complete temporally extended tasks, supporting the long-horizon part of the claim. It does not establish that information persists across separate tasks or sessions, or that these architectures address session continuity. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 17

  1. Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Pieter Abbeel, O., & Zaremba, W. (2017). Hindsight experience replay. Advances in Neural Information Processing Systems, 30. https://goo.gl/SMrQnI.
  2. Blundell, C., Uria, B., Pritzel, A., Li, Y., Ruderman, A., Leibo, J. Z., Rae, J., Wierstra, D., & Hassabis, D. (2016). Model-free episodic control. http://arxiv.org/abs/1606.04460 · arXiv
  3. Camp, B., Mandivarapu, J. K., & Estrada, R. (2020). Continual Learning with Deep Artificial Neurons. http://arxiv.org/abs/2011.07035 · arXiv
  4. Hafner, D., Pasukonis, J., Ba, J., Lillicrap, T., & Diamond, M. (2023). Mastering diverse domains through world models. https://arxiv.org/pdf/2301.04104v1.pdf · arXiv
  5. Kim, T., & Har, D. (2022). Failed Goal Aware Hindsight Experience Replay. http://arxiv.org/abs/2208.14741 · arXiv
  6. Kumar, R., & Nagaraj, D. (2022). Introspective Experience Replay: Look Back When Surprised. http://arxiv.org/abs/2206.03171 · arXiv
  7. Lee, S. Y., Choi, S., & Chung, S.-Y. (2018). Sample-Efficient Deep Reinforcement Learning via Episodic Backward Update. http://arxiv.org/abs/1805.12375 · arXiv
  8. Lengyel, M., & Dayan, P. (2008). Hippocampal contributions to control: The third way. Advances in Neural Information Processing Systems 20 -Proceedings of the 2007 Conference.
  9. Mccarthy, R., Wang, Q., & Redmond, S. J. (2023). Imaginary hindsight experience replay: Curious model-based learning for sparse reward tasks. https://arxiv.org/pdf/2110.02414.pdf · arXiv
  10. Pritzel, A., Uria, B., Srinivasan, S., Badia, A. P., Vinyals, O., Hassabis, D., Wierstra, D., & Blundell, C. (2017). Neural episodic control. 34th International Conference on Machine Learning, ICML 2017, 6.
  11. Ritter, S., Faulkner, R., Sartran, L., Santoro, A., Botvinick, M., & Raposo, D. (2021). Rapid task-solving in novel environments. ICLR 2021 -9th International Conference on Learning Representations.
  12. Ritter, S., Wang, J. X., Kurth-Nelson, Z., Jayakumar, S. M., Blundell, C., Pascanu, R., & Botvinick, M. (2018). Been there, done that: Meta-learning with episodic recall. 35th International Conference on Machine Learning, ICML 2018, 10.
  13. Rotinov, E. (2019). Reverse Experience Replay. http://arxiv.org/abs/1910.08780 · arXiv
  14. Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015). Prioritized Experience Replay. http://arxiv.org/abs/1511.05952 · arXiv
  15. Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016). Learning to reinforcement learn. https://arxiv.org/abs/1611.
  16. Yuan, Y., & Mattar, M. G. (2021). Improving Experience Replay with Successor Representation. http://arxiv.org/abs/2111.14331 · arXiv
  17. Zakharov, A., Crosby, M., & Fountas, S. (2020). Episodic memory for learning subjective- timescale models. https://arxiv.org/abs/2010.

Cited by

Sorted by citations

Related

Authors
Boyle, A., & Blomkvist, A.
Affiliations
  • Canadian Institute for Advanced Research (CA)
  • University of Glasgow (GB)
ORCID
Venue
Philosophical Transactions of the Royal Society B: Biological Sciences
Publisher
The Royal Society
Published
2024-09-16
Volume
379
Issue
1913
Pages
20230416-20230416
ISSN
  • 0962-8436 (ISSN-L)
  • 1471-2970
DOI
10.1098/rstb.2023.0416
PMC
PMC11449156
OpenAlex
W4402550125
Cited by
6
References
69
Citations by year
2026: 2 · 2025: 2 · 2024: 2
FWCI
0.39
Type
journal article
Language
en
Open access
Licence
Locations
  • Philosophical Transactions of the Royal Society B Biological Sciences (publishedVersion)
  • PubMed Central (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed (publishedVersion)
  • ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by)
  • London School of Economics and Political Science Theses Online (London School of Economics and Political Science) (acceptedVersion, cc-by)
  • London School of Economics and Political Science Research Online (London School of Economics and Political Science) (submittedVersion)
  • PhilPapers (PhilPapers Foundation) (submittedVersion)
Funders
  • British Academy
  • UK Research and Innovation
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
Keywords
episodic memory, artificial agents, strategic decision-making, fast learning, navigation, exploration, temporal distance, planning, adaptive memory
MeSH terms
Artificial Intelligence, Decision Making, Humans, Learning, Memory, Episodic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review