Episodic retrieval for model-based evaluation in sequential decision tasks.
Open accessAbstract
It has long been hypothesized that episodic memory supports adaptive decision making by enabling mental simulation of future events. Yet, attempts to characterize this process are surprisingly rare. On one hand, memory research is often carried out in settings that are far removed from ecological contexts of decision making. On the other hand, models of adaptive choice only invoke episodic memory in highly stylized terms, if at all. To address these gaps, we propose TCM-SR, a novel process-level model that grounds model-based evaluation in empirically informed dynamics of episodic recall. In this model, the probability of retrieving each available memory is governed by the successor representation, a biologically plausible world model in reinforcement learning. The evolution of these probabilities based on past retrievals, in turn, is dictated by the temporal context model, a prominent model of episodic retrieval. Through simulations and analytical derivations, we show that the patterns of episodic retrieval suggested by this model enables flexible computation of decision variables. On this basis, a number of previously described features of episodic memory might serve an adaptive purpose in sequential decision making. For instance, we show that the contiguity effect, a well-known bias in episodic retrieval, enables mental simulation via model-based rollouts to inform decisions. We also show that backward retrieval and emotional modulation improve generalization and the efficiency of decisions given limited experience. By bridging computational models across these two domains, we make several theoretical and empirical predictions linking episodic memory to adaptive choice in sequential tasks. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Part of this paper’s reference list could not be split into single entries.
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256 · see citing works
Hippocampal place-cell sequences depict future paths to remembered goals
Pfeiffer, B. E., & Foster, D. J. (2013) · Nature · doi:10.1038/nature12112 · Cited by 1,216 · see citing works
What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated
Kumaran, D., Hassabis, D., & McClelland, J. L. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.05.004 · Cited by 736 · see citing works
A context maintenance and retrieval model of organizational processes in free recall
Polyn, S. M., Norman, K. A., & Kahana, M. J. (2009) · Psychological Review · doi:10.1037/a0014420 · Cited by 726 · see citing works
Mather, M., Clewett, D., Sakaki, M., & Harley, C. W. (2016) · Behavioral and Brain Sciences · doi:10.1017/s0140525x15000667 · Cited by 695 · see citing works
Associative retrieval processes in free recall
Kahana, M. J. (1996) · Memory & Cognition · doi:10.3758/bf03197276 · Cited by 605 · see citing works
Précis of O'Keefe & Nadel's The hippocampus as a cognitive map
O'Keefe, J., & Nadel, L. (1979) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00063949 · Cited by 486 · see citing works
Prioritized memory access explains planning and hippocampal replay
Mattar, M. G., & Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0232-z · Cited by 393 · see citing works
Self-citationThe successor representation in human reinforcement learning
Momennejad, I., Russek, E. M., Cheong, J. H., Botvinick, M., Daw, N. D., & Gershman, S. J. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0180-8 · Cited by 384 · see citing works
Self-citationKumaran, D., & McClelland, J. L. (2012) · Psychological Review · doi:10.1037/a0028681 · Cited by 358 · see citing works
Speed/accuracy trade-off between the habitual and the goal-directed processes
Keramati, M., Dezfouli, A., & Piray, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002055 · Cited by 346 · see citing works
Parsimony, exhaustivity and balanced detection in neocortex
Romagnoni, A., Ribot, J., Bennequin, D., et al. (2016) · INRIA a CCSD electronic archive server · doi:10.1371/journal.pcbi · Cited by 297 · see citing works
Complementary learning systems
O’Reilly, R. C., Bhattacharyya, R., Howard, M. D., & Ketz, N. (2014) · Cognitive Science · doi:10.1111/j.1551-6709.2011.01214.x · Cited by 259 · see citing works
Dorsal hippocampus contributes to model-based planning
Miller, K. J., Botvinick, M., & Brody, C. D. (2017) · Nature Neuroscience · doi:10.1038/nn.4613 · Cited by 243 · see citing works
The successor representation: Its computational logic and neural substrates
Gershman, S. J. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.0151-18.2018 · Cited by 191 · see citing works
Experience replay is associated with efficient nonlocal learning
Liu, Y., Mattar, M. G., Behrens, T. E. J., Daw, N. D., & Dolan, R. J. (2021) · Science · doi:10.1126/science.abf1357 · Cited by 187 · see citing works
Self-citationExpanding the scope of memory search: Modeling intralist and interlist effects in free recall.
Lohnas, L. J., Polyn, S. M., & Kahana, M. J. (2015) · Psychological Review · doi:10.1037/a0039036 · Cited by 177 · see citing works
Lieder, F., Griffiths, T. L., & Hsu, M. (2017) · Psychological Review · doi:10.1037/rev0000074 · Cited by 174 · see citing works
Reliance on small samples, the wavy recency effect, and similarity-based learning.
Plonsky, O., Teodorescu, K., & Erev, I. (2015) · Psychological Review · doi:10.1037/a0039413 · Cited by 162 · see citing works
Declarative memory is critical for sustained advantageous complex decision-making
Gupta, R., Duff, M. C., Denburg, N. L., Cohen, N. J., Bechara, A., & Tranel, D. (2009) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2009.02.007 · Cited by 145 · see citing works
Interresponse times in single-trial free recall.
Murdock, B. B., & Okada, R. (1970) · Journal of Experimental Psychology · doi:10.1037/h0029993 · Cited by 143 · see citing works
Offline replay supports planning in human reinforcement learning
Momennejad, I., Otto, A. R., Daw, N. D., & Norman, K. A. (2018) · eLife · doi:10.7554/elife.32548 · Cited by 133 · see citing works
Self-citationDissociable effects of surprising rewards on learning and memory.
Rouhani, N., Norman, K. A., & Niv, Y. (2018) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0000518 · Cited by 131 · see citing works
Associative Retrieval Processes in Episodic Memory
Kahana, M. J., Howard, M. W., & Polyn, S. M. (2008) · Elsevier eBooks · doi:10.1016/b978-012370509-9.00185-6 · Cited by 124 · see citing works
The successor representation and temporal context
Gershman, S. J., Moore, C. D., Todd, M. T., Norman, K. A., & Sederberg, P. B. (2012) · Neural Computation · doi:10.1162/neco_a_00282 · Cited by 117 · see citing works
Mattar, M. G., & Lengyel, M. (2022) · Neuron · doi:10.1016/j.neuron.2021.12.018 · Cited by 113 · see citing works
Self-citationTask context and organization in free recall
Polyn, S. M., Norman, K. A., & Kahana, M. J. (2009) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2009.02.013 · Cited by 92 · see citing works
Linear reinforcement learning in planning, grid fields, and cognitive control
Piray, P., & Daw, N. D. (2021) · Nature Communications · doi:10.1038/s41467-021-25123-3 · Cited by 83 · see citing works
Self-citationRecall termination in free recall
Miller, J. F., Weidemann, C. T., & Kahana, M. J. (2012) · Memory & Cognition · doi:10.3758/s13421-011-0178-9 · Cited by 50 · see citing works
The role of reward and reward uncertainty in episodic memory
Mason, A., Farrell, S., Howard‐Jones, P. A., & Ludwig, C. J. H. (2017) · Journal of Memory and Language · doi:10.1016/j.jml.2017.05.003 · Cited by 49 · see citing works
Fundamental Law of Memory Recall
Naim, M., Katkov, M., Romani, S., & Tsodyks, M. V. (2020) · Physical Review Letters · doi:10.1103/physrevlett.124.018101 · Cited by 41 · see citing works
Advantages and Limitations of using Successor Features for Transfer in Reinforcement LearningPreprint
Lehnert, L., Tellex, S., & Littman, M. L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.00102 · Cited by 38 · see citing works
Uncertainty alters the balance between incremental learning and episodic memory
Nicholas, J., Daw, N. D., & Daphna Shohamy, A. (2022) · eLife · doi:10.7554/elife.81679 · Cited by 38 · see citing works
Self-citationMedial Temporal Lobe Amnesia Is Associated with a Deficit in Recovering Temporal Context
Palombo, D. J., Lascio, J. M. D., Howard, M. W., & Verfaellie, M. H. (2018) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01344 · Cited by 30 · see citing works
Time spent thinking in online chess reflects the value of computation
Russek, E., Acosta-Kane, D., van Opheusden, B., Mattar, M. G., & Griffiths, T. (2022) · doi:10.31234/osf.io/8j9zx · Cited by 18 · see citing works
Self-citationMemory mechanisms predict sampling biases in sequential decision tasks
Mattar, M., Talmi, D., & Daw, N. (2018) · 2018 Conference on Cognitive Computational Neuroscience · doi:10.32470/ccn.2018.1164-0
Self-citationFree association in a neural network.
Richie, R., Aka, A., & Bhatia, S. (2023) · Psychological Review · doi:10.1037/rev0000396
Flagged first, then by citations
Research works printed but not linked: 4
- [6]6. Kumaran, D. & McClelland, J. L. (2012). Generalization through the recurrent interaction of episodic memories: A model of the hippocampal system. Psychological Review , 119 (3), 573 -616. https://doi.org/10.1037/ a0028681
- [8]8. Lengyel, M. & Dayan, P. (2007). Hippocampal contributions to control: The third way. In J. Platt, D. Koller, Y. Singer, & S. Roweis (Eds.), Advances in neural information processing systems (Vol. 20, pp. 889 -896). Curran Associates.
- [33]33. Pritzel, A., Uria, B., Srinivasan, S., Badia, A. P., Vinyals, O., Hassabis, D., Wierstra, D., & Blundell, C. (2017). Neural episodic control [Conference session]. Proceedings of the International Conference on Machine Learning.
- [35]35. Ritter, S., Wang, J., Kurth-Nelson, Z., Jayakumar, S., Blundell, C., Pascanu, R., & Botvinick, M. (2018). Been there, done that: Meta-learning with episodic recall [Conference session]. Proceedings of the International Conference on Machine Learning.
Cited by
Sorted by citations
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 72 · see citing works
Predictive Representations: Building Blocks of Intelligence
Carvalho, W., Tomov, M. S., de Cothi, W., Barry, C., & Gershman, S. J. (2024) · Neural Computation · doi:10.1162/neco_a_01705 · Cited by 15 · see citing works
Human-inspired Perspectives: A Survey on AI Long-term Memory
He, Z., Lin, W., Zheng, H., Zhang, F., Jones, M. W., Aitchison, L., Xu, X., Liu, M., Kristensson, P. O., & Shen, J. (2024) · arXiv · doi:10.48550/arxiv.2411.00489 · Cited by 3 · see citing works
“…the learning process, while data that reduces accuracy serves as a negative example. retrieval adapter learns to trigger memory retrieval in context [209, 210, 28], adjusts parameters for various scenarios [304], and selects relevant memory based on effectiveness for the target task [209].”
- Authors
- Zhou, C. Y., Talmi, D., Daw, N. D., & Mattar, M. G.
- Affiliations
- University of California San Diego (US)
- University of Cambridge (GB)
- Princeton University (US)
- Venue
- Psychological Review
- Published
- 2025-01
- Volume
- 132
- Issue
- 1
- Pages
- 18-49
- ISSN
- 0033-295X (ISSN-L)
- 1939-1471
- OpenAlex
- W4405906861
- Cited by
- 21
- References
- 37
- Citations by year
- 2026: 5 · 2025: 6 · 2024: 8 · 2023: 2
- FWCI
- 4.62
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Psychological Review (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Apollo (University of Cambridge) (submittedVersion, cc-by-nc-nd)
- PsyArXiv (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Funders
- National Science Foundation (IIS-1822571)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
- Keywords
- episodic memory, episodic memory retrieval, model-based decision making, sequential decision making, successor representation, mental simulation, emotional modulation, adaptive choice, memory generalization
- MeSH terms
- Decision Making, Humans, Models, Psychological, Mental Recall, Memory, Episodic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
