Experience replay is associated with efficient nonlocal learning
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Replay supports planning: Learning from direct experience is easy—we can always use trial and error—but how do we learn from nondirect (nonlocal) experiences? For this, we need additional mechanisms that bridge time and space. In rodents, hippocampal replay is hypothesized to promote this function. Liu et al. measured high-temporal-resolution brain signals using human magnetoencephalography combined with a new model-based, visually oriented, multipath reinforcement memory task. This task was designed to differentiate local versus nonlocal learning episodes within the subject. They found that reverse sequential replay in the human medial temporal lobe supports nonlocal reinforcement learning and is the underlying mechanism for solving complex credit assignment problems such as value learning. Science , abf1357, this issue p. eabf1357
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Theta oscillations in the hippocampus
Buzsáki, G. (2002) · Neuron · doi:10.1016/s0896-6273(02)00586-x · Cited by 3,332
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citationReverse replay of behavioural sequences in hippocampal place cells during the awake state
Foster, D. J., & Wilson, M. A. (2006) · Nature · doi:10.1038/nature04587 · Cited by 1,727
Selective suppression of hippocampal ripples impairs spatial memory
Girardeau, G., Benchenane, K., Wiener, S. I., Buzsáki, G., & Zugaro, M. B. (2009) · Nature Neuroscience · doi:10.1038/nn.2384 · Cited by 1,560
Hippocampal place-cell sequences depict future paths to remembered goals
Pfeiffer, B. E., & Foster, D. J. (2013) · Nature · doi:10.1038/nature12112 · Cited by 1,216
Forward and reverse hippocampal place-cell sequences during ripples
Diba, K., & Buzsáki, G. (2007) · Nature Neuroscience · doi:10.1038/nn1961 · Cited by 1,205
Cited by
Spatial goal coding in the hippocampal formation
Nyberg, N., Duvelle, É., Barry, C., & Spiers, H. J. (2022) · Neuron · doi:10.1016/j.neuron.2021.12.012 · Cited by 141
Predictive maps in rats and humans for spatial navigation
de Cothi, W., Nyberg, N., Griesbauer, E.-M., Ghanamé, C., Zisch, F., Lefort, J. M., Fletcher, L., Newton, C., Renaudineau, S., Bendor, D., et al. (2022) · Current Biology · doi:10.1016/j.cub.2022.06.090 · Cited by 109
From remembering to reconstruction: The transformative neural representation of episodic memory.
Xue, G. (2022) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2022.102351 · Cited by 49
Episodic retrieval for model-based evaluation in sequential decision tasks.
Zhou, C. Y., Talmi, D., Daw, N. D., & Mattar, M. G. (2025) · Psychological Review · doi:10.1037/rev0000505 · Cited by 21
Self-citationEpisodic retrieval for model-based evaluation in sequential decision tasks
Zhou, C. Y., Talmi, D., Daw, N., & Mattar, M. G. (2023) · doi:10.31234/osf.io/3sqjh · Cited by 21
Self-citationOur brains sense the future through a new quantum-like implicit learning mechanism
Escolà-Gascón, Á. (2024) · Brain Research Bulletin · doi:10.1016/j.brainresbull.2024.111048 · Cited by 3
DeKraker, J., Eichert, N., Hong, S.-J., Evans, A. C., & Bernhardt, B. C. (2026) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2026.106890 · Cited by 1
Metadata
- Authors
- Liu, Y., Mattar, M. G., Behrens, T. E. J., Daw, N. D., & Dolan, R. J.
- Venue
- Science
- Published
- 2021-05-21
- Volume
- 372
- Issue
- 6544
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W3163789885
- Cited by
- 184
- References
- 22
- FWCI
- 15.86
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.