Hippocampal replays under the scrutiny of reinforcement learning models
Open accessThe DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
Multiple in vivo studies have shown that place cells from the hippocampus replay previously experienced trajectories. These replays are commonly considered to mainly reflect memory consolidation processes. Some data, however, have highlighted a functional link between replays and reinforcement learning (RL). This theory, extensively used in machine learning, has introduced efficient algorithms and can explain various behavioral and physiological measures from different brain regions. RL algorithms could constitute a mechanistic description of replays and explain how replays can reduce the number of iterations required to explore the environment during learning. We review the main findings concerning the different hippocampal replay types and the possible associated RL models (either model-based, model-free, or hybrid model types). We conclude by tying these frameworks together. We illustrate the link between data and RL through a series of model simulations. This review, at the frontier between informatics and biology, paves the way for future work on replays.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A logical calculus of the ideas immanent in nervous activity
McCulloch, W. S., & Pitts, W. (1943) · The Bulletin of Mathematical Biophysics · doi:10.1007/bf02478259 · Cited by 18,566
Cognitive maps in rats and men
Tolman, E. C. (1948) · Psychological Review · doi:10.1037/h0061626 · Cited by 6,641
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 2×
“This is because in most decision-making tasks in neuroscience, the inference phase of MF-RL simply consists of reading from a table the Q values of a small finite set of actions, to compare them and make a decision (Bavard et al. 2018; O’Doherty et al. 2004; Palminteri et al. 2015; Pessiglione et al. 2006).”
BackgroundMethodTaube, J., Muller, R., & Ranck, J. (1990) · The Journal of Neuroscience · doi:10.1523/jneurosci.10-02-00420.1990 · Cited by 1,809
Hippocampal sharp wave-ripple: A cognitive biomarker for episodic memory and planning
Buzsáki, G. (2015) · Hippocampus · doi:10.1002/hipo.22488 · Cited by 1,759
Reverse replay of behavioural sequences in hippocampal place cells during the awake state
Foster, D. J., & Wilson, M. A. (2006) · Nature · doi:10.1038/nature04587 · Cited by 1,727
Two-stage model of memory trace formation: A role for “noisy” brain states
Buzsáki, G. (1989) · Neuroscience · doi:10.1016/0306-4522(89)90423-5 · Cited by 1,709 · cited 1×
“What could be the role of these reactivations? Following the proposed role of SWRs in the two-stage memory model (Buzsáki, 1989), researchers have mostly interpreted replays as a sign of a consolidation of memory (Chen and Wilson, 2017; Diekelmann and Born, 2010; Ólafsdóttir et al.”
BackgroundCoordinated memory replay in the visual cortex and hippocampus during sleep
Ji, D., & Wilson, M. A. (2007) · Nature Neuroscience · doi:10.1038/nn1825 · Cited by 1,669 · cited 2×
“, 2018; Walker and Stickgold, 2006): they would copy volatile memories for long-term storage into the cortex (Ji and Wilson, 2007; McClelland et al., 1995; Peyrache et al., 2009).”
Background
Cited by
Shin, J. D., Tang, W., & Jadhav, S. P. (2019) · Neuron · doi:10.1016/j.neuron.2019.09.012 · Cited by 199
“The results of this study also connect well to machine learning literature based on hippocampal replay (Caze et al., 2018; Mattar and Daw, 2018), and thus may inspire improved algorithms.”
ResultLearning structures: Predictive representations, replay, and generalization
Momennejad, I. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.02.017 · Cited by 159
Replay in Deep Learning: Current Approaches and Missing Biological Elements
Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski, T. J., & Kanan, C. (2021) · Neural Computation · doi:10.1162/neco_a_01433 · Cited by 106
“…further suggests that selective and partial replay in tasks involving reward allow humans to perform generalization and model-based reinforcement learning (Momennejad, 2020), which has inspired several algorithms in machine learning (see Cazé, Khamassi, Aubin, & Girard, 2018, for a review).”
Metadata
- Authors
- Cazé, R., Khamassi, M., Aubin, L., & Girard, B.
- Venue
- Journal of Neurophysiology
- Publisher
- American Physiological Society
- Published
- 2018-12-01
- Volume
- 120
- Issue
- 6
- Pages
- 2877-2896
- ISSN
- 0022-3077 (ISSN-L)
- 1522-1598
- OpenAlex
- W2895961990
- Cited by
- 38
- References
- 45
- FWCI
- 3.18
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Hippocampal replays under the scrutiny of reinforcement learning models. https://app.cochat.ai/papers/10.1152/jn.00145.2018
- The paper
- Cazé, R., Khamassi, M., Aubin, L., & Girard, B. (2018). Hippocampal replays under the scrutiny of reinforcement learning models. Journal of Neurophysiology. https://doi.org/10.1152/jn.00145.2018
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.