The DOI matches its Crossref record; 107 of the 107 references it cites were found in a registry or an index.
Generalization of value in reinforcement learning by humans
Open accessAbstract
Research in decision‐making has focused on the role of dopamine and its striatal targets in guiding choices via learned stimulus–reward or stimulus–response associations, behavior that is well described by reinforcement learning theories. However, basic reinforcement learning is relatively limited in scope and does not explain how learning about stimulus regularities or relations may guide decision‐making. A candidate mechanism for this type of learning comes from the domain of memory, which has highlighted a role for the hippocampus in learning of stimulus–stimulus relations, typically dissociated from the role of the striatum in stimulus–response learning. Here, we used functional magnetic resonance imaging and computational model‐based analyses to examine the joint contributions of these mechanisms to reinforcement learning. Humans performed a reinforcement learning task with added relational structure, modeled after tasks used to isolate hippocampal contributions to memory. On each trial participants chose one of four options, but the reward probabilities for pairs of options were correlated across trials. This (uninstructed) relationship between pairs of options potentially enabled an observer to learn about option values based on experience with the other options and to generalize across them. We observed blood oxygen level‐dependent (BOLD) activity related to learning in the striatum and also in the hippocampus. By comparing a basic reinforcement learning model to one augmented to allow feedback to generalize between correlated options, we tested whether choice behavior and BOLD activity were influenced by the opportunity to generalize across correlated options. Although such generalization goes beyond standard computational accounts of reinforcement learning and striatal BOLD, both choices and striatal BOLD activity were better explained by the augmented model. Consistent with the hypothesized role for the hippocampus in this generalization, functional connectivity between the ventral striatum and hippocampus was modulated, across participants, by the ability of the augmented model to capture participants’ choice. Our results thus point toward an interactive model in which striatal reinforcement learning systems may employ relational representations typically associated with the hippocampus.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As expected, in contradistinction to the main study, appetitive PEs to monetary rewards were tracked by activity in the VS. (t(20) = 5.77, p < 0.001), but not the PAG (PAG–appetitive: t(20) = 1.54, p = 0.14; see Supplementary Fig. 2).”
From Representation of aversive prediction errors in the human periaqueductal gray (2014) · cites this paper as [23]
In the citing paper
“As expected, in contradistinction to the main study, appetitive PEs to monetary rewards were tracked by activity in the VS. (t(20) = 5.77, p < 0.001), but not the PAG (PAG–appetitive: t(20) = 1.54, p = 0.14; see Supplementary Fig. 2).”
From Representation of aversive prediction errors in the human periaqueductal gray (2014) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 20,012
Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., & Joliot, M. (2002) · NeuroImage · doi:10.1006/nimg.2001.0978 · Cited by 16,319
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Cragg, J. G., & Zarembka, P. (1975) · Canadian Journal of Economics/Revue canadienne d économique · doi:10.2307/134075 · Cited by 8,477
Memory and the hippocampus: A synthesis from findings with rats, monkeys, and humans.
Squire, L. R. (1992) · Psychological Review · doi:10.1037/0033-295x.99.2.195 · Cited by 5,376
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785
Psychophysiological and Modulatory Interactions in Neuroimaging
Friston, K. J., Buechel, C., Fink, G. R., Morris, J., Rolls, E., & Dolan, R. J. (1997) · NeuroImage · doi:10.1006/nimg.1997.0291 · Cited by 3,097
Research works printed but not linked: 2
- Eichenbaum H., 2001, From Conditioning to Conscious Recollection: Memory Systems of the Brain
- Thorndike E.L., 1911, Animal Intelligence
Cited by
Sorted by citations
Informatic parcellation of the network involved in the computation of subjective value
Clithero, J. A., & Rangel, A. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst106 · Cited by 706
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380
Self-citationProcess Account of Curiosity and Interest: A Reward-Learning Perspective
Murayama, K., FitzGibbon, L., & Sakaki, M. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09499-9 · Cited by 334
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280
Representation of aversive prediction errors in the human periaqueductal gray
Roy, M., Shohamy, D., Daw, N., Jepma, M., Wimmer, G. E., & Wager, T. D. (2014) · Nature Neuroscience · doi:10.1038/nn.3832 · Cited by 246
Self-citation
Metadata
- Authors
- Wimmer, G. E., Daw, N. D., & Shohamy, D.
- Affiliations
- Columbia University (US)
- New York University (US)
- Venue
- European Journal of Neuroscience
- Publisher
- Wiley
- Published
- 2012-04
- Volume
- 35
- Issue
- 7
- Pages
- 1092-1104
- ISSN
- 0953-816X (ISSN-L)
- 1460-9568
- PMC
- PMC3404618
- OpenAlex
- W1535087671
- Cited by
- 111
- References
- 107
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 7 · 2023: 8 · 2022: 9 · 2021: 9 · 2020: 8 · 2019: 11 · 2018: 9 · 2017: 1 · 2016: 14 · 2015: 5 · 2014: 9 · 2013: 6 · 2012: 5
- FWCI
- 4.09
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- European Journal of Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health (1r03da026957-01)
- National Institute on Drug Abuse (R03-DA026957)
- National Institute of Neurological Disorders and Stroke (R01NS078784)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- reinforcement learning, hippocampus, ventral striatum, relational learning, functional connectivity, computational modeling, choice behavior, reward probability, dopamine
- MeSH terms
- Adolescent, Adult, Basal Ganglia, Female, Generalization, Psychological, Humans, Learning, Male, Photic Stimulation, Psychomotor Performance, Reinforcement, Psychology, Reward, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
