Changes in corticostriatal connectivity during reinforcement learning in humans
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Many computational models assume that reinforcement learning relies on changes in synaptic efficacy between cortical regions representing stimuli and striatal regions involved in response selection, but this assumption has thus far lacked empirical support in humans. We recorded hemodynamic signals with fMRI while participants navigated a virtual maze to find hidden rewards. We fitted a reinforcement‐learning algorithm to participants' choice behavior and evaluated the neural activity and the changes in functional connectivity related to trial‐by‐trial learning variables. Activity in the posterior putamen during choice periods increased progressively during learning. Furthermore, the functional connections between the sensorimotor cortex and the posterior putamen strengthened progressively as participants learned the task. These changes in corticostriatal connectivity differentiated participants who learned the task from those who did not. These findings provide a direct link between changes in corticostriatal connectivity and learning, thereby supporting a central assumption common to several computational models of reinforcement learning. Hum Brain Mapp 36:793–803, 2015. © 2014 Wiley Periodicals, Inc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 3×
“…S-R learning [McDonald and Hong, 2004; Packard et al., 1989], as well as the remarkable fit between our findings and the predictions of “modelfree RL” [Daw et al., 2005] models such as the actor-critic (which work by S-R learning), provide some support our interpretation in terms of S-R learning.”
BackgroundResultInfluentialThe role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 1×
“More generally, our findings support the parallels between the dorsal striatum and the “actor” in the actor-critic model of action selection [Barto, 1995; O’Doherty et al., 2004].”
ResultA neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777
Balleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775
Learning and Memory Functions of the Basal Ganglia
Packard, M. G., & Knowlton, B. J. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142937 · Cited by 1,717 · cited 2×
“We used fMRI based on computational modeling (i.e., computational fMRI) [O’Doherty et al., 2007] to study changes in corticostriatal connectivity as human participants performed a task analogous to the “win-stay” version of a radial-arm maze task [Packard et al., 1989; Packard and Knowlton, 2002].”
MethodSelf-citationHaber, S. N., Fudge, J. L., & McFarland, N. R. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-06-02369.2000 · Cited by 1,610 · cited 4×
“Consistent with the known anatomy of the CBGTC sensorimotor loop [Draganski et al., 2008; Haber et al., 2000; Lehericy et al., 2004], we found that the posterolateral putamen is selectively connected to the premotor cortex-SMA, lateral thalamus, and sensory cortex.”
BackgroundInfluentialDopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · cited 1×
“Such coding of neutral outcomes relative to overall task expectations has been previously demonstrated [O’Doherty et al., 2003].”
BackgroundRapid formation and selective stabilization of synapses for enduring motor memories
Xu, T., Yu, X., Perlik, A. J., Tobin, W. F., Zweig, J. A., Tennant, K., Jones, T., & Zuo, Y. (2009) · Nature · doi:10.1038/nature08389 · Cited by 1,193 · cited 2×
“Because rapid changes in synaptic efficacy accompany learning-related plasticity [Xu et al., 2009], learning likely modifies the functional coupling between presynaptic and postsynaptic neurons—a phenomenon that can be studied using functional-connectivity tools in fMRI.”
Background
Cited by
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002
“Resource-light MF systems conversely learn values by iteratively updating them with prediction errors through experience, and are thought to capture habits and rely on sensorimotor CSTC loops 98,105–108 .”
BackgroundSelf-citationLearning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475
“Others have reported putative model-free value signals in the posterior putamen (Horga et al. 2015).”
BackgroundEvolutionary diversity as a catalyst for biological discovery
JOHNSON, Z. V., & YOUNG, L. J. (2018) · Integrative Zoology · doi:10.1111/1749-4877.12339 · Cited by 25
Metadata
- Authors
- Horga, G., Maia, T. V., Marsh, R., Hao, X., Xu, D., Duan, Y., Tau, G. Z., Graniello, B., Wang, Z., Kangarlu, A., Martinez, D., Packard, M. G., & Peterson, B. S.
- Venue
- Human Brain Mapping
- Publisher
- Wiley
- Published
- 2014-11-12
- Volume
- 36
- Issue
- 2
- Pages
- 793-803
- ISSN
- 1065-9471 (ISSN-L)
- 1097-0193
- OpenAlex
- W1580587320
- Cited by
- 43
- References
- 23
- FWCI
- 2.21
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Changes in corticostriatal connectivity during reinforcement learning in humans. https://app.cochat.ai/papers/10.1002/hbm.22665
- The paper
- Horga, G., Maia, T. V., Marsh, R., Hao, X., Xu, D., Duan, Y., Tau, G. Z., Graniello, B., Wang, Z., Kangarlu, A., Martinez, D., Packard, M. G., & Peterson, B. S. (2015). Changes in corticostriatal connectivity during reinforcement learning in humans. Human Brain Mapping. https://doi.org/10.1002/hbm.22665
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.