A universal role of the ventral striatum in reward-based learning: Evidence from human studies
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
Reinforcement learning enables organisms to adjust their behavior in order to maximize rewards. Electrophysiological recordings of dopaminergic midbrain neurons have shown that they code the difference between actual and predicted rewards, i.e., the reward prediction error, in many species. This error signal is conveyed to both the striatum and cortical areas and is thought to play a central role in learning to optimize behavior. However, in human daily life rewards are diverse and often only indirect feedback is available. Here we explore the range of rewards that are processed by the dopaminergic system in human participants, and examine whether it is also involved in learning in the absence of explicit rewards. While results from electrophysiological recordings in humans are sparse, evidence linking dopaminergic activity to the metabolic signal recorded from the midbrain and striatum with functional magnetic resonance imaging (fMRI) is available. Results from fMRI studies suggest that the human ventral striatum (VS) receives valuation information for a diverse set of rewarding stimuli. These range from simple primary reinforcers such as juice rewards over abstract social rewards to internally generated signals on perceived correctness, suggesting that the VS is involved in learning from trial-and-error irrespective of the specific nature of provided rewards. In addition, we summarize evidence that the VS can also be implicated when learning from observing others, and in tasks that go beyond simple stimulus-action-outcome learning, indicating that the reward system is also recruited in more complex learning tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“These responses are in line with the predictions of normative models of reinforcement learning (Montague et al., 1996; Schultz et al., 1997).”
ResultNeurophysiological investigation of the basis of the fMRI signal
Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001) · Nature · doi:10.1038/35084005 · Cited by 6,551
Brain magnetic resonance imaging with contrast dependent on blood oxygenation.
Ogawa, S., Lee, T. M., Kay, A. R., & Tank, D. W. (1990) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.87.24.9868 · Cited by 6,483
Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation.
Kwong, K. K., Belliveau, J. W., Chesler, D. A., Goldberg, I. E., Weisskoff, R. M., Poncelet, B. P., Kennedy, D. N., Hoppel, B. E., Cohen, M. S., & Turner, R. (1992) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.89.12.5675 · Cited by 4,212 · cited 1×
“…introduction in the early 1990s (Bandettini, Wong, Hinks, Tikofsky, & Hyde, 1992; Belliveau et al., 1991; Frahm, Bruhn, Merboldt, & H_nicke, 1992; Kwong, 1992; Ogawa, 1992; Ogawa, he ventral striatum in reward-based learning: Evidence from human…”
Ogawa, S., Tank, D. W., Menon, R., Ellermann, J. M., Kim, S. G., Merkle, H., & Ugurbil, K. (1992) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.89.13.5951 · Cited by 3,681
What we can do and what we cannot do with fMRI
Logothetis, N. K. (2008) · Nature · doi:10.1038/nature06976 · Cited by 3,612 · cited 3×
“…from such recordings suggest that the BOLD signal is slightly better explained by local field potentials than neuronal spiking activity (Goense & Logothetis, 2008; Kayser, Kim, Ugurbil, Kim, & K_nig, 2004; Logothetis, Pauls, Augath, Trinath, & Oeltermann, 2001; Mukamel et al., 2005; Raichle &…”
BackgroundOlds, J., & Milner, P. (1954) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0058775 · Cited by 3,176 · cited 1×
“The seminal experiments of Olds and Milner (1954) were a first step towards identifying a system in the mammalian brain that is dedicated to processing motivational information.”
BackgroundGetting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396 · cited 1×
“…rate of responding (Niv, Daw, Joel, & Dayan, 2007), and with signaling different aspects of salience, including appetitive and aversive information (Bromberg-Martin et al., 2010; Schultz, 2007), novelty (Bunzeck & D_zel, 2006; Ljungberg, 1992; Wittmann, Bunzeck, Dolan, & D_zel, 2007), and…”
The debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
Cited by
The Neuroscience of Drug Reward and Addiction
Volkow, N. D., Michaelides, M., & Baler, R. (2019) · Physiological Reviews · doi:10.1152/physrev.00014.2018 · Cited by 863
The striatum: where skills and habits meet
Graybiel, A. M., & Grafton, S. T. (2015) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a021691 · Cited by 463
Common brain mechanisms of chronic pain and addiction
Elman, I., & Borsook, D. (2016) · Neuron · doi:10.1016/j.neuron.2015.11.027 · Cited by 303
Emmert, K., Kopel, R., Sulzer, J., Brühl, A. B., Berman, B. D., Linden, D. E. J., Horovitz, S. G., Breimhorst, M., Caria, A., Frank, S., et al. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2015.09.042 · Cited by 245
The brain’s reward system in health and disease
Lewis, R. G., Florio, E., Punzo, D., & Borrelli, E. (2021) · Advances in experimental medicine and biology · doi:10.1007/978-3-030-81147-1_4 · Cited by 170
“Nevertheless, the NAcc is most appreciated for its involvement in reward processing and its role in evaluation and incentive-based learning (Schultz et al. 1992; Daniel and Pollmann 2014).”
BackgroundPfeiffer, U. J., Schilbach, L., Timmermans, B., Kuzmanovic, B., Georgescu, A. L., Bente, G., & Vogeley, K. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.06.061 · Cited by 155
Anterior insula reflects surprise in value-based decision-making and perception
Loued-Khenissi, L., Pfeuffer, A., Einhäuser, W., & Preuschoff, K. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116549 · Cited by 66
Abnormal social reward responses in anorexia nervosa: an fMRI study
Via, E., Soriano-Mas, C., Sánchez, I., Forcano, L., Harrison, B. J., Davey, C. G., Pujol, J., Martínez-Zalacaín, I., Menchón, J. M., Fernández-Aranda, F., et al. (2015) · PLoS ONE · doi:10.1371/journal.pone.0133539 · Cited by 56
Greene, R. K., Spanos, M., Alderman, C., Walsh, E., Bizzell, J., Mosner, M. G., Kinard, J. L., Stuber, G. D., Chandrasekhar, T., Politte, L. C., et al. (2018) · Journal of Neurodevelopmental Disorders · doi:10.1186/s11689-018-9228-y · Cited by 55
Sterpenich, V., Vidal, S., Hofmeister, J., Michalopoulos, G., Bancila, V., Warrot, D., Dayer, A., Desseilles, M., Aubry, J.-M., Kosel, M., et al. (2019) · Anesthesiology · doi:10.1097/aln.0000000000002667 · Cited by 53
Intelligence mindset shapes neural learning signals and memory
Bejjani, C., DePasque, S., & Tricomi, E. (2019) · Biological Psychology · doi:10.1016/j.biopsycho.2019.06.003 · Cited by 19
Metadata
- Authors
- Daniel, R., & Pollmann, S.
- Venue
- Neurobiology of Learning and Memory
- Publisher
- Elsevier BV
- Published
- 2014-10
- Volume
- 114
- Pages
- 90-100
- ISSN
- 1074-7427 (ISSN-L)
- OpenAlex
- W2019014607
- Cited by
- 208
- References
- 75
- FWCI
- 4.03
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.