Reinforcement Learning Signals in the Human Striatum Distinguish Learners from Nonlearners during Reward-Based Decision Making
Open accessAbstract
The computational framework of reinforcement learning has been used to forward our understanding of the neural mechanisms underlying reward learning and decision-making behavior. It is known that humans vary widely in their performance in decision-making tasks. Here, we used a simple four-armed bandit task in which subjects are almost evenly split into two groups on the basis of their performance: those who do learn to favor choice of the optimal action and those who do not. Using models of reinforcement learning we sought to determine the neural basis of these intrinsic differences in performance by scanning both groups with functional magnetic resonance imaging. We scanned 29 subjects while they performed the reward-based decision-making task. Our results suggest that these two groups differ markedly in the degree to which reinforcement learning signals in the striatum are engaged during task performance. While the learners showed robust prediction error signals in both the ventral and dorsal striatum during learning, the nonlearner group showed a marked absence of such signals. Moreover, the magnitude of prediction error signals in a region of dorsal striatum correlated significantly with a measure of behavioral performance across all subjects. These findings support a crucial role of prediction error signals, likely originating from dopaminergic midbrain neurons, in enabling learning of action selection preferences on the basis of obtained rewards. Thus, spontaneously observed individual differences in decision making performance demonstrate the suggested dependence of this type of learning on the functional integrity of the dopaminergic striatal system in humans.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The temporal-difference learning model fitted participants' choices with learning rate α=0.63±0.26, softmax inverse temperature β=4.74±2.74, and negative log likelihood 65±21, comparable to similar reinforcement-learning studies.
“The analysis revealed learning rates (α=0.63±0.26), softmax inverse temperatures (β=4.74±2.74), and model fits (negative log likelihood = 65 ± 21) comparable to those found in similar studies of reinforcement learning20.”
From Representation of aversive prediction errors in the human periaqueductal gray (2014) · cites this paper as [20]
People are more likely to stick with an advised choice in the face of negative outcomes.
“Much like people will stick to a decision when they believe the odds are higher 25, people are more likely to stick with an advised choice in the face of negative outcomes 53.”
From The social brain and reward: Social information processing in the human striatum (2014) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
An inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Individual differences in reasoning: Implications for the rationality debate?
Stanovich, K. E., & West, R. F. (2000) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00003435 · Cited by 4,255 · see citing works
Normal personality assessment in clinical practice: the NEO personality inventory
Costa, P. T., & McCrae, R. R. (1992) · Psychological Assessment · doi:10.1037/1040-3590.4.1.5 · Cited by 3,414 · see citing works
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan, P., & Abbott, L. F. (2001) · Cited by 3,153 · see citing works
The Obsessive-Compulsive Inventory: Development and validation of a short version.
Foa, E. B., Huppert, J. D., Leiberg, S., Langner, R., Kichic, R., Hajcak, G., & Salkovskis, P. M. (2002) · Psychological Assessment · doi:10.1037/1040-3590.14.4.485 · Cited by 2,959 · see citing works
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574 · see citing works
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · see citing works
Self-citationBarto, A. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,172 · see citing works
(2012) · Springer eBooks · doi:10.1007/springerreference_179754 · Cited by 2,095 · see citing works
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · see citing works
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584 · see citing works
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · see citing works
Self-citationReward representations and reward-related learning in the human brain: Insights from neuroimaging
O’Doherty, J. P. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.10.016 · Cited by 1,426 · see citing works
Self-citationFMRI visualization of brain activity during a monetary incentive delay task
Knutson, B., Westdorp, A., Kaiser, E., & Hommer, D. (2000) · NeuroImage · doi:10.1006/nimg.2000.0593 · Cited by 1,414 · see citing works
Tracking the Hemodynamic Responses to Reward and Punishment in the Striatum
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.6.3072 · Cited by 1,116 · see citing works
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · see citing works
Reward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855 · see citing works
Modulation of Caudate Activity by Action Contingency
Tricomi, E. M., Delgado, M. R., & Fiez, J. A. (2004) · Neuron · doi:10.1016/s0896-6273(03)00848-1 · Cited by 575 · see citing works
Is the short-latency dopamine response too short to signal reward error?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01373-3 · Cited by 506 · see citing works
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503 · see citing works
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500 · see citing works
Self-citationDYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485 · see citing works
The neural substrates of reward processing in humans: the modern role of FMRI
McClure, S. M., York, M. K., & Montague, P. R. (2004) · The Neuroscientist · doi:10.1177/1073858404263526 · Cited by 480 · see citing works
The computational neurobiology of learning and reward
Daw, N. D., & Doya, K. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.006 · Cited by 462 · see citing works
Self-citationActivity in human ventral striatum locked to errors of reward prediction
Pagnoni, G., Zink, C. F., Montague, P. R., & Berns, G. S. (2002) · Nature Neuroscience · doi:10.1038/nn802 · Cited by 451 · see citing works
The Roles of the Caudate Nucleus in Human Classification Learning
Seger, C. A., & Cincotta, C. M. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.3401-04.2005 · Cited by 403 · see citing works
Neural signature of fictive learning signals in a sequential investment task
Lohrenz, T. M., McCabe, K., Camerer, C. F., & Montague, P. R. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0608842104 · Cited by 263 · see citing works
Chemistry of the adaptive mind
Cools, R., & Robbins, T. W. (2004) · Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2004.1468 · Cited by 257 · see citing works
Representation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173 · see citing works
Self-citationHuman striatal activation reflects degree of stimulus saliency
Zink, C. F., Pagnoni, G., Chappelow, J., Martin-Skurski, M. E., & Berns, G. S. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.08.006 · Cited by 169 · see citing works
Newell, B. R. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.11.005 · Cited by 160 · see citing works
The Wisconsin Personality Disorders Inventory: Development, Reliability, and Validity
Klein, M. H., Benjamin, L. S., Rosenfeld, R., Treece, C., Husted, J. J., & Greist, J. H. (1993) · Journal of Personality Disorders · doi:10.1521/pedi.1993.7.4.285 · Cited by 147 · see citing works
Is probability matching smart? Associations between probabilistic choices and cognitive ability
West, R. F., & Stanovich, K. E. (2003) · Memory & Cognition · doi:10.3758/bf03194383 · Cited by 143 · see citing works
TD models of reward predictive responses in dopamine neurons
Suri, R. E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00046-1 · Cited by 137 · see citing works
Ventral–striatal/nucleus–accumbens sensitivity to prediction errors during classification learning
Rodriguez, P. F., Aron, A. R., & Poldrack, R. A. (2006) · Human Brain Mapping · doi:10.1002/hbm.20186 · Cited by 122 · see citing works
Joel, D., Zohar, O., Afek, M., Hermesh, H., LERNER, L., Kuperman, R. A., Gross‐Isseroff, R., Weizman, A., & Inzelberg, R. (2004) · Behavioural Brain Research · doi:10.1016/j.bbr.2004.07.006 · Cited by 67 · see citing works
Self-citationTesting the reference-dependent model: an experiment on asymmetrically dominated reference points
Herne, K. (1998) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199809)11:3 · Cited by 41 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779 · see citing works
Self-citationBalleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775 · see citing works
Self-citationLiu, X., Hairston, J., Schrier, M., & Fan, J. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.12.012 · Cited by 916 · see citing works
From prediction error to incentive salience: mesolimbic computation of reward motivation.
Berridge, K. C. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.07990.x · Cited by 700 · see citing works
Reinforcement Learning Signal Predicts Social Conformity
Klucharev, V., Hytönen, K., Rijpkema, M., Smidts, A., & Fernández, G. (2009) · Neuron · doi:10.1016/j.neuron.2008.11.027 · Cited by 594 · see citing works
Learning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475 · see citing works
Self-citationA computational and neural model of momentary subjective well-being
Rutledge, R. B., Skandali, N., Dayan, P., & Dolan, R. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1407535111 · Cited by 444 · see citing works
From Fear to Safety and Back: Reversal of Fear in the Human Brain
Schiller, D., Levy, I., Niv, Y., LeDoux, J. E., & Phelps, E. A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2265-08.2008 · Cited by 429 · see citing works
Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain
Niv, Y., Edlund, J. A., Dayan, P., & O'Doherty, J. P. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5498-10.2012 · Cited by 356 · see citing works
Self-citationReinforcement Learning in Multidimensional Environments Relies on Attention Mechanisms
Niv, Y., Daniel, R., Geana, A., Gershman, S. J., Leong, Y. C., Radulescu, A., & Wilson, R. C. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.2978-14.2015 · Cited by 362 · see citing works
How the Opinion of Others Affects Our Valuation of Objects
Campbell-Meiklejohn, D. K., Bach, D. R., Roepstorff, A., Dolan, R. J., & Frith, C. D. (2010) · Current Biology · doi:10.1016/j.cub.2010.04.055 · Cited by 339 · see citing works
The role of the striatum in aversive learning and aversive prediction errors
Delgado, M. R., Li, J., Schiller, D., & Phelps, E. A. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0161 · Cited by 328 · see citing works
Leotti, L. A., & Delgado, M. R. (2011) · Psychological Science · doi:10.1177/0956797611417005 · Cited by 318 · see citing works
Reward, Addiction, and Emotion Regulation Systems Associated With Rejection in Love
Fisher, H. E., Brown, L. L., Aron, A., Strong, G., & Mashek, D. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00784.2009 · Cited by 328 · see citing works
van den Bos, W., Cohen, M. X., Kahnt, T., & Crone, E. A. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr198 · Cited by 276 · see citing works
The effects of psychotherapy on neural responses to rewards in major depression
Dichter, G. S., Felder, J. N., Petty, C., Bizzell, J., Ernst, M., & Smoski, M. J. (2009) · Biological Psychiatry · doi:10.1016/j.biopsych.2009.06.021 · Cited by 278 · see citing works
Kokal, I., Engel, A., Kirschner, S., & Keysers, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027272 · Cited by 254 · see citing works
The social brain and reward: Social information processing in the human striatum
Bhanji, J. P., & Delgado, M. R. (2014) · WIREs Cognitive Science · doi:10.1002/wcs.1266 · Cited by 256 · see citing works
Representation of aversive prediction errors in the human periaqueductal gray
Roy, M., Shohamy, D., Daw, N., Jepma, M., Wimmer, G. E., & Wager, T. D. (2014) · Nature Neuroscience · doi:10.1038/nn.3832 · Cited by 246 · see citing works
Self-citationKrugel, L. K., Biele, G., Mohr, P. N. C., Li, S.-C., & Heekeren, H. R. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0905191106 · Cited by 229 · see citing works
Behavioral and Neural Properties of Social Reinforcement Learning
Jones, R. M., Somerville, L. H., Li, J., Ruberry, E. J., Libby, V., Glover, G., Voss, H. U., Ballon, D. J., & Casey, B. J. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.2972-11.2011 · Cited by 191 · see citing works
Dopaminergic Genes Predict Individual Differences in Susceptibility to Confirmation Bias
Doll, B. B., Hutchison, K. E., & Frank, M. J. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6486-10.2011 · Cited by 184 · see citing works
Social Network Modulation of Reward-Related Signals
Fareri, D. S., Niznikiewicz, M. A., Lee, V. K., & Delgado, M. R. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.0610-12.2012 · Cited by 175 · see citing works
Effects of Direct Social Experience on Trust Decisions and Neural Reward Circuitry
Fareri, D. S., Chang, L. J., & Delgado, M. R. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00148 · Cited by 147 · see citing works
Reward-related processing in the human brain: developmental considerations
Fareri, D. S., Martin, L. N., & Delgado, M. R. (2008) · Development and Psychopathology · doi:10.1017/s0954579408000576 · Cited by 141 · see citing works
Reputational Priors Magnify Striatal Responses to Violations of Trust
Fouragnan, E., Chierchia, G., Greiner, S., Neveu, R., Avesani, P., & Coricelli, G. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3086-12.2013 · Cited by 128 · see citing works
Neural mechanisms of reinforcement learning in unmedicated patients with major depressive disorder
Rothkirch, M., Tonn, J., Köhler, S., & Sterzer, P. (2017) · Brain · doi:10.1093/brain/awx025 · Cited by 124 · see citing works
Generalization of value in reinforcement learning by humans
Wimmer, G. E., Daw, N. D., & Shohamy, D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08017.x · Cited by 111 · see citing works
Self-citationZiegler, S., Pedersen, M. L., Mowinckel, A. M., & Biele, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.002 · Cited by 110 · see citing works
Gershman, S. J., Pesaran, B., & Daw, N. D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2469-09.2009 · Cited by 118 · see citing works
Self-citationUsing fMRI to study reward processing in humans: past, present, and future
Wang, K. S., Smith, D. V., & Delgado, M. R. (2016) · Journal of Neurophysiology · doi:10.1152/jn.00333.2015 · Cited by 105 · see citing works
The Influence of Emotion Regulation on Decision-making under Risk
Martin, L. N., & Delgado, M. R. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2011.21618 · Cited by 105 · see citing works
Mind matters: placebo enhances reward learning in Parkinson's disease
Schmidt, L., Braun, E. K., Wager, T. D., & Shohamy, D. (2014) · Nature Neuroscience · doi:10.1038/nn.3842 · Cited by 84 · see citing works
Sex differences in the functional connectivity of the amygdalae in association with cortisol
Kogler, L., Müller, V. I., Seidel, E.-M., Boubela, R., Kalcher, K., Moser, E., Habel, U., Gur, R. C., Eickhoff, S. B., & Derntl, B. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.03.064 · Cited by 78 · see citing works
Striatal structure and function predict individual biases in learning to avoid pain
Eldar, E., Hauser, T. U., Dayan, P., & Dolan, R. J. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1519829113 · Cited by 79 · see citing works
Dissociable reward and timing signals in human midbrain and ventral striatum
Klein-Flügge, M. C., Hunt, L. T., Bach, D. R., Dolan, R. J., & Behrens, T. E. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.024 · Cited by 75 · see citing works
Jarcho, J. M., Romer, A. L., Shechner, T., Galvan, A., Guyer, A. E., Leibenluft, E., Pine, D. S., & Nelson, E. E. (2015) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2015.03.002 · Cited by 70 · see citing works
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6 · see citing works
Self-citationAddressing altered anticipation as a transdiagnostic target through computational psychiatry
Sepúlveda, P., Aitsahalia, I., Kumar, K., Atkin, T., & Iigaya, K. (2025) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2025.02.014 · Cited by 3 · see citing works
- Authors
- Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P.
- Affiliations
- Tel Aviv University (IL)
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2007-11-21
- Volume
- 27
- Issue
- 47
- Pages
- 12860-12867
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- PMC
- PMC6673291
- OpenAlex
- W2004014356
- Cited by
- 390
- References
- 39
- Citations by year
- 2026: 7 · 2025: 12 · 2024: 13 · 2023: 8 · 2022: 14 · 2021: 16 · 2020: 10 · 2019: 52 · 2018: 22 · 2017: 24 · 2016: 21 · 2015: 29 · 2014: 29 · 2013: 24 · 2012: 25
- FWCI
- 9.18
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Neuroscience (publishedVersion, cc-by-nc-sa)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion, cc-by)
- Funders
- Gordon and Betty Moore Foundation
- United States-Israel Binational Science Foundation
- Tel Aviv University
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- reinforcement learning, prediction error, striatum, dorsal striatum, ventral striatum, reward-based decision-making, individual differences, dopaminergic midbrain, action selection
- MeSH terms
- Adult, Corpus Striatum, Decision Making, Female, Humans, Learning, Male, Psychomotor Performance, Reinforcement, Psychology, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
