Dissociable reward and timing signals in human midbrain and ventral striatum

Klein-Flügge, Miriam C.; Hunt, Laurence T.; Bach, Dominik R.; Dolan, Raymond J.; Behrens, Timothy E.J.

doi:10.1016/j.neuron.2011.08.024

Cited by 75Neurotransmitter Receptor Influence on BehaviorNeural and Behavioral Psychology StudiesMemory and Neural Mechanisms

Dissociable reward and timing signals in human midbrain and ventral striatum

Open access

Klein-Flügge, Miriam C., Hunt, Laurence T., Bach, Dominik R., Dolan, Raymond J., Behrens, Timothy E.J.·2011·Neuron

DOI 10.1016/j.neuron.2011.08.024PMC PMC3219831

Open this paper in CoChat — free account Open PDF

Abstract

Reward prediction error (RPE) signals are central to current models of reward-learning. Temporal difference (TD) learning models posit that these signals should be modulated by predictions, not only of magnitude but also timing of reward. Here we show that BOLD activity in the VTA conforms to such TD predictions: responses to unexpected rewards are modulated by a temporal hazard function and activity between a predictive stimulus and reward is depressed in proportion to predicted reward. By contrast, BOLD activity in ventral striatum (VS) does not reflect a TD RPE, but instead encodes a signal on the variable relevant for behavior, here timing but not magnitude of reward. The results have important implications for dopaminergic models of cortico-striatal learning and suggest a modification of the conventional view that VS BOLD necessarily reflects inputs from dopaminergic VTA neurons signaling an RPE.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1016/j.neuron.2011.08.024

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

When participants attributed outcomes to the neutral committee (their own performance), MBD activity was positively correlated with outcome.
“When participants attributed outcomes to the neutral committee (their own performance), MBD activity was, as has been previously reported,50”
From Two prefrontal-subcortical circuits estimate controllability and reflect its impact on learning (2026) · cites this paper as [52]
The Results (Relating VS Responses to Behavior) report larger ventral striatal responses when participants predicted outcome timing more accurately; the Discussion describes reward-prediction-error signals in the VTA. Neither passage addresses a neutral committee, participants attributing outcomes to their own performance, or an MBD activity–outcome correlation. Searches for those terms found no matching finding in our copy. (LLM, conf=0.74)
Experimental — accuracy not yet measured
This approach has been used to identify links between activity in neuromodulatory nuclei and computational trajectories, such as precision-weighted prediction errors or uncertainty.
“This approach has been used, for example, to identify links between activity in neuromodulatory nuclei and computational trajectories, such as (precision-weighted) prediction errors or uncertainty (111, 115–117).”
From Can Bayesian Theories of Autism Spectrum Disorder Help Improve Clinical Practice? (2016) · cites this paper as [116]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Advances in functional and structural MR image analysis and implementation as FSL
Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E. J., Johansen-Berg, H., Bannister, P. R., De Luca, M., Drobnjak, I., Flitney, D. E., et al. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.051 · Cited by 13,630 · see citing works
Self-citation
Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images
Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002) · NeuroImage · doi:10.1006/nimg.2002.1132 · Cited by 11,888 · see citing works
Fast robust automated brain extraction
Smith, S. M. (2002) · Human Brain Mapping · doi:10.1002/hbm.10062 · Cited by 10,496 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
A global optimisation method for robust affine registration of brain images
Jenkinson, M., & Smith, S. M. (2001) · Medical Image Analysis · doi:10.1016/s1361-8415(01)00036-6 · Cited by 6,805 · see citing works
Probabilistic Independent Component Analysis for Functional Magnetic Resonance Imaging
Beckmann, C. F., & Smith, S. M. (2004) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.822821 · Cited by 2,707 · see citing works
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · see citing works
Self-citation
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · see citing works
Self-citation
Multilevel linear modelling for FMRI group analysis using Bayesian inference
Woolrich, M. W., Behrens, T. E. J., Beckmann, C. F., Jenkinson, M., & Smith, S. M. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2003.12.023 · Cited by 1,719 · see citing works
Self-citation
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584 · see citing works
Self-citation
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · see citing works
Self-citation
General multilevel linear modeling for group analysis in FMRI
Beckmann, C. F., Jenkinson, M., & Smith, S. M. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00435-x · Cited by 1,464 · see citing works
Separating respiratory-variation-related fluctuations from neuronal-activity-related fluctuations in fMRI
Birn, R. M., Diamond, J. B., Smith, M. A., & Bandettini, P. A. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.02.048 · Cited by 1,342 · see citing works
States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166 · see citing works
Dopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,103 · see citing works
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018 · see citing works
Associative learning of social value
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. S. (2008) · Nature · doi:10.1038/nature07538 · Cited by 925 · see citing works
Self-citation
Dopamine-mediated regulation of corticostriatal synaptic plasticity
Calabresi, P., Picconi, B., Tozzi, A., & Di Filippo, M. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.001 · Cited by 865 · see citing works
Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors
Hare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792 · see citing works
Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging
Cools, R., Clark, L., Owen, A. M., & Robbins, T. W. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-11-04563.2002 · Cited by 784 · see citing works
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661 · see citing works
Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective
Botvinick, M. M., Niv, Y., & Barto, A. G. (2009) · Cognition · doi:10.1016/j.cognition.2008.08.011 · Cited by 646 · see citing works
Temporal difference models describe higher-order learning in humans
Seymour, B., O'Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., Friston, K. J., & Frackowiak, R. S. (2004) · Nature · doi:10.1038/nature02581 · Cited by 605 · see citing works
Self-citation
Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597 · see citing works
Reinforcement Learning Signal Predicts Social Conformity
Klucharev, V., Hytönen, K., Rijpkema, M., Smidts, A., & Fernández, G. (2009) · Neuron · doi:10.1016/j.neuron.2008.11.027 · Cited by 594 · see citing works
Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain
Yacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537 · see citing works
Dopamine-dependent plasticity of corticostriatal synapses
Reynolds, J. N. J., & Wickens, J. R. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00045-x · Cited by 528 · see citing works
Differential Encoding of Losses and Gains in the Human Striatum
Seymour, B., Daw, N. D., Dayan, P., Singer, T., & Dolan, R. J. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.0400-07.2007 · Cited by 455 · see citing works
Self-citation
Different Neural Correlates of Reward Expectation and Reward Expectation Error in the Putamen and Caudate Nucleus During Stimulus-Action-Reward Association Learning
Haruno, M., & Kawato, M. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00382.2005 · Cited by 433 · see citing works
Reinforcement Learning Signals in the Human Striatum Distinguish Learners from Nonlearners during Reward-Based Decision Making
Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390 · see citing works
How the Opinion of Others Affects Our Valuation of Objects
Campbell-Meiklejohn, D. K., Bach, D. R., Roepstorff, A., Dolan, R. J., & Frith, C. D. (2010) · Current Biology · doi:10.1016/j.cub.2010.04.055 · Cited by 339 · see citing works
Self-citation
The temporal precision of reward prediction in dopamine neurons
Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008) · Nature Neuroscience · doi:10.1038/nn.2159 · Cited by 310 · see citing works
Striatal Prediction Error Modulates Cortical Coupling
den Ouden, H. E. M., Daunizeau, J., Roiser, J., Friston, K. J., & Stephan, K. E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4458-09.2010 · Cited by 304 · see citing works
A unique adolescent response to reward prediction errors
Cohen, J. R., Asarnow, R. F., Sabb, F. W., Bilder, R. M., Bookheimer, S. Y., Knowlton, B. J., & Poldrack, R. A. (2010) · Nature Neuroscience · doi:10.1038/nn.2558 · Cited by 285 · see citing works
Neurocomputational models of basal ganglia function in learning, memory and choice
Cohen, M. X., & Frank, M. J. (2008) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.09.029 · Cited by 238 · see citing works
Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions
Krugel, L. K., Biele, G., Mohr, P. N. C., Li, S.-C., & Heekeren, H. R. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0905191106 · Cited by 229 · see citing works
The impact of physiological noise correction on fMRI at 7 T
Hutton, C., Josephs, O., Stadler, J., Featherstone, E., Reid, A., Speck, O., Bernarding, J., & Weiskopf, N. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.04.018 · Cited by 211 · see citing works
Human neural learning depends on reward prediction errors in the blocking paradigm.
Tobler, P. N., O’Doherty, J. P., Dolan, R. J., & Schultz, W. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00762.2005 · Cited by 209 · see citing works
Self-citation
Separate brain regions code for salience vs. valence during reward prediction in humans
Jensen, J., Smith, A. J., Willeit, M., Crawley, A. P., Mikulis, D. J., Vitcu, I., & Kapur, S. (2006) · Human Brain Mapping · doi:10.1002/hbm.20274 · Cited by 178 · see citing works
Representation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173 · see citing works
Signals in Human Striatum Are Appropriate for Policy Update Rather than Value Prediction
Li, J., & Daw, N. D. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.6316-10.2011 · Cited by 149 · see citing works
Human reinforcement learning subdivides structured action spaces by learning effector-specific values
Gershman, S. J., Pesaran, B., & Daw, N. D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2469-09.2009 · Cited by 118 · see citing works
Overlapping Prediction Errors in Dorsal Striatum During Instrumental Learning With Juice and Money Reward in the Human Brain
Valentin, V. V., & O’Doherty, J. P. (2009) · Journal of Neurophysiology · doi:10.1152/jn.91195.2008 · Cited by 117 · see citing works
Dissociating the contributions of independent corticostriatal systems to visual categorization learning through the use of reinforcement learning modeling and Granger causality modeling
Seger, C. A., Peterson, E. J., Cincotta, C. M., Lopez‐Paniagua, D., & Anderson, C. W. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.11.083 · Cited by 101 · see citing works
Comparing the Neural Basis of Monetary Reward and Cognitive Feedback during Information-Integration Category Learning
Daniel, R., & Pollmann, S. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.2205-09.2010 · Cited by 91 · see citing works
Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson's disease patients: evidence from a model-based fMRI study
Schönberg, T., O’Doherty, J. P., Joel, D., Inzelberg, R., Segev, Y., & Daw, N. D. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.08.011 · Cited by 81 · see citing works
Understanding the Neural Computations of Arbitrary Visuomotor Learning through fMRI and Associative Learning Theory
Brovelli, A., Laksiri, N., Nazarian, B., Meunier, M., & Boussaoud, D. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm198 · Cited by 74 · see citing works
Policy Adjustment in a Dynamic Economic Game
Li, J., McClure, S. M., King‐Casas, B., & Montague, P. R. (2006) · PLoS ONE · doi:10.1371/journal.pone.0000103 · Cited by 63 · see citing works
Neural regions that underlie reinforcement learning are also active for social expectancy violations
Harris, L. T., & Fiske, S. T. (2010) · Social Neuroscience · doi:10.1080/17470910903135825 · Cited by 50 · see citing works
Untitled work
Glover (2000) · doi:10.1002/1522-2594(200007)44:1

Flagged first, then by citations

Sorted by citations

The Nucleus Accumbens: An Interface Between Cognition, Emotion, and Action
Floresco, S. B. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010213-115159 · Cited by 938 · see citing works
Pupil diameter covaries with BOLD activity in human locus coeruleus
Murphy, P. R., O'Connell, R. G., O'Sullivan, M., Robertson, I. H., & Balsters, J. H. (2014) · Human Brain Mapping · doi:10.1002/hbm.22466 · Cited by 819 · see citing works
How prediction errors shape perception, attention, and motivation
den Ouden, H. E. M., Kok, P., & de Lange, F. P. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00548 · Cited by 512 · see citing works
Go and no-go learning in reward and punishment: Interactions between affect and effect
Guitart‐Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Düzel, E., & Dolan, R. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.024 · Cited by 418 · see citing works
Self-citation
Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning
Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.009 · Cited by 342 · see citing works
Dopamine restores reward prediction errors in old age
Chowdhury, R., Guitart‐Masip, M., Lambert, C., Dayan, P., Huys, Q. J. M., Düzel, E., & Dolan, R. J. (2013) · Nature Neuroscience · doi:10.1038/nn.3364 · Cited by 267 · see citing works
Self-citation
The Neural Network Underlying Incentive-Based Learning: Implications for Interpreting Circuit Disruptions in Psychiatric Disorders
Haber, S. N., & Behrens, T. E. J. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.031 · Cited by 230 · see citing works
Self-citation
Dopamine reward prediction errors reflect hidden-state inference across time
Starkweather, C. K., Babayan, B. M., Uchida, N., & Gershman, S. J. (2017) · Nature Neuroscience · doi:10.1038/nn.4520 · Cited by 206 · see citing works
Translational perspectives for computational neuroimaging
Stephan, K. E., Iglesias, S., Heinzle, J., & Diaconescu, A. O. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.008 · Cited by 169 · see citing works
Neuronal Computation Underlying Inferential Reasoning in Humans and Mice
Barron, H. C., Reeve, H. M., Koolschijn, R. S., Perestenko, P. V., Shpektor, A., Nili, H., Rothaermel, R., Campo-Urriza, N., O’Reilly, J. X., Bannerman, D. M., et al. (2020) · Cell · doi:10.1016/j.cell.2020.08.035 · Cited by 179 · see citing works
Self-citation
Effort and Valuation in the Brain: The Effects of Anticipation and Execution
Kurniawan, I. T., Guitart‐Masip, M., Dayan, P., & Dolan, R. J. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.4777-12.2013 · Cited by 171 · see citing works
Self-citation
Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta‐analysis
Fouragnan, E., Retzler, C., & Philiastides, M. G. (2018) · Human Brain Mapping · doi:10.1002/hbm.24047 · Cited by 176 · see citing works
Computational neuroimaging strategies for single patient predictions
Stephan, K. E., Schlagenhauf, F., Huys, Q. J. M., Raman, S., Aponte, E. A., Brodersen, K. H., Rigoux, L., Moran, R., Jean Daunizeau, A., Dolan, R. J., et al. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.06.038 · Cited by 166 · see citing works
Self-citation
Dissociable patterns of abnormal frontal cortical activation during anticipation of an uncertain reward or loss in bipolar versus major depression
Chase, H. W., Nusslock, R., Almeida, J., Forbes, E., Labarbara, E. J., & Phillips, M. L. (2013) · Bipolar Disorders · doi:10.1111/bdi.12132 · Cited by 156 · see citing works
The Behavioral and Neural Mechanisms Underlying the Tracking of Expertise
Boorman, E. D., O’Doherty, J. P., Adolphs, R., & Rangel, A. (2013) · Neuron · doi:10.1016/j.neuron.2013.10.024 · Cited by 147 · see citing works
Auditory rhythmic cueing in movement rehabilitation: findings and possible mechanisms
Schaefer, R. S. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0402 · Cited by 139 · see citing works
Hierarchical prediction errors in midbrain and septum during social learning
Diaconescu, A. O., Mathys, C. D., Weber, L. A. E., Kasper, L., Mauer, J., & Stephan, K. E. (2016) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsw171 · Cited by 130 · see citing works
Computational modelling of social cognition and behaviour—a reinforcement learning primer
Lockwood, P. L., & Klein-Flügge, M. C. (2020) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsaa040 · Cited by 125 · see citing works
Self-citation
Multiple Modes of Impulsivity in Parkinson's Disease
Nombela, C., Rittman, T., Robbins, T. W., & Rowe, J. B. (2014) · PLoS ONE · doi:10.1371/journal.pone.0085747 · Cited by 134 · see citing works
Can Bayesian Theories of Autism Spectrum Disorder Help Improve Clinical Practice?
Haker, H., Schneebeli, M., & Stephan, K. E. (2016) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2016.00107 · Cited by 130 · see citing works
Social rewards and social networks in the human brain
Fareri, D. S., & Delgado, M. R. (2014) · The Neuroscientist · doi:10.1177/1073858414521869 · Cited by 124 · see citing works
Individual differences in the proneness to have flow experiences are linked to dopamine D2-receptor availability in the dorsal striatum
de Manzano, Ö., Červenka, S., Jučaitė, A., Hellenäs, O., Farde, L., & Ullén, F. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.10.072 · Cited by 121 · see citing works
Two Anatomically and Computationally Distinct Learning Signals Predict Changes to Stimulus-Outcome Associations in Hippocampus
Boorman, E. D., Rajendran, V. G., O’Reilly, J. X., & Behrens, T. E. J. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.014 · Cited by 117 · see citing works
Self-citation
Temporal Specificity of Reward Prediction Errors Signaled by Putative Dopamine Neurons in Rat VTA Depends on Ventral Striatum
Takahashi, Y. K., Langdon, A., Niv, Y., & Schoenbaum, G. M. (2016) · Neuron · doi:10.1016/j.neuron.2016.05.015 · Cited by 116 · see citing works
Dopamine Modulates Adaptive Prediction Error Coding in the Human Midbrain and Striatum
Diederen, K. M. J., Ziauddeen, H., Vestergaard, M. D., Spencer, T., Schultz, W., & Fletcher, P. C. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.1979-16.2016 · Cited by 110 · see citing works
Adaptive prediction error coding in the human midbrain and striatum facilitates behavioral adaptation and learning efficiency
Diederen, K. M. J., Spencer, T., Vestergaard, M. D., Fletcher, P. C., & Schultz, W. (2016) · Neuron · doi:10.1016/j.neuron.2016.04.019 · Cited by 102 · see citing works
A comparison of neural responses to appetitive and aversive stimuli in humans and other mammals
Hayes, D. J., Duncan, N. W., Xu, J., & Northoff, G. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.06.018 · Cited by 98 · see citing works
Identity prediction errors in the human midbrain update reward-identity expectations in the orbitofrontal cortex
Howard, J. D., & Kahnt, T. (2018) · Nature Communications · doi:10.1038/s41467-018-04055-5 · Cited by 96 · see citing works
Local and global consequences of reward-evoked striatal dopamine release
Li, N., & Jasanoff, A. (2020) · Nature · doi:10.1038/s41586-020-2158-3 · Cited by 83 · see citing works
Neural signals encoding shifts in beliefs
Schwartenbeck, P., FitzGerald, T. H. B., & Dolan, R. J. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.10.067 · Cited by 82 · see citing works
Self-citation
Brainstem Modulation of Large-Scale Intrinsic Cortical Activity Correlations
van den Brink, R. L., Pfeffer, T. A., & Donner, T. H. (2019) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2019.00340 · Cited by 83 · see citing works
Hierarchical Learning Induces Two Simultaneous, But Separable, Prediction Errors in Human Basal Ganglia
Diuk, C., Tsai, K., Wallis, J. D., Botvinick, M., & Niv, Y. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5445-12.2013 · Cited by 79 · see citing works
Differential Modulation of Reinforcement Learning by D2 Dopamine and NMDA Glutamate Receptor Antagonism
Jocham, G., Klein, T. A., & Ullsperger, M. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.0757-14.2014 · Cited by 70 · see citing works
Impaired reward-related learning signals in remitted unmedicated patients with recurrent depression
Geugies, H., Mocking, R. J. T., Figueroa, C. A., Groot, P. F. C., Marsman, J.-B. C., Servaas, M. N., Steele, J. D., Schene, A. H., & Ruhé, H. G. (2019) · Brain · doi:10.1093/brain/awz167 · Cited by 67 · see citing works
The anticipation of events in time
Grabenhorst, M., Michalareas, G., Maloney, L. T., & Poeppel, D. (2019) · Nature Communications · doi:10.1038/s41467-019-13849-0 · Cited by 51 · see citing works
Anhedonia and Suicide
Auerbach, R. P., Pagliaccio, D., & Kirshenbaum, J. S. (2022) · Current topics in behavioral neurosciences · doi:10.1007/7854_2022_358 · Cited by 47 · see citing works
Heads for learning, tails for memory: reward, reinforcement and a role of dopamine in determining behavioral relevance across multiple timescales
Baudonnat, M., Huber, A., David, V., & Walton, M. E. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00175 · Cited by 42 · see citing works
Cholinergic and dopaminergic effects on prediction error and uncertainty responses during sensory associative learning
Iglesias, S., Kasper, L., Harrison, S. J., Manka, R., Mathys, C. D., & Stephan, K. E. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117590 · Cited by 45 · see citing works
Multiple associative structures created by reinforcement and incidental statistical learning mechanisms
Klein-Flügge, M. C., Wittmann, M. K., Shpektor, A., Jensen, D. E. A., & Rushworth, M. F. S. (2019) · Nature Communications · doi:10.1038/s41467-019-12557-z · Cited by 41 · see citing works
Self-citation
Prospect Theory and the Brain
Fox, C. R., & Poldrack, R. A. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00042-5 · Cited by 43 · see citing works
Mediofrontal Negativity Signals Unexpected Timing of Salient Outcomes
Garofalo, S., Timmermann, C., Battaglia, S., Maier, M. E., & di Pellegrino, G. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01074 · Cited by 41 · see citing works
Temporal prediction errors modulate cingulate–insular coupling
Limongi, R., Sutherland, S. C., Zhu, J. M., Young, M. E., & Habib, R. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.12.078 · Cited by 34 · see citing works
Reward-Related Activity in Ventral Striatum Is Action Contingent and Modulated by Behavioral Relevance
FitzGerald, T. H. B., Schwartenbeck, P., & Dolan, R. J. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.4389-13.2014 · Cited by 35 · see citing works
Self-citation
Expectation mismatch: Differences between self-generated and cue-induced expectations
Gaschler, R., Schwager, S., Umbach, V. J., Frensch, P. A., & Schubert, T. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.06.009 · Cited by 34 · see citing works
Reward signals in the motor cortex: from biology to neurotechnology
Derosière, G., Shokur, S., & Vassiliadis, P. (2025) · Nature Communications · doi:10.1038/s41467-024-55016-0 · Cited by 29 · see citing works
Multiplexing signals in reinforcement learning with internal models and dopamine
Nakahara, H. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.01.001 · Cited by 31 · see citing works
Models of neuromodulation for computational psychiatry
Iglesias, S., Tomiello, S., Schneebeli, M., & Stephan, K. E. (2016) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1420 · Cited by 26 · see citing works
Reward timing matters in motor learning
Vassiliadis, P., Lété, A., Duqué, J., & Derosière, G. (2022) · iScience · doi:10.1016/j.isci.2022.104290 · Cited by 21 · see citing works
Effects of social context on feedback-related activity in the human ventral striatum
Simon, D., Becker, M. P. I., Mothes‐Lasch, M., Miltner, W. H. R., & Straube, T. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.05.071 · Cited by 21 · see citing works
Assessing the Primacy of Human Amygdala-Inferotemporal Emotional Scene Discrimination with Rapid Whole-Brain fMRI
Sabatinelli, D., & Frank, D. W. (2019) · Neuroscience · doi:10.1016/j.neuroscience.2019.03.001 · Cited by 21 · see citing works
Modulating the map
Werlen, E., & Jones, M. W. (2015) · Progress in brain research · doi:10.1016/bs.pbr.2015.03.002 · Cited by 16 · see citing works
Selective maintenance of value information helps resolve the exploration/exploitation dilemma
Hallquist, M. N., & Dombrovski, A. Y. (2018) · Cognition · doi:10.1016/j.cognition.2018.11.004 · Cited by 16 · see citing works
The value of what’s to come: neural mechanisms coupling prediction error and reward anticipationPreprint
Iigaya, K., Hauser, T. U., Kurth-Nelson, Z. L., O’Doherty, J. P., Dayan, P., & Dolan, R. J. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/588699 · Cited by 15 · see citing works
Self-citation
Multiple systems in macaques for tracking prediction errors and other types of surprise
Grohn, J., Schüffelgen, U., Neubert, F.-X., Bongioanni, A., Verhagen, L., Sallet, J., Kolling, N., & Rushworth, M. F. S. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000899 · Cited by 13 · see citing works
PET-measured human dopamine synthesis capacity and receptor availability predict trading rewards and time-costs during foraging
Ianni, A. M., Eisenberg, D. P., Boorman, E. D., Constantino, S. M., Hegarty, C. E., Gregory, M. D., Masdeu, J. C., Kohn, P. D., Behrens, T. E. J., & Berman, K. F. (2023) · Nature Communications · doi:10.1038/s41467-023-41897-0 · Cited by 12 · see citing works
Self-citation
Complex sensorimotor transformation processes required for response selection are facilitated by the striatum
Stock, A., Vanessa Ness, A., & Beste, C. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.08.036 · Cited by 11 · see citing works
The power of price compels you: Behavioral economic insights into dopamine-based valuation of rewarding and aversively motivated behavior
Oleson, E. B., & Roberts, J. B. (2018) · Brain Research · doi:10.1016/j.brainres.2018.11.043 · Cited by 11 · see citing works
The contribution of striatal pseudo-reward prediction errors to value-based decision-making
Herrero, E. M., Sescousse, G., Cools, R., & Marco‐Pallarés, J. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.02.052 · Cited by 10 · see citing works
No unified reward prediction error in local field potentials from the human nucleus accumbens: evidence from epilepsy patients
Stenner, M., Rutledge, R. B., Zaehle, T., Schmitt, F. C., Kopitzki, K., Kowski, A. B., Voges, J., Heinze, H., & Dolan, R. J. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00260.2015 · Cited by 9 · see citing works
Self-citation
Money or funny: Effective connectivity during service recovery with a DCM-PEB approach
Chan, Y., Wang, C., & Chou, T. (2022) · Biological Psychology · doi:10.1016/j.biopsycho.2022.108464 · Cited by 9 · see citing works
Amphetamine disrupts haemodynamic correlates of prediction errors in nucleus accumbens and orbitofrontal cortex
Werlen, E., Shin, S.-L., Gastambide, F., François, J., Tricklebank, M. D., Marston, H. M., Huxter, J. R., Gilmour, G., & Walton, M. E. (2019) · Neuropsychopharmacology · doi:10.1038/s41386-019-0564-8 · Cited by 10 · see citing works
Treatment-resistant recurrent unipolar and bipolar depression: associative learning abnormalities
Suveges, S., Wang, Y., Tolomeo, S., Gilbertson, T., & Steele, J. D. (2025) · Brain · doi:10.1093/brain/awaf280 · Cited by 5 · see citing works
Distinct Action Signals by Subregions in the Nucleus Accumbens during STOP–Change Performance
Ashton, S. E., Sharalla, P., Kang, N., Brockett, A. T., McCarthy, M. M., & Roesch, M. R. (2024) · Journal of Neuroscience · doi:10.1523/jneurosci.0020-24.2024 · Cited by 5 · see citing works
Timing Intervals Using Population Synchrony and Spike Timing Dependent Plasticity
Xu, W., & Baker, S. N. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00123 · Cited by 4 · see citing works
Financial gain‐ and loss‐related BOLD signals in the human ventral tegmental area and substantia nigra pars compacta
Limbrick‐Oldfield, E. H., Leech, R., Wise, R. G., & Ungless, M. A. (2018) · European Journal of Neuroscience · doi:10.1111/ejn.14288 · Cited by 4 · see citing works
A VTA GABAergic computational model of dissociated reward prediction error computation in classical conditioning
Kaushik, P., Naudé, J., Bapi, R. S., & Alexandre, F. (2020) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2022.107653 · Cited by 3 · see citing works
Two prefrontal-subcortical circuits estimate controllability and reflect its impact on learning
Liu, Y., Spiering, L., Luo, S., Kingston, N., Grohn, J., & Rushworth, M. F. S. (2026) · Neuron · doi:10.1016/j.neuron.2026.07.029 · Cited by 1 · see citing works
Hippocampal-midbrain circuit enhances the pleasure of anticipation in the prefrontal cortex
Iigaya, K., Hauser, T. U., Kurth-Nelson, Z. L., O’Doherty, J. P., Dayan, P., & Dolan, R. J. (2019) · CaltechAUTHORS (California Institute of Technology) · Cited by 1 · see citing works
Self-citation
Action control in uncertain environments
Smittenaar, P. (2015) · UCL Discovery (University College London)
Investigating reward systems in the human midbrain with fMRI
Limbrick‐Oldfield, E. H. (2012) · Spiral (Imperial College London) · doi:10.25560/14511
Neural plasticity in decision making and memory formation
Garvert, M. M. (2016) · UCL Discovery (University College London)
Encoding Motivation Prediction Errors in the Human Dopaminergic Reward SystemPreprint
Ni, Y., Wang, S., Su, J., Li, J., & Wan, X. (2020) · Research Square · doi:10.21203/rs.3.rs-51287/v1
Faculty Opinions recommendation of Dopamine reward prediction errors reflect hidden-state inference across time.
Schultz, W. (2019) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.727374683.793565153
The One-Choice vs. Multi-Choice problem and the origin of incentive salience and outcome directed behaviorPreprint
Bosulu, J. (2022) · doi:10.31234/osf.io/wpqj7
Electrophysiological Correlates of Reward Processing in the Human Ventral Tegmental AreaPreprint
Ramaswamy, A., Steele, J. D., Roiser, J. P., Simmonds, L., Lagrata, S., Matharu, M. S., Vivekananda, U., Akram, H., Zrinzo, L. U., & Vladimir Litvak, A. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.27.701696

BEHAVIORAL NEUROSCIENCE Anticipatory reward signals in ventral striatal neurons of behaving rats
Khamassi, M., Mulder, A. B., Tabuchi, E., Douchamps, V., & Wiener, S. I. (2014)
Anticipatory reward signals in ventral striatal neurons of behaving rats
Khamassi, M., Mulder, A. B., Tabuchi, E., Douchamps, V., & Wiener, S. I. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06480.x · Cited by 43 · see citing works
Temporal Specificity of Reward Prediction Errors Signaled by Putative Dopamine Neurons in Rat VTA Depends on Ventral Striatum
Takahashi, Y. K., Langdon, A., Niv, Y., & Schoenbaum, G. M. (2016) · Neuron · doi:10.1016/j.neuron.2016.05.015 · Cited by 116 · see citing works
Time, Not Size, Matters for Striatal Reward Predictions to Dopamine
Burke, C. J., & Tobler, P. N. (2016) · Neuron · doi:10.1016/j.neuron.2016.06.029 · Cited by 5 · see citing works
A neuro-computational model showing the effects of ventral striatum lesion on the computation of reward prediction error in VTA
Kaushik, P., & Alexandre, F. (2019) · HAL

Authors: Klein-Flügge, M. C., Hunt, L. T., Bach, D. R., Dolan, R. J., & Behrens, T. E. J.
Affiliations: Wellcome Centre for Human Neuroimaging (GB)
University of Oxford (GB)
Corresponding author: Klein-Flügge, Miriam C.
ORCID: Klein-Flügge, Miriam C.
Hunt, Laurence T.
Bach, Dominik R.
Dolan, Raymond J.
Behrens, Timothy E.J.
Venue: Neuron
Publisher: Elsevier BV
Published: 2011-11
Volume: 72
Issue: 4
Pages: 654-664
ISSN: 0896-6273 (ISSN-L)
DOI: 10.1016/j.neuron.2011.08.024
PMC: PMC3219831
OpenAlex: W2126799239
Cited by: 75
References: 52
Citations by year: 2026: 2 · 2025: 2 · 2024: 1 · 2023: 1 · 2022: 4 · 2020: 7 · 2019: 10 · 2018: 5 · 2017: 2 · 2016: 10 · 2015: 6 · 2014: 12 · 2013: 9 · 2012: 4
FWCI: 3.31
Type: journal article
Language: en
Open access: hybrid
doi.org
Licence: cc-by
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
creativecommons.org/licenses/by/3.0/ (vor, 623 d)
Locations: Neuron (publishedVersion, cc-by)
Europe PMC (PubMed Central) (publishedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
UCL Discovery (University College London) (submittedVersion)
Max Planck Digital Library (submittedVersion, other-oa)
MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
Funders: Wellcome Trust
Topics: Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
Keywords: reward prediction error, temporal difference learning, ventral tegmental area, ventral striatum, reward timing, dopaminergic signaling, BOLD fMRI
MeSH terms: Adult, Basal Ganglia, Female, Humans, Magnetic Resonance Imaging, Male, Mesencephalon, Photic Stimulation, Psychomotor Performance, Reward, Time Factors, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Dissociable reward and timing signals in human midbrain and ventral striatum

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks