Cited by 244Neural and Behavioral Psychology StudiesNeurotransmitter Receptor Influence on BehaviorReinforcement Learning in Robotics

The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.

Testing the Reward Prediction Error Hypothesis with an Axiomatic Model

Open access

Rutledge, Robb B., Dean, Mark, Caplin, Andrew, Glimcher, Paul W.·2010·The Journal of Neuroscience

DOI 10.1523/jneurosci.1747-10.2010PMC PMC2957369

Open this paper in CoChat — free account Open PDF

Abstract

Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neural activity from the striatum, medial prefrontal cortex, amygdala, and posterior cingulate cortex satisfy necessary and sufficient conditions for the entire class of RPE models. However, activity measured from the anterior insula falsifies the axiomatic model, and therefore no RPE model can account for measured activity. Further analysis suggests the anterior insula might instead encode something related to the salience of an outcome. As cognitive neuroscience matures and models proliferate, formal approaches of this kind that assess entire model classes rather than specific model exemplars may take on increased significance.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1523/jneurosci.1747-10.2010

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
The Logic of Scientific Discovery.
Hutchison, T. W., & Popper, K. R. (1959) · Economica · doi:10.2307/2550489 · Cited by 7,629
Dissociable intrinsic connectivity networks for salience processing and executive control
Seeley, W. W., Menon, V., Schatzberg, A. F., Keller, J., Glover, G. H., Kenna, H., Reiss, A. L., & Greicius, M. D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.5587-06.2007 · Cited by 7,426
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
The Logic of Scientific Discovery
Popper, K. R., & Weiss, G. (1959) · Physics Today · doi:10.1063/1.3060577 · Cited by 2,717
Tests of Statistical Hypotheses Concerning Several Parameters When the Number of Observations is Large
Wald, A. (1943) · Transactions of the American Mathematical Society · doi:10.1090/s0002-9947-1943-0012401-3 · Cited by 2,522
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378

Show all 51 references

Research works printed but not linked: 1

Sutton RS Barto AG (1990) in Learning and computational neuroscience: foundations of adaptive networks, Time-derivative models of Pavlovian reinforcement, eds Gabriel M Moore J (MIT, Cambridge, MA), pp 497–537.

Sorted by citations

Processing of primary and secondary rewards: A quantitative meta-analysis and review of human functional neuroimaging studies
Sescousse, G., Caldú, X., Segura, B., & Dreher, J.-C. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.02.002 · Cited by 782
Significance of the insula for the evolution of human awareness of feelings from the body
(Bud) Craig, A. D. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.05990.x · Cited by 646
Neurophysiology of performance monitoring and adaptive behavior
Ullsperger, M., Danielmeier, C., & Jocham, G. (2014) · Physiological Reviews · doi:10.1152/physrev.00041.2012 · Cited by 634
Inflammation Effects on Motivation and Motor Activity: Role of Dopamine
Felger, J. C., & Treadway, M. T. (2016) · Neuropsychopharmacology · doi:10.1038/npp.2016.143 · Cited by 474
A computational and neural model of momentary subjective well-being
Rutledge, R. B., Skandali, N., Dayan, P., & Dolan, R. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1407535111 · Cited by 444
Self-citation
Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Garrison, J. R., Erdeniz, B., & Done, J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.023 · Cited by 460
The feedback-related negativity (FRN) revisited: New insights into the localization, meaning and network organization
Hauser, T. U., Iannaccone, R., Stämpfli, P., Drechsler, R., Brandeis, D., Walitza, S., & Brem, S. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.08.028 · Cited by 450
Rostrolateral Prefrontal Cortex and Individual Differences in Uncertainty-Driven Exploration
Badre, D., Doll, B. B., Long, N. M., & Frank, M. J. (2012) · Neuron · doi:10.1016/j.neuron.2011.12.025 · Cited by 316
Phasic Dopamine Release in the Rat Nucleus Accumbens Symmetrically Encodes a Reward Prediction Error Term
Hart, A. S., Rutledge, R. B., Glimcher, P. W., & Phillips, P. E. M. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.2489-13.2014 · Cited by 306
Self-citation
Frontal Theta Reflects Uncertainty and Unexpectedness during Exploration and Exploitation
Cavanagh, J. F., y Figueroa, C. M. L. F., Cohen, M. X., & Frank, M. J. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhr332 · Cited by 326
Habits without values.
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280
Anticipatory reward processing in addicted populations: a focus on the monetary incentive delay task
Balodis, I. M., & Potenza, M. N. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.08.020 · Cited by 267
Valuation of knowledge and ignorance in mesolimbic reward circuitry
Charpentier, C. J., Bromberg-Martin, E. S., & Sharot, T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1800547115 · Cited by 237
Representation of aversive prediction errors in the human periaqueductal gray
Roy, M., Shohamy, D., Daw, N., Jepma, M., Wimmer, G. E., & Wager, T. D. (2014) · Nature Neuroscience · doi:10.1038/nn.3832 · Cited by 246
Critical Roles for Anterior Insula and Dorsal Striatum in Punishment-Based Avoidance Learning
Palminteri, S., Justo, D., Jauffret, C., Pavlicek, B., Dauta, A., Delmaire, C., Czernecki, V., Karachi, C., Capelle, L., Dürr, A., et al. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.017 · Cited by 237
Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis
Chase, H. W., Kumar, P., Eickhoff, S. B., & Dombrovski, A. Y. (2015) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-015-0338-7 · Cited by 242
The Neural Network Underlying Incentive-Based Learning: Implications for Interpreting Circuit Disruptions in Psychiatric Disorders
Haber, S. N., & Behrens, T. E. J. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.031 · Cited by 230
Cognitive flexibility in adolescence: Neural and behavioral mechanisms of reward prediction error processing in adaptive decision making during development
Hauser, T. U., Iannaccone, R., Walitza, S., Brandeis, D., & Brem, S. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2014.09.018 · Cited by 203
A simple solution for model comparison in bold imaging: the special case of reward prediction error and reward outcomes
Erdeniz, B., Rohe, T., Done, J., & Seidler, R. D. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00116 · Cited by 216
Association of neural and emotional impacts of reward prediction errors with major depression
Rutledge, R. B., Moutoussis, M., Smittenaar, P., Zeidman, P., Taylor, T., Hrynkiewicz, L., Lam, J., Skandali, N., Siegel, J. Z., Ousdal, O. T., et al. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.1713 · Cited by 209
Self-citation
What is mood? A computational perspective
Clark, J. E., Watson, S., & Friston, K. J. (2018) · Psychological Medicine · doi:10.1017/s0033291718000430 · Cited by 204
“…shown that positive prediction errors (receiving more food than expected), show a strong positive correlation with dopaminergic change in the nucleus accumbens (Hart et al. 2014) with corresponding changes in functional brain activity in humans during a financial reward task (Rutledge et al. 2010).”
Background
Mechanisms of Hierarchical Reinforcement Learning in Cortico-Striatal Circuits 2: Evidence from fMRI
Badre, D., & Frank, M. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhr117 · Cited by 203
Musical reward prediction errors engage the nucleus accumbens and motivate learning
Gold, B. P., Herrero, E. M., Zeighami, Y., Benovoy, M., Dagher, A., & Zatorre, R. J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1809855116 · Cited by 192
Reward Processing, Neuroeconomics, and Psychopathology
Zald, D. H., & Treadway, M. T. (2017) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-044957 · Cited by 190
Learning to minimize efforts versus maximizing rewards: computational principles and neural correlates
Skvortsova, V., Palminteri, S., & Pessiglione, M. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.1350-14.2014 · Cited by 177
Role of the medial prefrontal cortex in impaired decision making in juvenile attention-deficit/hyperactivity disorder
Hauser, T. U., Iannaccone, R., Ball, J., Mathys, C., Brandeis, D., Walitza, S., & Brem, S. (2014) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1093 · Cited by 168
Effects of early life stress on amygdala and striatal development
Fareri, D. S., & Tottenham, N. (2016) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2016.04.005 · Cited by 177
What’s in a word? How instructions, suggestions, and social information change pain and emotion
Koban, L., Jepma, M., Geuter, S., & Wager, T. D. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.02.014 · Cited by 166
Brain networks for confidence weighting and hierarchical inference during probabilistic learning
Meyniel, F., & Dehaene, S. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1615773114 · Cited by 138
Disambiguating ventral striatum fMRI-related bold signal during reward prediction in schizophrenia
Morris, R. W., Vercammen, A., Lenroot, R., Moore, L., Langton, J. M., Short, B., Kulkarni, J., Curtis, J., O'Donnell, M., Weickert, C. S., et al. (2012) · Molecular Psychiatry · doi:10.1038/mp.2011.75 · Cited by 149
Separate mesocortical and mesolimbic pathways encode effort and reward learning signals
Hauser, T. U., Eldar, E., & Dolan, R. J. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1705643114 · Cited by 142
Within- and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory
Collins, A. G. E., & Frank, M. J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720963115 · Cited by 152
Anatomical and functional organization of the human substantia nigra and its connections
Zhang, Y., Larcher, K. M.-H., Mišić, B., & Dagher, A. (2017) · eLife · doi:10.7554/elife.26653 · Cited by 156
Behavioral, Physiological, and Neural Signatures of Surprise during Naturalistic Sports Viewing
Antony, J. W., Hartshorne, T. H., Pomeroy, K., Gureckis, T. M., Hasson, U., McDougle, S. D., & Norman, K. A. (2020) · Neuron · doi:10.1016/j.neuron.2020.10.029 · Cited by 150
The role of beta-gamma oscillations in unexpected rewards processing
HajiHosseini, A., Rodríguez‐Fornells, A., & Marco‐Pallarés, J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.125 · Cited by 151
Neural Evidence of Regret and Its Implications for Investor Behavior
Frydman, C., & Camerer, C. F. (2016) · Review of Financial Studies · doi:10.1093/rfs/hhw010 · Cited by 144
Amygdala and Ventral Striatum Make Distinct Contributions to Reinforcement Learning
Costa, V. D., Monte, O. D., Lucas, D. R., Murray, E. A., & Averbeck, B. B. (2016) · Neuron · doi:10.1016/j.neuron.2016.09.025 · Cited by 141
Positive reward prediction errors during decision-making strengthen memory encoding
Jang, A. I., Nassar, M. R., Dillon, D. G., & Frank, M. J. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0597-3 · Cited by 138
Auditory Hallucinations and the Brain’s Resting-State Networks: Findings and Methodological Observations
Alderson‐Day, B., Diederen, K. M. J., Fernyhough, C., Ford, J. M., Horga, G., Margulies, D. S., McCarthy‐Jones, S., Northoff, G., Shine, J. M., Turner, J. A., et al. (2016) · Schizophrenia Bulletin · doi:10.1093/schbul/sbw078 · Cited by 132
Computational Psychiatry of ADHD: Neural Gain Impairments across Marrian Levels of Analysis
Hauser, T. U., Fiore, V. G., Moutoussis, M., & Dolan, R. J. (2016) · Trends in Neurosciences · doi:10.1016/j.tins.2015.12.009 · Cited by 125
Cerebral Correlates of Salient Prediction Error for Different Rewards and Punishments
Metereau, E., & Dreher, J.-C. (2013) · Cerebral Cortex · doi:10.1093/cercor/bhs037 · Cited by 114
Dopamine, reward learning, and active inference
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00136 · Cited by 112
Distinct signals in medial and lateral VTA dopamine neurons modulate fear extinction at different times
Cai, L. X., Pizano, K., Gundersen, G. W., Hayes, C. L., Fleming, W., Holt, S., Cox, J., & Witten, I. B. (2020) · eLife · doi:10.7554/elife.54936 · Cited by 108
Deficits in attentional control: Cholinergic mechanisms and circuitry-based treatment approaches.
Sarter, M. F., & Paolone, G. (2011) · Behavioral Neuroscience · doi:10.1037/a0026227 · Cited by 111
Neural mechanisms for learning self and other ownership
Lockwood, P. L., Wittmann, M. K., Apps, M. A. J., Klein-Flügge, M. C., Crockett, M. J., Humphreys, G. W., & Rushworth, M. F. S. (2018) · Nature Communications · doi:10.1038/s41467-018-07231-9 · Cited by 98
Vigor in the Face of Fluctuating Rates of Reward: An Experimental Examination
Guitart‐Masip, M., Beierholm, U., Dolan, R. J., Düzel, E., & Dayan, P. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00090 · Cited by 89
The role of learning-related dopamine signals in addiction vulnerability
Huys, Q. J. M., Tobler, P. N., Hasler, G., & Flagel, S. B. (2014) · Progress in brain research · doi:10.1016/b978-0-444-63425-2.00003-9 · Cited by 92
Increased fronto-striatal reward prediction errors moderate decision making in obsessive–compulsive disorder
Hauser, T. U., Iannaccone, R., Dolan, R. J., Ball, J., Hättenschwiler, J., Drechsler, R., Rufer, M., Brandeis, D., Walitza, S., & Brem, S. (2017) · Psychological Medicine · doi:10.1017/s0033291716003305 · Cited by 86
Value Learning through Reinforcement
Daw, N. D., & Tobler, P. N. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00015-2 · Cited by 109
Dynamic shaping of dopamine signals during probabilistic Pavlovian conditioning
Hart, A. S., Clark, J. J., & Phillips, P. E. M. (2015) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.07.010 · Cited by 82
Altered ventral striatal–medial prefrontal cortex resting-state connectivity mediates adolescent social problems after early institutional care
Fareri, D. S., Gabard-Durnam, L. J., Goff, B., Flannery, J., Gee, D. G., Lumian, D. S., Caldera, C., & Tottenham, N. (2017) · Development and Psychopathology · doi:10.1017/s0954579417001456 · Cited by 90
The good, the bad and the brain: neural correlates of appetitive and aversive values underlying decision making
Pessiglione, M., & Delgado, M. R. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.08.006 · Cited by 89
Taming the beast: extracting generalizable knowledge from computational models of cognition
Nassar, M. R., & Frank, M. J. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.04.003 · Cited by 91
Dissociating neural learning signals in human sign- and goal-trackers
Schad, D. J., Rapp, M. A., Garbusow, M., Nebe, S., Sebold, M., Obst, E., Sommer, C., Deserno, L., Rabovsky, M., Friedel, E., et al. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0765-5 · Cited by 92
Motivational Deficits in Schizophrenia and the Representation of Expected Value
Waltz, J. A., & Gold, J. M. (2015) · Current topics in behavioral neurosciences · doi:10.1007/7854_2015_385 · Cited by 84
Past Makes Future: Role of pFC in Prediction
Fuster, J. M., & Bressler, S. L. (2014) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00746 · Cited by 83
Differential Modulation of Reinforcement Learning by D2 Dopamine and NMDA Glutamate Receptor Antagonism
Jocham, G., Klein, T. A., & Ullsperger, M. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.0757-14.2014 · Cited by 70
Anterior insula reflects surprise in value-based decision-making and perception
Loued-Khenissi, L., Pfeuffer, A., Einhäuser, W., & Preuschoff, K. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116549 · Cited by 66
Anatomical dissociation of intracerebral signals for reward and punishment prediction errors in humans
Gueguen, M. C. M., Lopez‐Persem, A., Billeke, P. B., Lachaux, J.-P., Rheims, S., Kahane, P., Minotti, L., Olivier David, A., Pessiglione, M., & Bastin, J. (2021) · Nature Communications · doi:10.1038/s41467-021-23704-w · Cited by 73
Modulation of feedback related activity in the rostral anterior cingulate cortex during trial and error exploration
Amiez, C., Sallet, J., Procyk, E., & Petrides, M. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.06.023 · Cited by 84
A systems identification approach using Bayes factors to deconstruct the brain bases of emotion regulation
Bo, K., Kraynak, T. E., Kwon, M., Sun, M. M. C., Gianaros, P. J., & Wager, T. D. (2024) · Nature Neuroscience · doi:10.1038/s41593-024-01605-7 · Cited by 80
Knowledge acquisition is governed by striatal prediction errors
Pine, A., Sadeh, N., Ben-Yakov, A., Dudai, Y., & Mendelsohn, A. (2018) · Nature Communications · doi:10.1038/s41467-018-03992-5 · Cited by 76
Understanding psychiatric disorder by capturing ecologically relevant features of learning and decision-making
Scholl, J., & Klein-Flügge, M. C. (2017) · Behavioural Brain Research · doi:10.1016/j.bbr.2017.09.050 · Cited by 69
Neural mechanisms of the link between giving social support and health
Inagaki, T. K. (2018) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13703 · Cited by 66
Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans
Fouragnan, E., Queirazza, F., Retzler, C., Mullinger, K. J., & Philiastides, M. G. (2017) · Scientific Reports · doi:10.1038/s41598-017-04507-w · Cited by 61
Mediofrontal event-related potentials in response to positive, negative and unsigned prediction errors
Sambrook, T. D., & Goslin, J. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.06.004 · Cited by 66
Identification of Neurotensin Receptor Expressing Cells in the Ventral Tegmental Area across the Lifespan
Woodworth, H. L., Perez‐Bonilla, P., Beekly, B. G., Lewis, T. J., & Leinninger, G. M. (2018) · eNeuro · doi:10.1523/eneuro.0191-17.2018 · Cited by 64
Anatomy and Connectivity of the Reward Circuit
Haber, S. N. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-805308-9.00001-4 · Cited by 67
Decision-making: from neuroscience to neuroeconomics—an overview
Serra, D. (2021) · Theory and Decision · doi:10.1007/s11238-021-09830-3 · Cited by 63
Beyond negative valence: 2-week administration of a serotonergic antidepressant enhances both reward and effort learning signals
Scholl, J., Kolling, N., Nelissen, N., Browning, M., Rushworth, M. F. S., & Harmer, C. J. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000756 · Cited by 58
The role of dopamine in risk taking: a specific look at Parkinson’s disease and gambling
Clark, C. A., & Dagher, A. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00196 · Cited by 69
Ghrelin Enhances Food Odor Conditioning in Healthy Humans: An fMRI Study
Han, J. E., Frasnelli, J., Zeighami, Y., Larcher, K. M.-H., Boyle, J., McConnell, T., Malik, S., Jones‐Gotman, M., & Dagher, A. (2018) · Cell Reports · doi:10.1016/j.celrep.2018.11.026 · Cited by 56
How mood tunes prediction: a neurophenomenological account of mood and its disturbance in major depression
Kiverstein, J. D., Miller, M., & Rietveld, E. (2020) · Neuroscience of Consciousness · doi:10.1093/nc/niaa003 · Cited by 60
Over a Decade of Neuroeconomics: What Have We Learned?
Konovalov, A., & Krajbich, I. M. (2016) · Organizational Research Methods · doi:10.1177/1094428116644502 · Cited by 55
A Selective Role for Dopamine in Learning to Maximize Reward But Not to Minimize Effort: Evidence from Patients with Parkinson's Disease
Skvortsova, V., Degos, B., Welter, M., Vidailhet, M., & Pessiglione, M. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.2081-16.2017 · Cited by 53
Dopamine-independent effect of rewards on choices through hidden-state inference
Blanco-Pozo, M., Akam, T., & Walton, M. E. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01542-x · Cited by 69
A unidirectional but not uniform striatal landscape of dopamine signaling for motivational stimuli
van Elzelingen, W., Goedhoop, J. N., Warnaar, P., Denys, D. A. J. P., Arbab, T., & Willuhn, I. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2117270119 · Cited by 54
Annual Research Review: Developmental computational psychiatry
Hauser, T. U., Will, G., Dubois, M., & Dolan, R. J. (2018) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12964 · Cited by 55
Forgetting in Reinforcement Learning Links Sustained Dopamine Signals to Motivation
Kato, A., & Morita, K. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005145 · Cited by 59
Reward Learning Through the Lens of RDoC: a Review of Theory, Assessment, and Empirical Findings in the Eating Disorders
Schaefer, L. M., & STEINGLASS, J. E. (2021) · Current Psychiatry Reports · doi:10.1007/s11920-020-01213-9 · Cited by 51
Alterations in functional brain networks associated with loss-chasing in gambling disorder and cocaine-use disorder
Worhunsky, P. D., Potenza, M. N., & Rogers, R. D. (2017) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2017.05.025 · Cited by 52
The human insula processes both modality-independent and pain-selective learning signals
Horing, B., & Büchel, C. (2022) · PLoS Biology · doi:10.1371/journal.pbio.3001540 · Cited by 50
Distinct Processing of Aversive Experience in Amygdala Subregions
Michely, J., Rigoli, F., Rutledge, R. B., Hauser, T. U., & Dolan, R. J. (2019) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2019.07.008 · Cited by 46
Self-citation
Reinforcement Learning and Tourette Syndrome
Palminteri, S., & Pessiglione, M. (2013) · International review of neurobiology · doi:10.1016/b978-0-12-411546-0.00005-6 · Cited by 47
An MEG signature corresponding to an axiomatic model of reward prediction error
Talmi, D., Fuentemilla, L., Vladimir Litvak, A., Düzel, E., & Dolan, R. J. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.06.051 · Cited by 53
Advanced Reinforcement Learning
Daw, N. D. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00016-4 · Cited by 46
Motivational Deficits in Schizophrenia Are Associated With Reduced Differentiation Between Gain and Loss-Avoidance Feedback in the Striatum
Waltz, J. A., Xu, Z., Brown, E. C., Ruiz, R., Frank, M. J., & Gold, J. M. (2017) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2017.07.008 · Cited by 46
Multiple associative structures created by reinforcement and incidental statistical learning mechanisms
Klein-Flügge, M. C., Wittmann, M. K., Shpektor, A., Jensen, D. E. A., & Rushworth, M. F. S. (2019) · Nature Communications · doi:10.1038/s41467-019-12557-z · Cited by 44
Expected Subjective Value Theory (ESVT): A Representation of Decision Under Risk and Certainty
Glimcher, P. W., & Tymula, A. A. (2016) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2022.12.013 · Cited by 60
Self-citation
Motivational Context Modulates Prediction Error Response in Schizophrenia
Reinen, J., Snellenberg, J. X. V., Horga, G., Abi‐Dargham, A., Daw, N. D., & Shohamy, D. (2016) · Schizophrenia Bulletin · doi:10.1093/schbul/sbw045 · Cited by 45
Breaking human social decision making into multiple components and then putting them together again
Suzuki, S., & O’Doherty, J. P. (2020) · Cortex · doi:10.1016/j.cortex.2020.02.014 · Cited by 48
Neural activity and fundamental learning, motivated by monetary loss and reward, are intact in mild to moderate major depressive disorder
Moutoussis, M., Rutledge, R. B., Prabhu, G., Hrynkiewicz, L., Lam, J., Ousdal, O. T., Guitart‐Masip, M., Fonagy, P., & Dolan, R. J. (2018) · PLoS ONE · doi:10.1371/journal.pone.0201451 · Cited by 42
Self-citation
Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges
Dennison, J. B., Sazhin, D., & Smith, D. V. (2022) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1589 · Cited by 50
Navigating complex decision spaces: Problems and paradigms in sequential choice.
Walsh, M. M., & Anderson, J. R. (2013) · Psychological Bulletin · doi:10.1037/a0033455 · Cited by 41
Beyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning
Ballard, I. C., Miller, E. M., Piantadosi, S. T., Goodman, N. D., & McClure, S. M. (2017) · Cerebral Cortex · doi:10.1093/cercor/bhx259 · Cited by 34
Automatic Sensory Predictions: A Review of Predictive Mechanisms in the Brain and Their Link to Conscious Processing
Tivadar, R. I., Knight, R. T., & Tzovara, A. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.702520 · Cited by 30
A multivariate brain signature for reward
Speer, S. P. H., Keysers, C., Barrios, J. C., Teurlings, C. J. S., Smidts, A., Boksem, M. A. S., Wager, T. D., & Gazzola, V. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.119990 · Cited by 36
Modulation of feedback-related negativity during trial-and-error exploration and encoding of behavioral shifts
Sallet, J., Camille, N., & Procyk, E. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00209 · Cited by 36
Antipsychotic dose modulates behavioral and neural responses to feedback during reinforcement learning in schizophrenia
Insel, C., Reinen, J., Weber, J., Wager, T. D., Jarskog, L. F., Shohamy, D., & Smith, E. E. (2014) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-014-0261-3 · Cited by 35
Of BOLD Claims and Excessive Fears: A Call for Caution and Patience Regarding Political Neuroscience
Theodoridis, A. G., & Nelson, A. J. (2012) · Political Psychology · doi:10.1111/j.1467-9221.2011.00860.x · Cited by 36

Authors: Rutledge, R. B., Dean, M., Caplin, A., & Glimcher, P. W.
Affiliations: New York University (US)
ORCID: Rutledge, Robb B.
Glimcher, Paul W.
Venue: The Journal of Neuroscience
Publisher: Society for Neuroscience
Published: 2010-10-06
Volume: 30
Issue: 40
Pages: 13525-13536
ISSN: 0270-6474 (ISSN-L)
1529-2401
DOI: 10.1523/jneurosci.1747-10.2010
PMC: PMC2957369
OpenAlex: W2035219495
Cited by: 244
References: 51
Citations by year: 2026: 8 · 2025: 11 · 2024: 9 · 2023: 23 · 2022: 11 · 2021: 11 · 2020: 40 · 2019: 10 · 2018: 16 · 2017: 25 · 2016: 15 · 2015: 12 · 2014: 23 · 2013: 13 · 2012: 9
FWCI: 5.61
Type: journal article
Language: en
Open access: hybrid
doi.org
Licence: cc-by-nc-sa
creativecommons.org/licenses/by-nc-sa/4.0/ (vor, 182 d)
Locations: Journal of Neuroscience (publishedVersion, cc-by-nc-sa)
Europe PMC (PubMed Central) (publishedVersion)
PubMed Central (publishedVersion)
PubMed (publishedVersion)
UCL Discovery (University College London) (submittedVersion)
Funders: National Institutes of Health (5r01ns054775-05)
National Defense Science and Engineering Graduate
National Institute on Aging (F31AG031656)
National Institute of Neurological Disorders and Stroke (R01-NS-054775)
Topics: Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords: reward prediction error, axiomatic model, reinforcement learning, model falsifiability, striatum, medial prefrontal cortex, amygdala, posterior cingulate cortex, anterior insula, neural coding, model comparison
MeSH terms: Adult, Algorithms, Behavior, Brain Mapping, Cognition, Female, Forecasting, Humans, Image Processing, Computer-Assisted, Functional Laterality, Learning, Male, Models, Neurological, Models, Psychological, Predictive Value of Tests, Regression Analysis, Reinforcement, Psychology, Reward, Prosencephalon

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Testing the Reward Prediction Error Hypothesis with an Axiomatic Model

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks