The DOI matches its Crossref record; 124 of the 124 references it cites were found in a registry or an index.
Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Open accessAbstract
Activation likelihood estimation (ALE) meta-analyses were used to examine the neural correlates of prediction error in reinforcement learning. The findings are interpreted in the light of current computational models of learning and action selection. In this context, particular consideration is given to the comparison of activation patterns from studies using instrumental and Pavlovian conditioning, and where reinforcement involved rewarding or punishing feedback. The striatum was the key brain area encoding for prediction error, with activity encompassing dorsal and ventral regions for instrumental and Pavlovian reinforcement alike, a finding which challenges the functional separation of the striatum into a dorsal 'actor' and a ventral 'critic'. Prediction error activity was further observed in diverse areas of predominantly anterior cerebral cortex including medial prefrontal cortex and anterior cingulate cortex. Distinct patterns of prediction error activity were found for studies using rewarding and aversive reinforcers; reward prediction errors were observed primarily in the striatum while aversive prediction errors were found more widely including insula and habenula.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Philosophical transactions of the Royal Society of London. B
Press, H. (1886) · Cited by 10,080
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 9,461
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Conflict monitoring and anterior cingulate cortex: an update
Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.10.003 · Cited by 3,623
Automated Talairach Atlas labels for functional brain mapping
Lancaster, J. L., Woldorff, M. G., Parsons, L. M., Liotti, M., Freitas, C. S., Rainey, L., Kochunov, P. V., Nickerson, D., Mikiten, S. A., & Fox, P. T. (2000) · Human Brain Mapping · doi:10.1002/1097-0193(200007)10:3<120::aid-hbm30>3.0.co;2-8 · Cited by 3,387
Anterior Cingulate Cortex, Error Detection, and the Online Monitoring of Performance
Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D., & Cohen, J. D. (1998) · Science · doi:10.1126/science.280.5364.747 · Cited by 3,273
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · cited 1×
“…Science, University of Hertfordshire, Hatfield, Hertfordshire AL10 9AB, UK r t i c l e i n f o rticle history: eceived 29 October 2012 eceived in revised form 19 March 2013 ccepted 27 March 2013 eywords: rediction error a b s t r a c t Activation likelihood estimation (ALE) meta-analyses were…”
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402
The debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
Psychological and neural mechanisms of the affective dimension of pain
Price, D. D. (2000) · Science · doi:10.1126/science.288.5472.1769 · Cited by 2,092
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
The integration of negative affect, pain and cognitive control in the cingulate cortex
Shackman, A. J., Salomons, T. V., Slagter, H. A., Fox, A. S., Winter, J. J., & Davidson, R. J. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn2994 · Cited by 2,082
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 3×
“Nearly all of the 35 studies in this review make reference to the role of dopamine (DA) in computing PE (the dopamine reward prediction hypothesis) as described in the seminal papers by Read Montague (e.g. Montague et al., 1996) and Wolfram Schultz (e.g. Schultz et al., 1997).”
(2020) · Machine Learning and Visual Perception · doi:10.1515/9783110595567-011 · Cited by 1,930
Eickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009) · Human Brain Mapping · doi:10.1002/hbm.20718 · Cited by 1,924
The free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831
Pain and emotion interactions in subregions of the cingulate gyrus
Vogt, B. A. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1704 · Cited by 1,810
Meta-analysis of the functional neuroanatomy of single-word reading: method and validation
Turkeltaub, P. E., Eden, G. F., Jones, K. M., & Zeffiro, T. A. (2002) · NeuroImage · doi:10.1006/nimg.2002.1131 · Cited by 1,604
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Reinforcement Learning: An Introduction
(1998) · Kybernetes · doi:10.1108/k.1998.27.9.1093.3 · Cited by 1,472
Neuronal Coding of Prediction Errors
Schultz, W., & Dickinson, A. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.473 · Cited by 1,427 · cited 3×
“The striatum was the key brain area encoding for prediction error, with activity encompassing dorsal and ventral regions for instrumental eward unishment triatum abenula ctor–critic einforcement learning and Pavlovian reinforcement alike, a finding which challenges the functional separation of the…”
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · cited 1×
“We identified activations in the midbrain or reward prediction errors, consistent with those areas reported n animal studies that showed increased spiking of DA neurons or the unexpected occurrence of rewards (Hollerman and Schultz, 998; Bayer and Glimcher, 2005).”
Lateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292
A common role of insula in feelings, empathy and uncertainty
Singer, T., Critchley, H. D., & Preuschoff, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.05.001 · Cited by 1,270
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Putting a spin on the dorsal-ventral divide of the striatum
Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., & Pennartz, C. M. A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.06.006 · Cited by 1,188
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Dopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,117
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955
The habenula: from stress evasion to value-based decision-making
Hikosaka, O. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2866 · Cited by 955
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · cited 4×
“Several studies (O’Doherty et al., 2003; Tobler et al., 2006; Brovelli et al., 2008; Morris et al., 2011; Pessiglione et al., 2006; McClure et al., 2003; Gläscher et al., 2009; Ramnani et al., 2004) reported deactivation of the BOLD signal relative to a baseline in striatum, predominantly ventral…”
InfluentialLiu, X., Hairston, J., Schrier, M., & Fan, J. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.12.012 · Cited by 916
ALE meta-analysis: controlling the false discovery rate and performing statistical contrasts
Laird, A. R., Fox, P. M., Price, C. J., Glahn, D. C., Uecker, A. M., Lancaster, J. L., Turkeltaub, P. E., Kochunov, P., & Fox, P. T. (2005) · Human Brain Mapping · doi:10.1002/hbm.20136 · Cited by 868
Value, pleasure and choice in the ventral prefrontal cortex
Grabenhorst, F., & Rolls, E. T. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.12.004 · Cited by 766
The short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762
From prediction error to incentive salience: mesolimbic computation of reward motivation.
Berridge, K. C. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.07990.x · Cited by 700
Dopamine signals for reward value and risk: basic and recent data
Schultz, W. (2010) · Behavioral and Brain Functions · doi:10.1186/1744-9081-6-24 · Cited by 653
A mathematical model for simple learning.
Bush, R. R., & Mosteller, F. (1951) · Psychological Review · doi:10.1037/h0054388 · Cited by 586
Postuma, R. B., & Dagher, A. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj088 · Cited by 579
Substantia nigra/ventral tegmental reward prediction error disruption in psychosis
Murray, G. K., Corlett, P. R., Clark, L., Pessiglione, M., Blackwell, A. D., Honey, G., Jones, P. B., Bullmore, E. T., Robbins, T. W., & Fletcher, P. C. (2008) · Molecular Psychiatry · doi:10.1038/sj.mp.4002058 · Cited by 532
Model‐Based fMRI and Its Application to Reward Learning and Decision Making
O'DOHERTY, J. P., HAMPTON, A., & KIM, H. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.022 · Cited by 508
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
Olshausen, B. A. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903321107891 · Cited by 499
Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498
Expected value and prediction error abnormalities in depression and schizophrenia
Gradin, V. B., Kumar, P., Waiter, G., Ahearn, T., Stickle, C., Milders, M., Reid, I., Hall, J., & Steele, J. D. (2011) · Brain · doi:10.1093/brain/awr059 · Cited by 485 · cited 1×
“…et al. (2008) RW 14 Cohen et al. (2007) RW 17 Delgado et al. (2008) TD 11 Gläscher et al. (2010) SARSA 18 Gläscher et al. (2009) RW 20 Gradin et al. (2011) SARSA 17 Hampton et al. (2006) RW 16 Howard-Jones et al. (2010) RW 16 Jocham et al. (2011) RW 16 Kahnt et al. (2011) RW 20 Kahnt…”
Eickhoff, S. B., Bzdok, D., Laird, A. R., Roski, C., Caspers, S., Zilles, K., & Fox, P. T. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.05.021 · Cited by 484
The neural substrates of reward processing in humans: the modern role of FMRI
McClure, S. M., York, M. K., & Montague, P. R. (2004) · The Neuroscientist · doi:10.1177/1073858404263526 · Cited by 480
Multiple model-based reinforcement learning
Doya, K., Samejima, K., Katagiri, K.-I., & Kawato, M. (2002) · Neural Computation · doi:10.1162/089976602753712972 · Cited by 476
Differential Encoding of Losses and Gains in the Human Striatum
Seymour, B., Daw, N., Dayan, P., Singer, T., & Dolan, R. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0400-07.2007 · Cited by 462
Chen, C.-H., Suckling, J., Lennox, B. R., Ooi, C., & Bullmore, E. T. (2011) · Bipolar Disorders · doi:10.1111/j.1399-5618.2011.00893.x · Cited by 458
Kim, H., Shimojo, S., & O'Doherty, J. P. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040233 · Cited by 453
Adaptive Critics and the Basal Ganglia
Barto, A. G. (1994) · Models of Information Processing in the Basal Ganglia · doi:10.7551/mitpress/4708.003.0018 · Cited by 446
Opponent appetitive-aversive neural processes underlie predictive learning of pain relief
Seymour, B., O'Doherty, J. P., Koltzenburg, M., Wiech, K., Frackowiak, R., Friston, K., & Dolan, R. (2005) · Nature Neuroscience · doi:10.1038/nn1527 · Cited by 439
Hlavin, L. (1990) · Neurology · doi:10.1212/wnl.40.2.388-c · Cited by 424
A computational substrate for incentive salience
McClure, S. M., Daw, N. D., & Read Montague, P. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00177-2 · Cited by 422
The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward.
Humphries, M. D., & Prescott, T. J. (2010) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2009.11.003 · Cited by 418
Salimi-Khorshidi, G., Smith, S. M., Keltner, J. R., Wager, T. D., & Nichols, T. E. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.12.039 · Cited by 388
Gläscher, J., Hampton, A. N., & O'Doherty, J. P. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn098 · Cited by 387 · cited 2×
“…studies (O’Doherty et al., 2003; Tobler et al., 2006; Brovelli et al., 2008; Morris et al., 2011; Pessiglione et al., 2006; McClure et al., 2003; Gläscher et al., 2009; Ramnani et al., 2004) reported deactivation of the BOLD signal relative to a baseline in striatum, predominantly ventral…”
Oliveira, F. T. P., McDonald, J. J., & Goodman, D. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2007.19.12.1994 · Cited by 387
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380
Abnormal temporal difference reward-learning signals in major depression
Kumar, P., Waiter, G., Ahearn, T., Milders, M., Reid, I., & Steele, J. D. (2008) · Brain · doi:10.1093/brain/awn136 · Cited by 374
The nucleus accumbens as part of a basal ganglia action selection circuit
Nicola, S. M. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0510-4 · Cited by 362
Differential roles of human striatum and amygdala in associative learning
Li, J., Schiller, D., Schoenbaum, G., Phelps, E. A., & Daw, N. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2904 · Cited by 353
Reward Encoding in the Monkey Anterior Cingulate Cortex
Amiez, C., Joseph, J. P., & Procyk, E. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj046 · Cited by 351
Dialogues on prediction errors
Niv, Y., & Schoenbaum, G. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.03.006 · Cited by 344
The role of the striatum in aversive learning and aversive prediction errors
Delgado, M. R., Li, J., Schiller, D., & Phelps, E. A. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0161 · Cited by 328
Nieuwenhuis, S., Slagter, H. A., Von Geusau, N. J. A., Heslenfeld, D. J., & Holroyd, C. B. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04152.x · Cited by 317
Martin, J. H. (1989) · Cited by 295
Suri, R. E., & Schultz, W. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00697-6 · Cited by 294
Jocham, G., Klein, T. A., & Ullsperger, M. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3904-10.2011 · Cited by 278
Arbuthnott, G. W., & Wickens, J. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2006.12.003 · Cited by 274
Neural signature of fictive learning signals in a sequential investment task
Lohrenz, T., McCabe, K., Camerer, C. F., & Montague, P. R. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0608842104 · Cited by 265
Prefrontal Cortex Fails to Learn from Reward Prediction Errors in Alcohol Dependence
Park, S. Q., Kahnt, T., Beck, A., Cohen, M. X., Dolan, R. J., Wrase, J., & Heinz, A. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.5587-09.2010 · Cited by 258
The Concept of the Ventral Striatum in Nonhuman Primates
HABER, S. N., & McFARLAND, N. R. (1999) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1999.tb09259.x · Cited by 253
Testing the Reward Prediction Error Hypothesis with an Axiomatic Model
Rutledge, R. B., Dean, M., Caplin, A., & Glimcher, P. W. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1747-10.2010 · Cited by 244
Guitart-Masip, M., Fuentemilla, L., Bach, D. R., Huys, Q. J. M., Dayan, P., Dolan, R. J., & Duzel, E. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6376-10.2011 · Cited by 216 · cited 2×
“…in RPE and APE might also result from a confound with he different action requirements of the learning tasks, namely ‘go’, vioral Reviews 37 (2013) 1297–1310 1307 or approach behaviours in rewarded trials as opposed to ‘no-go’ or avoid behaviours in punished trials (Guitart-Masip et al., 2011).”
Human neural learning depends on reward prediction errors in the blocking paradigm.
Tobler, P. N., O’Doherty, J. P., Dolan, R. J., & Schultz, W. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00762.2005 · Cited by 209
Choi, J.-K., Chen, Y. I., Hamel, E., & Jenkins, B. G. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2005.10.029 · Cited by 200
Learning of sequential movements by neural network model with dopamine-like reinforcement signal
Suri, R. E., & Schultz, W. (1998) · Experimental Brain Research · doi:10.1007/s002210050467 · Cited by 197
A Pallidus-Habenula-Dopamine Pathway Signals Inferred Stimulus Values
Bromberg-Martin, E. S., Matsumoto, M., Hong, S., & Hikosaka, O. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00158.2010 · Cited by 186
Perceptual Learning and Decision-Making in Human Medial Frontal Cortex
Kahnt, T., Grueschow, M., Speck, O., & Haynes, J.-D. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.054 · Cited by 179
How instructed knowledge modulates the neural systems of reward learning
Li, J., Delgado, M. R., & Phelps, E. A. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014938108 · Cited by 168
Peyron, R., García-Larrea, L., Grégoire, M. C., Convers, P., Richard, A., Lavenne, F., Barral, F. G., Mauguière, F., Michel, D., & Laurent, B. (2000) · Pain · doi:10.1016/s0304-3959(99)00190-6 · Cited by 162
Spoormaker, V. I., Schröter, M. S., Andrade, K. C., Dresler, M., Kiem, S. A., Goya‐Maldonado, R., Wetter, T. C., Holsboer, F., Sämann, P. G., & Czisch, M. (2012) · Human Brain Mapping · doi:10.1002/hbm.21369 · Cited by 153
Prediction error for free monetary reward in the human prefrontal cortex
Ramnani, N., Elliott, R., Athwal, B. S., & Passingham, R. E. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.028 · Cited by 151
Disambiguating ventral striatum fMRI-related bold signal during reward prediction in schizophrenia
Morris, R. W., Vercammen, A., Lenroot, R., Moore, L., Langton, J. M., Short, B., Kulkarni, J., Curtis, J., O'Donnell, M., Weickert, C. S., et al. (2012) · Molecular Psychiatry · doi:10.1038/mp.2011.75 · Cited by 149 · cited 3×
“Several studies (O’Doherty et al., 2003; Tobler et al., 2006; Brovelli et al., 2008; Morris et al., 2011; Pessiglione et al., 2006; McClure et al., 2003; Gläscher et al., 2009; Ramnani et al., 2004) reported deactivation of the BOLD signal relative to a baseline in striatum, predominantly ventral…”
BOLD responses to negative reward prediction errors in human habenula
Salas (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00036 · Cited by 145
Valentin, V. V., & O’Doherty, J. P. (2009) · Journal of Neurophysiology · doi:10.1152/jn.91195.2008 · Cited by 138
Multiple Forms of Value Learning and the Function of Dopamine
Balleine, B. W., Daw, N. D., & O’Doherty, J. P. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00024-5 · Cited by 135
Dopamine, Reward Prediction Error, and Economics*
Caplin, A., & Dean, M. R. W. (2008) · The Quarterly Journal of Economics · doi:10.1162/qjec.2008.123.2.663 · Cited by 132
Brain activity associated with painfully hot stimuli applied to the upper limb: A meta‐analysis
Farrell, M. J., Laird, A. R., & Egan, G. F. (2005) · Human Brain Mapping · doi:10.1002/hbm.20125 · Cited by 124 · cited 1×
“Our findings are largely consistent with the review of the PE animal literature undertaken by Schultz & Dickinson (2000) which indicated that most areas of the brain calculate a prediction error, and which suggested that PE might be a common form of neuronal processing that is not confined to updating the expected val…”
Ventral–striatal/nucleus–accumbens sensitivity to prediction errors during classification learning
Rodriguez, P. F., Aron, A. R., & Poldrack, R. A. (2006) · Human Brain Mapping · doi:10.1002/hbm.20186 · Cited by 122 · cited 3×
“N/A = no explicit mo ssumptions of RW prediction error.”
Kahnt, T., Park, S. Q., Cohen, M. X., Beck, A., Heinz, A., & Wrase, J. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21092 · Cited by 115
Volumes, spatial extents and a probabilistic atlas of the human basal ganglia and thalamus
Ahsan, R. L., Allom, R. J., Gousias, I. S., Habib, H., Turkheimer, F., Free, S. L., Lemieux, L., Myers, R., Duncan, J. S., Brooks, D. J., et al. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.06.004 · Cited by 114
Multiplicity of control in the basal ganglia: computational roles of striatal subregions
Bornstein, A. M., & Daw, N. D. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.02.009 · Cited by 114
Seger, C. A., Peterson, E. J., Cincotta, C. M., Lopez‐Paniagua, D., & Anderson, C. W. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.11.083 · Cited by 111
Segregated and Integrated Coding of Reward and Punishment in the Cingulate Cortex
Fujiwara, J., Tobler, P. N., Taira, M., Iijima, T., & Tsutsui, K. (2009) · Journal of Neurophysiology · doi:10.1152/jn.90909.2008 · Cited by 111
Ventral striatum: a critical look at models of learning and evaluation
van der Meer, M. A. A., & Redish, A. D. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.02.011 · Cited by 107
Jackson, A. A. (2000) · Nutrition Bulletin · doi:10.1046/j.1467-3010.2000.00024.x · Cited by 103
Handbook of Reward and Decision Making
Dreher, J., & Tremblay, L. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374620-7.x0001-5 · Cited by 94
The neural correlates of negative prediction error signaling in human fear conditioning
Spoormaker, V. I., Andrade, K. C., Schröter, M. S., Sturm, A., Goya‐Maldonado, R., Sämann, P. G., & Czisch, M. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.09.042 · Cited by 94
O’Sullivan, N., Szczepanowski, R., El‐Deredy, W., Mason, L., & Bentall, R. P. (2011) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2011.06.008 · Cited by 94
Goal-directed control and its antipodes
Dayan, P. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.004 · Cited by 93 · cited 1×
“Secondly, unlike machines, animals can accomplish a task via different routes through to the final action, referred to as ‘modelbased’ (goal directed) or ‘model-free’ (automatic) processes (see Daw et al., 2005; Dayan, 2009 for a discussion).”
Schönberg, T., O’Doherty, J. P., Joel, D., Inzelberg, R., Segev, Y., & Daw, N. D. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.08.011 · Cited by 93
Actor–Critic Models of Reinforcement Learning in the Basal Ganglia: From Natural to Artificial Rats
Khamassi, M., Lachèze, L., Girard, B., Berthoz, A., & Guillot, A. (2005) · Adaptive Behavior · doi:10.1177/105971230501300205 · Cited by 92
Brain mechanism of reward prediction under predictable and unpredictable environmental dynamics
Tanaka, S., Samejima, K., Okada, G., Ueda, K., Okamoto, Y., Yamawaki, S., & Doya, K. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.039 · Cited by 89
Axiomatic methods, dopamine and reward prediction error
Caplin, A., & Dean, M. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.07.007 · Cited by 88
Cohen, M. X. (2006) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsl021 · Cited by 85
A Temporal-Difference Model of Classical Conditioning
Sutton, R. S., & Barto, A. G. (1987) · eScholarship (California Digital Library) · Cited by 84
Expectancies in decision making, reinforcement learning, and ventral striatum
van der Meer, M. A. A., & Redish, A. D. (2010) · Frontiers in Neuroscience · doi:10.3389/neuro.01.006.2010 · Cited by 81
Brovelli, A., Laksiri, N., Nazarian, B., Meunier, M., & Boussaoud, D. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm198 · Cited by 81
All That Glitters … Dissociating Attention and Outcome Expectancy From Prediction Errors Signals
Roesch, M. R., Calu, D. J., Esber, G. R., & Schoenbaum, G. M. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00173.2010 · Cited by 76
Policy Adjustment in a Dynamic Economic Game
Li, J., McClure, S. M., King‐Casas, B., & Montague, P. R. (2006) · PLoS ONE · doi:10.1371/journal.pone.0000103 · Cited by 66
Baldassarre, G. (2002) · Cognitive Systems Research · doi:10.1016/s1389-0417(01)00039-0 · Cited by 63
Towards a General Theory of Neural Computation Based on Prediction by Single Neurons
Fiorillo, C. D. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003298 · Cited by 55
Dynamics of Prefrontal and Cingulate Activity during a Reward-Based Logical Deduction Task
Landmann, C., Dehaene, S., Pappatà, S., Jobert, A., Bottlaender, M., Roumenov, D., & Bihan, D. L. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhk028 · Cited by 42
The neural mechanisms of learning from competitors
Howard‐Jones, P. A., Bogacz, R., Yoo, J. H., Leonards, U., & Demetriou, S. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.06.027 · Cited by 35
Two-photon microscopy with diffractive optical elements and spatial light modulators
Watson, B. O. (2010) · Frontiers in Neuroscience · doi:10.3389/fnins.2010.00029 · Cited by 32
Cited by
The basal ganglia and the cerebellum: nodes in an integrated network
Bostan, A. C., & Strick, P. L. (2018) · Nature Reviews Neuroscience · doi:10.1038/s41583-018-0002-7 · Cited by 786
The striatum: where skills and habits meet
Graybiel, A. M., & Grafton, S. T. (2015) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a021691 · Cited by 463
A neural reward prediction error revealed by a meta-analysis of ERPs using great grand averages
Sambrook, T. D., & Goslin, J. (2015) · Psychological Bulletin · doi:10.1037/bul0000006 · Cited by 411
Behavioural and neural characterization of optimistic reinforcement learning
Lefebvre, G., Lebreton, M., Meyniel, F., Bourgeois-Gironde, S., & Palminteri, S. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0067 · Cited by 290
Contextual modulation of value signals in reward and punishment learning
Palminteri, S., Khamassi, M., Joffily, M., & Coricelli, G. (2015) · Nature Communications · doi:10.1038/ncomms9096 · Cited by 286
Metadata
- Authors
- Garrison, J., Erdeniz, B., & Done, J.
- Venue
- Neuroscience & Biobehavioral Reviews
- Publisher
- Elsevier BV
- Published
- 2013-08
- Volume
- 37
- Issue
- 7
- Pages
- 1297-1310
- ISSN
- 0149-7634 (ISSN-L)
- OpenAlex
- W2051226132
- Cited by
- 438
- References
- 124
- FWCI
- 1.77
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.