All That Glitters … Dissociating Attention and Outcome Expectancy From Prediction Errors Signals
Open accessThe DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
Initially reported in dopamine neurons, neural correlates of prediction errors have now been shown in a variety of areas, including orbitofrontal cortex, ventral striatum, and amygdala. Yet changes in neural activity to an outcome or cues that precede it can reflect other processes. We review the recent literature and show that although activity in dopamine neurons appears to signal prediction errors, similar activity in orbitofrontal cortex, basolateral amygdala, and ventral striatum does not. Instead, increased firing in basolateral amygdala to unexpected outcomes likely reflects attention, whereas activity in orbitofrontal cortex and ventral striatum is unaffected by prior expectations and may provide information on outcome expectancy. These results have important implications for how these areas interact to facilitate learning and guide behavior.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · cited 6×
“This pattern of firing does not meet the criteria for a Rescorla– Wagner or TDRL prediction error signal.”
BackgroundInfluentialPearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · cited 12×
“Classical learning theory postulates that learning to predict unexpected events is driven by errors in reward prediction (Pearce and Hall 1980; Rescorla and Wagner 1972; Sutton 1988).”
BackgroundMethodInfluentialThe amygdala: vigilance and emotion
Davis, M., & Whalen, P. J. (2001) · Molecular Psychiatry · doi:10.1038/sj.mp.4000812 · Cited by 2,882
Kapur, S. (2003) · American Journal of Psychiatry · doi:10.1176/appi.ajp.160.1.13 · Cited by 2,813
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Neuronal Coding of Prediction Errors
Schultz, W., & Dickinson, A. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.473 · Cited by 1,427
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378
Relative reward preference in primate orbitofrontal cortex
Tremblay, L., & Schultz, W. (1999) · Nature · doi:10.1038/19525 · Cited by 1,314
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Lateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292 · cited 3×
“…accumulating that other areas contribute to this process (Bayer and Glimcher 2005; Belova et al. 2007; D’Ardenne et al. 2008; Knutson et al. 2003; Matsumoto and Hikosaka 2007; McClure et al. 2003; Nobre et al. 1999; Roesch et al. 2006, 2007, 2009, 2010 [some of the most recently published study…”
BackgroundInfluentialAdaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276
Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex
Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2003) · Science · doi:10.1126/science.1087919 · Cited by 1,165
Neural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809
Quirk, G. J., Repa, J. C., & LeDoux, J. E. (1995) · Neuron · doi:10.1016/0896-6273(95)90092-6 · Cited by 808
Hare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792
Uniform Inhibition of Dopamine Neurons in the Ventral Tegmental Area by Aversive Stimuli
Ungless, M. A., Magill, P. J., & Bolam, J. P. (2004) · Science · doi:10.1126/science.1093360 · Cited by 782
The amygdala, reward and emotion
Murray, E. A. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.08.013 · Cited by 679
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661
Amygdala circuitry in attentional and representational processes
Holland, P. C., & Gallagher, M. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01271-6 · Cited by 618
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
Self-citationOrbitofrontal Cortex and Representation of Incentive Value in Associative Learning
Gallagher, M., McMahan, R. W., & Schoenbaum, G. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-15-06610.1999 · Cited by 593
Self-citationIzquierdo, A., Suda, R. K., & Murray, E. A. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.1921-04.2004 · Cited by 588
Schoenbaum, G., Chiba, A. A., & Gallagher, M. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-05-01876.1999 · Cited by 579 · cited 1×
“This could occur locally within ABL and may be evident in the rapid onset of cue-selective activity that characterizes firing in ABL in a variety of tasks (Quirk et al. 1995; Schoenbaum et al. 1999; Tye et al. 2008) or it may occur in downstream targets.”
BackgroundSelf-citationImportance of unpredictability for reward responses in primate dopamine neurons
Mirenowicz, J., & Schultz, W. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.2.1024 · Cited by 569
Yacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537
Is the short-latency dopamine response too short to signal reward error?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01373-3 · Cited by 506
Groenewegen, H. J., Berendse, H. W., Wolters, J. G., & Lohman, A. H. M. (1991) · Progress in brain research · doi:10.1016/s0079-6123(08)62677-1 · Cited by 506
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429
Processing of Temporal Unpredictability in Human and Animal Amygdala
Herry, C., Bach, D. R., Esposito, F., Di Salle, F., Perrig, W. J., Scheffler, K., Lüthi, A., & Seifritz, E. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.5218-06.2007 · Cited by 401
Orbitofrontal Cortex Mediates Outcome Encoding in Pavlovian But Not Instrumental Conditioning
Ostlund, S. B., & Balleine, B. W. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.5443-06.2007 · Cited by 379 · cited 1×
“…implicated in signaling information about specific outcomes (Burke et al. 2008; Gallagher et al. 1999; Izquierdo et al. 2004; McDannald et al. 2005; Ostlund and Balleine 2007), whereas VS is more clearly implicated in signaling information about the general affective or emotional value that a…”
BackgroundPrediction error as a linear function of reward probability is coded in human nucleus accumbens
Abler, B., Walter, H., Erk, S., Kammerer, H., & Spitzer, M. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.001 · Cited by 376
Mcdonald, A. J. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90248-m · Cited by 372
Le Pelley, M. E. (2004) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/02724990344000141 · Cited by 360
Hall, J., Parkinson, J. A., Connor, T. M., Dickinson, A., & Everitt, B. J. (2001) · European Journal of Neuroscience · doi:10.1046/j.0953-816x.2001.01577.x · Cited by 355
Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala
Belova, M. A., Paton, J. J., Morrison, S. E., & Salzman, C. D. (2007) · Neuron · doi:10.1016/j.neuron.2007.08.004 · Cited by 349
Linking nucleus accumbens dopamine and blood oxygenation
Knutson, B., & Gibbs, S. E. B. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0686-7 · Cited by 337
The temporal precision of reward prediction in dopamine neurons
Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008) · Nature Neuroscience · doi:10.1038/nn.2159 · Cited by 310
Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm
Tobler, P. N., Dickinson, A., & Schultz, W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-32-10402.2003 · Cited by 300 · cited 2×
“…error encoding (Bayer and Glimcher 2005; D’Ardenne et al. 2008; Fiorillo et al. 2003; Hollerman and Schultz 1998; Matsumoto and Hikosaka 2009; Mirenowicz and Schultz 1994; Montague et al. 1996; Morris et al. 2006; Pan et al. 2005; Roesch et al. 2007; Tobler et al. 2003; Waelti et al. 2001).”
BackgroundCorrelated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-30-09913.2003 · Cited by 298
Rapid strengthening of thalamo-amygdala synapses mediates cue–reward learning
Tye, K. M., Stuber, G. D., de Ridder, B., Bonci, A., & Janak, P. H. (2008) · Nature · doi:10.1038/nature06963 · Cited by 216
Human neural learning depends on reward prediction errors in the blocking paradigm.
Tobler, P. N., O’Doherty, J. P., Dolan, R. J., & Schultz, W. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00762.2005 · Cited by 209
Cited by
The ecology of human fear: survival optimization and the nervous system
Mobbs, D., Hagan, C. C., Dalgleish, T., Silston, B., & Prévost, C. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00055 · Cited by 498
Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Garrison, J., Erdeniz, B., & Done, J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.023 · Cited by 438
Emotion and cognition and the amygdala: From “what is it?” to “what's to be done?”
Pessoa, L. (2010) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2010.06.038 · Cited by 403
Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4652-10.2011 · Cited by 379
“This second effect indicates that behavioral changes are influenced by the unsigned reward prediction error of the outcome, similar to that predicted in attentional theories of learning (Courville et al., 2006; Lin and Nicolelis, 2008; Roesch et al., 2010).”
ResultStatistical learning of visual transitions in monkey inferotemporal cortex
Meyer, T., & Olson, C. R. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1112895108 · Cited by 325
Metadata
- Authors
- Roesch, M. R., Calu, D. J., Esber, G. R., & Schoenbaum, G.
- Venue
- Journal of Neurophysiology
- Publisher
- American Physiological Society
- Published
- 2010-08
- Volume
- 104
- Issue
- 2
- Pages
- 587-595
- ISSN
- 0022-3077 (ISSN-L)
- 1522-1598
- OpenAlex
- W1984500243
- Cited by
- 68
- References
- 52
- FWCI
- 5.61
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.