Cerebral Correlates of Salient Prediction Error for Different Rewards and Punishments
The DOI matches its Crossref record; 98 of the 98 references it cites were found in a registry or an index.
Abstract
Learning to predict rewarding and aversive outcomes is based on the comparison between predicted and actual outcomes (prediction error: PE). Recent electrophysiological studies reported that during a Pavlovian procedure some dopamine neurons code a classical PE signal while a larger population of dopaminergic neurons reflect a "salient" prediction error (SPE) signal, being excited both by unpredictable aversive events and by rewards. Yet, it is still unclear whether specific human brain structures receiving afferents from dopaminergic neurons code a SPE and whether this signal depends upon reinforcer type. Here, we used a model-based functional magnetic resonance imaging approach implementing a reinforcement learning model to compute the PE while subjects underwent a Pavlovian conditioning procedure with 2 types of rewards (pleasant juice and monetary gain) and 2 types of punishments (aversive juice and aversive picture). The results revealed that activity of a brain network composed of the striatum, anterior insula, and anterior cingulate cortex covaried with a SPE for appetitive and aversive juice. Moreover, amygdala activity correlated with a SPE for these 2 reinforcers and for aversive pictures. These results provide insights into the neurobiological mechanisms underlying the ability to learn stimuli-rewards and stimuli-punishments contingencies, by demonstrating that the network reflecting the SPE depends upon reinforcement's type.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The assessment and analysis of handedness: The Edinburgh inventory
Oldfield, R. C. (1971) · Neuropsychologia · doi:10.1016/0028-3932(71)90067-4 · Cited by 35,900
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 9,661
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 2×
“A current theory proposes that dopaminergic neurons encode a form of valence for learning and motivating rewardseeking behavior (Schultz et al. 1997; Berridge and Robinson 1998; Wise 2004) while another theory states that these neurons encode a form of salience (or ‘‘alerting’’) for shifting…”
LeDoux, J. E. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.155 · Cited by 8,444 · cited 2×
“Many of these processes, which include learning when to exhibit defensive as opposed to approach behavior, are thus likely to involve the amygdala (LeDoux 2000).”
(2005) · CTIT technical reports series · Cited by 5,307
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
The Behavior of Organisms: An Experimental Analysis
Jones, F. N., & Skinner, B. F. (1939) · The American Journal of Psychology · doi:10.2307/1416495 · Cited by 4,398
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data
Eickhoff, S. B., Stephan, K. E., Mohlberg, H., Grefkes, C., Fink, G. R., Amunts, K., & Zilles, K. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.034 · Cited by 4,033 · cited 2×
“Activations Localization and Reported Statistics Anatomic labeling of activated regions was done using the SPM Anatomy toolbox (Eickhoff et al. 2005) and the probabilistic atlas of Hammers et al.”
Dopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244 · cited 2×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 1×
“Keywords: appetitive conditioning, aversive conditioning, classical conditioning, fMRI, reinforcement learning model”
Valid conjunction inference with the minimum statistic
Nichols, T., Brett, M., Andersson, J., Wager, T., & Poline, J.-B. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.005 · Cited by 1,880 · cited 3×
“They were normalized to SPM5’s EPI template in Montreal Neurological Institute (MNI) space with a resampled voxel size of 3 3 3 3 3 mm and spatial smoothed using a Gaussian kernel with full-width at halfmaximum of 8 mm.”
InfluentialThe neural basis of loss aversion in decision-making under risk
Tom, S. M., Fox, C. R., Trepel, C., & Poldrack, R. A. (2007) · Science · doi:10.1126/science.1134239 · Cited by 1,733 · cited 3×
“Note that we did not include monetary losses in the experiment because monetary gains and losses are known to weigh differently (Tom et al. 2007) and because 1) it can be questioned whether monetary loss acts as a primary punishment like aversive liquids or aversive pictures; 2) monetary losses are the removal of a va…”
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584 · cited 6×
“…report midbrainrelated PE activity in different paradigms (Dreher et al. 2006; D’Ardenne et al. 2008) while others did not (McClure et al. 2003; O’Doherty, Dayan, et al. 2003; Abler et al. 2006; Li et al. 2006; Pessiglione et al. 2006; Behrens et al. 2008; Hare et al. 2008; Sescousse et al. 2010).”
InfluentialTemporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
FMRI visualization of brain activity during a monetary incentive delay task
Knutson, B., Westdorp, A., Kaiser, E., & Hommer, D. (2000) · NeuroImage · doi:10.1006/nimg.2000.0593 · Cited by 1,414 · cited 2×
“…stimuli, aversive juices, odors, tones, or visual stimuli (LaBar et al. 1998; Buchel et al. 1999; Gottfried, Deichmann, et al. 2002; Seymour et al. 2004; Knight et al. 2010; Sarinopoulos et al. 2010) or monetary losses (Delgado et al. 2000, 2008; Knutson et al. 2000; Nieuwenhuis et al. 2005).”
Neural systems responding to degrees of uncertainty in human decision-making
Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., & Camerer, C. F. (2005) · Science · doi:10.1126/science.1115327 · Cited by 1,411 · cited 2×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404 · cited 2×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
LaBar, K. S., Gatenby, J. C., Gore, J. C., LeDoux, J. E., & Phelps, E. A. (1998) · Neuron · doi:10.1016/s0896-6273(00)80475-4 · Cited by 1,402 · cited 2×
“…substrates of aversive conditioning using a variety of punishments, such as painful stimuli, aversive juices, odors, tones, or visual stimuli (LaBar et al. 1998; Buchel et al. 1999; Gottfried, Deichmann, et al. 2002; Seymour et al. 2004; Knight et al. 2010; Sarinopoulos et al. 2010) or…”
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378
Relative reward preference in primate orbitofrontal cortex
Tremblay, L., & Schultz, W. (1999) · Nature · doi:10.1038/19525 · Cited by 1,314 · cited 1×
“Indeed, a large body of literature reports context-dependent activity in different components of the reward system (Tremblay and Schultz 1999; Nieuwenhuis et al. 2005).”
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · cited 1×
“Yet, it is still unclear whether specific human brain structures receiving afferents from dopaminergic neurons code a SPE and whether this signal depends upon reinforcer type.”
Lateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292 · cited 2×
“In the same line, the neural activity of the lateral habenula has been reported to increase when expected rewards are not delivered or when unexpected punishments are received (Matsumoto and Hikosaka 2007, 2009a).”
Hammers, A., Allom, R., Koepp, M. J., Free, S. L., Myers, R., Lemieux, L., Mitchell, T. N., Brooks, D. J., & Duncan, J. S. (2003) · Human Brain Mapping · doi:10.1002/hbm.10123 · Cited by 1,286 · cited 5×
“Moreover, because the substantia nigra (SN) is known to be a key region in processing rewarding and aversive events (Schultz 1998; Matsumoto and Hikosaka 2009b), we used the probabilistic brain atlas of Hammers et al. (2003) to build bilateral ROIs in this region.”
InfluentialChanges in brain activity related to eating chocolate: From pleasure to aversion
Small, D. M. (2001) · Brain · doi:10.1093/brain/124.9.1720 · Cited by 1,179 · cited 2×
“Interestingly, previous reinforcer devaluation fMRI studies reported that the amygdala and orbitofrontal cortex showed reduced activation after feeding subjects to satiety with one type of food but not with another (Small et al. 2001; Gottfried et al. 2003; Kringelbach et al. 2003).”
Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex
Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2003) · Science · doi:10.1126/science.1087919 · Cited by 1,165 · cited 5×
“Interestingly, previous reinforcer devaluation fMRI studies reported that the amygdala and orbitofrontal cortex showed reduced activation after feeding subjects to satiety with one type of food but not with another (Small et al. 2001; Gottfried et al. 2003; Kringelbach et al. 2003).”
InfluentialTracking the Hemodynamic Responses to Reward and Punishment in the Striatum
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.6.3072 · Cited by 1,116 · cited 3×
“…stimuli, aversive juices, odors, tones, or visual stimuli (LaBar et al. 1998; Buchel et al. 1999; Gottfried, Deichmann, et al. 2002; Seymour et al. 2004; Knight et al. 2010; Sarinopoulos et al. 2010) or monetary losses (Delgado et al. 2000, 2008; Knutson et al. 2000; Nieuwenhuis et al. 2005).”
Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967
The Human Amygdala: An Evolved System for Relevance Detection
Sander, D., Grafman, J., & Zalla, T. (2003) · Reviews in the Neurosciences · doi:10.1515/revneuro.2003.14.4.303 · Cited by 944 · cited 2×
“with studies emphasizing the role of the amygdala as a detector of behaviorally relevant stimuli (Sander et al. 2003) and during conditioning procedures with both appetitive and aversive stimuli (Buchel et al.”
Associative learning of social value
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. S. (2008) · Nature · doi:10.1038/nature07538 · Cited by 925 · cited 3×
“…such activity in the lateral habenula (McClure et al. 2003; O’Doherty, Dayan, et al. 2003; Seymour et al. 2004; Abler et al. 2006; Dreher et al. 2006; Li et al. 2006; Pessiglione et al. 2006; Behrens et al. 2008; D’Ardenne et al. 2008; Hare et al. 2008; Rutledge et al. 2010; Sescousse et al. 2010).”
The primate amygdala represents the positive and negative value of visual stimuli during learning
Paton, J. J., Belova, M. A., Morrison, S. E., & Salzman, C. D. (2006) · Nature · doi:10.1038/nature04490 · Cited by 917 · cited 4×
“…in the primate amygdala support both valence-specific and salience-specific processes, with both common and distinct populations of neurons involved in learning the association between conditioned visual stimuli and rewards or punishments (Buchel et al. 1999; Paton et al. 2006; Belova et al. 2008).”
InfluentialTemporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · cited 6×
“such as juice (Berns et al. 2001; McClure et al. 2003; O’Doherty et al. 2004), odor (Gottfried et al.”
InfluentialA cellular mechanism of reward-related learning
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001) · Nature · doi:10.1038/35092560 · Cited by 879 · cited 2×
“(Reynolds et al. 2001) but also during aversive conditioning”
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844 · cited 2×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
Kringelbach, M. L. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.10.1064 · Cited by 824 · cited 2×
“Interestingly, previous reinforcer devaluation fMRI studies reported that the amygdala and orbitofrontal cortex showed reduced activation after feeding subjects to satiety with one type of food but not with another (Small et al. 2001; Gottfried et al. 2003; Kringelbach et al. 2003).”
Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli
Brischoux, F., Chakraborty, S., Brierley, D. I., & Ungless, M. A. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0811507106 · Cited by 801 · cited 2×
“Moreover, recent results in rodents confirm that, while some dopaminergic neurons of the VTA are inhibited by aversive stimuli, others are excited by these same stimuli (Brischoux et al. 2009).”
Hare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792 · cited 3×
“…such activity in the lateral habenula (McClure et al. 2003; O’Doherty, Dayan, et al. 2003; Seymour et al. 2004; Abler et al. 2006; Dreher et al. 2006; Li et al. 2006; Pessiglione et al. 2006; Behrens et al. 2008; D’Ardenne et al. 2008; Hare et al. 2008; Rutledge et al. 2010; Sescousse et al. 2010).”
Dissociation of Neural Representation of Intensity and Affective Valuation in Human Gustation
Small, D. M., Gregory, M. D., Mak, Y. E., Gitelman, D., Mesulam, M. M., & Parrish, T. (2003) · Neuron · doi:10.1016/s0896-6273(03)00467-7 · Cited by 788 · cited 2×
“valence (Small et al. 2003), and correlates with the PE for both aversive and appetitive stimuli when using pain and pain relief as reinforcers (Seymour et al.”
Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops
Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2004) · Nature Neuroscience · doi:10.1038/nn1279 · Cited by 779 · cited 2×
“…between conditioned stimuli and different types of rewards, such as juice (Berns et al. 2001; McClure et al. 2003; O’Doherty et al. 2004), odor (Gottfried et al. 2003), money (Tanaka et al. 2004; Abler et al. 2006; Pessiglione et al. 2006), and attractive faces (Bray and O’Doherty 2007).”
Predictability modulates human brain response to reward
Berns, G. S., McClure, S. M., Pagnoni, G., & Montague, P. R. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-08-02793.2001 · Cited by 711 · cited 2×
“…system show responses consistent with the RPE when learning associations between conditioned stimuli and different types of rewards, such as juice (Berns et al. 2001; McClure et al. 2003; O’Doherty et al. 2004), odor (Gottfried et al. 2003), money (Tanaka et al. 2004; Abler et al. 2006;…”
The amygdala, reward and emotion
Murray, E. A. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.08.013 · Cited by 679
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661 · cited 1×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
O'Doherty, J., Critchley, H., Deichmann, R., & Dolan, R. J. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-21-07931.2003 · Cited by 611 · cited 2×
“1C), while distinguishing between the modality (gustatory or visual) and the nature (primary/ immediate or secondary/delayed) of the reinforcers.”
Temporal difference models describe higher-order learning in humans
Seymour, B., O'Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., Friston, K. J., & Frackowiak, R. S. (2004) · Nature · doi:10.1038/nature02581 · Cited by 605 · cited 12×
“First, we identified several brain regions that have established roles during aversive and appetitive conditioning: the striatum, the insula, the anterior cingulate cortex (ACC) and the amygdala (Jensen et al. 2003; Seymour et al. 2004; Bray and O’Doherty 2007; Herwig et al. 2007; Sescousse et al. 2010).”
InfluentialRay Li, C.-S., Huang, C., Constable, R. T., & Sinha, R. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.3741-05.2006 · Cited by 600 · cited 4×
“…such activity in the lateral habenula (McClure et al. 2003; O’Doherty, Dayan, et al. 2003; Seymour et al. 2004; Abler et al. 2006; Dreher et al. 2006; Li et al. 2006; Pessiglione et al. 2006; Behrens et al. 2008; D’Ardenne et al. 2008; Hare et al. 2008; Rutledge et al. 2010; Sescousse et al. 2010).”
InfluentialRoesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597 · cited 2×
“…outcome is better than expected (unexpected reward or omission of expected punishment) and a negative response when it is worse than expected (unexpected punishment or omission of expected reward) (Schultz 1998; Bayer and Glimcher 2005; Pan et al. 2005; Roesch et al. 2007; Zaghloul et al. 2009).”
Modulation of Caudate Activity by Action Contingency
Tricomi, E. M., Delgado, M. R., & Fiez, J. A. (2004) · Neuron · doi:10.1016/s0896-6273(03)00848-1 · Cited by 574 · cited 3×
“Other studies have varied salience but have not independently varied valence across gains and losses (Jensen et al. 2003; Tricomi et al. 2004; Zink et al. 2006; Bjork and Hommer 2007).”
Representation of negative motivational value in the primate lateral habenula
Matsumoto, M., & Hikosaka, O. (2009) · Nature Neuroscience · doi:10.1038/nn.2233 · Cited by 547 · cited 5×
“Moreover, because the substantia nigra (SN) is known to be a key region in processing rewarding and aversive events (Schultz 1998; Matsumoto and Hikosaka 2009b), we used the probabilistic brain atlas of Hammers et al. (2003) to build bilateral ROIs in this region.”
InfluentialYacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537 · cited 5×
“striatum (O’Doherty Critchley, et al. 2003; Pessiglione et al. 2006; Yacubian et al. 2006) did so during instrumental learning paradigms.”
InfluentialMesolimbic dopaminergic pathways in fear conditioning
PEZZE, M., & FELDON, J. (2004) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2004.09.004 · Cited by 514 · cited 4×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
InfluentialIs the short-latency dopamine response too short to signal reward error?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01373-3 · Cited by 506 · cited 2×
“…of valence for learning and motivating rewardseeking behavior (Schultz et al. 1997; Berridge and Robinson 1998; Wise 2004) while another theory states that these neurons encode a form of salience (or ‘‘alerting’’) for shifting attention to unpredicted events (Redgrave et al. 1999; Horvitz 2000).”
Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-24-10829.2002 · Cited by 477
Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems
Tobler, P. N., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00745.2006 · Cited by 472 · cited 2×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
Role of the Amygdala in Decision‐Making
BECHARA, A., DAMASIO, H., & DAMASIO, A. R. (2003) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2003.tb07094.x · Cited by 464 · cited 2×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
Differential Encoding of Losses and Gains in the Human Striatum
Seymour, B., Daw, N., Dayan, P., Singer, T., & Dolan, R. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0400-07.2007 · Cited by 462 · cited 3×
“The study was approved by the local ethics committee (Lyon) and all subjects gave informed consent.”
Direct Activation of the Ventral Striatum in Anticipation of Aversive Stimuli
Jensen, J., McIntosh, A. R., Crawley, A. P., Mikulis, D. J., Remington, G., & Kapur, S. (2003) · Neuron · doi:10.1016/s0896-6273(03)00724-4 · Cited by 455 · cited 8×
“This result extends previous findings that anterior insula activity is often reported with various aversive events (Jensen et al. 2003; Liu et al. 2007), is driven by taste intensity irrespective of valence (Small et al. 2003), and correlates with the PE for both aversive and appetitive stimuli…”
InfluentialCritchley, H. D., Mathias, C. J., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00588-3 · Cited by 441 · cited 2×
“RTs have been shown to be good indicators of conditioning (Critchley et al. 2002; Gottfried et al. 2003) and to be correlated with the prediction v (t) estimated by a reinforcement learning model (Seymour et al. 2004).”
Opponent appetitive-aversive neural processes underlie predictive learning of pain relief
Seymour, B., O'Doherty, J. P., Koltzenburg, M., Wiech, K., Frackowiak, R., Friston, K., & Dolan, R. (2005) · Nature Neuroscience · doi:10.1038/nn1527 · Cited by 439 · cited 10×
“Our results are thus consistent with studies emphasizing the role of the amygdala as a detector of behaviorally relevant stimuli (Sander et al. 2003) and during conditioning procedures with both appetitive and aversive stimuli (Buchel et al. 1999; Seymour et al. 2005; Paton et al. 2006).”
InfluentialMedial prefrontal cell activity signaling prediction errors of action values
Matsumoto, M., Matsumoto, K., Abe, H., & Tanaka, K. (2007) · Nature Neuroscience · doi:10.1038/nn1890 · Cited by 437 · cited 2×
“action to make in a given context (Matsumoto et al. 2007) and the activity of this region has been shown to correlate with PE for both pain increase and pain relief (Seymour et al.”
Human Striatal Responses to Monetary Reward Depend On Saliency
Zink, C. F., Pagnoni, G., Martin-Skurski, M. E., Chappelow, J. C., & Berns, G. S. (2004) · Neuron · doi:10.1016/s0896-6273(04)00183-7 · Cited by 434 · cited 2×
“…but not Pavlovian conditioning, engage the striatum for monetary reward is directly supported by a study reporting higher striatal activation when the money delivery depended upon subject’s response as compared with when the receipt of money was independent of subject’s actions (Zink et al. 2004).”
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429
Büchel, C., Dolan, R. J., Armony, J. L., & Friston, K. J. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-24-10869.1999 · Cited by 405 · cited 5×
“Our results are thus consistent with studies emphasizing the role of the amygdala as a detector of behaviorally relevant stimuli (Sander et al. 2003) and during conditioning procedures with both appetitive and aversive stimuli (Buchel et al. 1999; Seymour et al. 2005; Paton et al. 2006).”
InfluentialPrediction error as a linear function of reward probability is coded in human nucleus accumbens
Abler, B., Walter, H., Erk, S., Kammerer, H., & Spitzer, M. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.001 · Cited by 376 · cited 6×
“…report midbrainrelated PE activity in different paradigms (Dreher et al. 2006; D’Ardenne et al. 2008) while others did not (McClure et al. 2003; O’Doherty, Dayan, et al. 2003; Abler et al. 2006; Li et al. 2006; Pessiglione et al. 2006; Behrens et al. 2008; Hare et al. 2008; Sescousse et al. 2010).”
InfluentialGottfried, J. A., Deichmann, R., Winston, J. S., & Dolan, R. J. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-24-10819.2002 · Cited by 354 · cited 1×
“Keywords: appetitive conditioning, aversive conditioning, classical conditioning, fMRI, reinforcement learning model”
The Architecture of Reward Value Coding in the Human Orbitofrontal Cortex
Sescousse, G., Redouté, J., & Dreher, J.-C. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.3501-10.2010 · Cited by 353 · cited 5×
“…report midbrainrelated PE activity in different paradigms (Dreher et al. 2006; D’Ardenne et al. 2008) while others did not (McClure et al. 2003; O’Doherty, Dayan, et al. 2003; Abler et al. 2006; Li et al. 2006; Pessiglione et al. 2006; Behrens et al. 2008; Hare et al. 2008; Sescousse et al. 2010).”
InfluentialSelf-citationSome observations on cortical inputs to the macaque monkey amygdala: An anterograde tracing study
Stefanacci, L., & Amaral, D. G. (2002) · Journal of Comparative Neurology · doi:10.1002/cne.10339 · Cited by 345 · cited 1×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
Methods and Software for fMRI Analysis of Clinical Subjects
Mazaika, P. K., Hoeft, F., Glover, G. H., & Reiss, A. L. (2009) · NeuroImage · doi:10.1016/s1053-8119(09)70238-1 · Cited by 339 · cited 1×
“First, outlier scans ( >1.5% variation in global intensity or >0.5 mm/time repetition scan-to-scan motion) were detected using the ArtRepair SPM toolbox http://spnl.stanford.edu/tools/ArtRepair/ArtRepair.htm (Mazaika et al. 2009).”
InfluentialThe role of the striatum in aversive learning and aversive prediction errors
Delgado, M. R., Li, J., Schiller, D., & Phelps, E. A. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0161 · Cited by 328 · cited 1×
“…stimuli, aversive juices, odors, tones, or visual stimuli (LaBar et al. 1998; Buchel et al. 1999; Gottfried, Deichmann, et al. 2002; Seymour et al. 2004; Knight et al. 2010; Sarinopoulos et al. 2010) or monetary losses (Delgado et al. 2000, 2008; Knutson et al. 2000; Nieuwenhuis et al. 2005).”
Activity in human reward-sensitive brain areas is strongly context dependent
Nieuwenhuis, S., Heslenfeld, D. J., Alting von Geusau, N. J., Mars, R. B., Holroyd, C. B., & Yeung, N. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.043 · Cited by 289 · cited 2×
“…stimuli, aversive juices, odors, tones, or visual stimuli (LaBar et al. 1998; Buchel et al. 1999; Gottfried, Deichmann, et al. 2002; Seymour et al. 2004; Knight et al. 2010; Sarinopoulos et al. 2010) or monetary losses (Delgado et al. 2000, 2008; Knutson et al. 2000; Nieuwenhuis et al. 2005).”
Human substantia nigra neurons encode unexpected financial rewards
Zaghloul, K. A., Blanco, J. A., Weidemann, C. T., McGill, K., Jaggi, J. L., Baltuch, G. H., & Kahana, M. J. (2009) · Science · doi:10.1126/science.1167342 · Cited by 264
Valence and salience contribute to nucleus accumbens activation
Cooper, J. C., & Knutson, B. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.08.009 · Cited by 256 · cited 3×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
Testing the Reward Prediction Error Hypothesis with an Axiomatic Model
Rutledge, R. B., Dean, M., Caplin, A., & Glimcher, P. W. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1747-10.2010 · Cited by 244 · cited 3×
“…such activity in the lateral habenula (McClure et al. 2003; O’Doherty, Dayan, et al. 2003; Seymour et al. 2004; Abler et al. 2006; Dreher et al. 2006; Li et al. 2006; Pessiglione et al. 2006; Behrens et al. 2008; D’Ardenne et al. 2008; Hare et al. 2008; Rutledge et al. 2010; Sescousse et al. 2010).”
Liu, X., Powell, D. K., Wang, H., Gold, B. T., Corbly, C. R., & Joseph, J. E. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.5227-06.2007 · Cited by 241 · cited 5×
“In most experiments involving monetary rewards, receipt of money is contingent upon subjects’ action (i.e., monetary decision making task, guessing task, and gambling task) and money delivery only follows a correct response (O’Doherty, Critchley, et al. 2003; Dreher et al. 2006; Liu et al. 2007).”
InfluentialUncertainty during anticipation modulates neural responses to aversion in human insula and amygdala
Sarinopoulos, I., Grupe, D. W., Mackiewicz, K. L., Herrington, J. D., Lor, M., Steege, E. E., & Nitschke, J. B. (2010) · Cerebral Cortex · doi:10.1093/cercor/bhp155 · Cited by 241 · cited 2×
“…stimuli, aversive juices, odors, tones, or visual stimuli (LaBar et al. 1998; Buchel et al. 1999; Gottfried, Deichmann, et al. 2002; Seymour et al. 2004; Knight et al. 2010; Sarinopoulos et al. 2010) or monetary losses (Delgado et al. 2000, 2008; Knutson et al. 2000; Nieuwenhuis et al. 2005).”
Moment-to-moment tracking of state value in the amygdala
Belova, M. A., Paton, J. J., & Salzman, C. D. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1400-08.2008 · Cited by 221 · cited 4×
“…in the primate amygdala support both valence-specific and salience-specific processes, with both common and distinct populations of neurons involved in learning the association between conditioned visual stimuli and rewards or punishments (Buchel et al. 1999; Paton et al. 2006; Belova et al. 2008).”
InfluentialActivation in the VTA and nucleus accumbens increases in anticipation of both gains and losses
Carter, R. M. (2009) · Frontiers in Behavioral Neuroscience · doi:10.3389/neuro.08.021.2009 · Cited by 195 · cited 3×
“Other studies reported that ventral striatal activation increases in anticipation of both gains and losses, supporting a saliency interpretation (Carter et al. 2009) or that some components of the reward system depend on valence (Seymour et al. 2007).”
Neural coding of distinct statistical properties of reward information in humans
Dreher, J.-C., Kohn, P., & Berman, K. F. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj004 · Cited by 180
Self-citationSeparate brain regions code for salience vs. valence during reward prediction in humans
Jensen, J., Smith, A. J., Willeit, M., Crawley, A. P., Mikulis, D. J., Vitcu, I., & Kapur, S. (2007) · Human Brain Mapping · doi:10.1002/hbm.20274 · Cited by 178 · cited 6×
“…been observed both in the human striatum, which shows increased activity for unexpected reward and for unexpected punishment (Seymour et al. 2005; Jensen et al. 2007) and in the striatum of nonhuman primates, which represent both appetitive and aversive events (Ravel et al. 2003) Together, our…”
InfluentialNeural Coding of Reward-Prediction Error Signals During Classical Conditioning With Attractive Faces
Bray, S., & O'Doherty, J. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01211.2006 · Cited by 172 · cited 6×
“First, we identified several brain regions that have established roles during aversive and appetitive conditioning: the striatum, the insula, the anterior cingulate cortex (ACC) and the amygdala (Jensen et al. 2003; Seymour et al. 2004; Bray and O’Doherty 2007; Herwig et al. 2007; Sescousse et al. 2010).”
InfluentialHuman striatal activation reflects degree of stimulus saliency
Zink, C. F., Pagnoni, G., Chappelow, J., Martin-Skurski, M., & Berns, G. S. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2005.08.006 · Cited by 168 · cited 5×
“Salient trials have also been reported to engage the dorsal striatum, when salience is defined by a monetary reception dependent upon a correct response (active condition) as opposed to a passive monetary reception (Tricomi et al. 2004; Zink et al. 2006).”
InfluentialThe role of serotonin in the regulation of patience and impulsivity
Miyazaki, K., Miyazaki, K. W., & Doya, K. (2012) · Molecular Neurobiology · doi:10.1007/s12035-012-8232-6 · Cited by 162 · cited 2×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
The roles of the cerebellum and basal ganglia in timing and error prediction
Dreher, J., & Grafman, J. (2002) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2002.02212.x · Cited by 162
Self-citationMenon, M., Jensen, J., Vitcu, I., Graff‐Guerrero, A., Crawley, A. P., Smith, M. A., & Kapur, S. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.10.020 · Cited by 157
Neural Correlates of Variations in Event Processing during Learning in Basolateral Amygdala
Roesch, M. R., Calu, D. J., Esber, G. R., & Schoenbaum, G. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.5781-09.2010 · Cited by 150 · cited 2×
“A recent study in rodents also demonstrated that the amygdala provides an unsigned error signal (Roesch et al. 2010) with characteristics consistent”
Distinct Tonic and Phasic Anticipatory Activity in Lateral Habenula and Dopamine Neurons
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.06.016 · Cited by 148 · cited 2×
“that dopamine neurons may follow both theories at different times during a single task (Bromberg-Martin et al. 2010) and our findings that some components of the reward system”
The Role of the Amygdala in Signaling Prospective Outcome of Choice
Kahn, I., Yeshurun, Y., Rotshtein, P., Fried, I., Ben-Bashat, D., & Hendler, T. (2002) · Neuron · doi:10.1016/s0896-6273(02)00626-8 · Cited by 141
BOLD responses to negative reward prediction errors in human habenula
Salas (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00036 · Cited by 136
Young, A. M. J. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2004.03.003 · Cited by 127
Machado, C. J., Kazama, A. M., & Bachevalier, J. (2009) · Emotion · doi:10.1037/a0014539 · Cited by 125 · cited 2×
“Consistent with its role in processing both emotional valence and intensity (Machado et al. 2009), it is in an ideal position to integrate appetitive and aversive events to direct emotional responses.”
Anticipating instrumentally obtained and passively-received rewards: A factorial fMRI investigation
Bjork, J. M., & Hommer, D. W. (2006) · Behavioural Brain Research · doi:10.1016/j.bbr.2006.10.034 · Cited by 119 · cited 1×
“Moreover, no brain region was found to elicit an activity modulated by the RPE signal for the gustatory or visual conditions when restricting the conjunctions analyses to one or the other of these 2 modalities.”
Ravel, S., Legallet, E., & Apicella, P. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-24-08489.2003 · Cited by 115 · cited 2×
“…punishment (Seymour et al. 2005; Jensen et al. 2007) and in the striatum of nonhuman primates, which represent both appetitive and aversive events (Ravel et al. 2003) Together, our results point to a general role of the striatum in coding a SPE signal across a broad range of reinforcer types and…”
Error-Related Functional Connectivity of the Habenula in Humans
Ide, J. S., & Li, C.-S. R. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00025 · Cited by 93
Expecting unpleasant stimuli–an fMRI study
Herwig, U., Abler, B., Walter, H., & Erk, S. (2007) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2006.02.007 · Cited by 92 · cited 2×
“First, we identified several brain regions that have established roles during aversive and appetitive conditioning: the striatum, the insula, the anterior cingulate cortex (ACC) and the amygdala (Jensen et al. 2003; Seymour et al. 2004; Bray and O’Doherty 2007; Herwig et al. 2007; Sescousse et al. 2010).”
Behavioral and neural predictors of upcoming decisions
Cohen, M. X., & Ranganath, C. (2005) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.5.2.117 · Cited by 78 · cited 2×
“…the amygdala, may also reflect greater saliency associated with risk-taking during neuroeconomics paradigms (Kahn et al. 2002; Bechara et al. 2003; Cohen and Ranganath 2005; Hsu et al. 2005) as well as during speeded as compared with delayed motor responses during the stop signal task, a…”
Li, C.-S. R., Chao, H. H.-A., & Lee, T.-W. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn132 · Cited by 76
Learning-related diminution of unconditioned SCR and fMRI signal responses
Knight, D. C., Waters, N. S., King, M. K., & Bandettini, P. A. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.07.012 · Cited by 61 · cited 2×
“…stimuli, aversive juices, odors, tones, or visual stimuli (LaBar et al. 1998; Buchel et al. 1999; Gottfried, Deichmann, et al. 2002; Seymour et al. 2004; Knight et al. 2010; Sarinopoulos et al. 2010) or monetary losses (Delgado et al. 2000, 2008; Knutson et al. 2000; Nieuwenhuis et al. 2005).”
Cited by
Precise minds in uncertain worlds: Predictive coding in autism.
Van de Cruys, S., Evers, K., Van der Hallen, R., Van Eylen, L., Boets, B., de-Wit, L., & Wagemans, J. (2014) · Psychological Review · doi:10.1037/a0037665 · Cited by 885
“In healthy subjects ACC activity is found for behaviorally relevant prediction errors (Ide, Shenoy, Yu, & Li, 2013; Metereau & Dreher, 2013).”
BackgroundSescousse, G., Caldú, X., Segura, B., & Dreher, J.-C. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.02.002 · Cited by 782
“coding, regardless of reinforcer type or valence (Météreau and Dreher, 2012).”
Self-citationEvent-related potential studies of outcome processing and feedback-guided learning
San Martín, R. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00304 · Cited by 354
“Similar conclusions have been reached using fMRI in humans (Walton et al., 2004; Aarts et al., 2008; Metereau and Dreher, 2012) and single-unit recordings inmonkeys (Niki andWatanabe, 1979; Akkal et al.”
ResultInteroception and drug addiction
Paulus, M. P., & Stewart, J. L. (2014) · Neuropharmacology · doi:10.1016/j.neuropharm.2013.07.002 · Cited by 281
“Some have argued that the insula provides a salience signal during reward related processing (Metereau and Dreher, 2013).”
BackgroundChase, H. W., Kumar, P., Eickhoff, S. B., & Dombrovski, A. Y. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0338-7 · Cited by 231
“…(13.15) Schonberg et al., 2010 (11.75) Gradin et al., 2011 (9.43) Madlon-Kay et al., 2013 (8.96) Howard-Jones et al., 2010 (7.83) O’Sullivan et al., 2011 (6.48) Metereau & Dreher, 2013 (5.80) Gershman et al., 2009 (2.57) Murray et al., 2008 (0.98) The studies contributing to each cluster and the…”
Influential
Metadata
- Authors
- Metereau, E., & Dreher, J.-C.
- Affiliations
- Lyon 1 Université (FR)
- ORCID
- Dreher, Jean-Claude
- Venue
- Cerebral Cortex
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-02-23
- Volume
- 23
- Issue
- 2
- Pages
- 477-487
- ISSN
- 1047-3211 (ISSN-L)
- 1460-2199
- OpenAlex
- W2140531843
- Cited by
- 114
- References
- 98
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 3 · 2023: 5 · 2022: 7 · 2021: 11 · 2020: 9 · 2019: 9 · 2018: 5 · 2017: 7 · 2016: 7 · 2015: 12 · 2014: 14 · 2013: 17 · 2012: 3
- FWCI
- 7.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Cerebral Cortex (publishedVersion)
- PubMed (publishedVersion)
- HAL (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- prediction error, classical conditioning, reward, punishment, aversive stimuli, dopaminergic neurons, striatum, anterior insula, anterior cingulate cortex, amygdala, reinforcement learning models, model-based fMRI, monetary gain
- MeSH terms
- Brain, Brain Mapping, Conditioning, Classical, Female, Humans, Learning, Magnetic Resonance Imaging, Male, Punishment, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.