The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Is Avoiding an Aversive Outcome Rewarding? Neural Substrates of Avoidance Learning in the Human Brain
Open accessAbstract
Avoidance learning poses a challenge for reinforcement-based theories of instrumental conditioning, because once an aversive outcome is successfully avoided an individual may no longer experience extrinsic reinforcement for their behavior. One possible account for this is to propose that avoiding an aversive outcome is in itself a reward, and thus avoidance behavior is positively reinforced on each trial when the aversive outcome is successfully avoided. In the present study we aimed to test this possibility by determining whether avoidance of an aversive outcome recruits the same neural circuitry as that elicited by a reward itself. We scanned 16 human participants with functional MRI while they performed an instrumental choice task, in which on each trial they chose from one of two actions in order to either win money or else avoid losing money. Neural activity in a region previously implicated in encoding stimulus reward value, the medial orbitofrontal cortex, was found to increase, not only following receipt of reward, but also following successful avoidance of an aversive outcome. This neural signal may itself act as an intrinsic reward, thereby serving to reinforce actions during instrumental avoidance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 7,342
Insensitivity to future consequences following damage to human prefrontal cortex
Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994) · Cognition · doi:10.1016/0010-0277(94)90018-3 · Cited by 6,128
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
The Behavior of Organisms: An Experimental Analysis
Jones, F. N., & Skinner, B. F. (1939) · The American Journal of Psychology · doi:10.2307/1416495 · Cited by 4,398
Emotion, Decision Making and the Orbitofrontal Cortex
Bechara, A. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.295 · Cited by 3,589
The psychology of fear and stress
(1988) · Choice Reviews Online · doi:10.5860/choice.26-2399 · Cited by 3,122
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · cited 1×
“at the time of outcome [3–9].”
Blood, A. J., & Zatorre, R. J. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.191355898 · Cited by 2,814
Cited by
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
Balleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775
Self-citationConflict monitoring and decision making: Reconciling two perspectives on anterior cingulate function
BOTVINICK, M. M. (2007) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.7.4.356 · Cited by 1,117
The Nucleus Accumbens: An Interface Between Cognition, Emotion, and Action
Floresco, S. B. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010213-115159 · Cited by 938
Liu, X., Hairston, J., Schrier, M., & Fan, J. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.12.012 · Cited by 916
Beckmann, M., Johansen-Berg, H., & Rushworth, M. F. S. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3328-08.2009 · Cited by 789
Informatic parcellation of the network involved in the computation of subjective value
Clithero, J. A., & Rangel, A. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst106 · Cited by 706
A specific role for posterior dorsolateral striatum in human habit learning
Tricomi, E., Balleine, B. W., & O’Doherty, J. P. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06796.x · Cited by 769
Self-citationReinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 704
Chib, V. S., Rangel, A., Shimojo, S., & O'Doherty, J. P. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2575-09.2009 · Cited by 643
Self-citationDetermining the neural substrates of goal-directed learning in the human brain
Valentin, V. V., Dickinson, A., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0564-07.2007 · Cited by 529
Self-citationNeural correlates of mentalizing-related computations during strategic interactions in humans
Hampton, A. N., Bossaerts, P., & O'Doherty, J. P. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0711099105 · Cited by 515
Self-citationModel‐Based fMRI and Its Application to Reward Learning and Decision Making
O'DOHERTY, J. P., HAMPTON, A., & KIM, H. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.022 · Cited by 508
Self-citationExpected value and prediction error abnormalities in depression and schizophrenia
Gradin, V. B., Kumar, P., Waiter, G., Ahearn, T., Stickle, C., Milders, M., Reid, I., Hall, J., & Steele, J. D. (2011) · Brain · doi:10.1093/brain/awr059 · Cited by 485
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456
Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Garrison, J., Erdeniz, B., & Done, J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.023 · Cited by 438
From Fear to Safety and Back: Reversal of Fear in the Human Brain
Schiller, D., Levy, I., Niv, Y., LeDoux, J. E., & Phelps, E. A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2265-08.2008 · Cited by 429
Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain
Niv, Y., Edlund, J. A., Dayan, P., & O'Doherty, J. P. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5498-10.2012 · Cited by 356
Self-citationAnticipation of monetary gain but not loss in healthy older adults
Samanez-Larkin, G. R., Gibbs, S. E. B., Khanna, K., Nielsen, L., Carstensen, L. L., & Knutson, B. (2007) · Nature Neuroscience · doi:10.1038/nn1894 · Cited by 399
Gläscher, J., Hampton, A. N., & O'Doherty, J. P. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn098 · Cited by 387
Self-citationEvent-related potential studies of outcome processing and feedback-guided learning
San Martín, R. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00304 · Cited by 354
The globus pallidus sends reward-related signals to the lateral habenula
Hong, S., & Hikosaka, O. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.035 · Cited by 346
The role of the striatum in aversive learning and aversive prediction errors
Delgado, M. R., Li, J., Schiller, D., & Phelps, E. A. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0161 · Cited by 328
Diekhof, E. K., Kaps, L., Falkai, P., & Gruber, O. (2012) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2012.02.007 · Cited by 331
The orbitofrontal cortex and emotion in health and disease, including depression
Rolls, E. T. (2019) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2017.09.021 · Cited by 329
Emotion, decision making, and the amygdala.
Seymour, B., & Dolan, R. (2008) · Neuron · doi:10.1016/j.neuron.2008.05.020 · Cited by 324
Gold, J. M. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.1269 · Cited by 317
The Role of the Orbitofrontal Cortex in Anxiety Disorders
MILAD, M. R., & RAUCH, S. L. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1401.006 · Cited by 316
Reward Improves Long-Term Retention of a Motor Memory through Induction of Offline Memory Gains
Abe, M., Schambra, H., Wassermann, E. M., Luckenbaugh, D., Schweighofer, N., & Cohen, L. G. (2011) · Current Biology · doi:10.1016/j.cub.2011.02.030 · Cited by 313
Engelmann, J. B. (2009) · Frontiers in Human Neuroscience · doi:10.3389/neuro.09.004.2009 · Cited by 300
Overlapping neural systems mediating extinction, reversal and regulation of fear
Schiller, D., & Delgado, M. R. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.04.002 · Cited by 294
Contextual modulation of value signals in reward and punishment learning
Palminteri, S., Khamassi, M., Joffily, M., & Coricelli, G. (2015) · Nature Communications · doi:10.1038/ncomms9096 · Cited by 286
The Role of Human Orbitofrontal Cortex in Value Comparison for Incommensurable Objects
FitzGerald, T. H. B., Seymour, B., & Dolan, R. J. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.0717-09.2009 · Cited by 279
Amygdala Reward Neurons Form and Store Fear Extinction Memory
Zhang, X., Kim, J., & Tonegawa, S. (2020) · Neuron · doi:10.1016/j.neuron.2019.12.025 · Cited by 276
The neural code of reward anticipation in human orbitofrontal cortex
Kahnt, T., Heinzle, J., Park, S. Q., & Haynes, J.-D. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0912838107 · Cited by 274
Human oscillatory activity associated to reward processing in a gambling task
Marco-Pallares, J., Cucurell, D., Cunillera, T., García, R., Andrés-Pueyo, A., Münte, T. F., & Rodríguez-Fornells, A. (2008) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2007.07.016 · Cited by 272
What is reinforced by phasic dopamine signals?
Redgrave, P., Gurney, K., & Reynolds, J. (2008) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.10.007 · Cited by 261
LIDA: A Systems-level Architecture for Cognition, Emotion, and Learning
Franklin, S., Madl, T., D'Mello, S., & Snaider, J. (2014) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2013.2277589 · Cited by 261
Tests of unconditioned anxiety — Pitfalls and disappointments
Ennaceur, A. (2014) · Physiology & Behavior · doi:10.1016/j.physbeh.2014.05.032 · Cited by 259
Serotonin Selectively Modulates Reward Value in Human Decision-Making
Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0053-12.2012 · Cited by 252
Finger, E. C., Marsh, A. A., Blair, K. S., Reid, M. E., Sims, C., Ng, P., Pine, D. S., & Blair, R. J. R. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2010.10010129 · Cited by 252
Representation of aversive prediction errors in the human periaqueductal gray
Roy, M., Shohamy, D., Daw, N., Jepma, M., Wimmer, G. E., & Wager, T. D. (2014) · Nature Neuroscience · doi:10.1038/nn.3832 · Cited by 246
The neurobiology of punishment
Seymour, B., Singer, T., & Dolan, R. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2119 · Cited by 240
Reward circuitry function in autism during face anticipation and outcomes
Dichter, G. S., Richey, J. A., Rittenberg, A. M., Sabatino, A., & Bodfish, J. W. (2012) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-011-1221-1 · Cited by 222
The convergence of information about rewarding and aversive stimuli in single neurons
Morrison, S. E., & Salzman, C. D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1815-09.2009 · Cited by 206
Relief as a reward: hedonic and neural responses to safety from pain
Leknes, S., Lee, M., Berna, C., Andersson, J., & Tracey, I. (2011) · PLoS ONE · doi:10.1371/journal.pone.0017870 · Cited by 207
Reward feedback alterations in unmedicated schizophrenia patients: Relevance for delusions
Schlagenhauf, F., Sterzer, P., Schmack, K., Ballmaier, M., Rapp, M., Wrase, J., Juckel, G., Gallinat, J., & Heinz, A. (2009) · Biological Psychiatry · doi:10.1016/j.biopsych.2008.12.016 · Cited by 195
A neurocomputational account of how inflammation enhances sensitivity to punishments versus rewards
Harrison, N. A., Voon, V., Cercignani, M., Cooper, E. A., Pessiglione, M., & Critchley, H. D. (2016) · Biological Psychiatry · doi:10.1016/j.biopsych.2015.07.018 · Cited by 185
The role of prediction in social neuroscience
Brown, E. C., & Brüne, M. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00147 · Cited by 190
“…case that learned aversive behavior is directly linked to reward processing, in that it has been shown to be modulated by monetary reward (Guo et al., 2011), and the avoidance of aversive outcomes may in itself be rewarding, and therefore reinforcing aversion avoiding behavior (Kim et al., 2006).”
BackgroundPain, decisions, and actions: a motivational perspective
Wiech, K., & Tracey, I. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00046 · Cited by 188
Venkatraman, V., Rosati, A. G., Taren, A. A., & Huettel, S. A. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2708-09.2009 · Cited by 186
Dichter, G. S., Kozink, R. V., McClernon, F. J., & Smoski, M. J. (2012) · Journal of Affective Disorders · doi:10.1016/j.jad.2011.09.048 · Cited by 170
Kim, H., Shimojo, S., & O'Doherty, J. P. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhq145 · Cited by 176
Self-citationRe-evaluating the role of the orbitofrontal cortex in reward and reinforcement
Noonan, M. P., Kolling, N., Walton, M. E., & Rushworth, M. F. S. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08023.x · Cited by 169
Attentional capture by signals of threat
Schmidt, L. J., Belopolsky, A. V., & Theeuwes, J. (2015) · Cognition and Emotion · doi:10.1080/02699931.2014.924484 · Cited by 173
Different neural systems adjust motor behavior in response to reward and punishment
Wrase, J., Kahnt, T., Schlagenhauf, F., Beck, A., Cohen, M. X., Knutson, B., & Heinz, A. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.04.001 · Cited by 172
Kohls, G., Perino, M. T., Taylor, J. M., Madva, E. N., Cayless, S. J., Troiani, V., Price, E., Faja, S., Herrington, J. D., & Schultz, R. T. (2013) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2013.07.020 · Cited by 170
Learning to predict and control harmful events: chronic pain and conditioning
Vlaeyen, J. W. S. (2015) · Pain · doi:10.1097/j.pain.0000000000000107 · Cited by 159
Neural systems underlying aversive conditioning in humans with primary and secondary reinforcers
Delgado, M. R., Jou, R. L., & Phelps, E. A. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00071 · Cited by 141
Avoidance of harm and anxiety: A role for the nucleus accumbens
Levita, L., Hoskin, R., & Champi, S. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.059 · Cited by 137
Generalization of value in reinforcement learning by humans
Wimmer, G. E., Daw, N. D., & Shohamy, D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08017.x · Cited by 111
Sonuga-Barke, E. J. S., & Fairchild, G. (2012) · Biological Psychiatry · doi:10.1016/j.biopsych.2012.04.004 · Cited by 124
Martins, D., Rademacher, L., Gabay, A. S., Taylor, R., Richey, J. A., Smith, D. V., Goerlich, K. S., Nawijn, L., Cremers, H. R., Wilson, R., et al. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.12.034 · Cited by 119
Selective involvement by the medial orbitofrontal cortex in biasing risky, but not impulsive, choice
Stopper, C. M., Green, E. B., & Floresco, S. B. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs297 · Cited by 117
The Value of Exercising Control Over Monetary Gains and Losses
Leotti, L. A., & Delgado, M. R. (2014) · Psychological Science · doi:10.1177/0956797613514589 · Cited by 110
“In contrast, in Experiment 2, avoiding a monetary loss may have been perceived as a positive outcome (Kim et al., 2006), thus enhancing perceived control (Thompson, Armstrong, & Thomas, 1998) and producing greater reward-related activity.”
BackgroundModel-based approaches to neuroimaging: combining reinforcement learning theory with fMRI data
Gläscher, J. P., & O'Doherty, J. P. (2010) · WIREs Cognitive Science · doi:10.1002/wcs.57 · Cited by 104
Self-citationHeightened aversion to risk and loss in depressed patients with a suicide attempt history
Baek, K., Kwon, J., Chae, J.-H., Chung, Y. A., Kralik, J. D., Min, J.-A., Huh, H., Choi, K. M., Jang, K.-I., Lee, N.-B., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-10541-5 · Cited by 98
Hedonic and informational functions of the human orbitofrontal cortex
Elliott, R., Agnew, Z., & Deakin, J. F. W. (2010) · Cerebral Cortex · doi:10.1093/cercor/bhp092 · Cited by 94
Cousijn, J., Goudriaan, A. E., Ridderinkhof, K. R., van den Brink, W., Veltman, D. J., & Wiers, R. W. (2012) · PLoS ONE · doi:10.1371/journal.pone.0042394 · Cited by 85
Whiting, D. L., Deane, F. P., Simpson, G. K., McLeod, H. J., & Ciarrochi, J. (2017) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2015.1062115 · Cited by 85
Fear as a translational mechanism in the psychopathology of anorexia nervosa
Murray, S. B., Strober, M., Craske, M. G., Griffiths, S., Levinson, C. A., & Strigo, I. A. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.10.013 · Cited by 75
Initial and sustained brain responses to contextual conditioned anxiety in humans
Andreatta, M., Glotzbach-Schoon, E., Mühlberger, A., Schulz, S. M., Wiemer, J., & Pauli, P. (2015) · Cortex · doi:10.1016/j.cortex.2014.09.014 · Cited by 73
Tricomi, E., & Fiez, J. A. (2012) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-011-0077-3 · Cited by 64
Decision making in avoidance–reward conflict: a paradigm for non-human primates and humans
Sierra-Mercado, D., Deckersbach, T., Arulpragasam, A. R., Chou, T., Rodman, A. M., Duffy, A., McDonald, E. J., Eckhardt, C. A., Corse, A. K., Kaur, N., et al. (2015) · Brain Structure and Function · doi:10.1007/s00429-014-0796-7 · Cited by 44
Approach and avoidance learning in obsessive-compulsive disorder
Endrass, T., Kloft, L., Kaufmann, C., & Kathmann, N. (2011) · Depression and Anxiety · doi:10.1002/da.20772 · Cited by 44
Neuroimaging the temporal dynamics of human avoidance to sustained threat
Schlund, M. W., Hudgins, C. D., Magee, S., & Dymond, S. (2013) · Behavioural Brain Research · doi:10.1016/j.bbr.2013.09.042 · Cited by 33
Borst, J. P., Taatgen, N. A., & van Rijn, H. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.05.084 · Cited by 33
Harlé, K. M., Zhang, S., Ma, N., Yu, A. J., & Paulus, M. P. (2016) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2016.06.008 · Cited by 29
Bolstad, I., Andreassen, O. A., Reckless, G. E., Sigvartsen, N. P., Server, A., & Jensen, J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068494 · Cited by 22
(2016) · Research Online (University of Wollongong) · Cited by 2
Metadata
- Authors
- Kim, H., Shimojo, S., & O'Doherty, J. P.
- Venue
- PLoS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2006-07-04
- Volume
- 4
- Issue
- 8
- Pages
- e233
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- OpenAlex
- W2004167381
- Cited by
- 453
- References
- 58
- FWCI
- 12.13
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.