Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain
Open accessAbstract
The acquisition of reward and the avoidance of punishment could logically be contingent on either emitting or withholding particular actions. However, the separate pathways in the striatum for go and no-go appear to violate this independence, instead coupling affect and effect. Respect for this interdependence has biased many studies of reward and punishment, so potential action-outcome valence interactions during anticipatory phases remain unexplored. In a functional magnetic resonance imaging study with healthy human volunteers, we manipulated subjects' requirement to emit or withhold an action independent from subsequent receipt of reward or avoidance of punishment. During anticipation, in the striatum and a lateral region within the substantia nigra/ventral tegmental area (SN/VTA), action representations dominated over valence representations. Moreover, we did not observe any representation associated with different state values through accumulation of outcomes, challenging a conventional and dominant association between these areas and state value representations. In contrast, a more medial sector of the SN/VTA responded preferentially to valence, with opposite signs depending on whether action was anticipated to be emitted or withheld. This dominant influence of action requires an enriched notion of opponency between reward and punishment.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The human lateral SN/VTA is activated during anticipation to acquire a reward or to avoid a punishment.
“The human lateral SN/VTA has been similarly shown to be activated during anticipation (i.e. before execution of motor responses) to acquire a reward or to avoid a punishment [53].”
From A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP (2011) · cites this paper as [53]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Self-citationGray, J. A. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00013066 · Cited by 4,477 · see citing works
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127 · see citing works
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718 · see citing works
Region of interest analysis using an SPM toolbox
Brett, M. (2002) · NeuroImage · Cited by 2,273 · see citing works
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244 · see citing works
Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration
Klein, A., Andersson, J., Ardekani, B. A., Ashburner, J., Avants, B., Chiang, M.-C., Christensen, G. E., Collins, D. L., Gee, J., Hellier, P., et al. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.12.037 · Cited by 2,227 · see citing works
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · see citing works
Self-citationDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · see citing works
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Self-citationBy Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964 · see citing works
Aron, A. R., & Poldrack, R. A. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4682-05.2006 · Cited by 1,630 · see citing works
Haber, S. N., Fudge, J. L., & McFarland, N. R. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-06-02369.2000 · Cited by 1,610 · see citing works
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · see citing works
Self-citationThe neostriatal mosaic: multiple levels of compartmental organization
Gerfen, C. R. (1992) · Trends in Neurosciences · doi:10.1016/0166-2236(92)90355-c · Cited by 1,471 · see citing works
Reward representations and reward-related learning in the human brain: Insights from neuroimaging
O’Doherty, J. P. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.10.016 · Cited by 1,426 · see citing works
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378 · see citing works
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · see citing works
Ungerstedt, U. (1971) · Acta Physiologica Scandinavica · doi:10.1111/j.1365-201x.1971.tb11001.x · Cited by 1,294 · see citing works
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · see citing works
Self-citationTracking the Hemodynamic Responses to Reward and Punishment in the Striatum
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.6.3072 · Cited by 1,116 · see citing works
Neural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076 · see citing works
Self-citation5-HT and mechanisms of defence
Deakin, J. F. W., & Graeff, F. G. (1991) · Journal of Psychopharmacology · doi:10.1177/026988119100500414 · Cited by 1,011 · see citing works
Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system
Lammel, S., Hetzel, A., Häckel, O., Jones, I., Liss, B., & Roeper, J. (2008) · Neuron · doi:10.1016/j.neuron.2008.01.022 · Cited by 976 · see citing works
(1947) · Harvard Educational Review · Cited by 967 · see citing works
Distributed Neural Representation of Expected Value
Knutson, B., Taylor, J., Kaufman, M., Peterson, R., & Glover, G. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0642-05.2005 · Cited by 955 · see citing works
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · see citing works
Rubia, K., Smith, A. B., Brammer, M. J., & Taylor, E. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00275-1 · Cited by 852 · see citing works
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844 · see citing works
Self-citationSalamone, J. D., & Correa, M. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00282-6 · Cited by 817 · see citing works
Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli
Brischoux, F., Chakraborty, S., Brierley, D. I., & Ungless, M. A. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0811507106 · Cited by 801 · see citing works
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661 · see citing works
Temporal difference models describe higher-order learning in humans
Seymour, B., O'Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., Friston, K. J., & Frackowiak, R. S. (2004) · Nature · doi:10.1038/nature02581 · Cited by 605 · see citing works
Self-citationModulation of Caudate Activity by Action Contingency
Tricomi, E. M., Delgado, M. R., & Fiez, J. A. (2004) · Neuron · doi:10.1016/s0896-6273(03)00848-1 · Cited by 575 · see citing works
Is the short-latency dopamine response too short to signal reward error?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01373-3 · Cited by 506 · see citing works
Absolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA
Bunzeck, N., & Düzel, E. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.021 · Cited by 503 · see citing works
Weiskopf, N., Hutton, C., Josephs, O., & Deichmann, R. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.07.029 · Cited by 488 · see citing works
van Kesteren, R. E., Syed, N. I., Munno, D. W., Bouwman, J., Feng, Z., Geraerts, W. P. M., & Smit, A. B. (2001) · Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0004.2001 · Cited by 471 · see citing works
Serotonin and dopamine: unifying affective, activational, and decision functions.
Cools, R., Nakamura, K., & Daw, N. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.121 · Cited by 464 · see citing works
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456 · see citing works
Self-citationBeyond the reward hypothesis: alternative functions of nucleus accumbens dopamine
Salamone, J. D., Correa, M., Mingote, S., & Weber, S. (2004) · Current Opinion in Pharmacology · doi:10.1016/j.coph.2004.09.004 · Cited by 454 · see citing works
Opponent appetitive-aversive neural processes underlie predictive learning of pain relief
Seymour, B., O'Doherty, J. P., Koltzenburg, M., Wiech, K., Frackowiak, R., Friston, K., & Dolan, R. (2005) · Nature Neuroscience · doi:10.1038/nn1527 · Cited by 439 · see citing works
Self-citationHuman Striatal Responses to Monetary Reward Depend On Saliency
Zink, C. F., Pagnoni, G., Martin-Skurski, M. E., Chappelow, J., & Berns, G. S. (2004) · Neuron · doi:10.1016/s0896-6273(04)00183-7 · Cited by 432 · see citing works
Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations
DELGADO, M. R., LOCKE, H. M., STENGER, V. A., & FIEZ, J. A. (2003) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.3.1.27 · Cited by 431 · see citing works
Serotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381 · see citing works
Self-citationLimbic-Striatal Memory Systems and Drug Addiction
Robbins, T. W. (2002) · Neurobiology of Learning and Memory · doi:10.1006/nlme.2002.4103 · Cited by 379 · see citing works
Pavlovian Conditioning: A Functional Perspective
Domjan, M. P. (2005) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.141409 · Cited by 359 · see citing works
The role of the striatum in aversive learning and aversive prediction errors
Delgado, M. R., Li, J., Schiller, D., & Phelps, E. A. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0161 · Cited by 328 · see citing works
Human striatal response to salient nonrewarding stimuli
Zink, C. F., Pagnoni, G., Martin, M. E., Dhamala, M., & Berns, G. S. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-22-08092.2003 · Cited by 312 · see citing works
Crockett, M. J., Clark, L., & Robbins, T. W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2513-09.2009 · Cited by 271 · see citing works
Valence and salience contribute to nucleus accumbens activation
Cooper, J. C., & Knutson, B. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.08.009 · Cited by 256 · see citing works
Motivation-dependent responses in the human caudate nucleus.
Delgado, M. R. (2004) · Cerebral Cortex · doi:10.1093/cercor/bhh062 · Cited by 246 · see citing works
The neostriatal mosaic: multiple levels of compartmental organization
Gerfen, C. R. (1992) · Springer eBooks · doi:10.1007/978-3-7091-9211-5_4 · Cited by 232 · see citing works
Functional imaging of the human dopaminergic midbrain
Düzel, E., Bunzeck, N., Guitart-Masip, M., Wittmann, B., Schott, B. H., & Tobler, P. N. (2009) · Trends in Neurosciences · doi:10.1016/j.tins.2009.02.005 · Cited by 216 · see citing works
Self-citationFaure, A., Reynolds, S. M., Richard, J. M., & Berridge, K. C. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4961-07.2008 · Cited by 193 · see citing works
Aron, A. R. (2003) · Brain · doi:10.1093/brain/awg067 · Cited by 185 · see citing works
McCullough, L. D., Sokolowski, J. D., & Salamone, J. D. (1993) · Neuroscience · doi:10.1016/0306-4522(93)90538-q · Cited by 180 · see citing works
Different neural systems adjust motor behavior in response to reward and punishment
Wrase, J., Kahnt, T., Schlagenhauf, F., Beck, A., Cohen, M. X., Knutson, B., & Heinz, A. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.04.001 · Cited by 172 · see citing works
Two-factor theory, the actor--critic model, and conditioned avoidance
Maia, T. V. (2010) · Learning & Behavior · doi:10.3758/lb.38.1.50 · Cited by 130 · see citing works
Elliott, R., Newman, J. L., Longe, O. A., & Deakin, J. F. W. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.10.010 · Cited by 127 · see citing works
Darvas, M., Fadok, J. P., & Palmiter, R. D. (2011) · Learning & Memory · doi:10.1101/lm.2041211 · Cited by 125 · see citing works
Young, A. M. J. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2004.03.003 · Cited by 123 · see citing works
A neural theory of punishment and avoidance, I: Qualitative theory
Grossberg, S. (1972) · Mathematical Biosciences · doi:10.1016/0025-5564(72)90062-4 · Cited by 122 · see citing works
NudC Associates with Lis1 and the Dynein Motor at the Leading Pole of Neurons
Aumais, J. P., Tunstead, J. R., McNeil, R. S., Schaar, B. T., McConnell, S. K., Lin, S.-H., Clark, G., & Yu‐Lee, L. (2001) · Journal of Neuroscience · doi:10.1523/jneurosci.21-24-j0002.2001 · Cited by 112 · see citing works
Contextual novelty changes reward representations in the striatum
Guitart-Masip, M., Bunzeck, N., Stephan, K. E., Dolan, R. J., & Düzel, E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.5331-09.2010 · Cited by 104 · see citing works
Self-citationPezze, M. A., Heidbreder, C. A., Feldon, J., & Murphy, C. A. (2001) · Neuroscience · doi:10.1016/s0306-4522(01)00403-1 · Cited by 96 · see citing works
A temporal difference account of avoidance learning
Moutoussis, M., Bentall, R. P., Williams, J. H., & Dayan, P. (2008) · Network Computation in Neural Systems · doi:10.1080/09548980802192784 · Cited by 83 · see citing works
Self-citationMotivated Reinforcement Learning
Dayan, P. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0006 · Cited by 50 · see citing works
Self-citationEscape, Avoidance, and Imitation: A Neural Network Approach
Schmajuk, N. A., & Zanutto, B. S. (1997) · Adaptive Behavior · doi:10.1177/105971239700600103 · Cited by 33 · see citing works
A Computational Model of Learned Avoidance Behavior in a One-Way Avoidance Experiment
Johnson, J. D., Li, W., Li, J., & Klopf, A. H. (2001) · Adaptive Behavior · doi:10.1177/105971230200900205 · Cited by 20 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Dayan P Dietterich TG Becker S Ghahramani Z NIPS 2001
- Gray JA McNaughton M The neuropsychology of anxiety: an inquiry into the function of the septohippocampal system 2000 Oxford University Press Oxford 2nd Edition
Cited by
Sorted by citations
States of Curiosity Modulate Hippocampus-Dependent Learning via the Dopaminergic Circuit
Gruber, M. J., Gelman, B. D., & Charan Ranganath, A. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.060 · Cited by 742 · see citing works
Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis
Huys, Q. J., Pizzagalli, D. A., Bogdan, R., & Dayan, P. (2013) · Biology of Mood & Anxiety Disorders · doi:10.1186/2045-5380-3-12 · Cited by 480 · see citing works
Self-citationCushman, F. A. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868313495594 · Cited by 472 · see citing works
A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP
Lisman, J., Grace, A. A., & Duzel, E. (2011) · Trends in Neurosciences · doi:10.1016/j.tins.2011.07.006 · Cited by 465 · see citing works
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456 · see citing works
Self-citationPrediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Garrison, J. R., Erdeniz, B., & Done, J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.023 · Cited by 430 · see citing works
Oldham, S., Murawski, C., Fornito, A., Youssef, G. J., Yücel, M., & Lorenzetti, V. (2018) · Human Brain Mapping · doi:10.1002/hbm.24184 · Cited by 425 · see citing works
Go and no-go learning in reward and punishment: Interactions between affect and effect
Guitart‐Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Düzel, E., & Dolan, R. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.024 · Cited by 418 · see citing works
Self-citationAction versus valence in decision making
Guitart-Masip, M., Duzel, E., Dolan, R., & Dayan, P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.01.003 · Cited by 270 · see citing works
Self-citationPost-learning hippocampal dynamics promote preferential retention of rewarding events
Gruber, M. J., Ritchey, M., Wang, S.-F., Doss, M. K., & Ranganath, C. (2016) · Neuron · doi:10.1016/j.neuron.2016.01.017 · Cited by 212 · see citing works
Haber, S. N., & Behrens, T. E. J. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.031 · Cited by 230 · see citing works
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219 · see citing works
Self-citationFrontal Theta Overrides Pavlovian Learning Biases
Cavanagh, J. F., Eisenberg, I. W., Guitart‐Masip, M., Huys, Q. J. M., & Frank, M. J. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5754-12.2013 · Cited by 201 · see citing works
Self-citationEffort and Valuation in the Brain: The Effects of Anticipation and Execution
Kurniawan, I. T., Guitart‐Masip, M., Dayan, P., & Dolan, R. J. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.4777-12.2013 · Cited by 171 · see citing works
Self-citationZhang, L., Lengersdorff, L., Mikuš, N., Gläscher, J. P., & Lamm, C. (2020) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsaa089 · Cited by 158 · see citing works
Kohls, G., Perino, M. T., Taylor, J. M., Madva, E. N., Cayless, S. J., Troiani, V., Price, E., Faja, S., Herrington, J. D., & Schultz, R. T. (2013) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2013.07.020 · Cited by 170 · see citing works
Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning
Mkrtchian, A., Aylward, J., Dayan, P., Roiser, J. P., & Robinson, O. J. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.01.017 · Cited by 151 · see citing works
Self-citationAvoidance of harm and anxiety: A role for the nucleus accumbens
Levita, L., Hoskin, R., & Champi, S. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.059 · Cited by 137 · see citing works
The Neurocognitive Bases of Human Volition.
Haggard, P. (2019) · Annual Review of Psychology · doi:10.1146/annurev-psych-010418-103348 · Cited by 126 · see citing works
Action controls dopaminergic enhancement of reward representations
Guitart‐Masip, M., Chowdhury, R., Sharot, T., Dayan, P., Düzel, E., & Dolan, R. J. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1202229109 · Cited by 125 · see citing works
Self-citationDistinct Midbrain and Habenula Pathways Are Involved in Processing Aversive Events in Humans
Hennigan, K., D’Ardenne, K., & McClure, S. M. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.0927-14.2015 · Cited by 119 · see citing works
Model averaging, optimal inference, and habit formation
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. J. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00457 · Cited by 121 · see citing works
Self-citationDopamine, reward learning, and active inference
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00136 · Cited by 112 · see citing works
Self-citationThe Value of Exercising Control Over Monetary Gains and Losses
Leotti, L. A., & Delgado, M. R. (2014) · Psychological Science · doi:10.1177/0956797613514589 · Cited by 110 · see citing works
Guitart‐Masip, M., Economides, M., Huys, Q. J. M., Frank, M. J., Chowdhury, R., Düzel, E., Dayan, P., & Dolan, R. J. (2013) · Psychopharmacology · doi:10.1007/s00213-013-3313-4 · Cited by 107 · see citing works
Self-citationNeural sensitivity to social reward and punishment anticipation in social anxiety disorder
Cremers, H. R., Veer, I. M., Spinhoven, P., Rombouts, S. A. R. B., & Roelofs, K. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00439 · Cited by 107 · see citing works
Dopaminergic basis for signaling belief updates, but not surprise, and the link to paranoia
Nour, M. M., Dahoun, T., Schwartenbeck, P., Adams, R. A., FitzGerald, T. H. B., Coello, C., Wall, M. B., Dolan, R. J., & Howes, O. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1809298115 · Cited by 101 · see citing works
Self-citationThe Role of the Striatum in Effort-Based Decision-Making in the Absence of Reward
Schouppe, N., Demanet, J., Boehler, C. N., Ridderinkhof, K. R., & Notebaert, W. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.1214-13.2014 · Cited by 100 · see citing works
The specificity of Pavlovian regulation is associated with recovery from depression
Huys, Q. J. M., Gölzer, M., Friedel, E., Heinz, A., Cools, R., Dayan, P., & Dolan, R. J. (2016) · Psychological Medicine · doi:10.1017/s0033291715002597 · Cited by 92 · see citing works
Self-citationDissociating motivation from reward in human striatal activity
Miller, E. M., Shankar, M. U., Knutson, B., & McClure, S. M. (2014) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00535 · Cited by 89 · see citing works
Hedonic value: enhancing adaptation for motivated agents
Cos, I., Cañamero, L. D., Hayes, G. M., & Gillies, A. (2013) · Adaptive Behavior · doi:10.1177/1059712313486817 · Cited by 85 · see citing works
Electrophysiological correlates of anticipating improbable but desired events
Fuentemilla, L., Cucurell, D., Marco-Pallarés, J., Guitart-Masip, M., Morís, J., & Rodríguez-Fornells, A. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.03.062 · Cited by 65 · see citing works
Self-citationMoutoussis, M., Bullmore, E. T., Goodyer, I. M., Fonagy, P., Jones, P. B., Dolan, R. J., & Dayan, P. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006679 · Cited by 79 · see citing works
Self-citationSchreiter, S., Spengler, S., Willert, A., Mohnke, S., Herold, D., Erk, S., Romanczuk‐Seiferth, N., Quinlivan, E., Attar, C. H., Banzhaf, C., et al. (2016) · Psychological Medicine · doi:10.1017/s0033291716001963 · Cited by 78 · see citing works
Motivational Deficits in Schizophrenia and the Representation of Expected Value
Waltz, J. A., & Gold, J. M. (2015) · Current topics in behavioral neurosciences · doi:10.1007/7854_2015_385 · Cited by 80 · see citing works
Kätsyri, J., Hari, R., Ravaja, N., & Nummenmaa, L. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00278 · Cited by 77 · see citing works
Instrumental vigour in punishment and reward
Dayan, P. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08026.x · Cited by 71 · see citing works
Self-citationMotivating inhibition – reward prospect speeds up response cancellation
Boehler, C. N., Hopf, J., Stoppel, C. M., & Krebs, R. M. (2012) · Cognition · doi:10.1016/j.cognition.2012.07.018 · Cited by 72 · see citing works
Learning and Choice in Mood Disorders: Searching for the Computational Parameters of Anhedonia
Robinson, O. J., & Chase, H. W. (2017) · Computational Psychiatry · doi:10.1162/cpsy_a_00009 · Cited by 66 · see citing works
A model of mood as integrated advantage.
Bennett, D., Davidson, G., & Niv, Y. (2021) · Psychological Review · doi:10.1037/rev0000294 · Cited by 69 · see citing works
Action-specific value signals in reward-related regions of the human brain
FitzGerald, T. H. B., Friston, K. J., & Dolan, R. J. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.3254-12.2012 · Cited by 68 · see citing works
Self-citationThe Subcortical Cocktail Problem; Mixed Signals from the Subthalamic Nucleus and Substantia Nigra
de Hollander, G., Keuken, M. C., & Forstmann, B. U. (2015) · PLoS ONE · doi:10.1371/journal.pone.0120572 · Cited by 67 · see citing works
Opponent Brain Systems for Reward and Punishment Learning
Palminteri, S., & Pessiglione, M. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-805308-9.00023-3 · Cited by 63 · see citing works
Wotruba, D., Heekeren, K., Michels, L., Buechler, R., Simon, J. J., Theodoridou, A., Kollias, S., Rössler, W., & Kaiser, S. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00382 · Cited by 61 · see citing works
Otmakhova, N. A., Düzel, E., Deutch, A. Y., & Lisman, J. E. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_10 · Cited by 57 · see citing works
Bustamante, J. C., Barrós‐Loscertales, A., Costumero, V., Fuentes‐Claramonte, P., Rosell‐Negre, P., Ventura‐Campos, N., Llopis, J., & Ávila, C. (2013) · Addiction Biology · doi:10.1111/adb.12041 · Cited by 54 · see citing works
Webber, H. E., Lopez-Gamundi, P., Stamatovich, S. N., de Wit, H., & Wardle, M. C. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.11.004 · Cited by 52 · see citing works
Pool, E. R., Pauli, W. M., Kress, C. S., & O’Doherty, J. P. (2019) · Nature Human Behaviour · doi:10.1038/s41562-018-0527-9 · Cited by 52 · see citing works
Neural processes mediating contextual influences on human choice behaviour
Rigoli, F., Friston, K. J., & Dolan, R. J. (2016) · Nature Communications · doi:10.1038/ncomms12416 · Cited by 52 · see citing works
Self-citationIsherwood, S. J., Keuken, M. C., Bazin, P., & Forstmann, B. U. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.07.021 · Cited by 50 · see citing works
The influence of emotions on cognitive control: feelings and beliefs—where do they meet?
Harlé, K. M., Shenoy, P., & Paulus, M. P. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00508 · Cited by 48 · see citing works
Dorsal striatal dopamine D1 receptor availability predicts an instrumental bias in action learning
de Boer, L., Axelsson, J., Chowdhury, R., Riklund, K., Dolan, R. J., Nyberg, L. H., Bäckman, L., & Guitart‐Masip, M. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1816704116 · Cited by 48 · see citing works
Self-citationAdolescents exhibit reduced Pavlovian biases on instrumental learning
Raab, H. A., & Hartley, C. A. (2020) · Scientific Reports · doi:10.1038/s41598-020-72628-w · Cited by 48 · see citing works
Opposing Effects of Appetitive and Aversive Cues on Go/No-go Behavior and Motor Excitability
Chiu, Y., Cools, R., & Aron, A. R. (2014) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00585 · Cited by 47 · see citing works
Effects of dopamine on reinforcement learning in Parkinson’s disease depend on motor phenotype
van Nuland, A. J. M., Helmich, R. C. G., Dirkx, M. F., Zach, H., Toni, I., Cools, R., & den Ouden, H. E. M. (2020) · Brain · doi:10.1093/brain/awaa335 · Cited by 47 · see citing works
Decision making in avoidance–reward conflict: a paradigm for non-human primates and humans
Sierra-Mercado, D., Deckersbach, T., Arulpragasam, A. R., Chou, T., Rodman, A. M., Duffy, A., McDonald, E. J., Eckhardt, C. A., Corse, A. K., Kaur, N., et al. (2015) · Brain Structure and Function · doi:10.1007/s00429-014-0796-7 · Cited by 44 · see citing works
Limbrick‐Oldfield, E. H., Brooks, J. C. W., Wise, R. G., Padormo, F., Hajnal, J. V., Beckmann, C. F., & Ungless, M. A. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.08.016 · Cited by 46 · see citing works
Krebs, R. M., & Woldorff, M. G. (2017) · Wiley eBooks · doi:10.1002/9781118920497.ch24 · Cited by 47 · see citing works
An empirical evaluation of active inference in multi-armed bandits
Marković, D., Stojić, H., Schwöbel, S., & Kiebel, S. J. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.018 · Cited by 44 · see citing works
Effect of relevance on amygdala activation and association with the ventral striatum
Ousdal, O. T., Reckless, G. E., Server, A., Andreassen, O. A., & Jensen, J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.035 · Cited by 43 · see citing works
Costumero, V., Barrós‐Loscertales, A., Bustamante, J. C., Ventura‐Campos, N., Fuentes‐Claramonte, P., & Ávila, C. (2013) · European Journal of Neuroscience · doi:10.1111/ejn.12234 · Cited by 44 · see citing works
Anderson, B. A., Folk, C. L., Garrison, R., & Rogers, L. L. (2016) · Journal of Experimental Psychology General · doi:10.1037/xge0000169 · Cited by 42 · see citing works
A computational account of threat-related attentional bias
Wise, T., Michely, J., Dayan, P., & Dolan, R. (2019) · PLoS Comput. Biol. · doi:10.1371/journal.pcbi.1007341 · Cited by 37 · see citing works
Self-citationChanges in corticostriatal connectivity during reinforcement learning in humans
Horga, G., Maia, T. V., Marsh, R., Hao, X., Xu, D., Duan, Y., Tau, G. Z., Graniello, B., Wang, Z., Kangarlu, A., et al. (2014) · Human Brain Mapping · doi:10.1002/hbm.22665 · Cited by 42 · see citing works
Individual differences in bodily freezing predict emotional biases in decision making
Ly, V., Huys, Q. J. M., Stins, J. F., Roelofs, K., & Cools, R. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00237 · Cited by 43 · see citing works
Self-citationMoutoussis, M., Rutledge, R. B., Prabhu, G., Hrynkiewicz, L., Lam, J., Ousdal, O. T., Guitart‐Masip, M., Fonagy, P., & Dolan, R. J. (2018) · PLoS ONE · doi:10.1371/journal.pone.0201451 · Cited by 42 · see citing works
Self-citationMeidawati, N., & Assidiqi, A. (2019) · Jurnal Akuntansi & Auditing Indonesia · doi:10.20885/jaai.vol23.iss2.art6 · Cited by 42 · see citing works
Ventral striatal hyperconnectivity during rewarded interference control in adolescents with ADHD
Ma, I., van Holstein, M., Mies, G. W., Mennes, M., Buitelaar, J. K., Cools, R., Cillessen, A. H. N., Krebs, R. M., & Scheres, A. (2016) · Cortex · doi:10.1016/j.cortex.2016.05.021 · Cited by 39 · see citing works
Neural signatures of arbitration between Pavlovian and instrumental action selection
Gershman, S. J., Guitart‐Masip, M., & Cavanagh, J. F. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008553 · Cited by 40 · see citing works
Self-citationChanging the incentive structure of social media platforms to halt the spread of misinformation
Globig, L. K., Holtz, N., & Sharot, T. (2023) · eLife · doi:10.7554/elife.85767 · Cited by 50 · see citing works
Wittmann, B. C., Tan, G. C.-Y., Lisman, J. E., Dolan, R. J., & Düzel, E. (2013) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2013.07.018 · Cited by 38 · see citing works
Self-citationApproach–Avoidance Processes Contribute to Dissociable Impacts of Risk and Loss on Choice
Wright, N. D., Symmonds, M., Hodgson, K., FitzGerald, T. H. B., Crawford, B., & Dolan, R. J. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.0049-12.2012 · Cited by 37 · see citing works
Self-citationValue and probability coding in a feedback-based learning task utilizing food rewards
Tricomi, E. M., & Lempert, K. M. (2014) · Journal of Neurophysiology · doi:10.1152/jn.00086.2014 · Cited by 37 · see citing works
Jones, S., Paul, E. S., Dayan, P., Robinson, E. S. J., & Mendl, M. (2017) · Behavioural Brain Research · doi:10.1016/j.bbr.2017.05.044 · Cited by 36 · see citing works
Self-citationThe Cue-Approach Task as a General Mechanism for Long-Term Non-Reinforced Behavioral Change
Salomon, T., Botvinik‐Nezer, R., Gutentag, T., Gera, R., Iwanir, R., Tamir, M., & Schönberg, T. (2018) · Scientific Reports · doi:10.1038/s41598-018-21774-3 · Cited by 36 · see citing works
FitzGerald, T. H. B., Schwartenbeck, P., & Dolan, R. J. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.4389-13.2014 · Cited by 35 · see citing works
Self-citationCrockett, M. J., Clark, L., Smillie, L. D., & Robbins, T. W. (2011) · Psychopharmacology · doi:10.1007/s00213-011-2577-9 · Cited by 34 · see citing works
Cortical dopamine reduces the impact of motivational biases governing automated behaviour
Scholz, V., Hook, R. W., Kandroodi, M. R., Algermissen, J., Ioannidis, K., Christmas, D. M., Valle, S., Robbins, T. W., Grant, J. E., Chamberlain, S. R., et al. (2022) · Neuropsychopharmacology · doi:10.1038/s41386-022-01291-8 · Cited by 33 · see citing works
Seeing through rose-colored glasses: How optimistic expectancies guide visual attention
Kress, L., Bristle, M., & Aue, T. (2018) · PLoS ONE · doi:10.1371/journal.pone.0193311 · Cited by 33 · see citing works
Humans Integrate Monetary and Liquid Incentives to Motivate Cognitive Task Performance
Yee, D. M., Krug, M. K., Allen, A. Z., & Braver, T. S. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.02037 · Cited by 31 · see citing works
Motivational sensitivity of outcome-response priming: Experimental research and theoretical models
Watson, P., Wiers, R. W. H. J., Hommel, B., & de Wit, S. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1449-2 · Cited by 31 · see citing works
The Dopaminergic Midbrain Mediates an Effect of Average Reward on Pavlovian Vigor
Rigoli, F., Chew, B., Dayan, P., & Dolan, R. J. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00972 · Cited by 29 · see citing works
Self-citationFreezing Behavior as a Response to Sexual Visual Stimuli as Demonstrated by Posturography
Mouras, H., Lelard, T., Ahmaidi, S., Godefroy, O., & Krystkowiak, P. C. (2015) · PLoS ONE · doi:10.1371/journal.pone.0127097 · Cited by 29 · see citing works
Medial Frontal Cortex Activity Predicts Information Sampling in Economic Choice
Kaanders, P., Nili, H., O'Reilly, J. X., & Hunt, L. (2021) · The Journal of Neuroscience · doi:10.1523/jneurosci.0392-21.2021 · Cited by 27 · see citing works
Neural correlates of reward-related response tendencies in an equiprobable Go/NoGo task
Asci, O., Braem, S., Park, H. R. P., Boehler, C. N., & Krebs, R. M. (2019) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-019-00692-5 · Cited by 27 · see citing works
Striatal BOLD and midfrontal theta power express motivation for action
Algermissen, J., Swart, J. C., Scheeringa, R., Cools, R., & den Ouden, H. E. M. (2021) · Cerebral Cortex · doi:10.1093/cercor/bhab391 · Cited by 27 · see citing works
Neural basis of decision making guided by emotional outcomes
Katahira, K., Matsuda, Y.-T., Fujimura, T., Ueno, K., Asamizuya, T., Suzuki, C., Cheng, K., Okanoya, K., & Okada, M. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00564.2014 · Cited by 26 · see citing works
Richter, A., Richter, S., Barman, A., Soch, J., Klein, M., Assmann, A., Libeau, C., Behnisch, G., Wüstenberg, T., Seidenbecher, C. I., et al. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00250 · Cited by 27 · see citing works
Basal Ganglia Activity Mirrors a Benefit of Action and Reward on Long-Lasting Event Memory
Koster, R., Guitart‐Masip, M., Dolan, R. J., & Düzel, E. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhv216 · Cited by 26 · see citing works
Self-citationWagenbreth, C., Zaehle, T., Galazky, I., Voges, J., Guitart‐Masip, M., Heinze, H., & Düzel, E. (2015) · Journal of Neurology · doi:10.1007/s00415-015-7749-9 · Cited by 26 · see citing works
Self-citationPrior fear conditioning and reward learning interact in fear and reward networks
Bulganin, L., Bach, D. R., & Wittmann, B. C. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00067 · Cited by 28 · see citing works
Self-citationOpposing effects of reward and punishment on human vigor
Griffiths, B. J., & Beierholm, U. (2017) · Scientific Reports · doi:10.1038/srep42287 · Cited by 25 · see citing works
Interactions between incentive valence and action information in a cued approach–avoidance task
Hoofs, V., Carsten, T., Boehler, C. N., & Krebs, R. M. (2018) · Psychological Research · doi:10.1007/s00426-018-0975-x · Cited by 24 · see citing works
Preparing for (valenced) action: The role of differential effort in the orthogonalized go/no‐go task
Schevernels, H., Bombeke, K., Krebs, R. M., & Boehler, C. N. (2015) · Psychophysiology · doi:10.1111/psyp.12558 · Cited by 23 · see citing works
Whole-Brain Neural Dynamics of Probabilistic Reward Prediction
Bach, D. R., Symmonds, M., Barnes, G. R., & Dolan, R. J. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.2943-16.2017 · Cited by 24 · see citing works
Self-citationNeural correlates of reward-directed action and inhibition of action
Le, T. M., Zhang, S., Zhornitsky, S., Wang, W., & Li, C.-S. R. (2019) · Cortex · doi:10.1016/j.cortex.2019.10.007 · Cited by 22 · see citing works
Motivation alters response bias and neural activation patterns in a perceptual decision-making task
Reckless, G. E., Bolstad, I., Nakstad, P., Andreassen, O. A., & Jensen, J. (2013) · Neuroscience · doi:10.1016/j.neuroscience.2013.02.015 · Cited by 21 · see citing works
Verbruggen, F., & McLaren, R. (2016) · Psychological Research · doi:10.1007/s00426-016-0827-5 · Cited by 21 · see citing works
Reward-Based Learning, Model-Based and Model-Free
Huys, Q. J. M., Cruickshank, A., & Seriès, P. (2014) · Encyclopedia of Computational Neuroscience · doi:10.1007/978-1-4614-7320-6_674-1 · Cited by 21 · see citing works
Self-citationAction and Valence Modulate Choice and Choice-Induced Preference Change
Koster, R., Düzel, E., & Dolan, R. J. (2015) · PLoS ONE · doi:10.1371/journal.pone.0119682 · Cited by 20 · see citing works
Self-citation
- Authors
- Guitart-Masip, M., Fuentemilla, L., Bach, D. R., Huys, Q. J. M., Dayan, P., Dolan, R. J., & Duzel, E.
- Affiliations
- Wellcome Centre for Human Neuroimaging (GB)
- UCL Queen Square Institute of Neurology (GB)
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2011-05-25
- Volume
- 31
- Issue
- 21
- Pages
- 7867-7875
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- PMC
- PMC3109549
- OpenAlex
- W2020682620
- Cited by
- 216
- References
- 71
- Citations by year
- 2026: 5 · 2025: 13 · 2024: 10 · 2023: 14 · 2022: 10 · 2021: 17 · 2020: 15 · 2019: 14 · 2018: 15 · 2017: 9 · 2016: 13 · 2015: 19 · 2014: 20 · 2013: 25 · 2012: 12
- FWCI
- 9.91
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Neuroscience (publishedVersion, cc-by-nc-sa)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (publishedVersion, other-oa)
- Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona) (submittedVersion, cc-by-nc-sa)
- UCL Discovery (University College London) (submittedVersion)
- Max Planck Digital Library (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Funders
- Wellcome Trust (078865/Z/05/Z, 078865)
- Gatsby Charitable Foundation
- European Commission (220139)
- Deutsche Forschungsgemeinschaft (TP A7, SFB 779)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- striatum, substantia nigra/ventral tegmental area, reward anticipation, action representation, state values, dopaminergic midbrain
- MeSH terms
- Adult, Corpus Striatum, Dopamine, Female, Humans, Magnetic Resonance Imaging, Male, Mesencephalon, Photic Stimulation, Reaction Time, Young Adult, Anticipation, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
