The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Go and no-go learning in reward and punishment: Interactions between affect and effect
Open accessAbstract
Decision-making invokes two fundamental axes of control: affect or valence, spanning reward and punishment, and effect or action, spanning invigoration and inhibition. We studied the acquisition of instrumental responding in healthy human volunteers in a task in which we orthogonalized action requirements and outcome valence. Subjects were much more successful in learning active choices in rewarded conditions, and passive choices in punished conditions. Using computational reinforcement-learning models, we teased apart contributions from putatively instrumental and Pavlovian components in the generation of the observed asymmetry during learning. Moreover, using model-based fMRI, we showed that BOLD signals in striatum and substantia nigra/ventral tegmental area (SN/VTA) correlated with instrumentally learnt action values, but with opposite signs for go and no-go choices. Finally, we showed that successful instrumental learning depends on engagement of bilateral inferior frontal gyrus. Our behavioral and computational data showed that instrumental learning is contingent on overcoming inherent and plastic Pavlovian biases, while our neuronal data showed this learning is linked to unique patterns of brain activity in regions implicated in action and inhibition respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationInformation theory, inference, and learning algorithms
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5949 · Cited by 6,362
Gray, J. A. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00013066 · Cited by 4,477
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
Region of interest analysis using an SPM toolbox
Brett, M. (2002) · NeuroImage · Cited by 2,273
Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration
Klein, A., Andersson, J., Ardekani, B. A., Ashburner, J., Avants, B., Chiang, M.-C., Christensen, G. E., Collins, D. L., Gee, J., Hellier, P., et al. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.12.037 · Cited by 2,227
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex
Cardinal, R. N., Parkinson, J. A., Hall, J., & Everitt, B. J. (2002) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(02)00007-6 · Cited by 2,129
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citationEvent-Related fMRI: Characterizing Differential Responses
Friston, K. J., Fletcher, P., Josephs, O., Holmes, A., Rugg, M. D., & Turner, R. (1998) · NeuroImage · doi:10.1006/nimg.1997.0306 · Cited by 2,029
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping
Ng, A. Y., Harada, D., & Russell, S. (1999) · Cited by 1,744
Aron, A. R., & Poldrack, R. A. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4682-05.2006 · Cited by 1,630
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Self-citationTemporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Self-citationThe neostriatal mosaic: multiple levels of compartmental organization
Gerfen, C. R. (1992) · Trends in Neurosciences · doi:10.1016/0166-2236(92)90355-c · Cited by 1,467
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Breland, K., & Breland, M. (1961) · American Psychologist · doi:10.1037/h0040090 · Cited by 1,181
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Self-citationTonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 8×
“, 2009), this pattern is consistentwith a suggestion that dopamine plays a role in action preparation and invigoration (Berridge and Robinson, 1998; Niv et al., 2007; Salamone et al., 2007), a role complementary to its established role in representing a reward prediction error.”
BackgroundInfluentialSelf-citationA selective role for dopamine in stimulus-reward learning
Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., Akers, C. A., Clinton, S. M., Phillips, P. E. M., & Akil, H. (2011) · Nature · doi:10.1038/nature09588 · Cited by 1,026
5-HT and mechanisms of defence
Deakin, J. F. W., & Graeff, F. G. (1991) · Journal of Psychopharmacology · doi:10.1177/026988119100500414 · Cited by 1,010
Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits
Salamone, J. D., Correa, M., Farrar, A., & Mingote, S. M. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0668-9 · Cited by 1,000
Reconciling the role of central serotonin neurons in human and animal behavior
Soubrié, P. (1986) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00022871 · Cited by 996
Everitt, B. J., Belin, D., Economidou, D., Pelloux, Y., Dalley, J. W., & Robbins, T. W. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0089 · Cited by 974
(1947) · Harvard Educational Review · Cited by 967
Associative learning of social value
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. S. (2008) · Nature · doi:10.1038/nature07538 · Cited by 925
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917
Information Theory, Inference, and Learning Algorithms
(2004) · IEEE Transactions on Information Theory · doi:10.1109/tit.2004.834752 · Cited by 866
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844
Self-citationAUTO‐MAINTENANCE IN THE PIGEON: SUSTAINED PECKING DESPITE CONTINGENT NON‐REINFORCEMENT2
Williams, D. R., & Williams, H. (1969) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1969.12-511 · Cited by 712
Image Distortion Correction in fMRI: A Quantitative Evaluation
Hutton, C., Bork, A., Josephs, O., Deichmann, R., Ashburner, J., & Turner, R. (2002) · NeuroImage · doi:10.1006/nimg.2001.1054 · Cited by 681
Ethoexperimental Approaches to the Biology of Emotion
Blanchard, D. C., & Blanchard, R. J. (1988) · Annual Review of Psychology · doi:10.1146/annurev.ps.39.020188.000355 · Cited by 624
Hikida, T., Kimura, K., Wada, N., Funabiki, K., & Nakanishi, S. (2010) · Neuron · doi:10.1016/j.neuron.2010.05.011 · Cited by 592 · cited 1×
“The decrease in activity within the striatum and SN/VTA as no-go choice value increased does not fit the classical view of cortico-striatal circuits, in which reward promotes the direct (go) pathway and the punishment promotes the indirect (no-go) pathway (Frank et al., 2004; Hikida et al., 2010).”
BackgroundDecision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
Self-citationParkinson, J. A., Olmstead, M. C., Burns, L. H., Robbins, T. W., & Everitt, B. J. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-06-02401.1999 · Cited by 555
Absolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA
Bunzeck, N., & Düzel, E. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.021 · Cited by 503
Weiskopf, N., Hutton, C., Josephs, O., & Deichmann, R. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.07.029 · Cited by 475
The computational neurobiology of learning and reward
Daw, N. D., & Doya, K. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.006 · Cited by 465
Serotonin and dopamine: unifying affective, activational, and decision functions.
Cools, R., Nakamura, K., & Daw, N. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.121 · Cited by 464
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456
Self-citationWhite, D. J. (1969) · Cited by 438
Serotonin Modulates Behavioral Reactions to Unfairness
Crockett, M. J., Clark, L., Tabibnia, G., Lieberman, M. D., & Robbins, T. W. (2008) · Science · doi:10.1126/science.1155577 · Cited by 421
Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding
Huys, Q. J. M., Cools, R., Gölzer, M., Friedel, E., Heinz, A., Dolan, R. J., & Dayan, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002028 · Cited by 416
Self-citationRobbins, T. W. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2097 · Cited by 413
Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
The misbehavior of value and the discipline of the will
Dayan, P., Niv, Y., Seymour, B., & D. Daw, N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.002 · Cited by 388
Self-citationSerotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381
Self-citationCools, R., Frank, M. J., Gibbs, S. E., Miyakawa, A., Jagust, W., & D'Esposito, M. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.4467-08.2009 · Cited by 364
The integrative function of the basal ganglia in instrumental conditioning
Balleine, B. W., Liljeholm, M., & Ostlund, S. B. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.10.034 · Cited by 356
Flagel, S. B., Robinson, T. E., Clark, J. J., Clinton, S. M., Watson, S. J., Seeman, P., Phillips, P. E. M., & Akil, H. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.142 · Cited by 339 · cited 1×
“This echoes recent evidence regarding individual differences in decision-making, and most particularly a prominent distinction between sign-tracking and goal-tracking in rodents (Flagel et al., 2010; 2011).”
BackgroundMarch, J. G. (1996) · Psychological Review · doi:10.1037//0033-295x.103.2.309 · Cited by 310
Human Pavlovian–Instrumental Transfer
Talmi, D., Seymour, B., Dayan, P., & Dolan, R. J. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4028-07.2008 · Cited by 301
Self-citationParkinson, J. A., Dalley, J. W., Cardinal, R. N., Bamford, A., Fehnert, B., Lachenal, G., Rudarakanchana, N., Halkerston, K. M., Robbins, T. W., & Everitt, B. J. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00291-7 · Cited by 288
Crockett, M. J., Clark, L., & Robbins, T. W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2513-09.2009 · Cited by 271
Separate neural substrates for skill learning and performance in the ventral and dorsal striatum
Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., & O'Reilly, R. C. (2007) · Nature Neuroscience · doi:10.1038/nn1817 · Cited by 241
How Humans Integrate the Prospects of Pain and Reward during Choice
Talmi, D., Dayan, P., Kiebel, S. J., Frith, C. D., & Dolan, R. J. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2026-09.2009 · Cited by 217
Self-citationGuitart-Masip, M., Fuentemilla, L., Bach, D. R., Huys, Q. J. M., Dayan, P., Dolan, R. J., & Duzel, E. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6376-10.2011 · Cited by 216
Self-citationFunctional imaging of the human dopaminergic midbrain
Düzel, E., Bunzeck, N., Guitart-Masip, M., Wittmann, B., Schott, B. H., & Tobler, P. N. (2009) · Trends in Neurosciences · doi:10.1016/j.tins.2009.02.005 · Cited by 216
Self-citationNeurogenetics and Pharmacology of Learning, Motivation, and Cognition
Frank, M. J., & Fossella, J. A. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.96 · Cited by 192
Overcoming status quo bias in the human brain
Fleming, S. M., Thomas, C. L., & Dolan, R. J. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0910380107 · Cited by 191
Self-citationStatistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate
Bayer, H. M., Lau, B., & Glimcher, P. W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01140.2006 · Cited by 187
McCullough, L. D., Sokolowski, J. D., & Salamone, J. D. (1993) · Neuroscience · doi:10.1016/0306-4522(93)90538-q · Cited by 180
Adaptive learning and risk taking.
Denrell, J. (2007) · Psychological Review · doi:10.1037/0033-295x.114.1.177 · Cited by 175
Niv, Y., Joel, D., Meilijson, I., & Ruppin, E. (2002) · Adaptive Behavior · doi:10.1177/10597123020101001 · Cited by 166
WICKENS, J. R., BUDD, C. S., HYLAND, B. I., & ARBUTHNOTT, G. W. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.016 · Cited by 165
Signals in Human Striatum Are Appropriate for Policy Update Rather than Value Prediction
Li, J., & Daw, N. D. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.6316-10.2011 · Cited by 163
Darvas, M., Fadok, J. P., & Palmiter, R. D. (2011) · Learning & Memory · doi:10.1101/lm.2041211 · Cited by 142
Two-factor theory, the actor--critic model, and conditioned avoidance
Maia, T. V. (2010) · Learning & Behavior · doi:10.3758/lb.38.1.50 · Cited by 139
A temporal difference account of avoidance learning
Moutoussis, M., Bentall, R. P., Williams, J. H., & Dayan, P. (2008) · Network Computation in Neural Systems · doi:10.1080/09548980802192784 · Cited by 86
Self-citationDissociable reward and timing signals in human midbrain and ventral striatum
Klein-Flügge, M. C., Hunt, L. T., Bach, D. R., Dolan, R. J., & Behrens, T. E. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.024 · Cited by 75
Self-citationUslaner, J. M., Dell’Orco, J., Pevzner, A., & Robinson, T. E. (2007) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301653 · Cited by 64
Beninger, R., & Phillips, A. (1981) · Pharmacology, Biochemistry and Behavior · doi:10.1016/0091-3057(81)90110-6 · Cited by 60
The Misbehaviour of Value and the Discipline of the Will
Dayan, P. (2006) · Cited by 60
Self-citationConditioned associations and economic decision biases.
Guitart-Masip, M., Talmi, D., & Dolan, R. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.06.021 · Cited by 36
Self-citation
Cited by
The mysterious motivational functions of mesolimbic dopamine
Salamone, J. D., & Correa, M. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.021 · Cited by 1,418
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002
Self-citationMotivation and cognitive control: from behavior to neural mechanism
Botvinick, M., & Braver, T. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010814-015044 · Cited by 909
Mechanisms of motivation–cognition interaction: challenges and opportunities
Braver, T. S., Krug, M. K., Chiew, K. S., Kool, W., Westbrook, J. A., Clement, N. J., Adcock, R. A., Barch, D. M., Botvinick, M. M., Carver, C. S., et al. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0300-0 · Cited by 391
Ahn, W.-Y., Haines, N., & Zhang, L. (2017) · Computational Psychiatry · doi:10.1162/cpsy_a_00002 · Cited by 362
Metadata
- Authors
- Guitart-Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Duzel, E., & Dolan, R. J.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2012-08
- Volume
- 62
- Issue
- 1
- Pages
- 154-166
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2142586783
- Cited by
- 421
- References
- 77
- FWCI
- 9.83
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.