By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Abstract
To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error. Here, we show, using two cognitive procedural learning tasks, that Parkinson's patients off medication are better at learning to avoid choices that lead to negative outcomes than they are at learning from positive outcomes. Dopamine medication reverses this bias, making patients more sensitive to positive than negative outcomes. This pattern was predicted by our biologically based computational model of basal ganglia–dopamine interactions in cognition, which has separate pathways for “Go” and “NoGo” responses that are differentially modulated by positive and negative reinforcement.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Neural mechanisms underlying decision making and learning involve fronto-striatal networks generating reward prediction errors which are highly sensitive to dopaminergic modulation (Frank et al. 2004; Kehagia et al. 2010; Schultz and Dickinson 2000).”
From Dopaminergic drug treatment remediates exaggerated cingulate prediction error responses in obsessive-compulsive disorder (2019) · cites this paper as Frank et al. 2004
In the citing paper
“Operant learning is, however, known to be impaired in PD (Avila et al., 2009; Frank et al., 2004; Rushworth et al., 1998; Rutledge et al., 2009; Shohamy et al., 2005).”
From Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models (2011) · cites this paper as Frank et al., 2004
In the citing paper
“Substantial work in conditions such as Parkinson’s disease, in which dopamine is reduced, shows that manipulations favoring D1 and D2 dopamine receptors result in effects that are most readily interpretable in a model-free manner (Frank et al., 2004).”
From Goals and Habits in the Brain (2013) · cites this paper as Frank et al., 2004
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS
MINK, J. W. (1996) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00042-1 · Cited by 2,486
Wise, R. A., & Rompre, P.-P. (1989) · Annual Review of Psychology · doi:10.1146/annurev.ps.40.020189.001203 · Cited by 1,968
Uneven Pattern of Dopamine Loss in the Striatum of Patients with Idiopathic Parkinson's Disease
Kish, S. J., Shannak, K., & Hornykiewicz, O. (1988) · New England Journal of Medicine · doi:10.1056/nejm198804073181402 · Cited by 1,953
A neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777
A neuropsychological theory of multiple systems in category learning.
Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998) · Psychological Review · doi:10.1037//0033-295x.105.3.442 · Cited by 1,223
The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia
Gerfen, C. R. (1992) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.15.030192.001441 · Cited by 1,004
Cools, R. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1136 · Cited by 906
Research works printed but not linked: 6
- 10.1037/0033-295x.105.3.442 · doi:10.1037/0033-295x.105.3.442 · Paper page
- M. Frank J. Cognit. Neurosci. in press.
- R. C. O'Reilly M. J. Frank ICS Tech. Rep. University of Colorado Institute of Cognitive Science (2004).
- 10.1088/0954-898x_8_4_001 · doi:10.1088/0954-898x_8_4_001 · Paper page
- M. Frank J. Rudy W. Levy R. O'Reilly Mem. Cognit. in press.
- 10.1037/0097-7403.17.3.334 · doi:10.1037/0097-7403.17.3.334 · Paper page
Cited by
Items 301–400 of 1,007
Adaptive properties of differential learning rates for positive and negative outcomes
Cazé, R. D., & van der Meer, M. A. A. (2013) · Biological Cybernetics · doi:10.1007/s00422-013-0571-5 · Cited by 120
Epigenetic Robotics Architecture (ERA)
Morse, A. F., de Greeff, J., Belpeame, T., & Cangelosi, A. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2087020 · Cited by 120
Task-related dissociation in ERN amplitude as a function of obsessive-compulsive symptoms
Gründler, T. O. J., Cavanagh, J. F., Figueroa, C. M., Frank, M. J., & Allen, J. J. B. (2009) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2009.03.010 · Cited by 119
Self-citationReward networks in the brain as captured by connectivity measures
Camara, E. (2009) · Frontiers in Neuroscience · doi:10.3389/neuro.01.034.2009 · Cited by 119
Lo, C.-C., Boucher, L., Pare, M., Schall, J. D., & Wang, X.-J. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.6164-08.2009 · Cited by 119
Feedback signals in the caudate reflect goal achievement on a declarative memory task
Tricomi, E. M., & Fiez, J. A. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.02.066 · Cited by 119
Reduced dopamine transporter binding predates impulse control disorders in Parkinson's disease
Vriend, C., Nordbeck, A. H., Booij, J., van der Werf, Y. D., Pattij, T., Voorn, P., Raijmakers, P. G. H. M., Foncke, E. M. J., van de Giessen, E., Berendse, H. W., et al. (2014) · Movement Disorders · doi:10.1002/mds.25886 · Cited by 118
A Reinforcement Learning Mechanism Responsible for the Valuation of Free Choice
Cockburn, J., Collins, A. G. E., & Frank, M. J. (2014) · Neuron · doi:10.1016/j.neuron.2014.06.035 · Cited by 118
Self-citationBanca, P., Vestergaard, M. D., Rankov, V., Baek, K., Mitchell, S., Lapa, T., Castelo-Branco, M., & Voon, V. (2015) · Neuropsychopharmacology · doi:10.1038/npp.2014.303 · Cited by 117
Habitual versus goal-directed action control in Parkinson disease
de Wit, S., Barker, R. A., Dickinson, A. D., & Cools, R. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21514 · Cited by 116
“Since then, similar feedback-based (probabilistic) learning impairments have been observed in a variety of instrumental learning paradigms (Frank, Samanta, Moustafa, & Sherman, 2007; Frank, Seeberger, & O ʼ Reilly, 2004; Shohamy et al., 2004; Shohamy, Myers, Onlaor, & Gluck, 2004).”
Theory of Mind in Parkinson's disease
Poletti, M., Enrici, I., Bonuccelli, U., & Adenzato, M. (2011) · Behavioural Brain Research · doi:10.1016/j.bbr.2011.01.010 · Cited by 116
How pupil responses track value-based decision-making during and after reinforcement learning
Slooten, J. C. V., Jahfari, S., Knapen, T. H. J., & Theeuwes, J. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006632 · Cited by 116
Lowe, C. J., Manocchio, F., Safati, A. B., & Hall, P. A. (2018) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2018.02.004 · Cited by 115
Christakou, A., Brammer, M., Giampietro, V., & Rubia, K. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1279-09.2009 · Cited by 114
Shin, J. H., Kim, D., & Jung, M. W. (2018) · Nature Communications · doi:10.1038/s41467-017-02817-1 · Cited by 114
A Role for the Medial Temporal Lobe in Feedback-Driven Learning: Evidence from Amnesia
Foerde, K., Race, E., Verfaellie, M., & Shohamy, D. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.5217-12.2013 · Cited by 113
Leyton, M., aan het Rot, M., Booij, L., Baker, G. B., Young, S. N., & Benkelfat, C. (2007) · Journal of Psychiatry and Neuroscience · doi:10.1139/jpn.0713 · Cited by 100
Dopamine, reward learning, and active inference
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00136 · Cited by 112
“…9 | Article 136 task (Figure 6), it is possible that studies that do not explicitly separate the effects of dopamine on performance and learning (Frank et al., 2004; Pessiglione et al., 2006; Moustafa et al., 2008; Rutledge et al., 2009; Voon et al., 2010; Nagy et al., 2012; Chowdhury et al.,…”
BackgroundThe interpretation of computational model parameters depends on the context
Eckstein, M. K., Master, S. L., Xia, L., Dahl, R. E., Wilbrecht, L., & Collins, A. G. (2022) · eLife · doi:10.7554/elife.75474 · Cited by 112
Collins, A. G. E., Albrecht, M. A., Waltz, J. A., Gold, J. M., & Frank, M. J. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.05.017 · Cited by 112
Self-citationBavard, S., Lebreton, M. P., Khamassi, M., Coricelli, G., & Palminteri, S. (2018) · Nature Communications · doi:10.1038/s41467-018-06781-2 · Cited by 112
Relating Natural Language Aptitude to Individual Differences in Learning Programming Languages
Prat, C. S., Madhyastha, T. M., Mottarella, M., & Kuo, C.-H. (2020) · Scientific Reports · doi:10.1038/s41598-020-60661-8 · Cited by 112
Learning and motivation in the human striatum
Shohamy, D. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.05.009 · Cited by 112
Winstanley, C. A., Cocker, P. J., & Rogers, R. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.230 · Cited by 112
Generalization of value in reinforcement learning by humans
Wimmer, G. E., Daw, N. D., & Shohamy, D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08017.x · Cited by 111
“dopamine system and its striatal targets in guiding choices via learned stimulus–reward or stimulus–response associations (Houk et al., 1995; Schultz et al., 1997; Frank et al., 2004; Everitt & Robbins, 2005; Daw & Doya, 2006; Schultz, 2006; Rangel et al., 2008).”
BackgroundPositive Affect Modulates Flexibility and Evaluative Control
van Wouwe, N. C., Band, G. P. H., & Ridderinkhof, K. R. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21380 · Cited by 111
Decision‐making in Parkinson’s disease patients with and without pathological gambling
Rossi, M., Gerschcovich, E. R., De Achaval, D., Perez‐Lloret, S., Cerquetti, D., Cammarota, A., Inés Nouzeilles, M., Fahrer, R., Merello, M., & Leiguarda, R. (2010) · European Journal of Neurology · doi:10.1111/j.1468-1331.2009.02792.x · Cited by 111
Virtual Reality as a Vehicle to Empower Motor-Cognitive Neurorehabilitation
Pérez-Marcos, D., Bieler-Aeschlimann, M., & Serino, A. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02120 · Cited by 111
Cerebral reorganization in Parkinson's disease.
Helmich, R. C. G. (2011) · Radboud Repository (Radboud University) · Cited by 111
Dopamine Modulates Adaptive Prediction Error Coding in the Human Midbrain and Striatum
Diederen, K. M. J., Ziauddeen, H., Vestergaard, M. D., Spencer, T., Schultz, W., & Fletcher, P. C. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.1979-16.2016 · Cited by 110
Ziegler, S., Pedersen, M. L., Mowinckel, A. M., & Biele, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.002 · Cited by 110
Updating beliefs under perceived threat
Garrett, N., González-Garzón, A. M., Foulkes, L., Levita, L., & Sharot, T. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.0716-18.2018 · Cited by 109
Reiter, A. M. F., Heinze, H., Schlagenhauf, F., & Deserno, L. (2016) · Neuropsychopharmacology · doi:10.1038/npp.2016.95 · Cited by 109
Caligiore, D., Mannella, F., Arbib, M. A., & Baldassarre, G. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005395 · Cited by 108
Guitart-Masip, M., Economides, M., Huys, Q. J. M., Frank, M. J., Chowdhury, R., Duzel, E., Dayan, P., & Dolan, R. J. (2014) · Psychopharmacology · doi:10.1007/s00213-013-3313-4 · Cited by 107
Self-citationEvent-related potentials and cognition in Parkinson’s disease: An integrative review
Seer, C., Lange, F., Georgiev, D., Jahanshahi, M., & Kopp, B. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.08.003 · Cited by 107
Dopamine and the motivation of cognitive control
Cools, R., Froböse, M. I., Aarts, E., & Hofmans, L. (2019) · Handbook of clinical neurology · doi:10.1016/b978-0-12-804281-6.00007-0 · Cited by 107
The role of learning-related dopamine signals in addiction vulnerability
Huys, Q. J. M., Tobler, P. N., Hasler, G., & Flagel, S. B. (2014) · Progress in brain research · doi:10.1016/b978-0-444-63425-2.00003-9 · Cited by 107
Depression and impulse control disorders in Parkinson׳s disease: two sides of the same coin?
Vriend, C., Pattij, T., van der Werf, Y. D., Voorn, P., Booij, J., Rutten, S., Berendse, H. W., & van den Heuvel, O. A. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.11.001 · Cited by 106
Enhanced negative feedback responses in remitted depression
Santesso, D. L., Steele, K. T., Bogdan, R., Holmes, A. J., Deveney, Meites, T. M., & Pizzagalli, D. A. (2008) · NeuroReport · doi:10.1097/wnr.0b013e3283036e73 · Cited by 106
Asaad, W. F., & Eskandar, E. N. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3793-11.2011 · Cited by 105
Variability in Dopamine Genes Dissociates Model-Based and Model-Free Reinforcement Learning
Doll, B. B., Bath, K. G., Daw, N. D., & Frank, M. J. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.1901-15.2016 · Cited by 105
Self-citationDynamic Flexibility in Striatal-Cortical Circuits Supports Reinforcement Learning
Gerraty, R. T., Davidow, J. Y., Foerde, K., Galvan, A., Bassett, D. S., & Shohamy, D. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.2084-17.2018 · Cited by 105
Distinct roles of dopamine and subthalamic nucleus in learning and probabilistic decision making
Coulthard, E. J., Bogacz, R., Javed, S., Mooney, L., Murphy, G., Keeley, S., & Whone, A. L. (2012) · Brain · doi:10.1093/brain/aws273 · Cited by 105
Decision making, impulsivity, and addictions: Do Parkinson's disease patients jump to conclusions?
Djamshidian, A., O’Sullivan, S. S., Sanotsky, Y., Sharman, S., Matviyenko, Y., Foltynie, T., Michalczuk, R., Avilés-Olmos, I., Fedoryshyn, L., Doherty, K. A., et al. (2012) · Movement Disorders · doi:10.1002/mds.25105 · Cited by 105
Kenemans, J. L., & Kähkönen, S. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.157 · Cited by 104
“Parkinson’s patients on L-dopa also manifest as ‘positive learners’: They learn more from positive than from negative feedback, apparently because elevated dopamine levels make them more sensitive to reward (Frank et al, 2004).”
Contextual novelty changes reward representations in the striatum
Guitart-Masip, M., Bunzeck, N., Stephan, K. E., Dolan, R. J., & Düzel, E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.5331-09.2010 · Cited by 104
“The basal ganglia, together with their dopaminergic afferents, provide a mechanism to learn about reward value of different behavioral options (Berridge and Robinson, 2003; Frank et al., 2004; Pessiglione et al., 2006).”
BackgroundRats and humans paying attention: Cross-species task development for translational research.
Demeter, E., Sarter, M., & Lustig, C. (2008) · Neuropsychology · doi:10.1037/a0013712 · Cited by 104
The risky business of dopamine agonists in Parkinson disease and impulse control disorders.
Claassen, D. O., van den Wildenberg, W. P. M., Ridderinkhof, K. R., Jessup, C. K., Harrison, M. B., Wooten, G. F., & Wylie, S. A. (2011) · Behavioral Neuroscience · doi:10.1037/a0023795 · Cited by 104
The Protective Action Encoding of Serotonin Transients in the Human Brain
Moran, R., Kishida, K. T., Lohrenz, T. M., Sáez, I., Laxton, A. W., Witcher, M. R., Tatter, S. B., Ellis, T. L., Phillips, P. E. M., Dayan, P., et al. (2018) · Neuropsychopharmacology · doi:10.1038/npp.2017.304 · Cited by 104
Choking on the money: reward-based performance decrements are associated with midbrain activity
Mobbs, D., Hassabis, D., Seymour, B., Marchant, J. L., Weiskopf, N., Dolan, R. J., & Frith, C. D. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02399.x · Cited by 103
Stress modulates reinforcement learning in younger and older adults.
Lighthall, N. R., Gorlick, M. A., Schoeke, A., Frank, M. J., & Mather, M. (2013) · Psychology and Aging · doi:10.1037/a0029823 · Cited by 103
Self-citationExplicit and implicit reinforcement learning across the psychosis spectrum.
Barch, D. M., Carter, C. S., Gold, J. M., Johnson, S. L., Kring, A. M., MacDonald, A. W., Pizzagalli, D. A., Ragland, J. D., Silverstein, S. M., & Strauss, M. E. (2017) · Journal of Abnormal Psychology · doi:10.1037/abn0000259 · Cited by 94
van der Schaaf, M. E., van Schouwenburg, M. R., Geurts, D. E. M., Schellekens, A. F. A., Buitelaar, J. K., Verkes, R. J., & Cools, R. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs344 · Cited by 102
Impulsivity and apathy in Parkinson's disease
Sinha, N., Manohar, S., & Husain, M. (2013) · Journal of Neuropsychology · doi:10.1111/jnp.12013 · Cited by 102
The Cognitive Neuroscience of Motivation and Learning
Daw, N. D., & Shohamy, D. (2008) · Social Cognition · doi:10.1521/soco.2008.26.5.593 · Cited by 102
Neural mechanisms of acquired phasic dopamine responses in learning
Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.11.019 · Cited by 102
Self-citationKahnt, T., Park, S. Q., Cohen, M. X., Beck, A., Heinz, A., & Wrase, J. (2009) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21092 · Cited by 102
Impulse Control and Related Disorders in Parkinson's Disease: Review
Lim, S., Evans, A. H., & Miyasaki, J. M. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1444.006 · Cited by 102
From Experienced Utility to Decision Utility
Berridge, K. C., & O’Doherty, J. P. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00018-8 · Cited by 102
Cognition in Parkinson's Disease
O’Callaghan, C., & Lewis, S. J. G. (2017) · International review of neurobiology · doi:10.1016/bs.irn.2017.05.002 · Cited by 102
A Genetically Defined Compartmentalized Striatal Direct Pathway for Negative Reinforcement
Xiao, X., Deng, H., Furlan, A., Yang, T., Zhang, X., Hwang, G.-R., Tucciarone, J., Wu, P., He, M., Palaniswamy, R., et al. (2020) · Cell · doi:10.1016/j.cell.2020.08.032 · Cited by 101
“Valence Representations and Valence-Specific Predictive Signals in Tshz1+ dMSNs The observations from Tshz1+ dMSNs thus far were rather unexpected, given the known and hypothesized functions of dMSNs (Frank et al., 2004; Hikosaka et al., 2019; Kravitz et al., 2010, 2012; Kravitz and Kreitzer, 2012; Nonomura et al., 20…”
BackgroundMethodResultInfluentialKanen, J. W., Ersche, K. D., Fineberg, N. A., Robbins, T. W., & Cardinal, R. N. (2019) · Psychopharmacology · doi:10.1007/s00213-019-05325-w · Cited by 101
Dowd, E. C., Frank, M. J., Collins, A., Gold, J. M., & Barch, D. M. (2016) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2016.05.005 · Cited by 101
Self-citationPerretta, J. G., Pari, G., & Beninger, R. J. (2005) · Cognitive and Behavioral Neurology · doi:10.1097/01.wnn.0000187939.81541.1d · Cited by 101
Frömer, R., Wolf, C. K. D., & Shenhav, A. (2019) · Nature Communications · doi:10.1038/s41467-019-12931-x · Cited by 101
Antonini, A., & Cilia, R. (2009) · Drug Safety · doi:10.2165/00002018-200932060-00004 · Cited by 101
The Cerebellum Is Involved in Reward-based Reversal Learning
Thoma, P., Bellebaum, C., Koch, B., Schwarz, M., & Daum, I. (2008) · The Cerebellum · doi:10.1007/s12311-008-0046-8 · Cited by 101
Ferré, S., Bonaventura, J., Zhu, W., Hatcher-Solis, C., Taura, J., Quiroz, C., Cai, N.-S., Moreno, E., Casadó-Anguera, V., Kravitz, A. V., et al. (2018) · Frontiers in Pharmacology · doi:10.3389/fphar.2018.00243 · Cited by 100
“…(positive reinforcement), while weak dopamine receptor activation promotes potentiation of corticostriatal synapses onto the indirect pathway and learning from negative outcomes (negative reinforcement) (Frank et al., 2004; Nakamura and Hikosaka, 2006; Shen et al., 2008; Voon et al., 2010).”
BackgroundProbabilistic learning and inference in schizophrenia
Averbeck, B. B., Evans, S., Chouhan, V., Bristow, E., & Shergill, S. S. (2011) · Schizophrenia Research · doi:10.1016/j.schres.2010.08.009 · Cited by 100
Impulse control disorders in Parkinson’s disease: seeking a roadmap toward a better understanding
Cilia, R., & van Eimeren, T. (2011) · Brain Structure and Function · doi:10.1007/s00429-011-0314-0 · Cited by 99
Moving Beyond ERP Components: A Selective Review of Approaches to Integrate EEG and Behavior
Bridwell, D. A., Cavanagh, J. F., Collins, A. G. E., Nunez, M. D., Srinivasan, R., Stober, S., & Calhoun, V. D. (2018) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2018.00106 · Cited by 99
Context-dependent outcome encoding in human reinforcement learning
Palminteri, S., & Lebreton, M. P. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.06.006 · Cited by 98
To Choose or to Avoid: Age Differences in Learning from Positive and Negative Feedback
Eppinger, B., & Kray, J. (2009) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21364 · Cited by 98
Oxytocin decreases aversion to angry faces in an associative learning task
Evans, S., Shergill, S. S., & Averbeck, B. B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2010.110 · Cited by 97
Probabilistic reinforcement learning in adults with autism spectrum disorders
Solomon, M., Smith, A. C., Frank, M. J., Ly, S., & Carter, C. S. (2011) · Autism Research · doi:10.1002/aur.177 · Cited by 97
Self-citationStress reduces use of negative feedback in a feedback-based learning task.
Petzold, A., Plessow, F., Goschke, T., & Kirschbaum, C. (2010) · Behavioral Neuroscience · doi:10.1037/a0018930 · Cited by 97
Meta-Learning through Hebbian Plasticity in Random Networks
Najarro, E., & Risi, S. (2020) · IT University Of Copenhagen (IT University of Copenhagen) · doi:10.48550/arxiv.2007.02686 · Cited by 97
Pessiglione, M., & Delgado, M. R. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.08.006 · Cited by 97
Julienne, H., Buhl, E., Leslie, D. S., & Hodge, J. J. L. (2017) · Neurobiology of Disease · doi:10.1016/j.nbd.2017.04.014 · Cited by 97
Experiential reward learning outweighs instruction prior to adulthood
Decker, J. H., Lourenco, F. S., Doll, B. B., & Hartley, C. A. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0332-5 · Cited by 96
Beyond valence and magnitude: A flexible evaluative coding system in the brain
Gu, R., Lei, Z., Broster, L., Wu, T., Jiang, Y., & Luo, Y.-J. (2011) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2011.10.006 · Cited by 96
Learning Reward Uncertainty in the Basal Ganglia
Mikhael, J. G., & Bogacz, R. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005062 · Cited by 95
A Role for Dopamine-Mediated Learning in the Pathophysiology and Treatment of Parkinson’s Disease
Beeler, J. A., Frank, M. J., McDaid, J., Alexander, E., Turkson, S., Sol Bernandez, M., McGehee, D. S., & Zhuang, X. (2012) · Cell Reports · doi:10.1016/j.celrep.2012.11.014 · Cited by 95
Self-citationEye Movements to Natural Images as a Function of Sex and Personality
Moss, F. J. M., Baddeley, R., & Canagarajah, N. C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0047870 · Cited by 95
Brown, J. W., & Braver, T. S. (2007) · Brain Research · doi:10.1016/j.brainres.2007.06.080 · Cited by 95
Synchronization of Midbrain Dopaminergic Neurons Is Enhanced by Rewarding Events
Joshua, M., Adler, A., Prut, Y., Vaadia, E., Wickens, J. R., & Bergman, H. (2009) · Neuron · doi:10.1016/j.neuron.2009.04.026 · Cited by 95
Franklin, N. T., & Frank, M. J. (2015) · eLife · doi:10.7554/elife.12029 · Cited by 95
Self-citationContributions of ERK signaling in the striatum to instrumental learning and performance
Shiflett, M. W., & Balleine, B. W. (2011) · Behavioural Brain Research · doi:10.1016/j.bbr.2010.12.010 · Cited by 94
“[1] Frank MJ, Seeberger LC, O’Reilly RC.”
The human ventromedial frontal lobe is critical for learning from negative feedback
Wheeler, E. Z., & Fellows, L. K. (2008) · Brain · doi:10.1093/brain/awn041 · Cited by 94
Transition to substance use disorders: impulsivity for reward and learning from reward
Poulton, A., & Hester, R. (2019) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsz077 · Cited by 94
Handbook of Reward and Decision Making
Dreher, J., & Tremblay, L. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374620-7.x0001-5 · Cited by 94
Markou, A., Salamone, J. D., Bussey, T. J., Mar, A. C., Brunner, D., Gilmour, G., & Balsam, P. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.08.007 · Cited by 93
Schönberg, T., O’Doherty, J. P., Joel, D., Inzelberg, R., Segev, Y., & Daw, N. D. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.08.011 · Cited by 93
Worthy, D. A., & Maddox, W. T. (2013) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2013.10.001 · Cited by 92
Interpreting the role of the striatum during multiple phases of motor learning
Cataldi, S., Stanley, A. T., Miniaci, M. C., & Sulzer, D. (2021) · FEBS Journal · doi:10.1111/febs.15908 · Cited by 92
Bornstein, A. M., & Daw, N. D. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003387 · Cited by 91
Comprehensive review: Computational modelling of schizophrenia
Valton, V., Romaniuk, L., Steele, J. D., Lawrie, S. M., & Seriès, P. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.08.022 · Cited by 91
A mosaic of cost–benefit control over cortico-striatal circuitry
Westbrook, A., Frank, M. J., & Cools, R. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.04.007 · Cited by 91
Self-citationPiray, P., Zeighami, Y., Bahrami, F., Eissa, A. M., Hewedi, D. H., & Moustafa, A. A. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.4063-13.2014 · Cited by 91
Metadata
- Authors
- Frank, M. J., Seeberger, L. C., & O'Reilly, R. C.
- Affiliations
- University of Colorado Boulder (US)
- Department of Psychology and Center for Neuroscience, University of Colorado Boulder, Boulder, CO 80309–0345, USA.
- Corresponding author
- Frank, Michael J.
- Venue
- Science
- Published
- 2004-12-10
- Volume
- 306
- Issue
- 5703
- Pages
- 1940-1943
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2163647009
- Cited by
- 1,964
- References
- 33
- Citations by year
- 2026: 57 · 2025: 73 · 2024: 83 · 2023: 79 · 2022: 91 · 2021: 99 · 2020: 134 · 2019: 104 · 2018: 125 · 2017: 111 · 2016: 115 · 2015: 111 · 2014: 134 · 2013: 111 · 2012: 108
- FWCI
- 11.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (R01MH069597, MH 069597-01)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Health Sciences › Medicine › Clinical Neurology › Parkinson's Disease Mechanisms and Treatments
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- Parkinson's disease, parkinsonism, dopamine, basal ganglia, positive reinforcement, negative reinforcement, avoidance learning, procedural learning, computational model, trial-and-error learning
- MeSH terms
- Aged, Antiparkinson Agents, Basal Ganglia, Cognition, Computer Simulation, Dopamine, Female, Frontal Lobe, Humans, Learning, Male, Middle Aged, Models, Neurological, Parkinson Disease, Probability, Reinforcement, Psychology, Matched-Pair Analysis, Feedback, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.