By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Abstract
To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error. Here, we show, using two cognitive procedural learning tasks, that Parkinson's patients off medication are better at learning to avoid choices that lead to negative outcomes than they are at learning from positive outcomes. Dopamine medication reverses this bias, making patients more sensitive to positive than negative outcomes. This pattern was predicted by our biologically based computational model of basal ganglia–dopamine interactions in cognition, which has separate pathways for “Go” and “NoGo” responses that are differentially modulated by positive and negative reinforcement.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Neural mechanisms underlying decision making and learning involve fronto-striatal networks generating reward prediction errors which are highly sensitive to dopaminergic modulation (Frank et al. 2004; Kehagia et al. 2010; Schultz and Dickinson 2000).”
From Dopaminergic drug treatment remediates exaggerated cingulate prediction error responses in obsessive-compulsive disorder (2019) · cites this paper as Frank et al. 2004
In the citing paper
“Operant learning is, however, known to be impaired in PD (Avila et al., 2009; Frank et al., 2004; Rushworth et al., 1998; Rutledge et al., 2009; Shohamy et al., 2005).”
From Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models (2011) · cites this paper as Frank et al., 2004
In the citing paper
“Substantial work in conditions such as Parkinson’s disease, in which dopamine is reduced, shows that manipulations favoring D1 and D2 dopamine receptors result in effects that are most readily interpretable in a model-free manner (Frank et al., 2004).”
From Goals and Habits in the Brain (2013) · cites this paper as Frank et al., 2004
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS
MINK, J. W. (1996) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00042-1 · Cited by 2,486
Wise, R. A., & Rompre, P.-P. (1989) · Annual Review of Psychology · doi:10.1146/annurev.ps.40.020189.001203 · Cited by 1,968
Uneven Pattern of Dopamine Loss in the Striatum of Patients with Idiopathic Parkinson's Disease
Kish, S. J., Shannak, K., & Hornykiewicz, O. (1988) · New England Journal of Medicine · doi:10.1056/nejm198804073181402 · Cited by 1,953
A neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777
A neuropsychological theory of multiple systems in category learning.
Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998) · Psychological Review · doi:10.1037//0033-295x.105.3.442 · Cited by 1,223
The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia
Gerfen, C. R. (1992) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.15.030192.001441 · Cited by 1,004
Cools, R. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1136 · Cited by 906
Research works printed but not linked: 6
- 10.1037/0033-295x.105.3.442 · doi:10.1037/0033-295x.105.3.442 · Paper page
- M. Frank J. Cognit. Neurosci. in press.
- R. C. O'Reilly M. J. Frank ICS Tech. Rep. University of Colorado Institute of Cognitive Science (2004).
- 10.1088/0954-898x_8_4_001 · doi:10.1088/0954-898x_8_4_001 · Paper page
- M. Frank J. Rudy W. Levy R. O'Reilly Mem. Cognit. in press.
- 10.1037/0097-7403.17.3.334 · doi:10.1037/0097-7403.17.3.334 · Paper page
Cited by
Items 801–900 of 1,007
Dunn, B. D., Widnall, E., Reed, N., Taylor, R. S., Owens, C. V., Spencer, A. E., Kraag, G., Kok, G., Geschwind, N., Wright, K. A., et al. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0438-1 · Cited by 36
Enhancing the psychometric properties of the Iowa gambling task using full generative modeling
Sullivan-Toole, H., Haines, N., Dale, K., & Olino, T. M. (2022) · Computational Psychiatry · doi:10.5334/cpsy.89 · Cited by 35
“For example, the ORL calculates separate gain and loss learning rates, in line with neurobiological and behavioral evidence for dissociable reward and punishment learning systems (Christakou, et al., 2013; Frank, Seeberger & O’Reilly, 2004; Gershman, 2015).”
Janssen, L. K., Sescousse, G., Hashemi, M. M., Timmer, M. H. M., ter Huurne, N. P., Geurts, D. E. M., & Cools, R. (2015) · Psychopharmacology · doi:10.1007/s00213-015-3986-y · Cited by 35
Skatova, A., Chan, P. A., & Daw, N. D. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00525 · Cited by 35
Alsiö, J., Phillips, B. U., Sala‐Bayo, J., Nilsson, S. R. O., Calafat, T., Rizwand, A., Plumbridge, J. M., López‐Cruz, L., Dalley, J. W., Cardinal, R. N., et al. (2019) · Psychopharmacology · doi:10.1007/s00213-019-05296-y · Cited by 35
Lin, L., Yu, L., Xiang, H.-C., Hu, X., Yuan, X., Zhu, H., Li, H., Zhang, H., Hou, T., Cao, J., et al. (2019) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2019.00239 · Cited by 35
The sense of agency for brain disorders: A comprehensive review and proposed framework
Malik, R. A., Galang, C. M., & Finger, E. C. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104759 · Cited by 35
Kobza, S., Ferrea, S., Schnitzler, A., Pollok, B., Südmeyer, M., & Bellebaum, C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0050250 · Cited by 35
The neural mechanisms of learning from competitors
Howard‐Jones, P. A., Bogacz, R., Yoo, J. H., Leonards, U., & Demetriou, S. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.06.027 · Cited by 35
Striatal action-learning based on dopamine concentration
Morris, G., Schmidt, R., & Bergman, H. (2009) · Experimental Brain Research · doi:10.1007/s00221-009-2060-6 · Cited by 35
The Effects of Evidence Bounds on Decision-Making: Theoretical and Empirical Developments
Zhang, J. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00263 · Cited by 35
Dopamine Modulation of Intertemporal Decision-making: Evidence from Parkinson Disease
Foerde, K., Figner, B., Doll, B. B., Woyke, I. C., Braun, E. K., Weber, E. U., & Shohamy, D. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00929 · Cited by 35
Reduction of Pavlovian Bias in Schizophrenia: Enhanced Effects in Clozapine-Administered Patients
Albrecht, M. A., Waltz, J. A., Cavanagh, J. F., Frank, M. J., & Gold, J. M. (2016) · PLoS ONE · doi:10.1371/journal.pone.0152781 · Cited by 35
Self-citationStocco, A., Murray, N. L., Yamasaki, B. L., Renno, T. J., Nguyen, J., & Prat, C. S. (2017) · Cognition · doi:10.1016/j.cognition.2017.03.001 · Cited by 35
Guardabassi, V., & Tomasetto, C. (2019) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2019.104706 · Cited by 35
Solomon, M., Ragland, J. D., Niendam, T. A., Lesh, T. A., Beck, J. S., Matter, J. C., Frank, M. J., & Carter, C. S. (2015) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2015.08.010 · Cited by 34
Self-citationSeymour, B., Barbe, M. T., Dayan, P., Shiner, T., Dolan, R. J., & Fink, G. R. (2016) · Scientific Reports · doi:10.1038/srep32509 · Cited by 34
Decision-making performance in Parkinson's disease correlates with lateral orbitofrontal volume
Kobayakawa, M., Tsuruya, N., & Kawamura, M. (2016) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2016.11.046 · Cited by 34
Jahfari, S., Ridderinkhof, K. R., Collins, A. G. E., Knapen, T. H. J., Waldorp, L., & Frank, M. J. (2018) · Cerebral Cortex · doi:10.1093/cercor/bhy076 · Cited by 34
Self-citationWalentowska, W., Severo, M. C., Moors, A., & Pourtois, G. (2019) · Psychophysiology · doi:10.1111/psyp.13456 · Cited by 34
Witt, K., Daniels, C., Schmitt-Eliassen, J., Kernbichler, J., Rehm, S., Volkmann, J., & Deuschl, G. (2006) · Brain Research · doi:10.1016/j.brainres.2006.07.055 · Cited by 34
Braunlich, K., & Seger, C. A. (2012) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1217 · Cited by 34
Methamphetamine-induced neurotoxicity disrupts naturally occurring phasic dopamine signaling
Howard, C. D., Daberkow, D. P., Ramsson, E. S., Keefe, K. A., & Garris, P. A. (2013) · European Journal of Neuroscience · doi:10.1111/ejn.12209 · Cited by 34
Reimers, L., Büchel, C., & Diekhof, E. K. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00401 · Cited by 34
Is the expression of stereotypic behavior a performance-limiting factor in animals?
Williams, J., & Randle, H. (2017) · Journal of Veterinary Behavior · doi:10.1016/j.jveb.2017.02.006 · Cited by 34
Learning the payoffs and costs of actions
Möller, M., & Bogacz, R. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006285 · Cited by 34
Modulating the Use of Multiple Memory Systems in Value-based Decisions with Contextual Novelty
Duncan, K. D., Semmler, A., & Shohamy, D. (2019) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01447 · Cited by 34
Adams, R. A., Moutoussis, M., Nour, M. M., Dahoun, T., Lewis, D. J., Illingworth, B. J. G., Veronese, M., Mathys, C. D., de Boer, L., Guitart‐Masip, M., et al. (2019) · Cerebral Cortex · doi:10.1093/cercor/bhz327 · Cited by 34
Dopaminergic medication increases reliance on current information in Parkinson’s disease
Vilares, I., & Kording, K. P. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0129 · Cited by 33
Reward prediction error in learning-related behaviors
Yujun, D., Song, D. F., Ni, J., Qing, H., & Quan, Z. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1171612 · Cited by 34
Group value learned through interactions with members: A reinforcement learning account
Hackel, L. M., Kogon, D., Amodio, D. M., & Wood, W. (2021) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2021.104267 · Cited by 33
Joint Modelling of Latent Cognitive Mechanisms Shared Across Decision-Making Domains
Stevenson, N., Innes, R. J., Boag, R. J., Miletić, S., Isherwood, S. J., Trutti, A. C., Heathcote, A., & Forstmann, B. U. (2024) · Computational Brain & Behavior · doi:10.1007/s42113-023-00192-3 · Cited by 33
Empathy and feedback processing in active and observational learning
Rak, N., Bellebaum, C., & Thoma, P. (2013) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-013-0187-1 · Cited by 33
Ryan, J. D., Moses, S. N., & Villate, C. (2008) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2008.09.006 · Cited by 33
Stocco, A. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00018 · Cited by 33
Impairments in both reward and punishment guided reinforcement learning in schizophrenia
Fervaha, G., Agid, O., Foussias, G., & Remington, G. J. (2013) · Schizophrenia Research · doi:10.1016/j.schres.2013.08.012 · Cited by 33
García‐García, I., Zeighami, Y., & Dagher, A. (2017) · Current Neurology and Neuroscience Reports · doi:10.1007/s11910-017-0755-9 · Cited by 33
Sewell, D. K., Jach, H. K., Boag, R. J., & Heer, C. V. (2019) · Psychonomic Bulletin & Review · doi:10.3758/s13423-019-01570-4 · Cited by 33
Online Learning in Iterated Prisoner’s Dilemma to Mimic Human Behavior
Lin, B., Bouneffouf, D., & Cecchi, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20868-3_10 · Cited by 33
Lourenco, F. S., Decker, J. H., Pedersen, G. A., Dellarco, D. V., Casey, B. J., & Hartley, C. A. (2015) · PLoS ONE · doi:10.1371/journal.pone.0128047 · Cited by 33
Pauli, W. M., Hazy, T. E., & O'Reilly, R. C. (2012) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00155 · Cited by 32
“The positive dopamine bursts cause go pathway neurons in the striatum to become more active (because of a preponderance of dopamine D1 receptors, which are excitatory) and no-go pathway neurons to become less active (from D2 receptors, which are inhibitory; Shen, Flajolet, Greengard, & Surmeier, 2008; Frank, 2005; Fra…”
BackgroundSelf-citationBrain function during probabilistic learning in relation to IQ and level of education
van den Bos, W., Crone, E. A., & Güroğlu, B. (2011) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2011.09.007 · Cited by 32
Contributions of the basal ganglia to action sequence learning and performance
Garr, E. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.09.017 · Cited by 32
Model-Free RL or Action Sequences?
Morris, A., & Cushman, F. A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02892 · Cited by 32
Neural Mechanisms of Human Decision-Making
Herd, S. A., Krueger, K. A., Nair, A., Mollick, J., & O’Reilly, R. C. (2021) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-020-00842-0 · Cited by 32
Self-citationCulture, Mind, and Brain: Emerging Concepts, Models, and Applications
(2020) · Cambridge University Press eBooks · doi:10.1017/9781108695374 · Cited by 32
The influence of psychological traits and prior experience on treatment expectations
Basedow, L. A., Fischer, A., Benson, S., Bingel, U., Brassen, S., Büchel, C., Engler, H., Mueller, E. M., Schedlowski, M., & Rief, W. (2023) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2023.152431 · Cited by 32
Reward processing and risky decision making in the aging brain.
Samanez-Larkin, G. R., & Knutson, B. D. (2014) · American Psychological Association eBooks · doi:10.1037/14322-006 · Cited by 32
Kim, Y., Matthews, M., & Moghaddam, B. (2010) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2010.07371.x · Cited by 32
Maril, S., Hassin‐Baer, S., Cohen, O. S., & Tomer, R. (2013) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2013.02.003 · Cited by 32
Enhanced avoidance learning in behaviorally inhibited young men and women
Sheynin, J., Shikari, S., Gluck, M. A., Moustafa, A. A., Servatius, R. J., & Myers, C. E. (2012) · Stress · doi:10.3109/10253890.2012.744391 · Cited by 32
Neural system interactions underlying human transitive inference
Moses, S. N., Brown, T. M., Ryan, J. D., & McIntosh, A. R. (2010) · Hippocampus · doi:10.1002/hipo.20735 · Cited by 32
Cockburn, J., & Frank, M. J. (2011) · The MIT Press eBooks · doi:10.7551/mitpress/9780262016438.003.0017 · Cited by 32
Self-citationAberg, K. C., Doell, K. C., & Schwartz, S. (2016) · PLoS ONE · doi:10.1371/journal.pone.0166675 · Cited by 32
Paradoxical Decision-Making: A Framework for Understanding Cognition in Parkinson’s Disease
Perugini, A., Ditterich, J., Shaikh, A. G., Knowlton, B. J., & Basso, M. A. (2018) · Trends in Neurosciences · doi:10.1016/j.tins.2018.04.006 · Cited by 32
Computational Modeling of Catecholamines Dysfunction in Alzheimer’s Disease at Pre-Plaque Stage
Caligiore, D., Silvetti, M., D’Amelio, M., Puglisi‐Allegra, S., & Baldassarre, G. (2020) · Journal of Alzheimer s Disease · doi:10.3233/jad-200276 · Cited by 32
Reinforcement learning in young adults with developmental language impairment
Lee, J. C., & Tomblin, J. B. (2012) · Brain and Language · doi:10.1016/j.bandl.2012.07.009 · Cited by 32
Computational approaches to schizophrenia: A perspective on negative symptoms
Deserno, L., Heinz, A., & Schlagenhauf, F. (2017) · Schizophrenia Research · doi:10.1016/j.schres.2016.10.004 · Cited by 31
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity
Miconi, T., Rawal, A., Clune, J., & Stanley, K. O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10585 · Cited by 31
Cognitive states influence dopamine-driven aberrant learning in Parkinson's disease
Cavanagh, J. F., Mueller, A. A., Brown, D. R., Janowich, J. R., Story-Remer, J., Wegele, A. R., & Richardson, S. P. (2017) · Cortex · doi:10.1016/j.cortex.2017.02.021 · Cited by 31
The asymmetric learning rates of murine exploratory behavior in sparse reward environments
Ohta, H., Satori, K., Takarada, Y., Arake, M., Ishizuka, T., Morimoto, Y., & Takahashi, T. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.05.030 · Cited by 31
Boroujeni, K. B., Sigona, M. K., Treuting, R. L., Manuel, T. J., Caskey, C. F., & Womelsdorf, T. (2022) · PLoS Biology · doi:10.1371/journal.pbio.3001785 · Cited by 31
Computational models of behavioral addictions: State of the art and future directions
Kato, A., Shimomura, K., Ognibene, D., Parvaz, M. A., Berner, L. A., Morita, K., & Fiore, V. G. (2022) · Addictive Behaviors · doi:10.1016/j.addbeh.2022.107595 · Cited by 31
Rmus, M., He, M., Baribault, B., Walsh, E. G., Festa, E. K., Collins, A. G. E., & Nassar, M. R. (2023) · eLife · doi:10.7554/elife.85243 · Cited by 31
Nishioka, T., Attachaipanich, S., Hamaguchi, K., Lazarus, M., d’Exaerde, A. D. K., Macpherson, T., & Hikida, T. (2023) · Nature Communications · doi:10.1038/s41467-023-38025-3 · Cited by 31
Dopamine and Norepinephrine Differentially Mediate the Exploration–Exploitation Tradeoff
Chen, C. S., Mueller, D., Knep, E., Ebitz, R. B., & Grissom, N. M. (2024) · Journal of Neuroscience · doi:10.1523/jneurosci.1194-23.2024 · Cited by 31
Linke, J. O., Sönnekes, C., & Wessa, M. (2011) · Bipolar Disorders · doi:10.1111/j.1399-5618.2011.00956.x · Cited by 31
Dorsal striatum is necessary for stimulus-value but not action-value learning in humans
Vo, K. D., Rutledge, R. B., Chatterjee, A. K., & Kable, J. W. (2014) · Brain · doi:10.1093/brain/awu277 · Cited by 31
Dopamine and Consolidation of Episodic Memory: Timing Is Everything
Grogan, J. P., Bogacz, R., Tsivos, D., Whone, A. L., & Coulthard, E. J. (2015) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00840 · Cited by 31
Acute effects of cocaine and cannabis on reversal learning as a function of COMT and DRD2 genotype
Spronk, D. B., van der Schaaf, M. E., Cools, R., de Bruijn, E. R. A., Franke, B., van Wel, J. H. P., Ramaekers, J. G., & Verkes, R. J. (2015) · Psychopharmacology · doi:10.1007/s00213-015-4141-5 · Cited by 31
Murty, V. P., Ballard, I. C., & Adcock, R. A. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhw005 · Cited by 31
Kroemer, N. B., Burrasch, C., & Hellrung, L. (2016) · Progress in brain research · doi:10.1016/bs.pbr.2016.06.009 · Cited by 31
The basal ganglia in reward and decision making
Doll, B. B., & Frank, M. J. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374620-7.00019-4 · Cited by 31
Self-citationBuelow, M. T., Frakey, L. L., Grace, J., & Friedman, J. H. (2013) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/act065 · Cited by 31
Psychopathy-related traits and the use of reward and social information: a computational approach
Brazil, I. A., Hunt, L. T., Bulten, B. H., Kessels, R. P. C., de Bruijn, E. R. A., & Mars, R. B. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00952 · Cited by 31
‘The Thorny and Arduous Path of Moral Progress’: Moral Psychology and Moral Enhancement
Zarpentine, C. (2012) · Neuroethics · doi:10.1007/s12152-012-9166-4 · Cited by 30
Basal Ganglia dysfunctions in movement disorders: What can be learned from computational simulations
Schroll, H., & Hamker, F. H. (2016) · Movement Disorders · doi:10.1002/mds.26719 · Cited by 30
Insensitivity to loss predicts apathy in huntington's disease
McLauchlan, D. J., Lancaster, T. M., Craufurd, D., Linden, D. E. J., & Rosser, A. (2019) · Movement Disorders · doi:10.1002/mds.27787 · Cited by 30
A Systematic Review of the Positive Valence System in Autism Spectrum Disorder
Tschida, J. E., & Yerys, B. E. (2020) · Neuropsychology Review · doi:10.1007/s11065-020-09459-z · Cited by 30
Vandendriessche, H., Demmou, A., Bavard, S., Yadak, J., Lemogne, C., Mauras, T., & Palminteri, S. (2022) · Psychological Medicine · doi:10.1017/s0033291722001593 · Cited by 30
Jepma, M., Roy, M., Ramlakhan, K., van Velzen, M., & Dahan, A. (2022) · eLife · doi:10.7554/elife.74149 · Cited by 30
Thoma, P., Edel, M., Suchan, B., & Bellebaum, C. (2014) · Psychiatry Research · doi:10.1016/j.psychres.2014.11.006 · Cited by 30
Strauss, G. P., Thaler, N. S., Matveeva, T. M., Vogel, S. J., Sutton, G. P., Lee, B. G., & Allen, D. N. (2015) · Journal of Abnormal Psychology · doi:10.1037/abn0000039 · Cited by 30
Working memory filtering continues to develop into late adolescence
Peverill, M., McLaughlin, K. A., Finn, A. S., & Sheridan, M. A. (2016) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2016.02.004 · Cited by 30
Cáceres, P., & Martín, R. S. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00204 · Cited by 30
Age-related variability in decision-making: Insights from neurochemistry
Berry, A. S., Jagust, W. J., & Hsu, M. (2018) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-018-00678-9 · Cited by 30
A biologically plausible embodied model of action discovery
Bolado-Gomez, R., & GURNEY, K. N. (2013) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2013.00004 · Cited by 30
Rosa, E. D., Schiff, S., Cagnolati, F., & Mapelli, D. (2015) · Aging Clinical and Experimental Research · doi:10.1007/s40520-015-0358-8 · Cited by 30
Reward-Based Learning, Model-Based and Model-Free
Huys, Q. J. M., Cruickshank, A., & Seriès, P. (2014) · Springer eBooks · doi:10.1007/978-1-4614-7320-6_674-1 · Cited by 30
Transitive inference in adults with autism spectrum disorders
Solomon, M., Frank, M. J., Smith, A. C., Ly, S., & Carter, C. S. (2011) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-011-0040-3 · Cited by 29
Self-citationClark, J. J., Nasrallah, N. A., Hart, A. S., Collins, A. L., Bernstein, I. L., & Phillips, P. E. M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0037357 · Cited by 29
Reduced susceptibility to confirmation bias in schizophrenia
Doll, B. B., Waltz, J. A., Cockburn, J., Brown, J. K., Frank, M. J., & Gold, J. M. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0250-6 · Cited by 29
Self-citationHybrid neural–cognitive models reveal how memory shapes human reward learning
Eckstein, M. K., Summerfield, C., Daw, N. D., & Miller, K. J. (2026) · Nature Human Behaviour · doi:10.1038/s41562-025-02324-0 · Cited by 29
Risk taking for potential losses but not gains increases with time of day
Bedder, R. L., Vaghi, M. M. S., Dolan, R. J., & Rutledge, R. B. (2023) · Scientific Reports · doi:10.1038/s41598-023-31738-x · Cited by 29
Bandit Models of Human Behavior: Reward Processing in Mental Disorders
Bouneffouf, D., Rish, I., & Cecchi, G. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-63703-7_22 · Cited by 29
Corlett, P. R., Canavan, S. V., Nahum, L., Appah, F., & MORGAN, P. T. (2014) · Cognitive Neuropsychiatry · doi:10.1080/13546805.2014.932685 · Cited by 29
Age-related changes in deterministic learning from positive versus negative performance feedback
van de Vijver, I., Ridderinkhof, K. R., & de Wit, S. (2015) · Aging Neuropsychology and Cognition · doi:10.1080/13825585.2015.1020917 · Cited by 29
CNTRICS Imaging Biomarkers Final Task Selection: Long-Term Memory and Reinforcement Learning
Ragland, J. D., Cohen, N. J., Cools, R., Frank, M., Hannula, D. E., & Ranganath, C. (2011) · Schizophrenia Bulletin · doi:10.1093/schbul/sbr168 · Cited by 29
Self-citationNatsheh, J. Y., & Shiflett, M. W. (2018) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2018.00045 · Cited by 29
A Novel Form of Memory for Auditory Fear Conditioning at a Low-Intensity Unconditioned Stimulus
Kishioka, A., Fukushima, F., Ito, T., Kataoka, H., Mori, H., Ikeda, T., Itohara, S., Sakimura, K., & Mishina, M. (2009) · PLoS ONE · doi:10.1371/journal.pone.0004157 · Cited by 29
Metadata
- Authors
- Frank, M. J., Seeberger, L. C., & O'Reilly, R. C.
- Affiliations
- University of Colorado Boulder (US)
- Department of Psychology and Center for Neuroscience, University of Colorado Boulder, Boulder, CO 80309–0345, USA.
- Corresponding author
- Frank, Michael J.
- Venue
- Science
- Published
- 2004-12-10
- Volume
- 306
- Issue
- 5703
- Pages
- 1940-1943
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2163647009
- Cited by
- 1,964
- References
- 33
- Citations by year
- 2026: 57 · 2025: 73 · 2024: 83 · 2023: 79 · 2022: 91 · 2021: 99 · 2020: 134 · 2019: 104 · 2018: 125 · 2017: 111 · 2016: 115 · 2015: 111 · 2014: 134 · 2013: 111 · 2012: 108
- FWCI
- 11.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (R01MH069597, MH 069597-01)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Health Sciences › Medicine › Clinical Neurology › Parkinson's Disease Mechanisms and Treatments
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- Parkinson's disease, parkinsonism, dopamine, basal ganglia, positive reinforcement, negative reinforcement, avoidance learning, procedural learning, computational model, trial-and-error learning
- MeSH terms
- Aged, Antiparkinson Agents, Basal Ganglia, Cognition, Computer Simulation, Dopamine, Female, Frontal Lobe, Humans, Learning, Male, Middle Aged, Models, Neurological, Parkinson Disease, Probability, Reinforcement, Psychology, Matched-Pair Analysis, Feedback, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.