Deficits in Positive Reinforcement Learning and Uncertainty-Driven Exploration Are Associated with Distinct Aspects of Negative Symptoms in Schizophrenia
Open accessThe DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Abstract
Background Negative symptoms are core features of schizophrenia; however, the cognitive and neural basis for individual negative symptom domains remains unclear. Converging evidence suggests a role for striatal and prefrontal dopamine in reward learning and the exploration of actions that might produce outcomes that are better than the status quo. The current study examines whether deficits in reinforcement learning and uncertainty-driven exploration predict specific negative symptoms domains. Methods We administered a temporal decision making task, which required trial-by-trial adjustment of reaction time (RT) to maximize reward receipt, to 51 patients with schizophrenia and 39 age-matched healthy controls. Task conditions were designed such that expected value (probability * magnitude) increased (IEV), decreased (DEV), or remained constant (CEV) with increasing response times. Computational analyses were applied to estimate the degree to which trial-by-trial responses are influenced by reinforcement history. Results Individuals with schizophrenia showed impaired Go learning, but intact NoGo learning relative to controls. These effects were pronounced as a function of global measures of negative symptom. Uncertainty-based exploration was substantially reduced in individuals with schizophrenia, and selectively correlated with clinical ratings of anhedonia. Conclusions Schizophrenia patients, particularly those with high negative symptoms, failed to speed RT's to increase positive outcomes and showed reduced tendency to explore when alternative actions could lead to better outcomes than the status quo. Results are interpreted in the context of current computational, genetic, and pharmacological data supporting the roles of striatal and prefrontal dopamine in these processes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Brief Psychiatric Rating Scale.
Overall, J. E., & Gorham, D. R. (1962) · Psychological Reports · doi:10.2466/pr0.1962.10.3.799 · Cited by 11,090
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785
Implications of Normal Brain Development for the Pathogenesis of Schizophrenia
Weinberger, D. R. (1987) · Archives of General Psychiatry · doi:10.1001/archpsyc.1987.01800190080012 · Cited by 3,724 · cited 1×
“We posit that these effects are related to degradations in prefrontal cortical DA function, often attributed as a source of negative symptoms (24,26,48,49).”
BackgroundEffect of COMT Val 108/158 Met genotype on frontal lobe function and risk for schizophrenia
Egan, M. F., Goldberg, T. E., Kolachana, B. S., Callicott, J. H., Mazzanti, C. M., Straub, R. E., Goldman, D., & Weinberger, D. R. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.111134598 · Cited by 2,381
The debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Chlorpromazine Equivalent Doses for the Newer Atypical Antipsychotics
Woods, S. W. (2003) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v64n0607 · Cited by 2,050
Abstract reward and punishment representations in the human orbitofrontal cortex
O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001) · Nature Neuroscience · doi:10.1038/82959 · Cited by 2,003
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964 · cited 2×
“When viewed in conjunction with neuocomputational models of corticostriatal circuitry in reinforcement earning (7,8), the current behavioral and modeling findings are sugestive of potential dysfunction in the direct D1-driven BG pathway eading to abnormalities in using positive feedback to guide behavior, ith relative…”
BackgroundSelf-citationThe Scale for the Assessment of Negative Symptoms (SANS): Conceptual and Theoretical Foundations
Andreasen, N. C. (1989) · British Journal of Psychiatry · doi:10.1192/s0007125000291496 · Cited by 1,530
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
Frank, M. J. (2005) · Journal of Cognitive Neuroscience · doi:10.1162/0898929052880093 · Cited by 925 · cited 3×
“This model has been applied to understand patterns of reard learning in Parkinson’s disease patients (7), who have deleted striatal DA levels because of the disease but increased triatal DA levels after DA medication (12,13).”
BackgroundMethodSelf-citationOrbitofrontal Cortex and Its Contribution to Decision-Making
Wallis, J. D. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094334 · Cited by 900
The brief negative symptom scale: Psychometric properties
Kirkpatrick, B., Strauss, G. P., Nguyen, L., Fischer, B. A., Daniel, D. G., Cienfuegos, A., & Marder, S. R. (2011) · Schizophrenia Bulletin · doi:10.1093/schbul/sbq059 · Cited by 895
Self-citationA cellular mechanism of reward-related learning
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001) · Nature · doi:10.1038/35092560 · Cited by 879 · cited 1×
“In reinforcement learning models of corticostriatal circuitry (7,8), phasic DA signals are proposed to modify synaptic plasticity in the corticostriatal pathway (9,10) and subsequently reinforce “Go” (learning to pursue actions that have high reward probability) and “NoGo” learning (learning to avoid actions with low…”
BackgroundPrefrontal Dopamine D1Receptors and Working Memory in Schizophrenia
Abi-Dargham, A., Mawlawi, O., Lombardo, I., Gil, R., Martinez, D., Huang, Y., Hwang, D.-R., Keilp, J., Kochan, L., Van Heertum, R., et al. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-09-03708.2002 · Cited by 724
Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning
Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0706111104 · Cited by 683 · cited 1×
“frontal damage (21) and in healthy participants with the val/val genotype of the catechol-O-methyltransferase (COMT) gene (15), who have reduced prefrontal (and particularly, orbitofrontal) DA levels (22).”
BackgroundSelf-citationBoorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665
Wallis, J. D., & Miller, E. K. (2003) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2003.02922.x · Cited by 628
Frank, M. J., & Claus, E. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.300 · Cited by 585 · cited 2×
“One-way ANOVAs indicated that the three groups failed to significantly differ on any parameter other than exploration [F (2,85) 5.”
MethodSelf-citationDysfunction of ventral striatal reward prediction in schizophrenia
Juckel, G., Schlagenhauf, F., Koslowski, M., Wüstenberg, T., Villringer, A., Knutson, B., Wrase, J., & Heinz, A. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2005.07.051 · Cited by 578
Neuronal Activity Related to Reward Value and Motivation in Primate Frontal Cortex
Roesch, M. R., & Olson, C. R. (2004) · Science · doi:10.1126/science.1093223 · Cited by 542
Emotional Experience in Patients With Schizophrenia Revisited: Meta-analysis of Laboratory Studies
Cohen, A. S., & Minor, K. S. (2010) · Schizophrenia Bulletin · doi:10.1093/schbul/sbn061 · Cited by 541
Substantia nigra/ventral tegmental reward prediction error disruption in psychosis
Murray, G. K., Corlett, P. R., Clark, L., Pessiglione, M., Blackwell, A. D., Honey, G., Jones, P. B., Bullmore, E. T., Robbins, T. W., & Fletcher, P. C. (2008) · Molecular Psychiatry · doi:10.1038/sj.mp.4002058 · Cited by 532
Elliott, R., Newman, J. L., Longe, O. A., & Deakin, J. F. W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-01-00303.2003 · Cited by 514 · cited 2×
“This was onfirmed statistically by separate repeated measures ANOVAs, which ndicated that groups significantly differed on the DEV condition F (1,88) 9.”
BackgroundMethodCognition and control in schizophrenia: a computational model of dopamine and prefrontal function
Braver, T. S., Barch, D. M., & Cohen, J. D. (1999) · Biological Psychiatry · doi:10.1016/s0006-3223(99)00116-x · Cited by 507
Frank, M. J., Doll, B. B., Oas-Terpstra, J., & Moreno, F. (2009) · Nature Neuroscience · doi:10.1038/nn.2342 · Cited by 471
Self-citationEmotional Response Deficits in Schizophrenia: Insights From Affective Science
Kring, A. M., & Moran, E. K. (2008) · Schizophrenia Bulletin · doi:10.1093/schbul/sbn071 · Cited by 458
Buchanan, R. W., Javitt, D. C., Marder, S. R., Schooler, N. R., Gold, J. M., McMahon, R. P., Heresco-Levy, U., & Carpenter, W. T. (2007) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2007.06081358 · Cited by 419
Self-citationComputational models of schizophrenia and dopamine modulation in the prefrontal cortex
Rolls, E. T., Loh, M., Deco, G., & Winterer, G. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2462 · Cited by 387
Variation in dopamine genes influences responsivity of the human reward system
Dreher, J.-C., Kohn, P., Kolachana, B., Weinberger, D. R., & Berman, K. F. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0805517106 · Cited by 376
Bodi, N., Keri, S., Nagy, H., Moustafa, A., Myers, C. E., Daw, N., Dibo, G., Takats, A., Bereczki, D., & Gluck, M. A. (2009) · Brain · doi:10.1093/brain/awp094 · Cited by 363
Waltz, J. A., Frank, M. J., Robinson, B. M., & Gold, J. M. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.09.042 · Cited by 327 · cited 3×
“We have argued hat this deficit in rapid acquisition associated with negative sympoms is likely to stem from prefrontal cortical dysfunction (19,20).”
BackgroundResultInfluentialSelf-citationWickens, J. R., Begg, A. J., & Arbuthnott, G. W. (1996) · Neuroscience · doi:10.1016/0306-4522(95)00436-m · Cited by 290
Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors
Voon, V., Pessiglione, M., Brezing, C., Gallea, C., Fernandez, H. H., Dolan, R. J., & Hallett, M. (2010) · Neuron · doi:10.1016/j.neuron.2009.12.027 · Cited by 280 · cited 1×
“Imaging studies show that these biases are ccompanied by medication-induced increased sensitivity to ositive prediction errors and reduced sensitivity to negative rediction errors in the ventral and dorsolateral striatum (17).”
BackgroundPrefrontal DA Transmission at D1 Receptors and the Pathology of Schizophrenia
Abi-Dargham, A., & Moore, H. (2003) · The Neuroscientist · doi:10.1177/1073858403252674 · Cited by 277
Neurocomputational models of basal ganglia function in learning, memory and choice
Cohen, M. X., & Frank, M. J. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.09.029 · Cited by 237 · cited 1×
“In reinforcement learning models of corticostriatal circuitry (7,8), phasic DA signals are proposed to modify synaptic plasticity in the corticostriatal pathway (9,10) and subsequently reinforce “Go” (learning to pursue actions that have high reward probability) and “NoGo” learning (learning to avoid actions with low…”
BackgroundSelf-citationFrank, M. J., Santamaria, A., O'Reilly, R. C., & Willcutt, E. (2007) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301278 · Cited by 227
Neural correlates of reward processing in schizophrenia — Relationship to apathy and depression
Simon, J. J., Biller, A., Walther, S., Roesch-Ely, D., Stippich, C., Weisbrod, M., & Kaiser, S. (2010) · Schizophrenia Research · doi:10.1016/j.schres.2009.11.007 · Cited by 221
Dalley, J. W., Lääne, K., Theobald, D. E. H., Armstrong, H. C., Corlett, P. R., Chudasama, Y., & Robbins, T. W. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0502080102 · Cited by 214
COMT genotype predicts cortical-limbic D1 receptor availability measured with [11C]NNC112 and PET
Slifstein, M., Kolachana, B., Simpson, E. H., Tabares, P., Cheng, B., Duvall, M., Gordon Frankle, W., Weinberger, D. R., Laruelle, M., & Abi-Dargham, A. (2008) · Molecular Psychiatry · doi:10.1038/mp.2008.19 · Cited by 195
Clinical correlates of levodopa-induced dopamine release in Parkinson disease: A PET study
Pavese, N., Evans, A. H., Tai, Y. F., Hotton, G., Brooks, D. J., Lees, A. J., & Piccini, P. (2006) · Neurology · doi:10.1212/01.wnl.0000242888.30755.5d · Cited by 191 · cited 1×
“This model has been applied to understand patterns of reard learning in Parkinson’s disease patients (7), who have deleted striatal DA levels because of the disease but increased triatal DA levels after DA medication (12,13).”
MethodWaltz, J. A., Schweitzer, J. B., Gold, J. M., Kurup, P. K., Ross, T. J., Jo Salmeron, B., Rose, E. J., McClure, S. M., & Stein, E. A. (2009) · Neuropsychopharmacology · doi:10.1038/npp.2008.214 · Cited by 186
Self-citationRole of Dopamine in the Primate Caudate Nucleus in Reward Modulation of Saccades
Nakamura, K., & Hikosaka, O. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4853-05.2006 · Cited by 178
Tedroff, J., Pedersen, M., Aquilonius, S.-M., Hartvig, P., Jacobsson, G., & Långström, B. (1996) · Neurology · doi:10.1212/wnl.46.5.1430 · Cited by 161
Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes
Palminteri, S., Lebreton, M., Worbe, Y., Grabli, D., Hartmann, A., & Pessiglione, M. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0904035106 · Cited by 154 · cited 1×
“[27]) and recent demonstration that D2 blockade enhances NoGo learning in Tourette’s syndrome patients (16).”
BackgroundA Bayesian formulation of behavioral control
Huys, Q. J. M., & Dayan, P. (2009) · Cognition · doi:10.1016/j.cognition.2009.01.008 · Cited by 145
A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism
Moustafa, A. A., Cohen, M. X., Sherman, S. J., & Frank, M. J. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3116-08.2008 · Cited by 144 · cited 1×
“In the current study, we examined reinforcement learning in SZ with a task that required subjects to modulate the response time of a single motor response, where both reinforcement probabilities and magnitudes varied as a function of reaction time (RT) (23).”
BackgroundSelf-citationWaltz, J. A., Frank, M. J., Wiecki, T. V., & Gold, J. M. (2011) · Neuropsychology · doi:10.1037/a0020882 · Cited by 132
Self-citationStrauss, G. P., Robinson, B. M., Waltz, J. A., Frank, M. J., Kasanova, Z., Herbener, E. S., & Gold, J. M. (2011) · Schizophrenia Bulletin · doi:10.1093/schbul/sbq047 · Cited by 50
Self-citationWiecki, T. V., Riedinger, K., von Ameln-Mayerhofer, A., Schmidt, W. J., & Frank, M. J. (2009) · Psychopharmacology · doi:10.1007/s00213-008-1457-4 · Cited by 47
Self-citationLinking Animal Models of Psychosis to Computational Models of Dopamine Function
Smith, A. J., Li, M., Becker, S., & Kapur, S. (2006) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301086 · Cited by 46
Abnormal prediction error is associated with negative and depressive symptoms in schizophrenia
Moran, P. M., Owen, L. H., Crookes, A. E., Al-Uzri, M. M., & Reveley, M. A. (2007) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2007.07.021 · Cited by 43
Associative learning in deficit and nondeficit schizophrenia
Farkas, M., Polgár, P., Kelemen, O., Réthelyi, J., Bitter, I., Myers, C. E., Gluck, M. A., & Kéri, S. (2008) · Neuroreport · doi:10.1097/wnr.0b013e3282f2dff6 · Cited by 42 · cited 1×
“In light of previous studies indicating that reinforcement learning impairments are most severe in high negative symptom patients (28,29), we therefore also examined the role of negative symptoms in Go and NoGo learning, with the prediction that high negative symptom patients would show the greatest Go learning impair…”
BackgroundPolgár, P., Farkas, M., Nagy, O., Kelemen, O., Réthelyi, J., Bitter, I., Myers, C. E., Gluck, M. A., & Kéri, S. (2007) · Schizophrenia Research · doi:10.1016/j.schres.2007.06.027 · Cited by 40
The role of the orbitofrontal cortex in human discrimination learning
Chase, H. W., Clark, L., Myers, C. E., Gluck, M. A., Sahakian, B. J., Bullmore, E. T., & Robbins, T. W. (2007) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2007.12.011 · Cited by 29 · cited 1×
“frontal damage (21) and in healthy participants with the val/val genotype of the catechol-O-methyltransferase (COMT) gene (15), who have reduced prefrontal (and particularly, orbitofrontal) DA levels (22).”
BackgroundParkinson's Disease: Fighting the Will?: Figure 1.
Niv, Y., & Rivlin‐Etzion, M. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.4010-07.2007 · Cited by 25
Kasanova, Z., Waltz, J. A., Strauss, G. P., Frank, M. J., & Gold, J. M. (2011) · Schizophrenia Research · doi:10.1016/j.schres.2010.12.003 · Cited by 19
Self-citation
Cited by
Schizophrenia, dopamine and the striatum: From biology to symptoms
McCutcheon, R. A., Abi-Dargham, A., & Howes, O. D. (2019) · Trends in Neurosciences · doi:10.1016/j.tins.2018.12.004 · Cited by 763
From reinforcement learning models to psychiatric and neurological disorders
Maia, T. V., & Frank, M. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2723 · Cited by 722
Self-citationDopamine and glutamate in schizophrenia: biology, symptoms and treatment
McCutcheon, R. A., Krystal, J. H., & Howes, O. D. (2020) · World Psychiatry · doi:10.1002/wps.20693 · Cited by 612
A Review of Reward Processing and Motivational Impairment in Schizophrenia
Strauss, G. P., Waltz, J. A., & Gold, J. M. (2014) · Schizophrenia Bulletin · doi:10.1093/schbul/sbt197 · Cited by 452
Self-citationNegative Symptoms of Schizophrenia Are Associated with Abnormal Effort-Cost Computations
Gold, J. M., Strauss, G. P., Waltz, J. A., Robinson, B. M., Brown, J. K., & Frank, M. J. (2013) · Biological Psychiatry · doi:10.1016/j.biopsych.2012.12.022 · Cited by 421
Self-citation
Metadata
- Authors
- Strauss, G. P., Frank, M. J., Waltz, J. A., Kasanova, Z., Herbener, E. S., & Gold, J. M.
- Venue
- Biological Psychiatry
- Publisher
- Elsevier BV
- Published
- 2011-03
- Volume
- 69
- Issue
- 5
- Pages
- 424-431
- ISSN
- 0006-3223 (ISSN-L)
- OpenAlex
- W2025810387
- Cited by
- 233
- References
- 58
- FWCI
- 8.97
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.