Cited by 233Neural and Behavioral Psychology StudiesNeurotransmitter Receptor Influence on BehaviorAnxiety, Depression, Psychometrics, Treatment, Cognitive Processes

The DOI matches its Crossref record; 361 of the 365 references it cites were found in a registry or an index.

Reinforcement learning in depression: A review of computational research

Chen, Chong, Takahashi, Taiki, Nakagawa, Shin, Inoue, Takeshi, Kusumi, Ichiro·2015·Neuroscience & Biobehavioral Reviews

DOI 10.1016/j.neubiorev.2015.05.005

Open this paper in CoChat — free account

Abstract

Despite being considered primarily a mood disorder, major depressive disorder (MDD) is characterized by cognitive and decision making deficits. Recent research has employed computational models of reinforcement learning (RL) to address these deficits. The computational approach has the advantage in making explicit predictions about learning and behavior, specifying the process parameters of RL, differentiating between model-free and model-based RL, and the computational model-based functional magnetic resonance imaging and electroencephalography. With these merits there has been an emerging field of computational psychiatry and here we review specific studies that focused on MDD. Considerable evidence suggests that MDD is associated with impaired brain signals of reward prediction error and expected value ('wanting'), decreased reward sensitivity ('liking') and/or learning (be it model-free or model-based), etc., although the causality remains unclear. These parameters may serve as valuable intermediate phenotypes of MDD, linking general clinical symptoms to underlying molecular dysfunctions. We believe future computational research at clinical, systems, and cellular/molecular/genetic levels will propel us toward a better understanding of the disease.

Integrity signals

Registry

Crossref · 10.1016/j.neubiorev.2015.05.005

Cites

361 of 361 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Prospect theory: An analysis of decision under risk
Kahneman, D. J., & Tversky, A. (1988) · Cambridge University Press eBooks · doi:10.1017/cbo9780511609220.014 · Cited by 32,720
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Untitled work
(2011) · Cited by 14,317
Feeling and thinking: Preferences need no inferences.
Zajonc, R. B. (1980) · American Psychologist · doi:10.1037/0003-066x.35.2.151 · Cited by 8,094
Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory
Shiffrin, R. M., & Schneider, W. (1977) · Psychological Review · doi:10.1037/0033-295x.84.2.127 · Cited by 7,335
Effects of stress throughout the lifespan on the brain, behaviour and cognition
Lupien, S. J., McEwen, B. S., Gunnar, M. R., & Heim, C. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2639 · Cited by 6,256
The Endophenotype Concept in Psychiatry: Etymology and Strategic Intentions
Gottesman, I. I., & Gould, T. D. (2003) · American Journal of Psychiatry · doi:10.1176/appi.ajp.160.4.636 · Cited by 5,615
Stress and the brain: from adaptation to disease
de Kloet, E. R., Joëls, M., & Holsboer, F. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1683 · Cited by 4,621

Flagged first, then by citations

Show all 365 references

4 of the 365 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

Takahashi, 2007, Salivary alpha-amylase levels and hyperbolic discounting in male humans, Neuroendocrinol. Lett., 28, 17

Sorted by citations

Neuroplasticity in cognitive and psychological mechanisms of depression: an integrative model
Price, R. B., & Duman, R. (2020) · Molecular Psychiatry · doi:10.1038/s41380-019-0615-x · Cited by 739
The orbitofrontal cortex: reward, emotion and depression
Rolls, E. T., Cheng, W., & Feng, J. (2020) · Brain Communications · doi:10.1093/braincomms/fcaa196 · Cited by 474
HPA Axis in the Pathomechanism of depression and schizophrenia: new therapeutic strategies based on its participation
Mikulska, J., Juszczyk, G., Gawrońska-Grzywacz, M., & Herbet, M. (2021) · Brain Sciences · doi:10.3390/brainsci11101298 · Cited by 450
Mental illness and well‐being: an affect regulation perspective
Gross, J. J., Uusberg, H., & Uusberg, A. (2019) · World Psychiatry · doi:10.1002/wps.20618 · Cited by 277
Neuromodulatory Systems and Their Interactions: A Review of Models, Theories, and Experiments
Avery, M. C., & Krichmar, J. L. (2017) · Frontiers in Neural Circuits · doi:10.3389/fncir.2017.00108 · Cited by 250
Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases
Pessiglione, M., Vinckier, F., Bouret, S., Jean Daunizeau, A., & Bouc, R. L. (2017) · Brain · doi:10.1093/brain/awx278 · Cited by 255
Reward-Processing Behavior in Depressed Participants Relative to Healthy Volunteers
Halahakoon, D. C., Kieslich, K., O’Driscoll, C., Nair, A., Lewis, G., & Roiser, J. P. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.2139 · Cited by 222
A Bayesian computational model reveals a failure to adapt interoceptive precision estimates across depression, anxiety, eating, and substance use disorders
Smith, R. S., Kuplicki, R., Feinstein, J. S., Forthman, K. L., Stewart, J. L., Paulus, M. P., Tulsa 1000 investigators, & Khalsa, S. S. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008484 · Cited by 207
The exercise-glucocorticoid paradox: How exercise is beneficial to cognition, mood, and the brain while increasing glucocorticoid levels
Chen, C., Nakagawa, S., An, Y., Ito, K., Kitaichi, Y., & Kusumi, I. (2016) · Frontiers in Neuroendocrinology · doi:10.1016/j.yfrne.2016.12.001 · Cited by 209
Physical activity for cognitive health promotion: An overview of the underlying neurobiological mechanisms
Chen, C., & Nakagawa, S. (2023) · Ageing Research Reviews · doi:10.1016/j.arr.2023.101868 · Cited by 178
Deep Learning and Neural Networks: Decision-Making Implications
Taherdoost, H. (2023) · Symmetry · doi:10.3390/sym15091723 · Cited by 178
Dopaminergic Enhancement of Striatal Response to Reward in Major Depression
Admon, R., Kaiser, R. H., Dillon, D. G., Beltzer, M., Goer, F., Olson, D. P., Vitaliano, G. D., & Pizzagalli, D. A. (2016) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2016.16010111 · Cited by 136
The neural underpinnings of cognitive flexibility and their disruption in psychotic illness
Waltz, J. A. (2017) · Neuroscience · doi:10.1016/j.neuroscience.2016.06.005 · Cited by 122
“Results in MDD have been mixed, as well (Chen et al., 2015), but it is interesting to note Waltz Page 17”
Background
Neural mechanisms of reinforcement learning in unmedicated patients with major depressive disorder
Rothkirch, M., Tonn, J., Köhler, S., & Sterzer, P. (2017) · Brain · doi:10.1093/brain/awx025 · Cited by 124
Regular Moderate- to Vigorous-Intensity Physical Activity Rather Than Walking Is Associated with Enhanced Cognitive Functions and Mental Health in Young Adults
Nakagawa, T., Koan, I., Chen, C., Matsubara, T., Hagiwara, K., Lei, H., Hirotsu, M., Yamagata, H., & Nakagawa, S. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17020614 · Cited by 124
Self-citation
Quantum reinforcement learning during human decision-making
Li, J., Dong, D., Wei, Z., Liu, Y., Pan, Y., Nori, F., & Zhang, X. (2020) · Nature Human Behaviour · doi:10.1038/s41562-019-0804-2 · Cited by 126
Reinforcement Learning Disruptions in Individuals With Depression and Sensitivity to Symptom Change Following Cognitive Behavioral Therapy
Brown, V. M., Zhu, L., Solway, A., Wang, J. M., McCurry, K. L., King-Casas, B., & Chiu, P. H. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.1844 · Cited by 112
Computational neuroscience approach to biomarkers and treatments for mental disorders
Yahata, N., Kasai, K., & Kawato, M. (2016) · Psychiatry and Clinical Neurosciences · doi:10.1111/pcn.12502 · Cited by 135
Probabilistic Reinforcement Learning in Patients With Schizophrenia: Relationships to Anhedonia and Avolition
Dowd, E. C., Frank, M. J., Collins, A. G. E., Gold, J. M., & Barch, D. M. (2016) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2016.05.005 · Cited by 101
Examining the effectiveness of gamification in mental health apps for depression: Systematic review and meta-analysis
Six, S. G., Byrne, K. A., Tibbett, T. P., & Pericot-Valverde, I. (2021) · JMIR Mental Health · doi:10.2196/32199 · Cited by 95
“People with depression frequently experience anhedonia, which may result in blunted sensitivity to reward [11-15].”
Background
Physical activity and mental health; it is more than just a prescription
Rebar, A. L., & Taylor, A. H. (2017) · Mental health and physical activity · doi:10.1016/j.mhpa.2017.10.004 · Cited by 102
Effects of transcranial direct current stimulation for treating depression: A modeling study
Csifcsák, G., Boayue, N. M., Puonti, O., Thielscher, A., & Mittner, M. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.02.077 · Cited by 82
Computational Neuroscience Approach to Psychiatry: A Review on Theory-driven Approaches
Khaleghi, A., Mohammadi, M. R., Shahi, K., & Nasrabadi, A. M. (2022) · Clinical Psychopharmacology and Neuroscience · doi:10.9758/cpn.2022.20.1.26 · Cited by 82
Amygdala-cortical collaboration in reward learning and decision making
Wassum, K. M. (2022) · eLife · doi:10.7554/elife.80926 · Cited by 88
The role of medial prefrontal corticosterone and dopamine in the antidepressant-like effect of exercise
Chen, C., Nakagawa, S., Kitaichi, Y., An, Y., Omiya, Y., Song, N., Koga, M., Kato, A., Inoue, T., & Kusumi, I. (2016) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2016.03.008 · Cited by 79
Anhedonia is associated with blunted reward sensitivity in first-degree relatives of patients with major depression
Liu, W., Roiser, J. P., Wang, L.-Z., Zhu, Y.-H., Huang, J., Neumann, D. L., Shum, D., Cheung, E. F. C., & Chan, R. C. K. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.10.050 · Cited by 84
Risk preference dynamics around life events
Kettlewell, N. (2019) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2019.04.018 · Cited by 82
Association Between Interleukin-6 and Striatal Prediction-Error Signals Following Acute Stress in Healthy Female Participants
Treadway, M. T., Admon, R., Arulpragasam, A. R., Mehta, M., Douglas, S., Vitaliano, G. D., Olson, D. P., Cooper, J. A., & Pizzagalli, D. A. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.02.1183 · Cited by 71
Reward and punishment reversal-learning in major depressive disorder.
Mukherjee, D., Filipowicz, A. L. S., Vo, K. D., Satterthwaite, T. D., & Kable, J. W. (2020) · Journal of Abnormal Psychology · doi:10.1037/abn0000641 · Cited by 76
Dopamine projections to the basolateral amygdala drive the encoding of identity-specific reward memories
Sias, A. C., Jafar, Y., Goodpaster, C. M., Ramírez-Armenta, K., Wrenn, T. M., Griffin, N. K., Patel, K., Lamparelli, A. C., Sharpe, M. J., & Wassum, K. M. (2024) · Nature Neuroscience · doi:10.1038/s41593-024-01586-7 · Cited by 86
Reward systems and cognitions in Major Depressive Disorder
Cléry-Melin, M.-L., Jollant, F., & Gorwood, P. A. P. M. (2018) · CNS Spectrums · doi:10.1017/s1092852918001335 · Cited by 73
Recent advances in the study of the neurobiological mechanisms behind the effects of physical activity on mood, resilience and emotional disorders
Chen, C., & Nakagawa, S. (2023) · Advances in Clinical and Experimental Medicine · doi:10.17219/acem/171565 · Cited by 77
Gut inference: A computational modelling approach
Smith, R. S., Mayeli, A., Taylor, S., Zoubi, O. A., Naegele, J., & Khalsa, S. S. (2021) · Biological Psychology · doi:10.1016/j.biopsycho.2021.108152 · Cited by 71
Understanding psychiatric disorder by capturing ecologically relevant features of learning and decision-making
Scholl, J., & Klein-Flügge, M. C. (2017) · Behavioural Brain Research · doi:10.1016/j.bbr.2017.09.050 · Cited by 69
Recent advances in the study of the comorbidity of depressive and anxiety disorders
Chen, C. (2022) · Advances in Clinical and Experimental Medicine · doi:10.17219/acem/147441 · Cited by 74
Biased belief updating in depression
Kube, T. (2023) · Clinical Psychology Review · doi:10.1016/j.cpr.2023.102298 · Cited by 61
Depression, Risk Preferences, and Risk-Taking Behavior
Cobb‐Clark, D. A., Dahmann, S. C., & Kettlewell, N. (2020) · The Journal of Human Resources · doi:10.3368/jhr.58.1.0419-10183r1 · Cited by 53
Major Depression Impairs the Use of Reward Values for Decision-Making
Rupprechter, S., Stankevicius, A., Huys, Q. J. M., Steele, J. D., & Seriès, P. (2018) · Scientific Reports · doi:10.1038/s41598-018-31730-w · Cited by 51
Psychological interventions for depression in Chinese university students: A systematic review and meta-analysis
Fu, Z., Zhou, S., Burger, H., Bockting, C. L. H., & Williams, A. D. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.11.058 · Cited by 44
Depressive symptoms are associated with blunted reward learning in social contexts
Safra, L., Chevallier, C., & Palminteri, S. (2019) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007224 · Cited by 40
Associations Between Daily Affective Instability and Connectomics in Functional Subnetworks in Remitted Patients with Recurrent Major Depressive Disorder
Servaas, M. N., Riese, H., Renken, R. J., Wichers, M., Bastiaansen, J. A., Figueroa, C. A., Geugies, H., Mocking, R. J. T., Geerligs, L., Marsman, J.-B. C., et al. (2017) · Neuropsychopharmacology · doi:10.1038/npp.2017.65 · Cited by 39
Neural activity and fundamental learning, motivated by monetary loss and reward, are intact in mild to moderate major depressive disorder
Moutoussis, M., Rutledge, R. B., Prabhu, G., Hrynkiewicz, L., Lam, J., Ousdal, O. T., Guitart‐Masip, M., Fonagy, P., & Dolan, R. J. (2018) · PLoS ONE · doi:10.1371/journal.pone.0201451 · Cited by 42
Not only serotonergic system, but also dopaminergic system involved in albiflorin against chronic unpredictable mild stress-induced depression-like behavior in rats
Song, J., Hou, X., Hu, X., Lu, C., Liu, C., Wang, J., Liu, W., Teng, L., & Wang, D. (2015) · Chemico-Biological Interactions · doi:10.1016/j.cbi.2015.10.001 · Cited by 40
Multiple Facets of Value-Based Decision Making in Major Depressive Disorder
Mukherjee, D., Lee, S., Kazinka, R., Satterthwaite, T. D., & Kable, J. W. (2020) · Scientific Reports · doi:10.1038/s41598-020-60230-z · Cited by 46
Like mother like daughter: putamen activation as a mechanism underlying intergenerational risk for depression
Colich, N. L., Ho, T. C., Ellwood-Lowe, M. E., Foland‐Ross, L. C., Sacchet, M. D., LeMoult, J., & Gotlib, I. H. (2017) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsx073 · Cited by 38
Latent variable analysis of positive and negative valence processing focused on symptom and behavioral units of analysis in mood and anxiety disorders
Paulus, M. P., Stein, M. B., Craske, M. G., Bookheimer, S. Y., Taylor, C. T., Simmons, A. N., Sidhu, N. K., Young, K. S., & Fan, B. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.12.046 · Cited by 39
Who are the placebo responders? A cross-sectional cohort study for psychological determinants
Wang, Y., Chan, E., Dorsey, S. G., Campbell, C. M., & Colloca, L. (2021) · Pain · doi:10.1097/j.pain.0000000000002478 · Cited by 42
Elevated Effort Cost Identified by Computational Modeling as a Distinctive Feature Explaining Multiple Behaviors in Patients With Depression
Vinckier, F., Jaffré, C., Gauthier, C., Smajda, S., Abdel-Ahad, P., Bouc, R. L., Jean Daunizeau, A., Fefeu, M., Borderies, N., Plaze, M., et al. (2022) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2022.07.011 · Cited by 44
Chronic psychosocial stressors are associated with alterations in salience processing and corticostriatal connectivity
McCutcheon, R. A., Bloomfield, M. A. P., Dahoun, T., Mehta, M., & Howes, O. D. (2019) · Schizophrenia Research · doi:10.1016/j.schres.2018.12.011 · Cited by 35
NPAS4 in the medial prefrontal cortex mediates chronic social defeat stress-induced anhedonia-like behavior and reductions in excitatory synapses
Hughes, B. W., Siemsen, B. M., Tsvetkov, E., Berto, S., Kumar, J., Cornbrooks, R. G., Akiki, R. M., Cho, J. Y., Carter, J. S., Snyder, K. K., et al. (2023) · eLife · doi:10.7554/elife.75631 · Cited by 35
Impaired social learning predicts reduced real-life motivation in individuals with depression: A computational fMRI study
Frey, A.-L., & McCabe, C. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.11.049 · Cited by 38
Enhancing the psychometric properties of the Iowa gambling task using full generative modeling
Sullivan-Toole, H., Haines, N., Dale, K., & Olino, T. M. (2022) · Computational Psychiatry · doi:10.5334/cpsy.89 · Cited by 35
Formalising social representation to explain psychiatric symptoms
Barnby, J. M., Dayan, P., & Bell, V. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.12.004 · Cited by 35
The computational psychiatry of antisocial behaviour and psychopathy
Pauli, R., & Lockwood, P. L. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104995 · Cited by 34
Computational models of subjective feelings in psychiatryPreprint
Kao, C.-H., Feng, G., Hur, J. K., Jarvis, H., & Rutledge, R. B. (2022) · PsyArXiv (OSF Preprints) · doi:10.1016/j.neubiorev.2022.105008 · Cited by 45
Revisiting the role of computational neuroimaging in the era of integrative neuroscience
Loosen, A. M., Kato, A., & Gu, X. (2024) · Neuropsychopharmacology · doi:10.1038/s41386-024-01946-8 · Cited by 38
The Role of MAPK and Dopaminergic Synapse Signaling Pathways in Antidepressant Effect of Electroacupuncture Pretreatment in Chronic Restraint Stress Rats
Yang, X., Guo, Z., Lü, J., Zhao, B., Fei, Y., Li, J., Jiang, H., Sun, L., Wang, Y., Sun, Y., et al. (2017) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2017/2357653 · Cited by 27
Angiotensin blockade enhances motivational reward learning via enhancing striatal prediction error signaling and frontostriatal communication
Xu, T., Zhou, X., Kanen, J. W., Wang, L., Li, J., Chen, Z., Zhang, R., Jiao, G., Zhou, F., Zhao, W., et al. (2023) · Molecular Psychiatry · doi:10.1038/s41380-023-02001-6 · Cited by 20
Invasive Computational Psychiatry
Sáez, I., & Gu, X. (2022) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.09.032 · Cited by 27
Single Photon in Hierarchical Architecture for Physical Decision Making: Photon Intelligence
Naruse, M., Berthel, M., Drezet, A., Huant, S., Hori, H., & Kim, S.-J. (2016) · ACS Photonics · doi:10.1021/acsphotonics.6b00742 · Cited by 25
Model-based estimation of subjective values using choice tasks with probabilistic feedback
Katahira, K., Yuki, S., & Okanoya, K. (2017) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2017.05.005 · Cited by 24
RETRACTED: Effects of L-DOPA Monotherapy on Psychomotor Speed and [11C]Raclopride Binding in High-Risk Older Adults With Depression
Rutherford, B. R., Slifstein, M., Chen, C., Abi‐Dargham, A., Brown, P. J., Wall, M. W., Vanegas‐Arroyave, N., Stern, Y., Bailey, V. S., Valente, E., et al. (2019) · Biological Psychiatry · doi:10.1016/j.biopsych.2019.04.007 · Cited by 24
Retracted
Beyond Description and Deficits: How Computational Psychiatry Can Enhance an Understanding of Decision-Making in Anorexia Nervosa
Haynos, A. F., Widge, A. S., Anderson, L. M., & Redish, A. D. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01320-9 · Cited by 25
Intermittent Absence of Control during Reinforcement Learning Interferes with Pavlovian Bias in Action Selection
Csifcsák, G., Melsæter, E., & Mittner, M. (2019) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01515 · Cited by 23
Investigation of reward learning and feedback sensitivity in non-clinical participants with a history of early life stress
Wilkinson, M. P., Slaney, C., Mellor, J. R., & Robinson, E. S. J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0260444 · Cited by 30
Artificial neural networks for model identification and parameter estimation in computational cognitive models
Rmus, M., Pan, T.-F., Xia, L., & Collins, A. G. E. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012119 · Cited by 26
Depression, Risk Preferences and Risk-Taking BehaviorPreprint
Cobb‐Clark, D. A., Dahmann, S. C., & Kettlewell, N. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3390275 · Cited by 21
Network changes in insula and amygdala connectivity accompany implicit suicidal associations
Ballard, E. D., Gilbert, J. R., Fields, J. S., Nugent, A. C., & Zarate, C. A. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.577628 · Cited by 20
Contextual influence of reinforcement learning performance of depression: evidence for a negativity bias?
Vandendriessche, H., Demmou, A., Bavard, S., Yadak, J., Lemogne, C., Mauras, T., & Palminteri, S. (2022) · Psychological Medicine · doi:10.1017/s0033291722001593 · Cited by 23
Computational models as statistical tools
Durstewitz, D., Koppe, G., & Toutounji, H. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.07.004 · Cited by 19
Computational approaches to treatment response prediction in major depression using brain activity and behavioral data: A systematic review
Karvelis, P., Charlton, C. E., Allohverdi, S. G., Bedford, P., Hauke, D. J., & Diaconescu, A. O. (2022) · Network Neuroscience · doi:10.1162/netn_a_00233 · Cited by 20
Symptom Patterns of the Occurrence of Depression and Anxiety in a Japanese General Adult Population Sample: A Latent Class Analysis
Lei, H., Chen, C., Hagiwara, K., Kusumi, I., Tanabe, H., Inoue, T., & Nakagawa, S. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.808918 · Cited by 19
Blunted diurnal interleukin-6 rhythm is associated with amygdala emotional hyporeactivity and depression: a modulating role of gene-stressor interactions
Hakamata, Y., Hori, H., Mizukami, S., Izawa, S., Yoshida, F., Moriguchi, Y., Hanakawa, T., Inoue, Y., & Tagaya, H. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1196235 · Cited by 20
Transdiagnostic inflexible learning dynamics explain deficits in depression and schizophrenia
Kirschner, H., Nassar, M. R., Fischer, A. G., Frodl, T., Meyer-Lotz, G., Froböse, S., Seidenbecher, S., Klein, T. A., & Ullsperger, M. (2023) · Brain · doi:10.1093/brain/awad362 · Cited by 20
Psychometric Properties and Validation of the EMOTICOM Test Battery in a Healthy Danish Population
Dam, V., Thystrup, C. K., Jensen, P., Bland, A., Mortensen, E. L., Elliott, R., Sahakian, B., Knudsen, G., Frokjaer, V., & Stenbæk, D. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02660 · Cited by 14
Revisiting the importance of model fitting for model-based fMRI: It does matter in computational psychiatry
Katahira, K., & Toyama, A. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008738 · Cited by 18
The neuroanatomy of social trust predicts depression vulnerability
Fermin, A. S. R., Kiyonari, T., Matsumoto, Y., Takagishi, H., Li, Y., Kanai, R., Sakagami, M., Akaishi, R., Ichikawa, N., Takamura, M., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-20443-w · Cited by 21
Computational reinforcement learning, reward (and punishment), and dopamine in psychiatric disorders
Liebenow, B., Jones, R., DiMarco, E. K., Trattner, J. D., Humphries, J. B., Sands, L. P., Spry, K. P., Johnson, C. K., Farkas, E. B., Jiang, A., et al. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.886297 · Cited by 23
Representation of Anticipated Rewards and Punishments in the Human Brain
Ben‐Zion, Z., & Levy, I. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-022324-042614 · Cited by 21
Can Understanding Reward Help Illuminate Anhedonia?
Kaya, S., & McCabe, C. (2019) · Current Behavioral Neuroscience Reports · doi:10.1007/s40473-019-00186-1 · Cited by 16
Pessimistically biased perception in panic disorder during risk learning
Kim, M., Kim, S., Lee, K., & Jeong, B. (2020) · Depression and Anxiety · doi:10.1002/da.23007 · Cited by 18
Decision‐making biases in suicide attempters with major depressive disorder: A computational modeling study using the balloon analog risk task (BART)
Liu, Q., Zhong, R., Ji, X., Law, S., Xiao, F., Wei, Y., Fang, S., Kong, X., Zhang, X., Yao, S., et al. (2022) · Depression and Anxiety · doi:10.1002/da.23291 · Cited by 19
Signed and unsigned effects of prediction error on memory: Is it a matter of choice?
Pupillo, F., & Bruckner, R. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105371 · Cited by 16
Sex difference in the weighting of expected uncertainty under chronic stress
Lei, H., Mochizuki, Y., Chen, C., Hagiwara, K., Hirotsu, M., Matsubara, T., & Nakagawa, S. (2021) · Scientific Reports · doi:10.1038/s41598-021-88155-1 · Cited by 15
Effects of subclinical depression on prefrontal–striatal model-based and model-free learning
Heo, S., Sung, Y., & Lee, S. W. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009003 · Cited by 16
Maladaptive Avoidance Learning in the Orbitofrontal Cortex in Adolescents With Major Depression
Willinger, D., Karipidis, I. I., Neuer, S., Emery, S., Rauch, C., Häberling, I. S., Berger, G. E., Walitza, S., & Brem, S. (2021) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2021.06.005 · Cited by 17
Spatiotemporal Precision of Neuroimaging in Psychiatry
McFadyen, J., & Dolan, R. J. (2022) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.08.016 · Cited by 16
Sex-dependent effects of early life stress on reinforcement learning and limbic cortico-striatal functional connectivity
Zühlsdorff, K., López‐Cruz, L., Dutcher, E. G., Jones, J. A., Pama, C., Sawiak, S. J., Khan, S. A., Milton, A. L., Robbins, T. W., Bullmore, E. T., et al. (2022) · Neurobiology of Stress · doi:10.1016/j.ynstr.2022.100507 · Cited by 15
The Algorithmic Agent Perspective and Computational Neuropsychiatry: From Etiology to Advanced Therapy in Major Depressive Disorder
Ruffini, G., Castaldo, F., Lopez-Sola, E., Sanchez-Todo, R., & Vohryzek, J. (2024) · Entropy · doi:10.3390/e26110953 · Cited by 20
The pursuit of happiness: A reinforcement learning perspective on habituation and comparisons
Dubey, R., Griffiths, T. L., & Dayan, P. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010316 · Cited by 18
Nonlinear Probability Weighting in Depression and Anxiety: Insights From Healthy Young Adults
Hagiwara, K., Mochizuki, Y., Chen, C., Lei, H., Hirotsu, M., Matsubara, T., & Nakagawa, S. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.810867 · Cited by 15
Anhedonia modulates benzodiazepine and opioid demand among persons in treatment for opioid use disorder
Greenwald, M. K., Moses, T. E. H., Lundahl, L. H., & Roehrs, T. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1103739 · Cited by 13
The association of positive emotions with absenteeism and presenteeism in Japanese workers
Chen, C., Okubo, R., Hagiwara, K., Mizumoto, T., Nakagawa, S., & Tabuchi, T. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.10.091 · Cited by 16
The Reason Why rTMS and tDCS Are Efficient in Treatments of Depression
Čukić, M. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02923 · Cited by 16
Reward Processing under Chronic Pain from the Perspective of “Liking” and “Wanting”: A Narrative Review
Liu, X., Wang, N., Gu, L., Guo, J., Wang, J., & Luo, F. (2019) · Pain Research and Management · doi:10.1155/2019/6760121 · Cited by 14
Psychiatric symptoms influence reward‐seeking and loss‐avoidance decision‐making through common and distinct computational processes
Suzuki, S., Yamashita, Y., & Katahira, K. (2021) · Psychiatry and Clinical Neurosciences · doi:10.1111/pcn.13279 · Cited by 16
On the Complexity of Brain Disorders: A Symptom-Based Approach
Moustafa, A. A., Phillips, J. R., Kéri, S., Misiak, B., & Frydecka, D. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00016 · Cited by 15
Applying Reinforcement Learning to Rodent Stress Research
Liao, C., & Kwan, A. C. (2021) · Chronic Stress · doi:10.1177/2470547020984732 · Cited by 13
The habenula in mood disorders: A systematic review of human studies
Fortin, J., Lafleur, M., Parisien, C., & Hétu, S. (2025) · Molecular Psychiatry · doi:10.1038/s41380-025-03105-x · Cited by 13
Adaptive Design Optimization as a Promising Tool for Reliable and Efficient Computational Fingerprinting
Kwon, M., Lee, S. H., & Ahn, W. (2022) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2022.12.003 · Cited by 14

Authors: Chen, C., Takahashi, T., Nakagawa, S., Inoue, T., & Kusumi, I.
Affiliations: Hokkaido University (JP)
Corresponding author: Chen, Chong
ORCID: Chen, Chong
Nakagawa, Shin
Inoue, Takeshi
Kusumi, Ichiro
Venue: Neuroscience & Biobehavioral Reviews
Publisher: Elsevier BV
Published: 2015-08
Volume: 55
Pages: 247-267
ISSN: 0149-7634 (ISSN-L)
DOI: 10.1016/j.neubiorev.2015.05.005
OpenAlex: W291512641
Cited by: 233
References: 365
Citations by year: 2026: 17 · 2025: 26 · 2024: 34 · 2023: 31 · 2022: 29 · 2021: 19 · 2020: 17 · 2019: 20 · 2018: 10 · 2017: 14 · 2016: 11 · 2015: 4
FWCI: 1.29
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: Neuroscience & Biobehavioral Reviews (publishedVersion)
PubMed (publishedVersion)
Funders: Ministry of Education, Culture, Sports, Science and Technology
Topics: Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
Keywords: major depressive disorder, reinforcement learning, computational psychiatry, model-free reinforcement learning, model-based reinforcement learning, reward prediction error, reward sensitivity, endophenotypes, decision-making deficits, cognitive deficits in depression
MeSH terms: Animals, Brain, Computer Simulation, Major Depressive Disorder, Humans, Motivation, Reinforcement, Psychology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reinforcement learning in depression: A review of computational research

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks