Evaluating the evidence for biotypes of depression: Methodological replication and extension of
Open accessAbstract
BACKGROUND: Psychiatric disorders are highly heterogeneous, defined based on symptoms with little connection to potential underlying biological mechanisms. A possible approach to dissect biological heterogeneity is to look for biologically meaningful subtypes. A recent study Drysdale et al. (2017) showed promising results along this line by simultaneously using resting state fMRI and clinical data and identified four distinct subtypes of depression with different clinical profiles and abnormal resting state fMRI connectivity. These subtypes were predictive of treatment response to transcranial magnetic stimulation therapy. OBJECTIVE: Here, we attempted to replicate the procedure followed in the Drysdale et al. study and their findings in a different clinical population and a more heterogeneous sample of 187 participants with depression and anxiety. We aimed to answer the following questions: 1) Using the same procedure, can we find a statistically significant and reliable relationship between brain connectivity and clinical symptoms? 2) Is the observed relationship similar to the one found in the original study? 3) Can we identify distinct and reliable subtypes? 4) Do they have similar clinical profiles as the subtypes identified in the original study? METHODS: We followed the original procedure as closely as possible, including a canonical correlation analysis to find a low dimensional representation of clinically relevant resting state fMRI features, followed by hierarchical clustering to identify subtypes. We extended the original procedure using additional statistical tests, to test the statistical significance of the relationship between resting state fMRI and clinical data, and the existence of distinct subtypes. Furthermore, we examined the stability of the whole procedure using resampling. RESULTS AND CONCLUSION: As in the original study, we found extremely high canonical correlations between functional connectivity and clinical symptoms, and an optimal three-cluster solution. However, neither canonical correlations nor clusters were statistically significant. On the basis of our extensive evaluations of the analysis methodology used and within the limits of comparison of our sample relative to the sample used in Drysdale et al., we argue that the evidence for the existence of the distinct resting state connectivity-based subtypes of depression should be interpreted with caution.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
To validate subtypes, one must use cross-validation, randomized holdouts, and/or independent testing to avoid overfitting.
“To validate subtypes, one must be very careful to use cross-validation, randomized holdouts, and/or independent testing to avoid overfitting (54).”
From Methods and challenges for assessing heterogeneity (2020) · cites this paper as [54]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 79,888 · see citing works
Hamilton, M. (1960) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.23.1.56 · Cited by 32,817 · cited 3× · see citing works
“This inventory was developed as an improvement over HAMD, aiming to improve the coverage of common MDD symptoms (Rush et al., 1996).”
MethodInfluentialAn inventory for measuring clinical anxiety: Psychometric properties.
Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.56.6.893 · Cited by 14,407 · see citing works
Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002) · NeuroImage · doi:10.1006/nimg.2002.1132 · Cited by 11,888 · see citing works
Zhang, Y., Brady, M., & Smith, S. M. (2001) · IEEE Transactions on Medical Imaging · doi:10.1109/42.906424 · Cited by 7,628 · see citing works
RELATIONS BETWEEN TWO SETS OF VARIATES
Hotelling, H. (1936) · Biometrika · doi:10.1093/biomet/28.3-4.321 · Cited by 5,429 · see citing works
Functional Network Organization of the Human Brain
Power, J. D., Cohen, A. L., Nelson, S. M., Wig, G. S., Barnes, K. A., Church, J. A., Vogel, A. C., Laumann, T. O., Miezin, F. M., Schlaggar, B. L., et al. (2011) · Neuron · doi:10.1016/j.neuron.2011.09.006 · Cited by 4,124 · see citing works
Robins, L. N. (1988) · Archives of General Psychiatry · doi:10.1001/archpsyc.1988.01800360017003 · Cited by 2,909 · see citing works
Circular analysis in systems neuroscience: the dangers of double dipping
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F., & Baker, C. I. (2009) · Nature Neuroscience · doi:10.1038/nn.2303 · Cited by 2,696 · see citing works
The Inventory of Depressive Symptomatology (IDS): Psychometric properties
Rush, A. J., Gullion, C. M., Basco, M. R., Jarrett, R. B., & Trivedi, M. H. (1996) · Psychological Medicine · doi:10.1017/s0033291700035558 · Cited by 2,177 · see citing works
Resting-state connectivity biomarkers define neurophysiological subtypes of depression
Drysdale, A. T., Grosenick, L., Downar, J., Dunlop, K., Mansouri, F., Meng, Y., Fetcho, R. N., Zebley, B., Oathes, D. J., Etkin, A., et al. (2017) · Nature Medicine · doi:10.1038/nm.4246 · Cited by 2,170 · see citing works
ICA-AROMA: A robust ICA-based strategy for removing motion artifacts from fMRI data
Pruim, R. H. R., Mennes, M., van Rooij, D., Llera, A., Buitelaar, J. K., & Beckmann, C. F. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.02.064 · Cited by 1,684 · see citing works
Self-citationThe NIMH Research Domain Criteria (RDoC) Project: Precision Medicine for Psychiatry
Insel, T. R. (2014) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.14020138 · Cited by 1,472 · see citing works
Miller, R. G. (1974) · Biometrika · doi:10.1093/biomet/61.1.1 · Cited by 1,391 · see citing works
Kapur, S., Phillips, A. G., & Insel, T. R. (2012) · Molecular Psychiatry · doi:10.1038/mp.2012.105 · Cited by 980 · see citing works
The Netherlands Study of Depression and Anxiety (NESDA): rationale, objectives and methods
Penninx, B. W. J. H., Beekman, A. T. F., Smit, J. H., Zitman, F. G., Nolen, W. A., Spinhoven, P., Cuijpers, P. J., de Jong, P. J., van Marwijk, H., Assendelft, W. J., et al. (2008) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.256 · Cited by 918 · see citing works
Self-citationThe inventory for depressive symptomatology (IDS): Preliminary findings
Rush, A. J., Giles, D. E., Schlesser, M. A., Fulton, C. L., Weissenburger, J. E., & Burns, C. (1986) · Psychiatry Research · doi:10.1016/0165-1781(86)90060-0 · Cited by 908 · see citing works
Smith, S. M., Nichols, T. E., Vidaurre, D., Winkler, A. M., Behrens, T. E. J., Glasser, M. F., Ugurbil, K., Barch, D. M., Van Essen, D. C., & Miller, K. L. (2015) · Nature Neuroscience · doi:10.1038/nn.4125 · Cited by 859 · see citing works
Parkes, L., Fulcher, B., Yücel, M., & Fornito, A. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.12.073 · Cited by 783 · see citing works
Sherry, A. R., & Henson, R. K. (2005) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8401_09 · Cited by 768 · see citing works
Kabacoff, R. I., Segal, D. L., Hersen, M., & Van Hasselt, V. B. (1997) · Journal of Anxiety Disorders · doi:10.1016/s0887-6185(96)00033-3 · Cited by 500 · see citing works
Chekroud, A. M., Gueorguieva, R., Krumholz, H. M., Trivedi, M. H., Krystal, J. H., & McCarthy, G. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0025 · Cited by 276 · see citing works
Statistical Significance of Clustering for High-Dimension, Low–Sample Size Data
Liu, Y., Hayes, D. N., Nobel, A. B., & Marron, J. S. (2008) · Journal of the American Statistical Association · doi:10.1198/016214508000000454 · Cited by 269 · see citing works
Marquand, A. F., Wolfers, T., Mennes, M., Buitelaar, J. K., & Beckmann, C. F. (2016) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2016.04.002 · Cited by 254 · see citing works
Self-citationReplication in behavioral research.
Rosenthal, R. (1990) · Journal of social behavior and personality · Cited by 206 · see citing works
Cusin, C., Yang, H., Yeung, A., & D, M. F. M. (2009) · Humana Press eBooks · doi:10.1007/978-1-59745-387-5_2 · Cited by 166 · see citing works
Functional corticostriatal connection topographies predict goal-directed behaviour in humans
Marquand, A. F., Haak, K. V., & Beckmann, C. F. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0146 · Cited by 153 · see citing works
Self-citationFloch, É. L., Guillemot, V., Frouin, V., Pinel, P., Lalanne, C., Trinchera, L., Tenenhaus, A., Moreno, A., Zilbovicius, M., Bourgeron, T., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.06.061 · Cited by 104 · see citing works
Toward a generalizable model of symptoms in major depressive disorder
Gullion, C. M., & Rush, A. J. (1998) · Biological Psychiatry · doi:10.1016/s0006-3223(98)00235-2 · Cited by 91 · see citing works
Chang, C., Dong, D., Jiang, Y., Ming, Q., Zhong, X. B., Sun, X., Xiong, G., Gao, Y., & Yao, S. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00245 · Cited by 46 · see citing works
Functional MRI correlates of visuospatial planning in out-patient depression and anxiety
van Tol, M., van der Wee, N. J. A., Demenescu, L. R., Nielen, M. M. A., Aleman, A., Renken, R. J., van Buchem, M. A., Zitman, F. G., & Veltman, D. J. (2011) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2011.01702.x · Cited by 43 · see citing works
Self-citationDong, D., Li, C., Ming, Q., Zhong, X., Zhang, X., Sun, X., Jiang, Y., Gao, Y., Wang, X., & Yao, S. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.03.030 · Cited by 26 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The promise of machine learning in predicting treatment outcomes in psychiatry
Chekroud, A. M., Bondar, J., Delgadillo, J., Doherty, G., Wasil, A., Fokkema, M., Cohen, Z., Belgrave, D., DeRubeis, R., Iniesta, R., et al. (2021) · World Psychiatry · doi:10.1002/wps.20882 · Cited by 531 · see citing works
The future of rodent models in depression research
Gururajan, A., Reif, A., Cryan, J. F., & Slattery, D. A. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0221-6 · Cited by 269 · see citing works
Deep brain stimulation for psychiatric disorders: From focal brain targets to cognitive networks
Sullivan, C. R. P., Olsen, S., & Widge, A. S. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117515 · Cited by 112 · see citing works
Serotonin regulation of behavior via large-scale neuromodulation of serotonin receptor networks
Salvan, P., Fonseca, M., Winkler, A. M., Beauchamp, A., Lerch, J. P., & Johansen-Berg, H. (2023) · Nature Neuroscience · doi:10.1038/s41593-022-01213-3 · Cited by 101 · see citing works
Methods and challenges for assessing heterogeneity
Feczko, E., & Fair, D. A. (2020) · Biological Psychiatry · doi:10.1016/j.biopsych.2020.02.015 · Cited by 96 · see citing works
Identifying schizophrenia subgroups using clustering and supervised learning
Talpalaru, A., Bhagwat, N., Devenyi, G. A., Lepage, M., & Chakravarty, M. M. (2019) · Schizophrenia Research · doi:10.1016/j.schres.2019.05.044 · Cited by 53 · see citing works
Toward Robust Anxiety Biomarkers: A Machine Learning Approach in a Large-Scale Sample
Boeke, E. A., Holmes, A. J., & Phelps, E. A. (2020) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2019.05.018 · Cited by 50 · see citing works
Moment-to-Moment Brain Signal Variability Reliably Predicts Psychiatric Treatment Outcome
Månsson, K. N. T., Waschke, L., Manzouri, A., Furmark, T., Fischer, H., & Garrett, D. D. (2022) · Biological Psychiatry · doi:10.1016/j.biopsych.2021.09.026 · Cited by 49 · see citing works
Nugent, A. C., Ballard, E. D., Gilbert, J. R., Tewarie, P. K., Brookes, M. J., & Zarate, C. A. (2020) · NeuroImage: Clinical · doi:10.1016/j.nicl.2020.102378 · Cited by 34 · see citing works
- Authors
- Dinga, R., Schmaal, L., Penninx, B. W. J. H., van Tol, M. J., Veltman, D. J., van Velzen, L., Mennes, M., van der Wee, N. J. A., & Marquand, A. F.
- Affiliations
- Amsterdam University Medical Centers (NL)
- The University of Melbourne (AU)
- Radboud University Nijmegen (NL)
- Corresponding author
- Dinga, Richard
- Venue
- NeuroImage Clinical
- Publisher
- Elsevier BV
- Published
- 2019
- Volume
- 22
- Pages
- 101796
- ISSN
- 2213-1582 (ISSN-L)
- PMC
- PMC6543446
- OpenAlex
- W2923495490
- Cited by
- 313
- References
- 32
- Citations by year
- 2026: 19 · 2025: 29 · 2024: 50 · 2023: 38 · 2022: 37 · 2021: 72 · 2020: 48 · 2019: 18 · 2018: 2
- FWCI
- 25.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- NeuroImage Clinical (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, other-oa)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion, other-oa)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (publishedVersion, cc-by)
- Radboud Repository (Radboud University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, cc-by)
- Funders
- Rivierduinen
- Nederlands Instituut voor Onderzoek van de Gezondheidszorg
- ZonMw (10-000-1002)
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (016.156.077, 016.156.415)
- Universitair Medisch Centrum Groningen
- Leids Universitair Medisch Centrum
- GGZ Friesland
- GGZ inGeest
- Institute of Neurosciences, Mental Health and Addiction
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Health Sciences › Medicine › Pharmacology › Treatment of Major Depression
- Keywords
- depression subtypes, resting-state functional connectivity, canonical correlation analysis, hierarchical clustering, anxiety
- MeSH terms
- Machine Learning, Adult, Anxiety Disorders, Brain, Major Depressive Disorder, Female, Humans, Magnetic Resonance Imaging, Male, Middle Aged, Connectome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
