CoChat
Sign Up
Cited by 96Mental Health Research TopicsAnxiety, Depression, Psychometrics, Treatment, Cognitive ProcessesPersonality Disorders and Psychopathology

Methods and challenges for assessing heterogeneity

Open access
Feczko, Eric, Fair, Damien A.·2020·Biological Psychiatry
DOI 10.1016/j.biopsych.2020.02.015PMC PMC8404882
Open this paper in CoChat — free account

Abstract

The widely acknowledged homogeneity assumption limits progress in refining clinical diagnosis, understanding mechanisms, and developing new treatments for mental health disorders. This homogeneity assumption drives both a comorbidity and a heterogeneity problem, where two different approaches tackle the problems. One, a unifying approach, tackles the comorbidity problem by assuming that a single general psychopathology factor underlies multiple disorders. Another, a multifactorial approach, tackles the heterogeneity problem by assuming that disorders comprise multiple subtypes driven by multiple discrete factors. We show how each of these approaches can make useful contributions to mental health–related research and clinical practice. For example, the unifying approach can develop a rapid assessment tool that may be clinically valuable for triaging cases. The multifactorial approach can reveal subtypes that are differentially responsive to treatments and highlight distinct mechanisms leading to similar phenotypes. Because both approaches tackle different problems, both have different limitations. We describe the statistical frameworks that incorporate and adjudicate between both approaches (e.g., the bifactor model, normative modeling, and the functional random forest). Such frameworks can identify whether sets of disorders are more affected by heterogeneity or comorbidity. Therefore, future studies that incorporate such frameworks can provide further insight into the nature of psychopathology.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper argues that the homogeneity assumption in psychiatric research drives both a comorbidity problem and a heterogeneity problem, which are best addressed by unifying and multifactorial approaches, respectively. It reviews bifactor models, normative modeling, and the functional random forest as frameworks that can incorporate both problems, concluding that no single approach solves both and that large-scale comparisons are needed.

Key findings

  • The homogeneity assumption is described as unfounded; comorbidity and heterogeneity problems emerge from it. Unifying approaches model a general psychopathology (p) factor, while multifactorial approaches model discrete subtypes driven by different factors.
  • Evidence for a p factor includes bifactor model fits in over 30,000 individuals from NESARC and 1037 children from the Dunedin study; a p rapid screening tool showed an area under the curve of 0.76 and a large association with trauma exposure (r = .3); genetic studies of 3650 and 11,551 children found polygenic risk scores more associated with p than specific factors.
  • Subtyping examples include ADHD (106 children), ASD (250 and 220 children), temperament (437 children), executive function (1000 transdiagnostic children: 320 discovery, 692 replication), and treatment-resistant depression (122 cases; over 1100 across 710 and 477 datasets); some subtypes were nested within typical variation and responded differentially to treatment.
  • Hybrid frameworks—bifactor models, normative modeling, and the functional random forest—can assess both comorbidity and heterogeneity, but the paper states that no approach solves both problems; normative modeling studies used 491, 474, and 527 participants, and the FRF uses random forest plus infomap community detection.

Methods

  • Narrative/conceptual review, not a primary empirical study; it synthesizes prior literature and describes statistical frameworks for assessing heterogeneity and comorbidity.
  • Draws on cited cohorts and samples (e.g., NESARC >30,000; Dunedin 1037; Philadelphia Neurodevelopmental Cohort >1000; Human Connectome Project 491; Thematically Organized Psychosis 474; Longitudinal European Autism Project 527) to illustrate approaches.
  • Main analysis is a conceptual comparison of unifying versus multifactorial approaches and of frameworks including bifactor models, normative modeling, and the functional random forest.

Limitations

  • Stated: the unifying approach has a heterogeneity problem, and the multifactorial approach has a comorbidity problem; bifactor models are supervised, cannot discover new factors, and tend to fit better even with misspecified factors; normative modeling assumes Gaussian clinical features; longitudinal FRF struggles when the first or last time point is missing; community detection tweaks can affect identified subtypes; both hybrid frameworks require large datasets.
  • Stated: most subtyping methods cannot discover unknown subtypes tied to a question, validation must avoid overfitting, and appropriate subtypes likely differ by whether the interest is diagnosis, prognosis, treatment response, or mechanism.
  • Evident from the text: as a review, it provides no new empirical validation or direct comparison of the frameworks and explicitly calls for future large-scale datasets and direct comparisons against unified, multifactor, and bifactor models.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.biopsych.2020.02.015

Cites

Statistics

Nothing to recompute.

Claims the paper makes

0 of 98 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    Diagnosis of generalized anxiety or major depression increases the risk of receiving the other diagnosis in the future.

    “For example, diagnosis of generalized anxiety or major depression increases the risk of receiving the other diagnosis in the future (10–12).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [10] Moffitt, 2007, Depression and generalized anxiety disorder: Cumulative and sequential comorbidity in a birth cohort followed prospectively to age 32 years, Arch Gen Psychiatry source

      Not checked in the free run.

    • cites [11] Copeland, 2011, Cumulative prevalence of psychiatric disorders by young adulthood: A prospective cohort analysis from the Great Smoky Mountains Study, J Am Acad Child Adolesc Psychiatry source

      Not checked in the free run.

    • cites [12] Kessler, 2011, Development of lifetime comorbidity in the World Health Organization world mental health surveys, Arch Gen Psychiatry source

      Not checked in the free run.

  • Premisecitation

    The same genetic risk factors can underlie many diagnoses, from autism spectrum disorder to schizophrenia.

    “Furthermore, a common source may drive such comorbidity; for example, the same genetic risk factors can underlie many diagnoses (4,15,16)—from autism spectrum disorder (ASD) to schizo- phrenia.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [4] Wang, 2017, Polygenic risk for five psychiatric disorders and cross-disorder and disorder-specific neural connectivity in two independent populations, Neuroimage Clin source

      Not checked in the free run.

    • cites [15] Guo, 2017, Polygenic risk score and heritability estimates reveals a genetic relationship between ASD and OCD, Eur Neuropsychopharmacol source

      Not checked in the free run.

    • cites [16] Gaugler, 2014, Most genetic risk for autism resides with common variation, Nat Genet source

      Not checked in the free run.

  • Interpretationnone given

    The homogeneity assumption drives both a comorbidity problem and a heterogeneity problem.

    “This homogeneity assumption drives both a comorbidity and a heterogeneity problem, where two different approaches tackle the problems.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The homogeneity assumption limits progress in refining clinical diagnosis, understanding mechanisms, and developing new treatments for mental health disorders.
  • Interpretationreasoning

    Because the unifying and multifactorial approaches tackle different problems, each has different limitations.

    “Because both approaches tackle different problems, both have different limitations.”
    Quote in the paperSubstantial claim
  • Premisenone given

    Statistical frameworks such as the bifactor model, normative modeling, and the functional random forest incorporate and adjudicate between both approaches.

    “We describe the statistical frameworks that incorporate and adjudicate between both approaches (e.g., the bifactor model, normative modeling, and the functional random forest).”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 19
  • May combine two claims. 37

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    The unifying approach can develop a rapid assessment tool that may be clinically valuable for triaging cases.

    “For example, the unifying approach can develop a rapid assessment tool that may be clinically valuable for triaging cases.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Profound comorbidity exists across diagnostic labels thought to separate internalizing and externalizing disorders, with high phenotypic correlations between diagnosis pairs.

    “Profound comorbidity exists across diagnostic labels thought to separate internalizing and externalizing disorders; such studies reveal high phenotypic correlations between diagnosis pairs (13,14).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [13] Lichtenstein, 2009, Common genetic determinants of schizophrenia and bipolar disorder in Swedish families: A population-based study, Lancet source

      Not checked in the free run.

    • cites [14] Sartor, 2010, Common genetic contributions to alcohol and cannabis use and dependence symptomatology, Alcohol Clin Exp Res source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Most psychopathological processes and biological links to most mental health disorders remain unknown.

    “Despite efforts over the past century (1–3), most psycho- pathological processes and biological links to most mental health disorders remain unknown.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [1] Kendler, 2009, An historical framework for psychiatric nosology, Psychol Med source

      Not checked in the free run.

    • cites [2] Nigg, 2006, Temperament and developmental psychopathology, J Child Psychol Psychiatry source

      Not checked in the free run.

    • cites [3] Mason, 2018, “A more perfect arrangement of plants”: The botanical model in psychiatric nosology, 1676 to the present day, Hist Psychiatry source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning

    From the homogeneity assumption, the comorbidity and heterogeneity problems emerge and limit psychiatric research.

    “From the homo- geneity assumption, the comorbidity (17) and heterogeneity (18,19) problems emerge and limit psychiatric research (20–26).”
    Quote in the paperSubstantial claim0 of 10 sources checked
    • cites [17] Murray, 2016, The development of the general factor of psychopathology “p factor” through childhood and adolescence, J Abnorm Child Psychol source

      Not checked in the free run.

    • cites [18] Feczko, 2019, The heterogeneity problem: Approaches to identify psychiatric subtypes, Trends Cogn Sci source

      Not checked in the free run.

    • cites [19] Marquand, 2016, Understanding heterogeneity in clinical cohorts using normative models: Beyond case-control studies, Biol Psychiatry source

      Not checked in the free run.

    • cites [20] Anttila, 2018, Analysis of shared heritability in common disorders of the brain, Science

      Not checked in the free run.

    • cites [21] Constantino, 2016, Diagnosis of autism spectrum disorder: Reconciling the syndrome, its diverse origins, and variation in expression, Lancet Neurol source

      Not checked in the free run.

    • cites [22] Regier, 2013, DSM-5 field trials in the United States and Canada, part II: Test-retest reliability of selected categorical diagnoses, Am J Psychiatry source

      Not checked in the free run.

    • cites [23] Fried, 2015, Problematic assumptions have slowed down depression research: Why symptoms, not syndromes are the way forward, Front Psychol source

      Not checked in the free run.

    • cites [24] Uddin, 2017, Progress and roadblocks in the search for brain-based biomarkers of autism and attention-deficit/hyperactivity disorder, Transl Psychiatry source

      Not checked in the free run.

    • cites [25] Arbabshirani, 2017, Single subject prediction of brain disorders in neuroimaging: Promises and pitfalls, Neuroimage source

      Not checked in the free run.

    • cites [26] Friston, 2017, Computational nosology and precision psychiatry, Comput Psychiatry source

      Not checked in the free run.

    • Rests on an assumption: The homogeneity assumption limits progress in refining clinical diagnosis, understanding mechanisms, and developing new treatments for mental health disorders.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    A multifactorial approach tackles the heterogeneity problem by assuming that disorders comprise multiple subtypes driven by multiple discrete factors.

    “Another, a multifactorial approach, tackles the heterogeneity problem by assuming that disorders comprise multiple subtypes driven by multiple discrete factors.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The multifactorial approach can reveal subtypes that are differentially responsive to treatments and highlight distinct mechanisms leading to similar phenotypes.

    “The multifactorial approach can reveal subtypes that are differentially responsive to treatments and highlight distinct mechanisms leading to similar phenotypes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The homogeneity assumption is unfounded; disorders are not homogeneous discrete entities.

    “These issues show that the homogeneity assumption is unfounded; disorders are not homogeneous discrete en- tities.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    A unifying approach tackles the comorbidity problem by assuming that a single general psychopathology factor underlies multiple disorders.

    “One, a unifying approach, tackles the comorbidity problem by assuming that a single general psychopathology factor underlies multiple disorders.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning

    Elevated blood pressure is an excellent clinical sign in many cardiovascular diseases, but by itself it does not provide information regarding mechanisms of any given disorder.

    “However, despite blood pressure’s association with many unique dis- eases, by itself it does not provide information regarding the mechanisms of any given disorder.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [42] Oparil, 2015, New approaches in the treatment of hypertension, Circ Res source

      Not checked in the free run.

    • cites [43] Hamrahian, 2017, Management of hypertension in patients with chronic kidney disease, Curr Hypertens Rep source

      Not checked in the free run.

    • cites [44] Liu, 2017, Association between psychosocial stress and hypertension: A systematic review and meta-analysis, Neurol Res source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Taken together, these studies suggest that the same individual belongs in multiple subtypes, and some subtypes nest within typical variation.

    “Taken together, these studies suggest that the same individual belongs in multiple subtypes, and some subtypes nest within typical variation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premiseown data

    Through whole-brain measures of cortical shape, three ASD subtypes were discovered: one with cortical thickening and mild symptoms, another with cortical thinning and language impairments, and a third with altered shape and language impairments but not thinning.

    “One subtype presented with cortical thickening and mild symptoms, another with cortical thinning and additional language impairments, and a third with altered shape and additional language impairments but not thinning.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    A p rapid screening assessment was calibrated and validated; the area under the ROC curve was 0.76, suggesting some generalizability for the instrument.

    “The area under the curve was 0.76, sug- gesting some generalizability for the instrument.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [41] Moore, 2019, Development of a computerized adaptive screening tool for overall psychopathology (“p”), J Psychiatr Res source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Executive function subtypes, but not clinical subtypes, showed differential brain activation in the right intraparietal gyrus during a dual-attention task.

    “Using brain imaging data to examine biological links to the three groups, the investigators found that executive function subtypes, but not clinical subtypes, observed differ- ential brain activation in the right intraparietal gyrus during a dual-attention task.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [51] Vaidya, 2019, Data-driven identification of subtypes of executive function across typical development, attention deficit hyperactivity disorder, and autism spectrum disorders, J Child Psychol Psychiatry source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Despite evidence for the p factor across multiple disorders, many investigators caution against its existence across every disorder.

    “Despite evidence for the p factor across multiple disorders, many investigators caution against its existence across every disorder (32,33,41).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [32] Caspi, 2014, The p factor, Clin Psychol Sci source

      Not checked in the free run.

    • cites [33] Lahey, 2012, Is there a general factor of prevalent psychopathology during adulthood?, J Abnorm Psychol source

      Not checked in the free run.

    • cites [41] Moore, 2019, Development of a computerized adaptive screening tool for overall psychopathology (“p”), J Psychiatr Res source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    To validate subtypes, one must use cross-validation, randomized holdouts, and/or independent testing to avoid overfitting.

    “To validate subtypes, one must be very careful to use cross-validation, randomized holdouts, and/or independent testing to avoid overfitting (54).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [54] Dinga, 2019, Evaluating the evidence for biotypes of depression: Methodological replication and extension of, Neuroimage Clin source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In the Dunedin study, inclusion of the p factor fit the data better than exclusion, suggesting a common mechanism.

    “The investigators found that inclusion of the p factor fit the data better than exclusion, suggesting a common mechanism.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [32] Caspi, 2014, The p factor, Clin Psychol Sci source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    A study of 122 treatment-resistant depression cases revealed two subtypes: one with negative subcallosal cingulate cortex to left insula connectivity responded to medication but not CBT; the other with positive connectivity responded to therapy but not medication.

    “For example, a study of 122 treatment-resistant depression cases revealed two subtypes (53). One had negative sub- callosal cingulate cortex to left insula connectivity and responded to medication but not cognitive behavioral therapy. The other had positive connectivity and responded to therapy but not medication.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [53] Dunlop, 2017, Functional connectivity of the subcallosal cingulate cortex and differential outcomes to treatment with cognitive-behavioral therapy or antidepressant medication for major depressive disorder, Am J Psychiatry source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In over 1100 participants with treatment-resistant depression, four subtypes were identified using functional connectivity MRI, and these subtypes responded differentially to transcranial magnetic stimulation treatment.

    “Investigators identified four subtypes using functional connectivity magnetic resonance imaging in over 1100 participants with treatment-resistant depression. The investigators further demonstrated that these four subtypes responded differentially to transcranial magnetic stimulation treatment, suggesting that these identified subtypes are clini- cally useful.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [52] Drysdale, 2017, Resting-state connectivity biomarkers define neurophysiological subtypes of depression, Nat Med source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The p factor is associated with genetic and cognitive links: neuroticism correlated more with the general bifactor than specific factors, the association was partly heritable, and the general bifactor was associated with IQ and negative affectivity.

    “The general bifactor associated with IQ and negative affectivity and was partly heritable.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [39] Tackett, 2013, Common genetic influences on negative emotionality and a p factor in childhood and adolescence, J Abnorm Psychol source

      Not checked in the free run.

    • cites [28] Neumann, 2016, Single nucleotide polymorphism heritability of a p factor in children, J Am Acad Child Adolesc Psychiatry source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Three temperament subtypes (mild, irritable, and surgent) were identified from 437 children with or without ADHD followed longitudinally; subtypes showed distinct amygdala functional connectivity patterns, and children in the irritable subtype were more likely to receive a new comorbid diagnosis.

    “Three temperament subtypes, “mild,” “irrita- ble,” and “surgent,” were identified from 437 children with or without ADHD followed longitudinally (50).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [50] Karalunas, 2014, Subtyping attention-deficit/hyperactivity disorder using temperament dimensions, JAMA Psychiatry source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The heterogeneity problem comprises two tenets: different underlying mechanisms may drive the same disorder, and factors important for mental health outcomes may be unrelated to such mechanisms depending on the outcome of interest.

    “The heterogeneity problem comprises two tenets (18). First, different underlying mechanisms may drive the same disorder. Second, factors important for mental health outcomes may be unrelated to such mechanisms, depending on the outcome of interest.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [18] Feczko, 2019, The heterogeneity problem: Approaches to identify psychiatric subtypes, Trends Cogn Sci source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    ADHD subtyping using resting-state functional connectivity MRI and ADHD symptoms revealed similar subtypes within ADHD and typical samples, associated with impulsive characteristics, independent of diagnosis.

    “Specifically, the in- vestigators found that similar subtypes existed within ADHD and typical samples and were associated with impulsive characteristics, independent of diagnosis.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31] Costa Dias, 2015, Characterizing heterogeneity in children with and without ADHD based on reward system connectivity, Dev Cogn Neurosci source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The p factor is associated with altered perfusion of the dorsal anterior cingulate cortex, reduced gray matter volume across the brain, and reduced brain activation in task control systems during a working memory task.

    “Using cross-sectional samples of over 1000 children ages 11 to 23 years, in- vestigators found that increased p showed altered perfusion of the dorsal anterior cingulate cortex (34), reduced gray matter volume across the brain (35), and reduced brain activation in task control systems when engaged in a working memory task (36).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [34] Kaczkurkin, 2018, Common and dissociable regional cerebral blood flow differences associate with dimensions of psychopathology across categorical diagnoses, Mol Psychiatry source

      Not checked in the free run.

    • cites [35] Kaczkurkin, 2019, Evidence for dissociable linkage of dimensions of psychopathology to brain structure in youths, Am J Psychiatry source

      Not checked in the free run.

    • cites [36] Shanmugan, 2016, Common and dissociable mechanisms of executive system dysfunction across psychiatric disorders in youth, Am J Psychiatry source

      Not checked in the free run.

  • May combine two claims.

    Premisenone given

    Many individuals are diagnosed with different disorders across the life span, while profound comorbidity is present across multiple diagnoses.

    “Consistent with the idea of a common mechanism, many individuals are diagnosed with different disorders across the life span, while profound comorbidity is present across multiple diagnoses.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    It is often unclear whether comorbidity, heterogeneity, or both problems may be present across a subset of mental health disorders.

    “While it seems like the approaches tackle different problems, it is often unclear whether comorbidity, heterogeneity, or both problems may be present across a subset of mental health disorders.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Identified subtypes can share underlying mechanisms, whereas others remain distinct; the multifactorial approach has a comorbidity problem.

    “Indeed, identified subtypes can share underlying mech- anisms, whereas others remain distinct; the multifactorial approach has a comorbidity problem.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    Normative models allow researchers to identify shared mechanisms, potential subtypes or subfactors linked to an outcome, and to make predictions for individual cases.

    “Normative models (59) enable researchers to identify common (i.e., comorbid) mechanisms, identify potential subtypes or subfactors tied to an outcome of interest, and make pre- dictions on individual cases.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [59] Marquand, 2019, Conceptualizing mental disorders as deviations from normative functioning, Mol Psychiatry source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Other empirical studies employing the bifactor model have failed to find evidence for p or have found that a more specific factor fit just as well.

    “Other empirical studies employing the bifactor model have failed to find evidence for p (17) or have found that a more specific factor fit just as well (58).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [17] Murray, 2016, The development of the general factor of psychopathology “p factor” through childhood and adolescence, J Abnorm Child Psychol source

      Not checked in the free run.

    • cites [58] Haltigan, 2018, “P” and “DP:” Examining symptom-level bifactor models of psychopathology and dysregulation in clinically referred children and adolescents, J Am Acad Child Adolesc Psychiatry source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    These studies adopt analytic frameworks that enable an investigator to assess whether comorbidity and/or heterogeneity are important for the question of interest.

    “These studies adopt analytic frameworks that enable an investigator to assess whether comorbidity and/or heteroge- neity are important for the question of interest.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Small sample sizes stopped those investigators from performing subtyping with neuroimaging data.

    “Although a good future direction, limited sample sizes prevented the investigators from subtyping using neuroimaging data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [60] Zabihi, 2019, Dissecting the heterogeneous cortical anatomy of autism spectrum disorder using normative models, Biol Psychiatry Cogn Neurosci Neuroimaging

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Those investigators created normative models in which age and sex predicted structural MRI measures and then examined patient variation by diagnosis.

    “The in- vestigators created normative models where age and sex predicted structural magnetic resonance imaging measures and then examined patient variation by diagnosis.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [61] Wolfers, 2018, Mapping the heterogeneous phenotype of schizophrenia and bipolar disorder using normative models, JAMA Psychiatry source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Newer hybrid models such as normative modeling and the functional random forest may overcome the limitations of the bifactor framework and allow new inferences in psychopathology research.

    “Newer hybrid models, such as normative modeling (19,59–61) or the functional random forest (FRF) (18,62), may overcome these limitations and enable new inferences from psychopathology studies.”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [19] Marquand, 2016, Understanding heterogeneity in clinical cohorts using normative models: Beyond case-control studies, Biol Psychiatry source

      Not checked in the free run.

    • cites [59] Marquand, 2019, Conceptualizing mental disorders as deviations from normative functioning, Mol Psychiatry source

      Not checked in the free run.

    • cites [60] Zabihi, 2019, Dissecting the heterogeneous cortical anatomy of autism spectrum disorder using normative models, Biol Psychiatry Cogn Neurosci Neuroimaging

      Not checked in the free run.

    • cites [61] Wolfers, 2018, Mapping the heterogeneous phenotype of schizophrenia and bipolar disorder using normative models, JAMA Psychiatry source

      Not checked in the free run.

    • cites [18] Feczko, 2019, The heterogeneity problem: Approaches to identify psychiatric subtypes, Trends Cogn Sci source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    To study schizophrenia and bipolar disorder heterogeneity nested within age and sex, investigators applied normative modeling to 474 participants from the Thematically Organized Psychosis study.

    “To characterize heterogeneity of schizophrenia and bipolar disorder nested within age and sex, investigators applied normative modeling to 474 participants from the Thematically Organized Psychosis study (61).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [61] Wolfers, 2018, Mapping the heterogeneous phenotype of schizophrenia and bipolar disorder using normative models, JAMA Psychiatry source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Most studies consider both comorbidity and heterogeneity problems within a single study.

    “Instead, most studies consider both co- morbidity and heterogeneity problems within the single study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The FRF consists of several approaches that have been detailed elsewhere.

    “The FRF comprises several ap- proaches, which have been explicitly detailed elsewhere (18).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18] Feczko, 2019, The heterogeneity problem: Approaches to identify psychiatric subtypes, Trends Cogn Sci source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Internalizing and externalizing are common traits due to extensive replication and validation.

    “For example, internalizing (inward behaviors such as disparaging oneself) and externalizing (outward behaviors such as disparaging another) are common traits due to extensive replication and validation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Subtypes from the unsupervised alternative approach are less tied to any outcome and require greater validation than those from the hybrid approach.

    “Unlike the hybrid approach, such subtypes are less tied to any outcome and require greater validation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Individuals with ASD showed greater variability, but their heterogeneity in brain structure development was still largely nested within typical heterogeneity.

    “Individuals with ASD were more variable; however, ASD heterogeneity in brain structure development remained largely nested within typical heterogeneity.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [60] Zabihi, 2019, Dissecting the heterogeneous cortical anatomy of autism spectrum disorder using normative models, Biol Psychiatry Cogn Neurosci Neuroimaging

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    That study found one subtype in which dorsolateral prefrontal activity predicted high hyperactive symptoms and another subtype not related to such activity.

    “The investigators found one sub- type, where dorsolateral prefrontal activity predicted high hy- peractive symptoms and another subtype unrelated to such activity.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] Marquand, 2016, Understanding heterogeneity in clinical cohorts using normative models: Beyond case-control studies, Biol Psychiatry source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The bifactor framework has limits: because it is a supervised approach, it requires researchers to specify specific factors, so the researcher cannot discover new ones.

    “Because it is a supervised approach, the bifactor framework requires researchers to specify specific factors, such as internalizing and externalizing. Therefore, the researcher cannot discover new ones (18).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [18] Feczko, 2019, The heterogeneity problem: Approaches to identify psychiatric subtypes, Trends Cogn Sci source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The authors previously proposed the functional random forest to identify subtypes tied to an outcome.

    “More recently, we proposed the FRF to identify subtypes tied to an outcome (18,62).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [18] Feczko, 2019, The heterogeneity problem: Approaches to identify psychiatric subtypes, Trends Cogn Sci source

      Not checked in the free run.

    • cites [62] Feczko, 2018, Subtyping cognitive profiles in autism spectrum disorder using a random forest algorithm, Neuroimage source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Frameworks that mix supervised and unsupervised learning, including FRF and normative modeling, let investigators test the biological bases of common outcomes and whether subtypes underlie that mapping.

    “By combining supervised and unsupervised learning, hybrid statistical frameworks, such as FRF and normative modeling, enable investigators to test biological underpinnings of com- mon outcomes and whether subtypes underlie such mapping.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Simulations show that bifactor models tend to be robust to misspecified factors and therefore always have fits superior to those of correlated factor models.

    “Simulations show that bifactor models tend to be robust to misspecified factors (57) and, therefore, always have fits superior to those of correlated factor models.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [57] Greene, 2019, Are fit indices used to test psychopathology structure biased? A simulation study, J Abnorm Psychol source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Careful selection of outcomes, measures, and cohort in normative modeling can address the comorbidity problem.

    “Careful selection of outcomes, measures, and cohort can address the comorbidity problem.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Parallel-design RCTs allow measurement of between-treatment variation, but not variation among patients, clusters, or cluster-by-patient treatment interactions.

    “In such designs, one can measure the variation be- tween treatments; however, the variation between patients, clusters, or cluster/patient-treatment interactions cannot be measured.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Both methods discussed are limited and are specialized toward specific questions.

    “Both methods outlined here are limited, specializing them toward specific questions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    Both FRF and normative modeling include out-of-sample validation, which helps draw inferences about comorbidity and heterogeneity.

    “Out-of-sample validation is built in to both frameworks, which aids in deriving inferences from comorbidity and het- erogeneity problems.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    The longitudinal FRF has trouble finding individual trajectories if the first or last time point is absent, and changes to community detection algorithms can alter the subtypes found.

    “The longitudinal FRF struggles to identify individual trajectories when missing the first or last time point, while tweaking community detection algorithms may affect identified subtypes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    Studies that tackle comorbidity and heterogeneity might produce new understanding of pathophysiological processes and their biological connections.

    “Research studies that tackle the comorbid and heterogeneity problem may identify new insights into pathophysiological processes and their biological links.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Each framework for overcoming comorbidity and heterogeneity has strengths and limitations.

    “Each framework has strengths and limitations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    Both FRF and normative modeling require large datasets.

    “More critically, both frameworks require large datasets.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    The supervised part of these hybrid frameworks guarantees that biological measures truly relate to the outcome of interest.

    “The supervised component ensures that the biological mea- sures are actually related to the outcome of interest.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    The homogeneity assumption drives both the comorbidity problem and the heterogeneity problem.

    “The homogeneity assumption drives a comorbid and a het- erogeneity problem.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    More complex RCTs, like repeated cross-over designs, assign patients randomly to different treatments across periods and can measure patient/cluster-treatment interactions.

    “More complex RCTs, such as repeated cross-over designs (69), where patients are randomly assigned to different treatments at different periods throughout the study, can measure patient/cluster-treatment interactions.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [69] Senn, 2016, Mastering variation: Variance components and personalised medicine, Stat Med source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Current tools for the homogeneity assumption might not help with treatment-effect heterogeneity in randomized clinical trials, where several factors can drive variation in treatment response.

    “For example, our current tools to tackle the homogeneity assumption may not help address the treatment-effect het- erogeneity problem (67–69) in research clinical trials (RCTs), where multiple factors may drive treatment-response variation.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [67] Cortés, 2019, Does evidence support the high expectations placed in precision medicine? A bibliographic review, F1000Research source

      Not checked in the free run.

    • cites [68] Senn, 2009, Measurement in clinical trials: A neglected issue for statisticians?, Stat Med source

      Not checked in the free run.

    • cites [69] Senn, 2016, Mastering variation: Variance components and personalised medicine, Stat Med source

      Not checked in the free run.

55 of 55

  • The paper relies on this without support.

    The homogeneity assumption limits progress in refining clinical diagnosis, understanding mechanisms, and developing new treatments for mental health disorders.

    “The widely acknowledged homogeneity assumption limits progress in refining clinical diagnosis, understanding mechanisms, and developing new treatments for mental health disorders.”

  • The paper relies on this without support.

    Most studies still treat typical and diagnosed cohorts as homogeneous discrete entities.

    “However, most studies still treat typical and diagnosed cohorts as homogeneous discrete entities.”

  • The authors state this as an assumption.

    The comorbidity problem assumes that the same mechanism may underlie two different diagnoses.

    “The comorbidity problem assumes that the same mechanism may underlie two different diagnoses.”

  • The authors state this as an assumption.

    The multifactorial approach assumes that psychopathology comprises discrete conditions driven by different factors.

    “The multifactorial approach (Figure 1B) tackles het- erogeneity by assuming that psychopathology comprises discrete conditions driven by different factors (29).”

  • The authors state this as an assumption.

    One can assume that diagnoses and/or phenotypes can be driven by multiple distinct factors independent of a p factor.

    “Alternatively, one can assume that diagnoses and/or pheno- types can be driven by multiple distinct factors independent of a p factor.”

  • The authors state this as an assumption.

    One must also assume that factor misspecifications due to our current knowledge drive the heterogeneity problem.

    “By extension, one must also assume that factor misspecifications due to our current knowledge drive the het- erogeneity problem.”

  • The authors state this as an assumption.

    The normative model assumes that clinical features are Gaussian.

    “the normative model assumes that clinical features are all Gaussian and therefore has difficulty incorporating categorical information into the model.”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The areas of FCS decrease are prominent hubs of global and local functional connectivity, and age-related decrease could underlie performance decline in working memory and visual sustained attention.

    “The areas of FCS decrease are the prominent hubs of global and local functional connectivity, and the age-related decrease could underlie the performance decline in working memory and visual sustained attention, which are the most affected cognitive functions that occur with aging [39-41].”

    From Mapping neurophysiological subtypes of major depressive disorder using normative models of the functional connectome (2023) · cites this paper as [39]

The citation lists haven’t been fetched yet

References 72 · Cited by 96

Authors
Feczko, E., & Fair, D. A.
Affiliations
Oregon Health & Science University (US)
Corresponding author
Feczko, Eric
ORCID
Venue
Biological Psychiatry
Publisher
Elsevier BV
Published
2020-07
Volume
88
Issue
1
Pages
9-17
ISSN
0006-3223 (ISSN-L)
DOI
10.1016/j.biopsych.2020.02.015
PMC
PMC8404882
OpenAlex
W3007747757
Semantic Scholar
3941abf6cf3189bb1551a7ccb5f87473dda26414
Cited by
96
References
72
Citations by year
2026: 13 · 2025: 21 · 2024: 24 · 2023: 19 · 2022: 6 · 2021: 7 · 2020: 6
FWCI
1.39
Type
journal article
Language
en
Open access
Licence
Locations
  • Biological Psychiatry (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • National Institute of Mental Health (R01 MH096773-03S1, R01 MH 096773–05, R01 MH59107, R01 MH086654, R01 MH096773, R00 MH091238)
  • U.S. National Library of Medicine (T15 LM007088)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
  • Social Sciences › Psychology › Clinical Psychology › Personality Disorders and Psychopathology
Keywords
comorbidities, bifactor model, normative modeling, general psychopathology factor, treatment response
MeSH terms
Mental Disorders, Humans, Mental Health, Psychopathology, Comorbidity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review