Cited by 275Schizophrenia research and treatmentBipolar Disorder and TreatmentPsychopathy, Forensic Psychiatry, Sexual Offending

Multimodal Machine Learning Workflows for Prediction of Psychosis in Patients With Clinical High-Risk Syndromes and Recent-Onset Depression

Open access
Koutsouleris, Nikolaos, Dwyer, Dominic B., Degenhardt, Franziska, Maj, Carlo, Urquijo-Castro, Maria Fernanda, Sanfelici, Rachele, Popovic, David, Oeztuerk, Oemer +45 more·2021·JAMA Psychiatry

Abstract

Importance: Diverse models have been developed to predict psychosis in patients with clinical high-risk (CHR) states. Whether prediction can be improved by efficiently combining clinical and biological models and by broadening the risk spectrum to young patients with depressive syndromes remains unclear. Objectives: To evaluate whether psychosis transition can be predicted in patients with CHR or recent-onset depression (ROD) using multimodal machine learning that optimally integrates clinical and neurocognitive data, structural magnetic resonance imaging (sMRI), and polygenic risk scores (PRS) for schizophrenia; to assess models' geographic generalizability; to test and integrate clinicians' predictions; and to maximize clinical utility by building a sequential prognostic system. Design, Setting, and Participants: This multisite, longitudinal prognostic study performed in 7 academic early recognition services in 5 European countries followed up patients with CHR syndromes or ROD and healthy volunteers. The referred sample of 167 patients with CHR syndromes and 167 with ROD was recruited from February 1, 2014, to May 31, 2017, of whom 26 (23 with CHR syndromes and 3 with ROD) developed psychosis. Patients with 18-month follow-up (n = 246) were used for model training and leave-one-site-out cross-validation. The remaining 88 patients with nontransition served as the validation of model specificity. Three hundred thirty-four healthy volunteers provided a normative sample for prognostic signature evaluation. Three independent Swiss projects contributed a further 45 cases with psychosis transition and 600 with nontransition for the external validation of clinical-neurocognitive, sMRI-based, and combined models. Data were analyzed from January 1, 2019, to March 31, 2020. Main Outcomes and Measures: Accuracy and generalizability of prognostic systems. Results: A total of 668 individuals (334 patients and 334 controls) were included in the analysis (mean [SD] age, 25.1 [5.8] years; 354 [53.0%] female and 314 [47.0%] male). Clinicians attained a balanced accuracy of 73.2% by effectively ruling out (specificity, 84.9%) but ineffectively ruling in (sensitivity, 61.5%) psychosis transition. In contrast, algorithms showed high sensitivity (76.0%-88.0%) but low specificity (53.5%-66.8%). A cybernetic risk calculator combining all algorithmic and human components predicted psychosis with a balanced accuracy of 85.5% (sensitivity, 84.6%; specificity, 86.4%). In comparison, an optimal prognostic workflow produced a balanced accuracy of 85.9% (sensitivity, 84.6%; specificity, 87.3%) at a much lower diagnostic burden by sequentially integrating clinical-neurocognitive, expert-based, PRS-based, and sMRI-based risk estimates as needed for the given patient. Findings were supported by good external validation results. Conclusions and Relevance: These findings suggest that psychosis transition can be predicted in a broader risk spectrum by sequentially integrating algorithms' and clinicians' risk estimates. For clinical translation, the proposed workflow should undergo large-scale international validation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1001/jamapsychiatry.2020.3604

Cites

74 of 74 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Koutsouleris, N., Dwyer, D. B., Degenhardt, F., Maj, C., Urquijo-Castro, M. F., Sanfelici, R., Popovic, D., Oeztuerk, O., Haas, S. S., Weiske, J., Ruef, A., Kambeitz-Ilankovic, L., Antonucci, L. A., Neufang, S., Schmidt-Kraepelin, C., Ruhrmann, S., Penzel, N., Kambeitz, J., Haidl, T. K., Rosen, M., Chisholm, K., Riecher-Rössler, A., Egloff, L., Schmidt, A., Andreou, C., Hietala, J., Schirmer, T., Romer, G., Walger, P., Franscini, M., Traber-Walker, N., Schimmelmann, B. G., Flückiger, R., Michel, C., Rössler, W., Borisov, O., Krawitz, P. M., Heekeren, K., Buechler, R., Pantelis, C., Falkai, P., Salokangas, R. K. R., Lencer, R., Bertolino, A., Borgwardt, S., Noethen, M., Brambilla, P., Wood, S. J., Upthegrove, R., Schultze-Lutter, F., Theodoridou, A., Meisenzahl, E., & PRONIA Consortium
Affiliations
  • King's College London (GB)
  • Ludwig-Maximilians-Universität München (DE)
  • University of Bonn (DE)
  • Max Planck Institute for Human Cognitive and Brain Sciences (DE)
  • Max Planck Society (DE)
  • Icahn School of Medicine at Mount Sinai (US)
  • Department of Psychiatry and Psychotherapy, Faculty of Medicine and University Hospital, University of Cologne, Cologne, Germany
  • University of Bari Aldo Moro (IT)
  • Heinrich Heine University Düsseldorf (DE)
  • University of Cologne (DE)
  • University of Birmingham (GB)
  • Department of Psychiatry, Psychiatric University Hospital, University of Basel, Switzerland
  • University of Basel (CH)
  • University of Turku (FI)
  • GE Healthcare GmbH (previously GE Global Research GmbH), Munich, Germany
  • University of Münster (DE)
  • Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, LVR Clinic Düsseldorf, Düsseldorf, Germany
  • University of Zurich (CH)
  • Department of Child and Adolescent Psychiatry and Psychotherapy, University of Zürich, Zürich, Switzerland
  • University of Bern (CH)
  • Department of Psychiatry, Psychotherapy and Psychosomatics, University Hospital of Psychiatry Zurich, Zurich, Switzerland
  • The University of Melbourne (AU)
  • Max Planck Institute of Psychiatry (DE)
  • Institute of Human Genetics, Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, Germany
  • University of Milan (IT)
  • Department of Psychiatry and Psychotherapy, Medical Faculty, Heinrich-Heine University, Düsseldorf, Germany
Corresponding author
Koutsouleris, Nikolaos
Venue
JAMA Psychiatry
Published
2021-02-01
Volume
78
Issue
2
Pages
195
ISSN
2168-622X (ISSN-L)
OpenAlex
W3108636332
Cited by
275
References
74
Citations by year
2026: 46 · 2025: 59 · 2024: 56 · 2023: 49 · 2022: 38 · 2021: 27
FWCI
20.67
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JAMA Psychiatry (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Kölner Universitäts PublikationsServer (Universität zu Köln) (publishedVersion)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
  • UTUPub (University of Turku) (submittedVersion)
  • Archivio Istituzionale della Ricerca (Universita Degli Studi Di Milano) (submittedVersion, other-oa)
  • Bern Open Repository and Information System (University of Bern) (submittedVersion)
  • edoc (University of Basel) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • Research Portal (King's College London) (submittedVersion, other-oa)
  • University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
  • MPG.PuRe (Max Planck Society) (submittedVersion)
  • CINECA IRIS Institutional Research Information System (University of Bari Aldo Moro) (submittedVersion)
  • CINECA IRIS Institutional Research Information System (University of Bari Aldo Moro) (submittedVersion, cc-by-nc-nd)
  • UTUPub (University of Turku) (submittedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
Funders
  • Universität Basel
  • University of Bern
  • Università degli Studi di Milano
  • Universität zu Lübeck
  • Westfälische Wilhelms-Universität Münster
  • Universita degli Studi di Bari Aldo Moro
  • Turun Yliopisto
  • Universität zu Köln
  • Università degli Studi di Udine
  • Universitas Airlangga
Topics
  • Health Sciences › Medicine › Psychiatry and Mental health › Schizophrenia research and treatment
  • Health Sciences › Medicine › Psychiatry and Mental health › Bipolar Disorder and Treatment
  • Social Sciences › Psychology › Clinical Psychology › Psychopathy, Forensic Psychiatry, Sexual Offending
Keywords
psychosis prediction, recent-onset depression, multimodal learning, polygenic risk score, schizophrenia, structural MRI, neuropsychological performance, prognostic model, model generalization, psychosis spectrum, young adults, external validation
MeSH terms
Machine Learning, Adult, Depressive Disorder, Disease Susceptibility, Europe, Female, Follow-Up Studies, Humans, Longitudinal Studies, Male, Prognosis, Psychotic Disorders, Schizophrenia, Sensitivity and Specificity, Time Factors, Comorbidity, Young Adult, Workflow

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review