Clinical prediction models in psychiatry: a systematic review of two decades of progress and challenges
Open accessAbstract
Recent years have seen the rapid proliferation of clinical prediction models aiming to support risk stratification and individualized care within psychiatry. Despite growing interest, attempts to synthesize current evidence in the nascent field of precision psychiatry have remained scarce. This systematic review therefore sought to summarize progress towards clinical implementation of prediction modeling for psychiatric outcomes. We searched MEDLINE, PubMed, Embase, and PsychINFO databases from inception to September 30, 2020, for English-language articles that developed and/or validated multivariable models to predict (at an individual level) onset, course, or treatment response for non-organic psychiatric disorders (PROSPERO: CRD42020216530). Individual prediction models were evaluated based on three key criteria: (i) mitigation of bias and overfitting; (ii) generalizability, and (iii) clinical utility. The Prediction model Risk Of Bias ASsessment Tool (PROBAST) was used to formally appraise each study’s risk of bias. 228 studies detailing 308 prediction models were ultimately eligible for inclusion. 94.5% of developed prediction models were deemed to be at high risk of bias, largely due to inadequate or inappropriate analytic decisions. Insufficient internal validation efforts (within the development sample) were also observed, while only one-fifth of models underwent external validation in an independent sample. Finally, our search identified just one published model whose potential utility in clinical practice was formally assessed. Our findings illustrated significant growth in precision psychiatry with promising progress towards real-world application. Nevertheless, these efforts have been inhibited by a preponderance of bias and overfitting, while the generalizability and clinical utility of many published models has yet to be formally established. Through improved methodological rigor during initial development, robust evaluations of reproducibility via independent validation, and evidence-based implementation frameworks, future research has the potential to generate risk prediction tools capable of enhancing clinical decision-making in psychiatric care.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Published prediction modelling studies often have severe methodological limitations that undermine their usefulness.
“Empirical evidence suggests that published studies often have severe methodological limitations, which undermine their usefulness.”
From Developing clinical prediction models: a step-by-step guide (2024) · cites this paper as [20]
A review of 308 prediction models in psychiatry found that most were at high risk of bias.
“Another review of 308 prediction models in psychiatry found that most were at high risk of bias.20”
From Developing clinical prediction models: a step-by-step guide (2024) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., & Walters, E. E. (2005) · Archives of General Psychiatry · doi:10.1001/archpsyc.62.6.593 · Cited by 20,391 · see citing works
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · see citing works
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388 · see citing works
Age of onset of mental disorders: a review of recent literature
Kessler, R. C., Amminger, G. P., Aguilar-Gaxiola, S., Alonso, J., Lee, S., & ??st??n, T. B. (2007) · Current Opinion in Psychiatry · doi:10.1097/yco.0b013e32816ebc8c · Cited by 3,457 · cited 1× · see citing works
“The resulting diagnostic variation generally reflected developmental trends for each disorder [29, 30].”
BackgroundCollins, P. G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,312 · see citing works
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281 · see citing works
Self-citationPROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984 · see citing works
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119 · see citing works
Self-citationMoons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018 · see citing works
Riley, R. D., Snell, K. I., Ensor, J., Burke, D. L., Harrell Jr, F. E., Moons, K. G., & Collins, G. S. (2019) · Statistics in Medicine · doi:10.1002/sim.7992 · Cited by 1,409 · see citing works
Prediction models for cardiovascular disease risk in the general population: systematic review
Damen, J. A. A. G., Hooft, L., Schuit, E., Debray, T. P. A., Collins, G. S., Tzoulaki, I., Lassale, C. M., Siontis, G. C. M., Chiocchia, V., Roberts, C., et al. (2016) · BMJ · doi:10.1136/bmj.i2416 · Cited by 935 · see citing works
Single subject prediction of brain disorders in neuroimaging: Promises and pitfalls
Arbabshirani, M. R., Plis, S., Sui, J., & Calhoun, V. D. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.02.079 · Cited by 912 · see citing works
Collins, G. S., Dhiman, P., Andaur Navarro, C. L., Ma, J., Hooft, L., Reitsma, J. B., Logullo, P., Beam, A. L., Peng, L., Van Calster, B., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048008 · Cited by 852 · see citing works
Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints
van der Ploeg, T., Austin, P. C., & Steyerberg, E. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-137 · Cited by 809 · see citing works
Self-citationCollins, G. S., Ogundimu, E. O., & Altman, D. G. (2016) · Statistics in Medicine · doi:10.1002/sim.6787 · Cited by 665 · see citing works
Debray, T. P. A., Vergouwe, Y., Koffijberg, H., Nieboer, D., Steyerberg, E. W., & Moons, K. G. M. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.06.018 · Cited by 597 · see citing works
Self-citationThe new field of ‘precision psychiatry’
Fernandes, B. S., Williams, L. M., Steiner, J. H., Leboyer, M., Carvalho, A. F., & Berk, M. (2017) · BMC Medicine · doi:10.1186/s12916-017-0849-x · Cited by 552 · see citing works
Adequate sample size for developing prediction models is not simply related to events per variable
Ogundimu, E. O., Altman, D. G., & Collins, G. S. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.02.031 · Cited by 546 · see citing works
The promise of machine learning in predicting treatment outcomes in psychiatry
Chekroud, A. M., Bondar, J., Delgadillo, J., Doherty, G., Wasil, A., Fokkema, M., Cohen, Z., Belgrave, D., DeRubeis, R., Iniesta, R., et al. (2021) · World Psychiatry · doi:10.1002/wps.20882 · Cited by 531 · see citing works
Austin, P. C., & Steyerberg, E. W. (2017) · Statistical Methods in Medical Research · doi:10.1177/0962280214558972 · Cited by 516 · see citing works
Self-citationFusar-Poli, P., Rutigliano, G., Stahl, D., Davies, C., Bonoldi, I., Reilly, T., & McGuire, P. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0284 · Cited by 246 · see citing works
Self-citationDetecting Neuroimaging Biomarkers for Psychiatric Disorders: Sample Size Matters
Schnack, H. G., & Kahn, R. S. (2016) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2016.00050 · Cited by 228 · see citing works
de Pablo, G. S., Studerus, E., Vaquerizo‐Serrano, J., Irving, J., Catalán, A., Oliver, D., Baldwin, H., Danese, A., Fazel, S., Steyerberg, E. W., et al. (2020) · Schizophrenia Bulletin · doi:10.1093/schbul/sbaa120 · Cited by 221 · see citing works
Self-citationEvaluating the impact of prediction models: lessons learned, challenges, and recommendations
Kappen, T. H., van Klei, W. A., van Wolfswinkel, L., Kalkman, C. J., Vergouwe, Y., & Moons, K. G. M. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0033-6 · Cited by 215 · see citing works
The Science of Prognosis in Psychiatry: A Review
Fusar-Poli, P., Hijazi, Z., Stahl, D., & Steyerberg, E. W. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.2530 · Cited by 197 · see citing works
Self-citationTowards a brain‐based predictome of mental illness
Rashid, B., & Calhoun, V. D. (2020) · Human Brain Mapping · doi:10.1002/hbm.25013 · Cited by 160 · see citing works
Machine learning for genetic prediction of psychiatric disorders: a systematic review
Bracher‐Smith, M., Crawford, K. L., & Escott‐Price, V. (2020) · Molecular Psychiatry · doi:10.1038/s41380-020-0825-2 · Cited by 159 · see citing works
A review of statistical updating methods for clinical prediction models
Su, T.-L., Jaki, T., Hickey, G. L., Buchan, I., & Sperrin, M. (2018) · Statistical Methods in Medical Research · doi:10.1177/0962280215626466 · Cited by 147 · see citing works
Riley, R. D., Snell, K. I. E., Martin, G. P., Whittle, R., Archer, L., Sperrin, M., & Collins, G. S. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.12.005 · Cited by 142 · see citing works
A systematic review and quality assessment of individualised breast cancer risk prediction models
Louro, J., Posso, M., Boon, M. H., Román, M., Domingo, L., Castells, X., & Sala, M. (2019) · British Journal of Cancer · doi:10.1038/s41416-019-0476-8 · Cited by 140 · see citing works
An empirical assessment of validation practices for molecular classifiers
Castaldi, P. J., Dahabreh, I. J., & Ioannidis, J. P. A. (2011) · Briefings in Bioinformatics · doi:10.1093/bib/bbq073 · Cited by 98 · see citing works
Fusar-Poli, P., Werbeloff, N., Rutigliano, G., Oliver, D., Davies, C., Stahl, D., McGuire, P., & Osborn, D. (2019) · Schizophrenia Bulletin · doi:10.1093/schbul/sby070 · Cited by 95 · see citing works
Self-citationStuderus, E., Ramyead, A., & Riecher‐Rössler, A. (2017) · Psychological Medicine · doi:10.1017/s0033291716003494 · Cited by 91 · see citing works
Navarro, C. L. A., Damen, J. A., Takada, T., Nijman, S. W. J., Dhiman, P., Ma, J., Collins, P. G. S., Bajpai, R. C., Riley, R. D., Moons, K. G. M., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-038832 · Cited by 82 · see citing works
Rocha, T. B.-M., Fisher, H. L., Caye, A., Anselmi, L., Arseneault, L., Barros, F. C., Caspi, A., Danese, A., Gonçalves, H., Harrington, H., et al. (2020) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2019.12.004 · Cited by 78 · see citing works
Self-citationRisk Prediction Models in Psychiatry
Bernardini, F., Attademo, L., Cleary, S. D., Luther, C. W., Shim, R. S., Quartesan, R., & Compton, M. T. (2016) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.15r10003 · Cited by 70 · see citing works
Raket, L. L., Jaskolowski, J., Kinon, B. J., Brasen, J. C., Jönsson, L., Wehnert, A., & Fusar-Poli, P. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30024-8 · Cited by 60 · see citing works
Self-citationOliver, D., Spada, G., Colling, C., Broadbent, M., Baldwin, H., Patel, R., Stewart, R., Stahl, D., Dobson, R., McGuire, P., et al. (2021) · Schizophrenia Research · doi:10.1016/j.schres.2020.05.007 · Cited by 49 · see citing works
Self-citationFusar‐Poli, P., Stringer, D., Durieux, A. M. S., Rutigliano, G., Bonoldi, I., Micheli, A. D., & Stahl, D. R. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0600-9 · Cited by 37 · see citing works
Self-citationWang, T., Oliver, D., Msosa, Y. J., Colling, C., Spada, G., Roguski, Ł., Folarin, A., Stewart, R. J., Roberts, A., Dobson, R., et al. (2020) · Journal of Visualized Experiments · doi:10.3791/60794 · Cited by 36 · see citing works
Self-citationRisk Calculators in Bipolar Disorder: A Systematic Review
Silva Ribeiro, J., Pereira, D., Salagre, E., Coroa, M., Santos Oliveira, P., Santos, V., Madeira, N., Grande, I., & Vieta, E. (2020) · Brain Sciences · doi:10.3390/brainsci10080525 · Cited by 26 · see citing works
Latham, R. M., Temple, R., Romeo, C., Danese, A., & Fisher, H. L. (2020) · Journal of Mental Health · doi:10.1080/09638237.2020.1844869 · Cited by 7 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Developing clinical prediction models: a step-by-step guide
Efthimiou, O., Seo, M., Chalkou, K., Debray, T., Egger, M., & Salanti, G. (2024) · BMJ · doi:10.1136/bmj-2023-078276 · Cited by 544 · see citing works
Chen, Z., Liu, X., Yang, Q., Wang, Y.-J., Miao, K., Gong, Z., Yu, Y., Leonov, A., Liu, C., Feng, Z., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.1671 · Cited by 43 · see citing works
Postpartum depression: a developed and validated model predicting individual risk in new mothers
Munk-Olsen, T., Liu, X., Madsen, K. B., Kjeldsen, M.-M. Z., Petersen, L. V., Bergink, V., Skalkidou, A., Vigod, S. N., Frokjaer, V. G., Pedersen, C. B., et al. (2022) · Translational Psychiatry · doi:10.1038/s41398-022-02190-8 · Cited by 36 · see citing works
Perfalk, E., Damgaard, J. G., Bernstorff, M., Hansen, L., Danielsen, A. A., & Østergaard, S. D. (2024) · Psychological Medicine · doi:10.1017/s0033291724002642 · Cited by 12 · see citing works
Child and adolescent psychiatry: challenges, solutions, opportunities, and future directions
Cortese, S., Arango, C., Aymerich, C., Catalan, A., Chetouani, M., Cohen, D., Coghill, D., Gabellone, A., Iniesta, R., Kadan, A., et al. (2026) · World Psychiatry · doi:10.1002/wps.70044 · Cited by 4 · see citing works
Self-citation
- Authors
- Meehan, A. J., Lewis, S. J., Fazel, S., Fusar-Poli, P., Steyerberg, E. W., Stahl, D., & Danese, A.
- Affiliations
- King's College London (GB)
- University of Oxford (GB)
- Leiden University Medical Center (NL)
- Corresponding author
- Danese, Andrea
- ORCID
- Venue
- Molecular Psychiatry
- Published
- 2022-04-01
- Volume
- 27
- Issue
- 6
- Pages
- 2700-2708
- ISSN
- 1359-4184 (ISSN-L)
- 1476-5578
- PMC
- PMC9156409
- OpenAlex
- W4221067412
- Cited by
- 219
- References
- 43
- Citations by year
- 2026: 62 · 2025: 54 · 2024: 54 · 2023: 36 · 2022: 13
- FWCI
- 5.75
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Molecular Psychiatry (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, cc-by)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Research Portal (King's College London) (submittedVersion, cc-by)
- Funders
- RCUK | Medical Research Council (MR/P005918/1)
- DH | National Institute for Health Research
- Wellcome Trust (202836/Z/16/Z)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- clinical prediction model, precision psychiatry, risk stratification, mental illness, overfitting, internal validation, external validation, clinical utility, PROBAST, model generalization, treatment response prediction, clinical implementation
- MeSH terms
- Humans, Prognosis, Psychiatry, Reproducibility of Results, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
