The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Towards better clinical prediction models: seven steps for development and an ABCD for validation
Open accessAbstract
Clinical prediction models provide risk estimates for the presence of disease (diagnosis) or an event in the future course of disease (prognosis) for individual patients. Although publications that present and evaluate such models are becoming more frequent, the methodology is often suboptimal. We propose that seven steps should be considered in developing prediction models: (i) consideration of the research question and initial data inspection; (ii) coding of predictors; (iii) model specification; (iv) model estimation; (v) evaluation of model performance; (vi) internal validation; and (vii) model presentation. The validity of a prediction model is ideally assessed in fully independent data, where we propose four key measures to evaluate model performance: calibration-in-the-large, or the model intercept (A); calibration slope (B); discrimination, with a concordance statistic (C); and clinical usefulness, with decision-curve analysis (D). As an application, we develop and validate prediction models for 30-day mortality in patients with an acute myocardial infarction. This illustrates the usefulness of the proposed framework to strengthen the methodological rigour and quality for prediction models in cardiovascular research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“and is best informed - a priori - by medical expert knowledge or existing literature13,17,76.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“Unlike discrimination and calibration, DCA derives the clinical utility from the predictive performance5,6,17,49,68,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“Similar to internal validation of the AIPM, external AIPM model validation can be based on discrimination (area under the receiver operating characteristic curve, sensitivity, specificity, positive and negative predictive values), calibration (calibration plot)5,6,13,17,49,56,78, and Decision Curve Analysis5,6,17,49,68,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“The literature recommends to assess generalizability on external data from a different time period, place, or healthcare setting3,7,8,10,16,17,27,56,68,72,79.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [17]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“It is therefore considered important to compose a diverse development team17,22–25, create awareness and involve stakeholders in design choices22,24,26,27,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [17]
Experimental — accuracy not yet measured
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Harrell, F. E. (2001) · Springer Series in Statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,195
Authors/Task Force Members, Steg, P. G., James, S. K., Atar, D., Badano, L. P., Lundqvist, C. B., Borger, M. A., Di Mario, C., Dickstein, K., Ducrocq, G., et al. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs215 · Cited by 6,829
Decision curve analysis: a novel method for evaluating prediction models
Vickers, A. J., & Elkin, E. B. (2006) · Medical Decision Making · doi:10.1177/0272989x06295361 · Cited by 6,341
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
Self-citationRegression Shrinkage and Selection via The Lasso: A Retrospective
Tibshirani, R. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2011.00771.x · Cited by 3,589
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Self-citationInternal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
Self-citationShmueli, G. (2010) · Statistical Science · doi:10.1214/10-sts330 · Cited by 2,517
Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers
Pencina, M. J., D'Agostino, R. B., & Steyerberg, E. W. (2011) · Statistics in Medicine · doi:10.1002/sim.4085 · Cited by 2,453
Self-citation
Cited by
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationLiu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719
Prediction models need appropriate internal, internal-external, and external validation
Steyerberg, E. W., & Harrell, F. E. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.04.005 · Cited by 1,304
Self-citationVariable selection strategies and its importance in clinical prediction modelling.
Chowdhury, M. Z. I., & Turin, T. C. (2020) · Family Medicine and Community Health · doi:10.1136/fmch-2019-000262 · Cited by 1,011
Metadata
- Authors
- Steyerberg, E. W., & Vergouwe, Y.
- Affiliations
- Erasmus MC (NL)
- Corresponding author
- Steyerberg, E. W.
- ORCID
- Steyerberg, E. W.
- Venue
- European Heart Journal
- Publisher
- Oxford University Press (OUP)
- Published
- 2014-06-04
- Volume
- 35
- Issue
- 29
- Pages
- 1925-1931
- ISSN
- 0195-668X (ISSN-L)
- 1522-9645
- PMC
- PMC4155437
- OpenAlex
- W2154286581
- Cited by
- 1,928
- References
- 56
- Citations by year
- 2026: 271 · 2025: 173 · 2024: 222 · 2023: 236 · 2022: 263 · 2021: 248 · 2020: 188 · 2019: 106 · 2018: 72 · 2017: 80 · 2016: 43 · 2015: 21 · 2014: 2
- FWCI
- 51.38
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- European Heart Journal (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
- RePub (Erasmus University Rotterdam) (submittedVersion)
- Funders
- National Institute of Neurological Disorders and Stroke (1U01-NS086294-01, U01NS086294)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- clinical prediction model, prediction model development, model validation, calibration, discrimination (statistics), decision curve analysis, 30-day mortality, acute myocardial infarction, prognostic model, methodological quality
- MeSH terms
- Calibration, Decision Support Techniques, Diagnosis, Humans, Prognosis, Reproducibility of Results, Models, Statistical, Risk Assessment, Clinical Coding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Towards better clinical prediction models: seven steps for development and an ABCD for validation. https://app.cochat.ai/papers/10.1093/eurheartj/ehu207
- The paper
- Steyerberg, E. W., & Vergouwe, Y. (2014). Towards better clinical prediction models: seven steps for development and an ABCD for validation. European Heart Journal. https://doi.org/10.1093/eurheartj/ehu207
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.