Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker
The DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index.
Abstract
Prediction models are increasingly used to complement clinical reasoning and decision making in modern medicine in general, and in the cardiovascular domain in particular. Developed models first and foremost need to provide accurate and (internally and externally) validated estimates of probabilities of specific health conditions or outcomes in targeted patients. The adoption of such models must guide physician's decision making and an individual's behaviour, and consequently improve individual outcomes and the cost-effectiveness of care. In a series of two articles we review the consecutive steps generally advocated for risk prediction model research. This first article focuses on the different aspects of model development studies, from design to reporting, how to estimate a model's predictive performance and the potential optimism in these estimates using internal validation techniques, and how to quantify the added or incremental value of new predictors or biomarkers (of whatever type) to existing predictors. Each step is illustrated with empirical examples from the cardiovascular field.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It is therefore essential that key details of how a prediction model was developed and validated be clearly reported to enable synthesis and critical appraisal of all relevant information (14,(33)(34)(35)(36).”
From Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [14]
In the citing paper
“The TRIPOD Statement covers studies that report solely development [12,15], both development and external validation, and solely external validation (with or without updating), of a prediction model [14] (Figure 1).”
From Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement (2015) · cites this paper as [15]
In the citing paper
“The TRIPOD Statement covers studies that report solely development,12,15 both development and external validation, and solely external validation (with or without updating), of a prediction model14 (Figure 3).”
From Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [15]
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
Hosmer, D. W., & Lemeshow, S. (2000) · doi:10.1002/0471722146 · Cited by 20,302
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,265
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Prediction of coronary heart disease using risk factor categories
Wilson, P. W. F., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998) · Circulation · doi:10.1161/01.cir.97.18.1837 · Cited by 9,581
Harrell, F. E. (2001) · Springer Series in Statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,195
Research works printed but not linked: 2
- Wells, 1997, A simple clinical model to categorize pretest probability in patients with suspected pulmonary embolism, Blood, 90, 424
- Hosmer DW Lemeshow S . Applied Survival Analysis: Regression Modelling of Time to Event Data. New York: Wiley, 1999.
Cited by
Items 201–300 of 1,000
Liu, Q., Chu, H., LaValley, M. P., Hunter, D. J., Zhang, H., Tao, L., Zhan, S., Lin, J., & Zhang, Y. (2021) · The Lancet Rheumatology · doi:10.1016/s2665-9913(21)00324-6 · Cited by 44
McShane, L. M., & Polley, M. C. (2013) · Clinical Trials · doi:10.1177/1740774513499458 · Cited by 44
Refining Prognosis in Lung Cancer
Mahar, A. L., Compton, C. C., McShane, L. M., Halabi, S., Asamura, H., Rami–Porta, R., & Groome, P. A. (2015) · Journal of Thoracic Oncology · doi:10.1097/jto.0000000000000652 · Cited by 43
for the Individualised Screening for Diabetic Retinopathy (ISDR) Study Group, Eleuteri, A., Fisher, A. C., Broadbent, D. M., García-Fiñana, M., Cheyne, C. P., Wang, A., Stratton, I. M., Gabbay, M., Seddon, D., et al. (2017) · Diabetologia · doi:10.1007/s00125-017-4386-0 · Cited by 39
The Role of Risk Prediction Models in Prevention and Management of AKI
Hodgson, L. E., Selby, N. M., Huang, T., & Forni, L. G. (2019) · Seminars in Nephrology · doi:10.1016/j.semnephrol.2019.06.002 · Cited by 45
A clinical prediction model to assess risk for pancreatic cancer among patients with prediabetes
Boursi, B., Finkelman, B. S., Giantonio, B. J., Haynes, K., Rustgi, A. K., Rhim, A. D., Mamtani, R., & Yang, Y.-X. (2021) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000002052 · Cited by 45
Key Aspects of Prognostic Model Development and Interpretation From a Clinical Perspective
Hoesseini, A., van Leeuwen, N., Sewnaik, A., Steyerberg, E. W., de Jong, R. J. B., Lingsma, H. F., & Offerman, M. P. J. (2021) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2021.3505 · Cited by 43
Risk factors of recurrent anal sphincter ruptures: a population‐based cohort study
Jangö, H., Langhoff‐Roos, J., Rosthøj, S., & Sakse, A. (2012) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.2012.03486.x · Cited by 43
Rakers, M. M., van Buchem, M. M., Kucenko, S., de Hond, A., Kant, I. M. J., van Smeden, M., Moons, K. G. M., Leeuwenberg, A., Chavannes, N. H., Villalobos-Quesada, M. J., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.32990 · Cited by 42
Self-citationShi, C., Dumville, J. C., & Cullum, N. A. (2018) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2018.08.005 · Cited by 42
A clinical predictive model of renal injury in children with congenital solitary functioning kidney
Poggiali, I. V., Silva, A. C. S. E., Vasconcelos, M. A., Dias, C. S., Gomes, I. R. R., Carvalho, R. A., Oliveira, M. C. L., Pinheiro, S. V. B., MAK, R. H. K., & OLIVEIRA, E. A. (2018) · Pediatric Nephrology · doi:10.1007/s00467-018-4111-3 · Cited by 42
Cardiovascular Disease Prognostic Models in Latin America and the Caribbean: A Systematic Review
Carrillo-Larco, R. M., Altez-Fernandez, C., Pacheco-Barrios, N. V., Claudia Bambs, S., Irazola, V. E., Miranda, J. J., Danaei, G., & Perel, P. (2019) · Global Heart · doi:10.1016/j.gheart.2019.03.001 · Cited by 42
A Roadmap for the Development of Applied Computational Psychiatry
Paulus, M. P., Huys, Q. J. M., & Maia, T. V. (2016) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2016.05.001 · Cited by 41
Introduction into PPPM as a new paradigm of public health service: an integrative view
Bodrova, T. A., Kostyushev, D. S., Antonova, E., Slavin, S., Gnatenko, D., Bocharova, M., Legg, M., Pozzilli, P., Paltsev, M. A., & Suchkov, S. V. (2012) · The EPMA Journal · doi:10.1186/1878-5085-3-16 · Cited by 41
Wang, Q.-L., Ness‐Jensen, E., Santoni, G., Xie, S.-H., & Lagergren, J. (2020) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000001094 · Cited by 42
van Giessen, A., Peters, J., Wilcher, B., Hyde, C., Moons, C. G. M., de Wit, G. A., & Koffijberg, H. (2017) · Value in Health · doi:10.1016/j.jval.2017.01.001 · Cited by 41
Pharmacogenomic scores in psychiatry: systematic review of current evidence
Sharew, N. T., Clark, S. R., Schubert, K. O., & Amare, A. T. (2024) · Translational Psychiatry · doi:10.1038/s41398-024-02998-6 · Cited by 40
Predictive modelling of Ross River virus notifications in southeastern Australia
Cutcher, Z., Williamson, E., Lynch, S., Rowe, S. L., Clothier, H. J., & Firestone, S. M. (2016) · Epidemiology and Infection · doi:10.1017/s0950268816002594 · Cited by 40
Mao, S., Yu, X., Sun, J., Yang, Y., Shan, Y., Sun, J., Mugaanyi, J., Fan, R., Wu, S., & Lu, C. (2022) · BMC Cancer · doi:10.1186/s12885-022-09345-2 · Cited by 40
Systematic review of risk prediction models for diabetes after bariatric surgery
Zhang, R., Borisenko, O., Telegina, I., Hargreaves, J., Ahmed, A. R., Santos, R. S., Pring, C., Funch-Jensen, P. M., Dillemans, B., & Hedenbro, J. (2016) · British journal of surgery · doi:10.1002/bjs.10255 · Cited by 40
CCL21 as a Potential Serum Biomarker for Pulmonary Arterial Hypertension in Systemic Sclerosis
Hoffmann‐Vold, A., Hesselstrand, R., Fretheim, H., Ueland, T., Andreassen, A. K., Brunborg, C., Palchevskiy, V., Midtvedt, Ø., Garen, T., Aukrust, P., et al. (2018) · Arthritis & Rheumatology · doi:10.1002/art.40534 · Cited by 40
Predictors of dizziness in older persons: a 10-year prospective cohort study in the community
Maarsingh, O. R., Stam, H., van de Ven, P. M., van Schoor, N. M., Ridd, M. J., & van der Wouden, J. C. (2014) · BMC Geriatrics · doi:10.1186/1471-2318-14-133 · Cited by 39
Kengne, A. P. (2013) · Cardiovascular journal of South Africa/Cardiovascular journal of Southern Africa · doi:10.5830/cvja-2013-078 · Cited by 41
Self-citationBjerre, M., Pedersen, S. H., Møgelvang, R., Lindberg, S. Ø., Jensen, J. S., Galatius, S., & Flyvbjerg, A. (2013) · European Journal of Preventive Cardiology · doi:10.1177/2047487313487083 · Cited by 39
A predictive model of progressive chronic kidney disease in idiopathic nephrotic syndrome
Mendonça, A. C. Q., OLIVEIRA, E. A., Fróes, B. P. E., Faria, L. D. D. C., Pinto, J. S., Nogueira, M. M. I., Lima, G. O., Resende, P. I., Assis, N. S., Silva, A. C. S. E., et al. (2015) · Pediatric Nephrology · doi:10.1007/s00467-015-3136-0 · Cited by 39
Nazha, A., Komrokji, R. S., Garcia-Manero, G., Barnard, J., Roboz, G. J., Steensma, D. P., Dezern, A. E., Zell, K., Zimmerman, C., Ali, N. E. A., et al. (2016) · Haematologica · doi:10.3324/haematol.2015.140962 · Cited by 38
Bamia, C. (2018) · European Journal of Clinical Nutrition · doi:10.1038/s41430-018-0128-8 · Cited by 38
Screening for hyperglycemia in the developing world: Rationale, challenges and opportunities
Echouffo-Tcheugui, J. B., Mayige, M. T., Ogbera, A. O., Sobngwi, E., & Kengne, A. P. (2012) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2012.08.003 · Cited by 38
Self-citationYone, E. W. P., Kengne, A. P., Moifo, B., & Kuaban, C. (2012) · Scandinavian Journal of Infectious Diseases · doi:10.3109/00365548.2012.714905 · Cited by 38
Self-citationEcho and heart failure: when do people need an echo, and when do they need natriuretic peptides?
Modin, D., Andersen, D. M., & Biering‐Sørensen, T. (2018) · Echo Research and Practice · doi:10.1530/erp-18-0004 · Cited by 38
Dodds, R. M., Kuh, D. J., Sayer, A. A., & Cooper, R. (2018) · Experimental Gerontology · doi:10.1016/j.exger.2018.06.001 · Cited by 38
Con: Most clinical risk scores are useless
Dekker, F. W., Ramspek, C. L., & van Diepen, M. (2017) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfx073 · Cited by 37
Conversion From Off-Pump Coronary Artery Bypass Grafting to On-Pump Coronary Artery Bypass Grafting
Keeling, B., Thourani, V. H., Aliawadi, G., Kim, S., Cyr, D. D., Badhwar, V., Jacobs, J. P., Brennan, J. M., Meza, J. M., Matsouaka, R., et al. (2017) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2017.03.032 · Cited by 37
Archer, R. J., Everson-Hock, E., Hamilton, J., Stevens, J., Essat, M., Poku, E. N., Clowes, M., Pandor, A., & Stevenson, M. (2018) · Health Technology Assessment · doi:10.3310/hta22660 · Cited by 39
Assessing Participation in Community-Based Physical Activity Programs in Brazil
Reis, R. S., Yan, Y., Parra, D. C., & Brownson, R. C. (2013) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e3182a365ae · Cited by 38
Postpartum depression: a developed and validated model predicting individual risk in new mothers
Munk-Olsen, T., Liu, X., Madsen, K. B., Kjeldsen, M.-M. Z., Petersen, L. V., Bergink, V., Skalkidou, A., Vigod, S. N., Frokjaer, V. G., Pedersen, C. B., et al. (2022) · Translational Psychiatry · doi:10.1038/s41398-022-02190-8 · Cited by 36
Kom, I. Y., Dongelmans, D. A., de Keizer, N. F., Jager, K. J., Schut, M. C., Abu–Hanna, A., & Klopotowska, J. E. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad014 · Cited by 36
Goerdten, J., Carrière, I., & Terrera, G. M. (2020) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1002/trc2.12041 · Cited by 36
Haarhaus, M. L., Santos-Araújo, C., Haase, M., Veiga, P. M., Lucas, C., & Macario, F. J. G. R. M. (2021) · Clinical Kidney Journal · doi:10.1093/ckj/sfab037 · Cited by 36
Ahmed, A., Ashour, O. M., Ali, H., & Firouz, M. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117314 · Cited by 37
Pigazzani, F., Gorni, D., Dyar, K. A., Pedrelli, M., Kennedy, G., Costantino, G., Bruno, A., Mackenzie, I. S., MacDonald, T. M., Tietge, U. J. F., et al. (2022) · Antioxidants · doi:10.3390/antiox11081541 · Cited by 36
Graphics and statistics for cardiology: clinical prediction rules
Woodward, M., Tunstall‐Pedoe, H., & Peters, S. A. E. (2017) · Heart · doi:10.1136/heartjnl-2016-310210 · Cited by 36
Self-citationHeart failure risk predictions in adult patients with congenital heart disease: a systematic review
Wang, F., Harel-Sterling, L., Cohen, S., Liu, A., Brophy, J. M., Paradis, G., & Marelli, A. J. (2019) · Heart · doi:10.1136/heartjnl-2019-314977 · Cited by 36
on behalf of the GOCE Group, Louvigné, M., Rakotonjanahary, J., Goumy, L., Tavenard, A., Brasme, J., Rialland, F., Baruchel, A., Auclerc, M.-F., Despert, V., et al. (2020) · Pediatric Rheumatology · doi:10.1186/s12969-019-0376-8 · Cited by 36
Pendlebury, S. T., Lovett, N. G., Smith, S. C., Cornish, E. F., Mehta, Z., & Rothwell, P. M. (2016) · Age and Ageing · doi:10.1093/ageing/afv177 · Cited by 35
Osteoprotegerin improves risk detection by traditional cardiovascular risk factors and hsCRP
Møgelvang, R., Haahr-Pedersen, S. A., Bjerre, M., Frystyk, J., Iversen, A. Z., Galatius, S., Flyvbjerg, A., & Jensen, J. S. (2012) · Heart · doi:10.1136/heartjnl-2012-302240 · Cited by 34
Williams, B. A., Chagin, K. M., Bash, L. D., Boden, W. E., Duval, S., Fowkes, F. G. R., Mahaffey, K. W., Patel, M. D., D’Agostino, R. B., Peterson, E. D., et al. (2018) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2018.03.026 · Cited by 34
Prognostic models of diabetic microvascular complications: a systematic review and meta-analysis
Saputro, S. A., Pattanaprateep, O., Pattanateepapon, A., Karmacharya, S., & Thakkinstian, A. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01841-z · Cited by 34
Prognostic risk stratification of gliomas using deep learning in digital pathology images
Chunduru, P., Phillips, J. J., & Molinaro, A. M. (2022) · Neuro-Oncology Advances · doi:10.1093/noajnl/vdac111 · Cited by 34
Huang, Z., Klarić, L., Krasauskaite, J., Khalid, W., Strachan, M. W. J., Wilson, J. F., & Price, J. F. (2023) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwad160 · Cited by 34
Development and Application of Chronic Disease Risk Prediction Models
Oh, S. M., Stefani, K. M., & Kim, H. C. (2014) · Yonsei Medical Journal · doi:10.3349/ymj.2014.55.4.853 · Cited by 34
Development and validation of a prognostic score during tuberculosis treatment
Pefura-Yone, E. W., Balkissou, A. D., Poka-Mayap, V., Fatime-Abaicho, H. K., Enono-Edende, P., & Kengne, A. P. (2017) · BMC Infectious Diseases · doi:10.1186/s12879-017-2309-9 · Cited by 34
Self-citationDevelopment of a prediction model for inpatient gout flares in people with comorbid gout
Jatuworapruk, K., Grainger, R., Dalbeth, N., & Taylor, W. J. (2019) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2019-216277 · Cited by 33
Nadarajah, R., Wu, J., Frangi, A. F., Hogg, D., Cowan, C., & Gale, C. P. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-052887 · Cited by 34
A Review of Cancer Risk Prediction Models with Genetic Variants
Wang, X., Oldani, M. J., Zhao, X., Huang, X., & Qian, D. (2014) · Cancer Informatics · doi:10.4137/cin.s13788 · Cited by 33
Kerr, K. F., Meisner, A., Thiessen‐Philbrook, H., Coca, S. G., & Parikh, C. R. (2014) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.10351013 · Cited by 33
Takeuchi, M., Inuzuka, R., Hayashi, T., Shindo, T., Hirata, Y., Shimizu, N., Inatomi, J., Yokoyama, Y., Namai, Y., Oda, Y., et al. (2017) · The Pediatric Infectious Disease Journal · doi:10.1097/inf.0000000000001621 · Cited by 33
A clinical predictive model of chronic kidney disease in children with posterior urethral valves
Vasconcelos, M. A., Silva, A. C. S. E., Gomes, I. R. R., Carvalho, R. A., Pinheiro, S. V. B., Colosimo, E. A., Yorgin, P. D., MAK, R. H. K., & OLIVEIRA, E. A. (2018) · Pediatric Nephrology · doi:10.1007/s00467-018-4078-0 · Cited by 33
Ramspek, C. L., Verberne, W. R., van Buren, M., Dekker, F. W., Bos, W. J. W., & van Diepen, M. (2020) · Clinical Kidney Journal · doi:10.1093/ckj/sfaa021 · Cited by 32
Past, present, and future of mortality risk scores in the contemporary cardiac intensive care unit
Jentzer, J. C., & Rosselló, X. (2021) · European Heart Journal Acute Cardiovascular Care · doi:10.1093/ehjacc/zuab072 · Cited by 32
Wang, R., Miao, Z., Liu, T., Liu, M., Grdinovac, K., Song, X., Liang, Y., Delen, D., & Paiva, W. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10071473 · Cited by 32
Risk Prediction in Chronic Kidney Disease
Samuels, J., Connelly, K., & Tangri, N. (2022) · Wiley eBooks · doi:10.1002/9781119105954.ch5 · Cited by 32
Computational approaches and machine learning for individual-level treatment predictions
Paulus, M. P., & Thompson, W. K. (2019) · Psychopharmacology · doi:10.1007/s00213-019-05282-4 · Cited by 31
van der Sluijs, R., Debray, T. P. A., Poeze, M., Leenen, L. P. H., & van Heijl, M. (2019) · Diagnostic and Prognostic Research · doi:10.1186/s41512-019-0058-5 · Cited by 31
Carrillo-Larco, R. M., Aparcana-Granda, D. J., Mejia, J. R., Barengo, N. C., & Bernabé‐Ortiz, A. (2019) · Diabetic Medicine · doi:10.1111/dme.14114 · Cited by 32
Mao, S., Yu, X., Yang, Y., Shan, Y., Mugaanyi, J., Wu, S., & Lu, C. (2021) · Scientific Reports · doi:10.1038/s41598-021-93528-7 · Cited by 31
Gao, W., Ho, Y. K., Verne, J., Gordon, E., & Higginson, I. J. (2014) · Health Services and Delivery Research · doi:10.3310/hsdr02420 · Cited by 31
Foley, K., Shi, Z., Whybra, P., Kalendralis, P., Larue, R. T. H. M., Berbée, M., Sosef, M. N., Parkinson, C., Staffurth, J. N., Crosby, T. D. L., et al. (2018) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2018.10.033 · Cited by 31
Obokata, M., Negishi, K., Ohyama, Y., Okada, H., Imai, K., & Kurabayashi, M. (2015) · PLoS ONE · doi:10.1371/journal.pone.0133884 · Cited by 31
Statistical considerations on prognostic models for glioma
Molinaro, A. M., Wrensch, M., Jenkins, R. B., & Eckel‐Passow, J. E. (2015) · Neuro-Oncology · doi:10.1093/neuonc/nov255 · Cited by 31
Hampton, J. S., Kenny, R. P. W., Rees, C. J., Hamilton, W. T., Eastaugh, C., Richmond, C., & Sharp, L. (2023) · EClinicalMedicine · doi:10.1016/j.eclinm.2023.102204 · Cited by 30
Zhang, C., Yan, R., Liu, X., Nie, X., Peng, Y., & Peng, X. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.112101 · Cited by 31
Validation of a Pretransplant Risk Score for New-Onset Diabetes After Kidney Transplantation
Chakkera, H. A., Chang, Y.-H. H., Ayub, A., Gonwa, T. A., Weil, E. J., & Knowler, W. C. (2013) · Diabetes Care · doi:10.2337/dc13-0428 · Cited by 30
van Hoorn, F., Koster, M. P. H., Naaktgeboren, C. A., Groenendaal, F., Kwee, A., Ruiter, M. L.-D., Franx, A., & Bekker, M. N. (2020) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16446 · Cited by 30
Hoogendam, L., Bakx, J. A. C., Souer, J. S., Slijper, H. P., Andrinopoulou, E.-R., & Selles, R. W. (2021) · Neurosurgery · doi:10.1227/neu.0000000000001749 · Cited by 30
Predictive model for long COVID in children 3 months after a SARS-CoV-2 PCR test
Nugawela, M. D., Stephenson, T. J., Shafran, R., Stavola, B. L. D., Ladhani, S. N., Simmons, R., McOwat, K., Rojas, N. K., Dalrymple, E., Cheung, E. Y., et al. (2022) · BMC Medicine · doi:10.1186/s12916-022-02664-y · Cited by 30
Thromboxane biosynthesis and future events in diabetes: the ASCEND trial
Petrucci, G., Buck, G. A. N., Rocca, B., Parish, S., Baigent, C., Hatem, D., Mafham, M. M., Habib, A., Bowman, L. J., Armitage, J., et al. (2024) · European Heart Journal · doi:10.1093/eurheartj/ehad868 · Cited by 30
Echouffo-Tcheugui, J. B., & Kengne, A. P. (2013) · Diabetes & Metabolism · doi:10.1016/j.diabet.2013.07.002 · Cited by 30
Self-citationMassaut, J., Valles, P., Ghismonde, A., Jacques, C., Louis, L., Zakir, A., den Bergh, R. V., Santiague, L., Massenat, R., & Edema, N. (2017) · BMC Health Services Research · doi:10.1186/s12913-017-2541-4 · Cited by 30
Klompmaker, S., de Rooij, T., Koerkamp, B. G., Shankar, A. H., Siebert, U., Besselink, M. G., & Moser, A. J. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000003659 · Cited by 30
Greiff, G., Pleym, H., Stenseth, R., Berg, K. S., Wahba, A., & Videm, V. (2014) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2014.08.002 · Cited by 30
Gehringer, C. K., Martin, G. P., CALSTER, B. V., Hyrich, K. L., Verstappen, S. M. M., & Sergeant, J. C. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111481 · Cited by 30
Twelve practical recommendations for developing and applying clinical predictive models
Feng, G., Xu, H., Wan, S., Wang, H., Chen, X., Magari, R. T., Han, Y. K., Wei, Y., & Gu, H. (2024) · The Innovation Medicine · doi:10.59717/j.xinn-med.2024.100105 · Cited by 32
New directions in cardiovascular risk assessment: the role of secondary risk stratification markers
Wierzbicki, A. S. (2012) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2012.02956.x · Cited by 29
Ajnakina, O., Agbedjro, D., Lally, J., Forti, M. D., Trotta, A., Mondelli, V., Pariante, C. M., Dazzan, P., Gaughran, F., Fisher, H. L., et al. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2020.113527 · Cited by 30
Prediction Models for Bronchopulmonary Dysplasia in Preterm Infants: A Systematic Review
Peng, H., Zhan, Y.-L., Chen, Y., Jin, Z.-C., Liu, F., Wang, B., & Yu, Z. (2022) · Frontiers in Pediatrics · doi:10.3389/fped.2022.856159 · Cited by 30
Validation of a novel risk score to predict early and late recurrence in solitary fibrous tumour
Georgiesh, T., Pedersen, N. A., Schöffski, P., Zhang, Y., Napolitano, A., Bovée, J. V. M. G., Hjelle, Å., Tang, G., Spalek, M. J., Nannini, M., et al. (2022) · British Journal of Cancer · doi:10.1038/s41416-022-01959-4 · Cited by 29
Willemsen, A. C. H., Kok, A., van Kuijk, S. M. J., Baijens, L. W. J., de Bree, R., Devriese, L. A., Hoebers, F. J. P., Lalisang, R. I., Schols, A. M. W. J., Terhaard, C. H. J., et al. (2019) · Clinical Nutrition · doi:10.1016/j.clnu.2019.11.033 · Cited by 29
Models Predicting Psychosis in Patients With High Clinical Risk: A Systematic Review
Montemagni, C., Bellino, S., Bracale, N., Bozzatello, P., & Rocca, P. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00223 · Cited by 30
Rosenberg, N. E., Kudowa, E., Price, J. T., Pettifor, A. E., Bekker, L., Hosseinipour, M. C., & Chagomerana, M. (2020) · Sexually Transmitted Diseases · doi:10.1097/olq.0000000000001242 · Cited by 29
Development and Validation of a Prediction Model to Estimate Individual Risk of Pancreatic Cancer
Yu, A., Woo, S. M., Joo, J., Yang, H.-R., Lee, W. J., Park, S.-J., & Nam, B.-H. (2016) · PLOS ONE · doi:10.1371/journal.pone.0146473 · Cited by 28
“This approach is meant to supplement the reasoning and decision making of healthcare professionals by providing more objectively estimated probabilities [16].”
BackgroundPrediction models for self-harm and suicide: a systematic review and critical appraisal
Seyedsalehi, A., Bailey, J. P. M., Ogonah, M. G. T., Fanshawe, T. R., & Fazel, S. (2025) · BMC Medicine · doi:10.1186/s12916-025-04367-6 · Cited by 30
Koole, K., Clausen, M. J. A. M., van Es, R. J. J., van Kempen, P. M. W., Melchers, L. J., Koole, R., Langendijk, J. A., van Diest, P. J., Roodenburg, J. L. N., Schuuring, E. M. D., et al. (2016) · Molecular Diagnosis & Therapy · doi:10.1007/s40291-016-0204-5 · Cited by 28
Vucicevic, D., Honoris, L., Raia, F., & Deng, M. C.-H. K. (2018) · Annals of Cardiothoracic Surgery · doi:10.21037/acs.2017.12.01 · Cited by 28
Zhang, X., Neelam, K., Low, S. K., Ang, K., Yeo, D., Yeoh, L. Y., Liu, A. Y. L., Kwan, P. Y., Tang, W. E., Subramaniam, T., et al. (2018) · Diabetes and Vascular Disease Research · doi:10.1177/1479164118786969 · Cited by 28
Quan, J., Pang, D., Li, T. K., Choi, C. H., Siu, S. C., Tang, S. Y. H., Wat, N. M. S., Woo, J., Lau, Z. Y., Tan, K. B., et al. (2019) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2019-00731 · Cited by 28
Development and Validation of a Risk Prediction Model for Cesarean Delivery After Labor Induction
Danilack, V. A., Hutcheon, J. A., Triche, E. W., Dore, D. D., Muri, J. H., PHIPPS, M. G., & Savitz, D. A. (2019) · Journal of Women s Health · doi:10.1089/jwh.2019.7822 · Cited by 28
Goto, T., Yoshida, K., Faridi, M. K., Camargo, C. A., & Hasegawa, K. (2020) · BMC Pulmonary Medicine · doi:10.1186/s12890-020-1136-8 · Cited by 28
de la Torre, J., Marín, J., Ilarri, S., & Marín, J. J. (2020) · Applied Sciences · doi:10.3390/app10175942 · Cited by 28
Prediction Models for Gastrointestinal and Liver Diseases: Too Many Developed, Too Few Validated
Thrift, A. P., Kanwal, F., & El‐Serag, H. B. (2016) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2016.08.026 · Cited by 28
Metadata
- Authors
- Moons, K. G. M., Kengne, A. P., Woodward, M., Royston, P., Vergouwe, Y., Altman, D. G., & Grobbee, D. E.
- Affiliations
- University Medical Center Utrecht (NL)
- The University of Sydney (AU)
- MRC Clinical Trials Unit at UCL (GB)
- University of Oxford (GB)
- Corresponding author
- Moons, Karel G M
- Venue
- Heart
- Publisher
- BMJ
- Published
- 2012-03-07
- Volume
- 98
- Issue
- 9
- Pages
- 683-690
- ISSN
- 1355-6037 (ISSN-L)
- 1468-201X
- OpenAlex
- W2146404132
- Cited by
- 889
- References
- 100
- Citations by year
- 2026: 42 · 2025: 52 · 2024: 59 · 2023: 59 · 2022: 92 · 2021: 87 · 2020: 75 · 2019: 46 · 2018: 76 · 2017: 53 · 2016: 58 · 2015: 68 · 2014: 56 · 2013: 44 · 2012: 18
- FWCI
- 9.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Heart (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- Medical Research Council (MC_EX_G0800814)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- risk prediction model, model development, internal validation, optimism bias, cardiovascular risk prediction, predictive performance, clinical decision-making, cost-effectiveness
- MeSH terms
- Cardiovascular Diseases, Cost-Benefit Analysis, Decision Making, Decision Support Techniques, Humans, Models, Theoretical, Predictive Value of Tests, Risk, Reproducibility of Results, Biomarkers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.