Methodological conduct of prognostic prediction models developed using machine learning in oncology: a systematic review
Open accessAbstract
BACKGROUND: Describe and evaluate the methodological conduct of prognostic prediction models developed using machine learning methods in oncology. METHODS: We conducted a systematic review in MEDLINE and Embase between 01/01/2019 and 05/09/2019, for studies developing a prognostic prediction model using machine learning methods in oncology. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement, Prediction model Risk Of Bias ASsessment Tool (PROBAST) and CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) to assess the methodological conduct of included publications. Results were summarised by modelling type: regression-, non-regression-based and ensemble machine learning models. RESULTS: Sixty-two publications met inclusion criteria developing 152 models across all publications. Forty-two models were regression-based, 71 were non-regression-based and 39 were ensemble models. A median of 647 individuals (IQR: 203 to 4059) and 195 events (IQR: 38 to 1269) were used for model development, and 553 individuals (IQR: 69 to 3069) and 50 events (IQR: 17.5 to 326.5) for model validation. A higher number of events per predictor was used for developing regression-based models (median: 8, IQR: 7.1 to 23.5), compared to alternative machine learning (median: 3.4, IQR: 1.1 to 19.1) and ensemble models (median: 1.7, IQR: 1.1 to 6). Sample size was rarely justified (n = 5/62; 8%). Some or all continuous predictors were categorised before modelling in 24 studies (39%). 46% (n = 24/62) of models reporting predictor selection before modelling used univariable analyses, and common method across all modelling types. Ten out of 24 models for time-to-event outcomes accounted for censoring (42%). A split sample approach was the most popular method for internal validation (n = 25/62, 40%). Calibration was reported in 11 studies. Less than half of models were reported or made available. CONCLUSIONS: The methodological conduct of machine learning based clinical prediction models is poor. Guidance is urgently needed, with increased awareness and education of minimum prediction modelling standards. Particular focus is needed on sample size estimation, development and validation analysis methods, and ensuring the model is available for independent validation, to improve quality of machine learning based clinical prediction models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Calibration measures are still under-reported.
“Unfortunately, calibration measures are still under-reported.”
From Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance (2025) · cites this paper as [94]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
1 retracted
Alcantud, J. C. R., Varela, G., Santos-Buitrago, B., Santos‐García, G., & Jiménez, M. F. (2019) · PLoS ONE · doi:10.1371/journal.pone.0218283 · Cited by 41 · see citing works
RetractedLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823 · see citing works
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · see citing works
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.08.010 · Cited by 51,384 · see citing works
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092 · see citing works
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,759 · see citing works
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · see citing works
Sterne, J. A. C., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., Wood, A. M., & Carpenter, J. R. (2009) · BMJ · doi:10.1136/bmj.b2393 · Cited by 7,299 · see citing works
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878 · see citing works
Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project
Conroy, R. (2003) · European Heart Journal · doi:10.1016/s0195-668x(03)00114-3 · Cited by 5,290 · see citing works
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147 · see citing works
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · see citing works
The APACHE III Prognostic System
Knaus, W. A., Wagner, D. P., Draper, E. A., Zimmerman, J. E., Bergner, M., BASTOS, P. G., Sirio, C., Murphy, D. J., Lotring, T., Damiano, A., et al. (1991) · CHEST Journal · doi:10.1378/chest.100.6.1619 · Cited by 3,524 · see citing works
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281 · see citing works
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267 · see citing works
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984 · see citing works
Nashef, S. A. M., Roques, F., Sharples, L. D., Nilsson, J., Smith, C., Goldstone, A. R., & Lockowandt, U. (2012) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezs043 · Cited by 2,971 · see citing works
Calculating the sample size required for developing a clinical prediction model
Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M. (2020) · BMJ · doi:10.1136/bmj.m441 · Cited by 2,917 · see citing works
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119 · see citing works
Self-citationMoons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018 · see citing works
Self-citationHippisley-Cox, J., Coupland, C., & Brindle, P. (2017) · BMJ · doi:10.1136/bmj.j2099 · Cited by 1,703 · see citing works
Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1377 · Cited by 1,669 · see citing works
Self-citationA Preoperative Nomogram for Disease Recurrence Following Radical Prostatectomy for Prostate Cancer
Kattan, M. W., Eastham, J. A., Stapleton, A. M. F., Wheeler, T. M., & Scardino, P. T. (1998) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/90.10.766 · Cited by 1,243 · see citing works
Machine Learning and Prediction in Medicine — Beyond the Peak of Inflated Expectations
Chen, J. H., & Asch, S. M. (2017) · New England Journal of Medicine · doi:10.1056/nejmp1702071 · Cited by 1,171 · see citing works
Nagendran, M., Chen, Y., Lovejoy, C. A., Gordon, A. C., Komorowski, M., Harvey, H., Topol, E. J., Ioannidis, J. P. A., Collins, G. S., & Maruthappu, M. (2020) · BMJ · doi:10.1136/bmj.m689 · Cited by 1,152 · see citing works
Prediction models for cardiovascular disease risk in the general population: systematic review
Damen, J. A. A. G., Hooft, L., Schuit, E., Debray, T. P. A., Collins, G. S., Tzoulaki, I., Lassale, C. M., Siontis, G. C. M., Chiocchia, V., Roberts, C., et al. (2016) · BMJ · doi:10.1136/bmj.i2416 · Cited by 935 · see citing works
Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis
Sun, G.-W., Shook, T. L., & Kay, G. L. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(96)00025-x · Cited by 933 · see citing works
Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints
van der Ploeg, T., Austin, P. C., & Steyerberg, E. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-137 · Cited by 809 · see citing works
Reporting of artificial intelligence prediction models
Collins, G. S., & Moons, K. G. M. (2019) · The Lancet · doi:10.1016/s0140-6736(19)30037-6 · Cited by 701 · see citing works
Reporting and Methods in Clinical Prediction Research: A Systematic Review
Bouwmeester, W., Zuithoff, N. P. A., Mallett, S., Geerlings, M. I., Vergouwe, Y., Steyerberg, E. W., Altman, D. G., & Moons, K. G. M. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001221 · Cited by 600 · cited 2× · see citing works
“However, compared to the number of prediction models that are developed, very few are used in clinical practice and many models contribute to research waste [14–17].”
BackgroundNo rationale for 1 variable per 10 events criterion for binary logistic regression analysis
van Smeden, M., de Groot, J. D., Moons, K., Collins, G., Altman, D., Eijkemans, M., & Reitsma, J. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0267-3 · Cited by 563 · see citing works
Groenwold, R. H. H., White, I. R., Donders, A. R. T., Carpenter, J. R., Altman, D. G., & Moons, K. G. M. (2012) · Canadian Medical Association Journal · doi:10.1503/cmaj.110977 · Cited by 538 · see citing works
Self-citationCollins, G. S., Mallett, S., Omar, O., & Yu, L.-M. (2011) · BMC Medicine · doi:10.1186/1741-7015-9-103 · Cited by 484 · see citing works
Andaur Navarro, C. L., Damen, J. A. A., Takada, T., Nijman, S. W. J., Dhiman, P., Ma, J., Collins, G. S., Bajpai, R., Riley, R. D., Moons, K. G. M., et al. (2021) · BMJ · doi:10.1136/bmj.n2281 · Cited by 415 · see citing works
Deep learning-based survival prediction of oral cancer patients
Kim, D. W., Lee, S., Kwon, S., Nam, W., Cha, I., & Kim, H. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-43372-7 · Cited by 324 · see citing works
Song, X., Liu, X., Liu, F., & Wang, C. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104484 · Cited by 301 · see citing works
Balki, I., Amirabadi, A., Levman, J., Martel, A. L., Emersic, Z., Meden, B., Garcia-Pedrero, A., Ramirez, S. C., Kong, D., Moody, A. R., et al. (2019) · Canadian Association of Radiologists Journal · doi:10.1016/j.carj.2019.06.002 · Cited by 299 · see citing works
Use of machine learning to analyse routinely collected intensive care unit data: a systematic review
Shillan, D., Sterne, J. A. C., Champneys, A., & Gibbison, B. (2019) · Critical Care · doi:10.1186/s13054-019-2564-9 · Cited by 251 · see citing works
Prognostic models for breast cancer: a systematic review
Phung, M. T., Tin, S. T., & Elwood, M. (2019) · BMC Cancer · doi:10.1186/s12885-019-5442-6 · Cited by 214 · see citing works
Reporting methods in studies developing prognostic models in cancer: a review
Mallett, S., Royston, P., Dutton, S., Waters, R., & Altman, D. G. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-20 · Cited by 206 · see citing works
Wang, W., Kiik, M., Peek, N. B., Ćurčin, V., Marshall, I. J., Rudd, A. G., Wang, Y., Douiri, A., Wolfe, C. D. A., & Bray, B. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234722 · Cited by 205 · see citing works
Collins, G. S., Omar, O., Shanyinde, M., & Yu, L.-M. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.06.020 · Cited by 192 · see citing works
Knol, M. J., Janssen, K. J. M., Donders, A. R. T., Egberts, T. C. G., Heerdink, E. R., Grobbee, D. E., Moons, K. G. M., & Geerlings, M. I. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.08.028 · Cited by 190 · see citing works
Collins, G. S., Ogundimu, E. O., Cook, J. A., Manach, Y. L., & Altman, D. G. (2016) · Statistics in Medicine · doi:10.1002/sim.6986 · Cited by 185 · see citing works
Risk Prediction Models for Colorectal Cancer: A Systematic Review
Usher‐Smith, J. A., Walter, F. M., Emery, J. D., Win, A. K., & Griffin, S. J. (2015) · Cancer Prevention Research · doi:10.1158/1940-6207.capr-15-0274 · Cited by 182 · see citing works
Survival outcome prediction in cervical cancer: Cox models vs deep-learning model
Matsuo, K., Purushotham, S., Jiang, B., Mandelbaum, R. S., Takiuchi, T., Liu, Y., & Roman, L. D. (2018) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2018.12.030 · Cited by 179 · see citing works
Karhade, A., Thio, Q., Ogink, P., Bono, C. M., Ferrone, M., Oh, K., Saylor, P., Schoenfeld, A. J., Shin, J. H., Harris, M. B., et al. (2019) · Neurosurgery · doi:10.1093/neuros/nyz070 · Cited by 179 · see citing works
Mushkudiani, N. A., Hukkelhoven, C. W. P. M., Hernández, A. V., Murray, G. D., Choi, S. C., Maas, A. I. R., & Steyerberg, E. W. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.06.011 · Cited by 177 · see citing works
Hippisley‐Cox, J., & Coupland, C. A. C. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-007825 · Cited by 177 · see citing works
Karhade, A., Thio, Q., Ogink, P., Shah, A. A., Bono, C. M., Oh, K., Saylor, P., Schoenfeld, A. J., Shin, J. H., Harris, M. B., et al. (2018) · Neurosurgery · doi:10.1093/neuros/nyy469 · Cited by 151 · see citing works
Thamer, M., Kaufman, J. S., Zhang, Y., Zhang, Q., Cotter, D. J., & Bang, H. (2015) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2015.05.014 · Cited by 140 · see citing works
Predicting Breast Cancer Recurrence Using Machine Learning Techniques
Abreu, P. H., Santos, M. S., Abreu, M. H., Andrade, B., & Silva, D. C. (2016) · ACM Computing Surveys · doi:10.1145/2988544 · Cited by 135 · see citing works
Sauerbrei, W., Boulesteix, A., & Binder, H. (2011) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2011.629890 · Cited by 134 · see citing works
Merath, K., Hyer, J. M., Mehta, R., Farooq, A., Bagante, F., Sahara, K., Tsilimigras, D. I., Beal, E., Paredes, A. Z., Wu, L., et al. (2020) · Journal of Gastrointestinal Surgery · doi:10.1007/s11605-019-04338-2 · Cited by 131 · see citing works
Predicting breast cancer 5-year survival using machine learning: A systematic review
Li, J., Zhou, Z., Dong, J., Fu, Y., Li, Y., Luan, Z., & Peng, X. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250370 · Cited by 123 · see citing works
Taninaga, J., Nishiyama, Y., Fujibayashi, K., Gunji, T., Sasabe, N., Iijima, K., & Naito, T. (2019) · Scientific Reports · doi:10.1038/s41598-019-48769-y · Cited by 120 · see citing works
Tseng, Y., Huang, C.-E., Wen, C.-N., Lai, P.-Y., Wu, M., Sun, Y.-C., Wang, H., & Lu, J. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.05.003 · Cited by 117 · see citing works
Karadaghy, O. A., Shew, M. A., New, J., & Bur, A. M. (2019) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2019.0981 · Cited by 114 · see citing works
Alabi, R. O., Elmusrati, M. S., Sawazaki‐Calone, Í., Kowalski, L. P., Haglund, C. H., Coletta, R. D., Mäkitie, A. A., Salo, T. A., Leivo, I. V. V., & Almangush, A. (2019) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-019-02642-5 · Cited by 109 · see citing works
Dhiman, P., Ma, J., Navarro, C. L. A., Speich, B., Bullock, G. S., Damen, J. A., Kirtley, S., Hooft, L., Riley, R. D., CALSTER, B. V., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.06.024 · Cited by 109 · see citing works
Heus, P., Damen, J. A., Pajouheshnia, R., Scholten, R. J. P. M., Reitsma, J. B., Collins, P. G. S., Altman, D. G., Moons, K. G. M., & Hooft, L. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-025611 · Cited by 108 · see citing works
Use of machine learning to predict early biochemical recurrence after robot‐assisted prostatectomy
Wong, N. C., Lam, C. J., Patterson, L., & Shayegan, B. (2018) · British Journal of Urology · doi:10.1111/bju.14477 · Cited by 101 · see citing works
Luna, J. M., Chao, H., Diffenderfer, E. S., Valdés, G., Chinniah, C., Grace, X., Cengel, K. A., Solberg, T. D., Berman, A. T., & Simone, C. B. (2019) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2019.01.003 · Cited by 99 · see citing works
Fahey, M., Crayton, E., Wolfe, C. D. A., & Douiri, A. (2018) · PLoS ONE · doi:10.1371/journal.pone.0185402 · Cited by 96 · see citing works
Extent of Resection in Meningioma: Predictive Factors and Clinical Implications
D, J.-M. L. M., Corniola, M. V., Broi, M. D., Joswig, H., Scheie, D., Schaller, K., Helseth, E., & Meling, T. R. (2019) · Scientific Reports · doi:10.1038/s41598-019-42451-z · Cited by 96 · see citing works
A risk score for predicting perioperative blood transfusion in liver surgery
Pulitanò, C., Arru, M., Bellio, L., Rossini, S., Ferla, G., & Aldrighetti, L. A. (2007) · British journal of surgery · doi:10.1002/bjs.5731 · Cited by 86 · see citing works
Muhlestein, W. E., Akagi, D. S., Davies, J. M., & Chambless, L. B. (2018) · Neurosurgery · doi:10.1093/neuros/nyy343 · Cited by 84 · see citing works
Shung, D. L., Simonov, M., Gentry, M., Au, B., & Laine, L. (2019) · Digestive Diseases and Sciences · doi:10.1007/s10620-019-05645-z · Cited by 79 · see citing works
Banerjee, A., Chen, S., Fatemifar, G., Zeina, M., Lumbers, R. T., Mielke, J., Gill, S. K., Kotecha, D., Freitag, D. F., Denaxas, S., et al. (2021) · BMC Medicine · doi:10.1186/s12916-021-01940-7 · Cited by 78 · see citing works
Sahle, B. W., Owen, A. J., Chin, K. L., & Reid, C. M. (2017) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2017.03.005 · Cited by 77 · see citing works
Paik, E. S., Lee, J., Park, J., Kim, J.-H., Kim, M., Kim, T., Choi, C. H., Kim, B., Bae, D., & Seo, S. W. (2019) · Journal of Gynecologic Oncology · doi:10.3802/jgo.2019.30.e65 · Cited by 75 · see citing works
Wang, H.-H., Wang, Y.-H., Liang, C.-W., & Li, Y. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.2335 · Cited by 75 · see citing works
Markaki, M., Tsamardinos, I., Langhammer, A., Lagani, V., Hveem, K., & Røe, O. D. (2018) · EBioMedicine · doi:10.1016/j.ebiom.2018.03.027 · Cited by 71 · see citing works
Fong, Y., Evans, J., Brook, D., Kenkre, J. E., Jarvis, P., & Gower-Thomas, K. (2015) · Annals of The Royal College of Surgeons of England · doi:10.1308/003588414x14055925060514 · Cited by 66 · see citing works
Wang, X., Zhang, Y., Hao, S., Zheng, L., Liao, J., Ye, C., Xia, M., Wang, O., Liu, M., Weng, C. H., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/13260 · Cited by 62 · see citing works
Auffenberg, G. B., Ghani, K. R., Ramani, S., Usoro, E., Denton, B. T., ROGERS, C. G., Stockton, B. R., Miller, D. C., & Singh, K. (2018) · European Urology · doi:10.1016/j.eururo.2018.09.050 · Cited by 61 · see citing works
Khalaf, M. H., H, V. S. M. P., Abdelrazek, M. A., Shah, R. P., Khosla, A., Jackson, K., Desai, M., & Kothary, N. (2018) · Radiology · doi:10.1148/radiol.2018180257 · Cited by 58 · see citing works
Use of the Gail Model and Breast Cancer Preventive Therapy Among Three Primary Care Specialties
Corbelli, J. A., Borrero, S. B., Bonnema, R. A., McNamara, M., Kraemer, K. L., Rubio, D. M., Karpov, I. O., & McNeil, M. (2014) · Journal of Women s Health · doi:10.1089/jwh.2014.4742 · Cited by 57 · see citing works
Lebrett, M. B., Balata, H., Evison, M., Colligan, D., Duerden, R., Elton, P., Greaves, M., Howells, J., Irion, K., Karunaratne, D., et al. (2020) · Thorax · doi:10.1136/thoraxjnl-2020-214626 · Cited by 57 · see citing works
A novel prediction method for lymph node involvement in endometrial cancer: machine learning
Günakan, E., Suat, A. T. A. N., Haberal, N., Küçükyıldız, İ. A., Gökçe, E., & Ayhan, A. (2019) · International Journal of Gynecological Cancer · doi:10.1136/ijgc-2018-000033 · Cited by 50 · see citing works
Nartowt, B. J., Hart, G. R., Roffman, D. A., Llor, X., Ali, I. M., Muhammad, W., Liang, Y., & Deng, J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0221421 · Cited by 50 · see citing works
Oyaga-Iriarte, E., Insausti, A., Sayar, O., & Aldaz, A. (2019) · Journal of Pharmacological Sciences · doi:10.1016/j.jphs.2019.03.004 · Cited by 49 · see citing works
van Niftrik, C. H. B., van der Wouden, F., Staartjes, V. E., Fierstra, J., Stienen, M. N., Akeret, K., Sebök, M., Fedele, T., Sarnthein, J., Bozinov, O., et al. (2019) · Neurosurgery · doi:10.1093/neuros/nyz145 · Cited by 49 · see citing works
Xu, Y., Kong, S., Cheung, W. Y., Bouchard‐Fortier, A., Dort, J. C., Quan, H., Buie, E. M., McKinnon, G., & Quan, M. L. (2019) · BMC Cancer · doi:10.1186/s12885-019-5432-8 · Cited by 46 · see citing works
Artificial neural network models to predict nodal status in clinically node-negative breast cancer
Dihge, L., Ohlsson, M., Edén, P., Bendahl, P., & Rydén, L. (2019) · BMC Cancer · doi:10.1186/s12885-019-5827-6 · Cited by 44 · see citing works
Development of Deep Learning Algorithm for Detection of Colorectal Cancer in EHR Data
Yu-Hsiang, W., Phung-Anh, N., Mohaimenul, I. M., Li, Y., & Yang, H. (2019) · Studies in health technology and informatics · doi:10.3233/shti190259 · Cited by 41 · see citing works
Yan, P., Huang, R., Hu, P., Liu, F., Zhu, X., Hu, P., Yin, H., Zhang, J., Meng, T., & Huang, Z. (2019) · Journal of Neuro-Oncology · doi:10.1007/s11060-019-03181-4 · Cited by 40 · see citing works
Ryu, S. M., Lee, S.-H., Kim, E., & Eoh, W. (2018) · World Neurosurgery · doi:10.1016/j.wneu.2018.12.091 · Cited by 39 · see citing works
Creating Prognostic Systems for Well-Differentiated Thyroid Cancer Using Machine Learning
Yang, C., Gardiner, L. A., Wang, H., Hueman, M. T., & Chen, D. (2019) · Frontiers in Endocrinology · doi:10.3389/fendo.2019.00288 · Cited by 39 · see citing works
Kaviarasi, R., & Gandhi Raj, R. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1297-2 · Cited by 39 · see citing works
Semi-supervised learning to improve generalizability of risk prediction models
Chi, S., Li, X., Tian, Y., Li, J., Kong, X., Ding, K., Weng, C., & Li, J. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103117 · Cited by 38 · see citing works
Elarre, P. S., Oyaga-Iriarte, E., Yu, K. H., Baudin, V., Moreno, L. A., Carranza, O. E., Ortega, A. C., Ponz‐Sarvisé, M., Sosa, L., Rotellar, F., et al. (2019) · Cancers · doi:10.3390/cancers11050606 · Cited by 38 · see citing works
Bridge, J., Blakey, J., & Bonnett, L. J. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-0913-7 · Cited by 37 · see citing works
Fuse, K., Uemura, S., Tamura, S., Suwabe, T., Katagiri, T., Tanaka, T. N., Ushiki, T., Shibasaki, Y., Sato, N., Yano, T., et al. (2019) · Cancer Medicine · doi:10.1002/cam4.2401 · Cited by 34 · see citing works
Collins, S. D., Peek, N. B., Riley, R. D., & Martin, G. P. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.12.011 · Cited by 31 · see citing works
Tian, Z., Yen, A., Zhou, Z., Shen, C., Albuquerque, K., & Hrycushko, B. A. (2019) · Brachytherapy · doi:10.1016/j.brachy.2019.04.004 · Cited by 30 · see citing works
A prospective study examining cachexia predictors in patients with incurable cancer
Vagnildhaug, O. M., Brunelli, C., Hjermstad, M. J., Strasser, F., Baracos, V. E., Wilcock, A., Nabal, M., Kaasa, S., Laird, B. J. A., & Solheim, T. S. (2019) · BMC Palliative Care · doi:10.1186/s12904-019-0429-2 · Cited by 30 · see citing works
Facciorusso, A., Del Prete, V., Antonino, M., Buccino, V. R., & Muscatiello, N. (2019) · Pancreatology · doi:10.1016/j.pan.2019.07.038 · Cited by 30 · see citing works
Egger, M. E., Stevenson, M. A., Bhutiani, N., Jordan, A. C., Scoggins, C. R., Philips, P., Martin, R. C. G., & McMasters, K. M. (2019) · Annals of Surgical Oncology · doi:10.1245/s10434-019-07690-4 · Cited by 28 · see citing works
Greene, M. Z., Hughes, T. L., Hanlon, A. L., Huang, L., Sommers, M. S., & Meghani, S. H. (2018) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2018.11.007 · Cited by 27 · see citing works
Survivability prediction of colon cancer patients using neural networks
Al-Bahrani, R., Agrawal, A., & Choudhary, A. (2017) · Health Informatics Journal · doi:10.1177/1460458217720395 · Cited by 26 · see citing works
Gait speed and survival of older surgical patient with cancer: Prediction after machine learning
Sasani, K., Catanese, H. N., Ghods, A., Rokni, S. A., Ghasemzadeh, H. G., Downey, R. J., & Shahrokni, A. (2018) · Journal of Geriatric Oncology · doi:10.1016/j.jgo.2018.06.012 · Cited by 25 · see citing works
Corniola, M. V., D, J.-M. L. M., Broi, M. D., Joswig, H., Schaller, K., Helseth, E., & Meling, T. R. (2019) · Acta Neurochirurgica · doi:10.1007/s00701-019-03862-z · Cited by 25 · see citing works
Feng, S., Li, H.-B., Fan, F., Li, J., Cao, H., Xia, Z., Yang, K., Zhu, X., Cheng, T., & Cheng, Q. (2019) · Scientific Reports · doi:10.1038/s41598-019-47211-7 · Cited by 25 · see citing works
Can machine learning predict resecability of a peritoneal carcinomatosis?
Maubert, A., Birtwisle, L., Bernard, J., Benizri, E. I., & Bereder, J. M. (2019) · Surgical Oncology · doi:10.1016/j.suronc.2019.04.008 · Cited by 24 · see citing works
Predicting Overall Survival in Patients with Metastatic Rectal Cancer: a Machine Learning Approach
Zhao, B., Gabriel, R. A., Vaida, F., Lopez, N., Eisenstein, S. G., & Clary, B. M. (2019) · Journal of Gastrointestinal Surgery · doi:10.1007/s11605-019-04373-z · Cited by 24 · see citing works
Grigore, B., Lewis, R. A., Peters, J., Robinson, S., & Hyde, C. (2020) · BMC Cancer · doi:10.1186/s12885-020-07572-z · Cited by 23 · see citing works
Bradley, A., Meer, R. V. D., & McKay, C. J. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-027192 · Cited by 19 · see citing works
Li, M., Zhan, C., Sui, X., Jiang, W., Shi, Y., Yang, X., Feng, M., Wang, J., & Wang, Q. (2019) · Frontiers in Oncology · doi:10.3389/fonc.2019.00771 · Cited by 16 · see citing works
Tighe, D. F., Lewis-Morris, T., & Freitas, A. A. (2019) · British Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.bjoms.2019.05.026 · Cited by 16 · see citing works
Velázquez, N., Press, B. H., Renson, A., Wysock, J. S., Taneja, S. S., Huang, W. C. W., & Bjurlin, M. A. (2018) · Clinical Genitourinary Cancer · doi:10.1016/j.clgc.2018.09.018 · Cited by 15 · see citing works
Beachler, D. C., de Luise, C., Yin, R., Gangemi, K., Cochetti, P. T., & Lanes, S. (2018) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4681 · Cited by 15 · see citing works
Obrzut, B., Kusy, M., Semczuk, A., Obrzut, M., & Kluska, J. (2019) · Journal of Cancer · doi:10.7150/jca.33945 · Cited by 14 · see citing works
Hammer, J., Geinitz, H., Nieder, C., Track, C., Thames, H. D., Seewald, D. H., Petzer, A. L., Helfgott, R., Spiegl, K. J., Heck, D., et al. (2018) · Clinical Breast Cancer · doi:10.1016/j.clbc.2018.08.001 · Cited by 13 · see citing works
Lindsay, W. D., Ahern, C. A., Tobias, J. S., Berlind, C. G., Chinniah, C., Gabriel, P. E., Gee, J. C., & Simone, C. B. (2018) · Medical Physics · doi:10.1002/mp.13314 · Cited by 11 · see citing works
Yang, X., Wang, F., Feng, J., Hu, Y., Lun, D., Hua, K., Liu, Y.-H., Zhang, H., & Yang, L. (2019) · World Neurosurgery · doi:10.1016/j.wneu.2019.02.183 · Cited by 9 · see citing works
Zheng, B., Lin, J., Li, Y., Zhuo, X., Huang, X., Shen, Q., & Tang, Y. (2019) · Supportive Care in Cancer · doi:10.1007/s00520-019-04699-z · Cited by 7 · see citing works
Iraji, M. S. (2019) · Journal of Applied Biomedicine · doi:10.32725/jab.2018.007 · Cited by 7 · see citing works
Thapa, S., Fischbach, L. A., Delongchamp, R. R., Faramawi, M. F., & Orloff, M. S. (2019) · Gastroenterology Research and Practice · doi:10.1155/2019/8321942 · Cited by 7 · see citing works
Mahmoudian, M., Seyednasrollah, F. S., Koivu, L., Hirvonen, O., Jyrkkiö, S. M., & Elo, L. L. (2019) · F1000Research · doi:10.12688/f1000research.8192.2 · Cited by 5 · see citing works
Zhou, H.-F., Lü, J., Zhu, H., Guo, J., Huang, M. X., Ji, J., Lv, W., Li, Y., Xu, H., Chen, L., et al. (2019) · CardioVascular and Interventional Radiology · doi:10.1007/s00270-019-02331-5 · Cited by 5 · see citing works
Sim, J.-A., & Yun, Y. H. (2019) · Studies in health technology and informatics · doi:10.3233/shti190548 · Cited by 5 · see citing works
Mahmoudian, M., Seyednasrollah, F. S., Koivu, L., Hirvonen, O., Jyrkkiö, S. M., & Elo, L. L. (2016) · F1000Research · doi:10.12688/f1000research.8192.1 · Cited by 5 · see citing works
Xu, Y., Kong, S., Cheung, W. Y., Quan, M. L., Nakoneshny, S. C., & Dort, J. C. (2019) · Head & Neck · doi:10.1002/hed.25682 · Cited by 4 · see citing works
Flagged first, then by citations
1 of the 126 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-078378 · Cited by 4,718 · see citing works
“…models have shown that they are often poorly conducted (including deficiencies in study design or data collection 37 38 ); use poor methodology 37 38 ; are incompletely reported with key details missing 39-54 ; are consequently at high risk of bias 41 49 55-57 ; rarely adhere to open science…”
Method(2024) · BMJ · doi:10.1136/bmj.q902 · Cited by 416 · see citing works
“…models have shown that they are often poorly conducted (including deficiencies in study design or data collection 37 38 ); use poor methodology 37 38 ; are incompletely reported with key details missing 39-54 ; are consequently at high risk of bias 41 49 55-57 ; rarely adhere to open science…”
Stability of clinical prediction models developed using statistical or machine learning methods
Riley, R. D., & Collins, G. S. (2023) · Biometrical Journal · doi:10.1002/bimj.202200302 · Cited by 211 · see citing works
APPRAISE-AI Tool for Quantitative Evaluation of AI Studies for Clinical Decision Support
Kwong, J. C. C., Khondker, A., Lajkosz, K., McDermott, M. B. A., Borrat, X., McCradden, M. D., Mamdani, M., Kulkarni, G. S., & Johnson, A. E. W. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.35377 · Cited by 151 · see citing works
Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, P. G. S., & Barnett, A. G. (2023) · BMC Medicine · doi:10.1186/s12916-023-03048-6 · Cited by 149 · see citing works
Andaur Navarro, C. L., Damen, J. A. A., van Smeden, M., Takada, T., Nijman, S. W. J., Dhiman, P., Ma, J., Collins, G. S., Bajpai, R., Riley, R. D., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.11.015 · Cited by 121 · see citing works
Self-citationVan Calster, B., Collins, G. S., Vickers, A. J., Wynants, L., Kerr, K. F., Barreñada, L., Varoquaux, G., Singh, K., Moons, K. G., Hernandez-Boussard, T., et al. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100916 · Cited by 112 · see citing works
Dhiman, P., Ma, J., Qi, C., Bullock, G. S., Sergeant, J. C., Riley, R. D., & Collins, P. G. S. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-02008-1 · Cited by 99 · see citing works
Importance of sample size on the quality and utility of AI-based prediction models for healthcare
Riley, R. D., Ensor, J., Snell, K. I. E., Archer, L., Whittle, R., Dhiman, P., Alderman, J. E., Liu, X., Kirton, L., Manson-Whitton, J., et al. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.01.013 · Cited by 93 · see citing works
López‐Ayala, P., Riley, R. D., Collins, P. G. S., & Zimmermann, T. (2025) · BMJ · doi:10.1136/bmj-2024-082440 · Cited by 88 · see citing works
Liu, W., Laranjo, L., Klimis, H., Chiang, J., Yue, J., Marschner, S. L., Quiroz, J. C., Jorm, L. R., & Chow, C. K. (2023) · European Heart Journal - Quality of Care and Clinical Outcomes · doi:10.1093/ehjqcco/qcad017 · Cited by 84 · see citing works
Ma, J., Dhiman, P., Qi, C., Bullock, G. S., van Smeden, M., Riley, R. D., & Collins, P. G. S. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.07.017 · Cited by 78 · see citing works
The Improved Kidney Risk Score in ANCA-Associated Vasculitis for Clinical Practice and Trials
Bate, S. G., McGovern, D., Costigliolo, F., Tan, P. G., Krátký, V., Scott, J., Chapman, G. B., Brown, N., Floyd, L., Brilland, B., et al. (2023) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000000000274 · Cited by 78 · see citing works
Artificial intelligence in ovarian cancer histopathology: a systematic review
Breen, J., Allen, K., Zucker, K., Adusumilli, P., Scarsbrook, A. F., Hall, G. D., Orsi, N. M., & Ravikumar, N. (2023) · npj Precision Oncology · doi:10.1038/s41698-023-00432-6 · Cited by 75 · see citing works
Uncertainty of risk estimates from clinical prediction models: rationale, challenges, and approaches
Riley, R. D., Collins, P. G. S., Kirton, L., Snell, K. I. E., Ensor, J., Whittle, R., Dhiman, P., van Smeden, M., Liu, X., Alderman, J. E., et al. (2025) · BMJ · doi:10.1136/bmj-2024-080749 · Cited by 54 · see citing works
AI in imaging: the regulatory landscape
Hill, D. L. G. (2024) · British Journal of Radiology · doi:10.1093/bjr/tqae002 · Cited by 52 · see citing works
dos Santos, C. F., & Amorim‐Lopes, M. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02463-y · Cited by 59 · see citing works
Digital twins in oncology: From predictive modelling to personalised treatment strategies
Olawade, D. B., Oisakede, E. O., Bello, O. J., Analikwu, C. C., Egbon, E., & Ojo, A. (2026) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2026.105171 · Cited by 47 · see citing works
Dhiman, P., Ma, J., Andaur Navarro, C. L., Speich, B., Bullock, G., Damen, J. A. A., Hooft, L., Kirtley, S., Riley, R. D., Van Calster, B., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.03.012 · Cited by 41 · see citing works
Self-citationTsegaye, B., Snell, K. I. E., Archer, L., Kirtley, S., Riley, R. D., Sperrin, M., CALSTER, B. V., Collins, P. G. S., & Dhiman, P. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111675 · Cited by 36 · see citing works
Troiano, G., Nibali, L., Petsos, H., Eickholz, P., Saleh, M. H. A., Santamaria, P., Jiao, J., Shi, S., Meng, H., Zhurakivska, K., et al. (2022) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.13739 · Cited by 35 · see citing works
Kom, I. Y., Dongelmans, D. A., de Keizer, N. F., Jager, K. J., Schut, M. C., Abu–Hanna, A., & Klopotowska, J. E. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad014 · Cited by 34 · see citing works
Kwong, J. C. C., Wu, J., Malik, S., Khondker, A., Gupta, N. K., Bodnariuc, N., Narayana, K., Malik, M., van der Kwast, T., Johnson, A. E. W., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01088-7 · Cited by 35 · see citing works
Kastrup, N., Holst‐Kristensen, A. W., & Valentin, J. B. (2024) · BMC Digital Health · doi:10.1186/s44247-024-00088-7 · Cited by 34 · see citing works
Li, L. K., Pan, H., Liang, Y., Shao, M., Xie, S., Lu, S., & Liao, S. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111502 · Cited by 30 · see citing works
Li, H., Sun, Y., & Qiao, S. (2025) · Measurement · doi:10.1016/j.measurement.2025.116762 · Cited by 29 · see citing works
Zaka, A., Mutahar, D., Gorcilov, J., Gupta, A. K., Kovoor, J. G., Stretton, B., Mridha, N., Sivagangabalan, G., Thiagalingam, A., Chow, C. K., et al. (2024) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztae074 · Cited by 28 · see citing works
Zhong, J., Liu, X., Lu, J. A., Yang, J., Zhang, G., Mao, S., Chen, H., Yin, Q., Cen, Q., Jiang, R., et al. (2025) · European Radiology · doi:10.1007/s00330-024-11331-0 · Cited by 28 · see citing works
Akano, T. T., & James, C. C. (2022) · Beni-Suef University Journal of Basic and Applied Sciences · doi:10.1186/s43088-022-00327-8 · Cited by 27 · see citing works
Twelve practical recommendations for developing and applying clinical predictive models
Feng, G., Xu, H., Wan, S., Wang, H., Chen, X., Magari, R. T., Han, Y. K., Wei, Y., & Gu, H. (2024) · The Innovation Medicine · doi:10.59717/j.xinn-med.2024.100105 · Cited by 27 · see citing works
Clinical prediction models using machine learning in oncology: challenges and recommendations
Collins, G. S., Chester-Jones, M., Gerry, S., Ma, J., Matos, J., Sehjal, J., Tsegaye, B., & Dhiman, P. (2025) · BMJ Oncology · doi:10.1136/bmjonc-2025-000914 · Cited by 27 · see citing works
Yoong, S. Q., Porock, D. C., Whitty, D., Tam, W., & Zhang, H. (2023) · Palliative Medicine · doi:10.1177/02692163231180657 · Cited by 22 · see citing works
Nayebirad, S., Hassanzadeh, A., Vahdani, A. M., Mohamadi, A., Forghani, S., Shafiee, A., & Masoudkabir, F. (2025) · BMC Cardiovascular Disorders · doi:10.1186/s12872-025-04746-0 · Cited by 22 · see citing works
Kwong, J. C. C., Khondker, A., Meng, E., Taylor, N., Kuk, C., Perlis, N., Kulkarni, G. S., Hamilton, R. J., Fleshner, N. E., Finelli, A., et al. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00067-5 · Cited by 21 · see citing works
Application of machine learning in tax prediction: A review with practical approaches
Akinrinola, O., Addy, W. A., Ajayi-Nifise, A. O., Odeyemi, O., & Falaiye, T. O. (2024) · Global Journal of Engineering and Technology Advances · doi:10.30574/gjeta.2024.18.2.0028 · Cited by 19 · see citing works
Krepper, D., Cesari, M., Hubel, N. J., Zelger, P., & Sztankay, M. J. (2024) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-024-00808-7 · Cited by 19 · see citing works
Integration of Federated Learning and Blockchain in Healthcare: A TutorialPreprint
Shahsavari, Y., Dambri, O. A., Baseri, Y., Hafid, A. S., & Makrakis, D. (2024) · doi:10.36227/techrxiv.171340632.25905526/v1 · Cited by 18 · see citing works
A systematic review of machine learning applications in predicting opioid associated adverse events
Medina, C. R. R., Benitez-Aurioles, J., Jenkins, D. A., & Jani, M. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01312-4 · Cited by 17 · see citing works
Zhuang, Q., Zhang, A. Y., Cong, R. S. T. Y., Yang, G. M., Neo, P. S. H., Tan, D. S. W., Chua, M. L. K., Tan, I. B., Wong, F. Y., Ong, M. E. H., et al. (2024) · BMC Palliative Care · doi:10.1186/s12904-024-01457-9 · Cited by 17 · see citing works
Mäenpää, S. M., & Korja, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105523 · Cited by 17 · see citing works
Prognostic models for surgical-site infection in gastrointestinal surgery: systematic review
McLean, K. A., Goel, T., Lawday, S., Riad, A., Simoes, J. F. F., Knight, S. R., Ghosh, D., Glasbey, J. C., Bhangu, A. A., & Harrison, E. M. (2023) · British journal of surgery · doi:10.1093/bjs/znad187 · Cited by 17 · see citing works
Aly, F., Hansen, C. R., Mouiee, D. A., Sundaresan, P., Haidar, A., Vinod, S. K., & Holloway, L. C. (2023) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2023.109629 · Cited by 11 · see citing works
Clinical prediction models for treatment outcomes in newly diagnosed epilepsy: A systematic review
Ratcliffe, C., Pradeep, V., Marson, A. G., Keller, S. S., & Bonnett, L. J. (2024) · Epilepsia · doi:10.1111/epi.17994 · Cited by 11 · see citing works
Austin, P. C., Lee, D. S., & Wang, B. (2024) · Diagnostic and Prognostic Research · doi:10.1186/s41512-024-00179-z · Cited by 11 · see citing works
Shahsavari, Y., Baseri, Y., Hafid, A. S., Dambri, O. A., & Makrakis, D. (2026) · Journal of Medical Internet Research · doi:10.2196/80178 · Cited by 23 · see citing works
Li, H., Dong, D., Fang, M., He, B., Liu, S., Hu, C. Y., Liu, Z., Wang, H., Tang, L., & Tian, J. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3484991 · Cited by 9 · see citing works
de Mattos, B. P., Mattjie, C., Ravazio, R., Barros, R. C., & Grassi‐Oliveira, R. (2024) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-024-01403-z · Cited by 9 · see citing works
Kalaycıoğlu, O., Pavlou, M., Akhanlı, S. E., de Belder, M., Ambler, G., & Omar, R. (2025) · Statistical Methods in Medical Research · doi:10.1177/09622802251338983 · Cited by 8 · see citing works
Chen, D. C., Arnold, K., Sukhdeo, R., Alla, J. F., & Raman, S. (2025) · BMJ Oncology · doi:10.1136/bmjonc-2025-000733 · Cited by 13 · see citing works
Ugwu, G. C., Jalali, F., Liu, G., Li, G., Langendijk, J. A., & Alizadeh, B. Z. (2025) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2025.111350 · Cited by 7 · see citing works
Held, U., Forzy, T., Signorell, A., Deforth, M., Burgstaller, J., & Wertli, M. (2023) · Pain · doi:10.1097/j.pain.0000000000003023 · Cited by 6 · see citing works
Bioinformatics in urology — molecular characterization of pathophysiology and response to treatment
Gheinani, A. H., Kim, J., You, S., & Adam, R. M. (2023) · Nature Reviews Urology · doi:10.1038/s41585-023-00805-3 · Cited by 9 · see citing works
Evaluation of changes in prediction modelling in biomedicine using systematic reviews
Lusa, L., Kappenberg, F., Collins, P. G. S., Schmid, M., Sauerbrei, W., Rahnenführer, J., & on behalf of the Topic Group for High-dimensional data of the STRATOS initiative (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02605-2 · Cited by 6 · see citing works
Riley, R. D., Collins, P. G. S., Whittle, R., Archer, L., Snell, K. I. E., Dhiman, P., Kirton, L., Legha, A., Liu, X., Denniston, A. K., et al. (2025) · Diagnostic and Prognostic Research · doi:10.1186/s41512-025-00193-9 · Cited by 6 · see citing works
Liu, D., Kababji, S. E., Mitsakakis, N., Pilgram, L., Walters, T., Clemons, M., Pond, G., El-Hussuna, A., & Emam, K. E. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03266-3 · Cited by 15 · see citing works
Lyman, G. H., & Kuderer, N. M. (2024) · Cancer Investigation · doi:10.1080/07357907.2024.2354991 · Cited by 5 · see citing works
Lopez-Perez, L., Georga, E. I., Conti, C. A., Vicente, V., García, R., Pecchia, L., Fotiadis, D. I., Licitra, L. F., Cabrera-Umpiérrez, M. F., Arredondo, M. T., et al. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106067 · Cited by 10 · see citing works
dos Santos, A. L., Pinhati, C. C., Perdigão, J., Galante, S. C., Silva, L., Veloso, I. Y., Silva, A. C. S. E., & OLIVEIRA, E. A. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102824 · Cited by 8 · see citing works
Pilgram, L., Kababji, S. E., Liu, D., & Emam, K. E. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf169 · Cited by 5 · see citing works
Externally validated yet undertrained: sample size deficits in machine learning-based radiomics
Koçak, B., Banaz, T., Söyleyici, M., Köse, F., Keleş, A., & Meşe, İ. (2026) · European Radiology · doi:10.1007/s00330-026-12543-2 · Cited by 5 · see citing works
White, N. M., Parsons, R., Borg, D. N., Collins, P. G. S., & Barnett, A. G. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111433 · Cited by 5 · see citing works
Talimtzi, P., Ntolkeras, A., Kostopoulos, G., Bougioukas, K. I., Pagkalidou, E., Ouranidis, A., Pataka, A., & Haidich, A. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111264 · Cited by 5 · see citing works
Dhiman, P., Ma, J., Kirtley, S., Mouka, E., Waldron, C., Whittle, R., & Collins, P. G. S. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111287 · Cited by 5 · see citing works
Legha, A., Ensor, J., Whittle, R., Archer, L., CALSTER, B. V., Christodoulou, E., Snell, K. I. E., Sadatsafavi, M., Collins, P. G. S., & Riley, R. D. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.112117 · Cited by 4 · see citing works
Shanmugaraj, A., Khalid, B., Kumar, M. V., Kunze, K. N., & Sheth, U. (2026) · Shoulder & Elbow · doi:10.1177/17585732251412368 · Cited by 4 · see citing works
When Is a Prognostic Prediction Model Ready for Clinical Use? A Primer for Clinicians
Feller, D., & Chiarotto, A. (2026) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2026.13868 · Cited by 4 · see citing works
Hancox, Z., Kingsbury, S. R., Clegg, A. P., Conaghan, P. G., & Relton, S. D. (2024) · International Conference on Machine Learning and Applications · doi:10.1109/icmla61862.2024.00041 · Cited by 4 · see citing works
LIANG, W., ZHANG, Z., YANG, K., HU, H., Luo, Q., YANG, A., CHANG, L., & ZENG, Y. (2023) · Biocell · doi:10.32604/biocell.2023.027373 · Cited by 4 · see citing works
Hjärtström, M., Dihge, L., Bendahl, P., Skarping, I. D., Ellbrant, J., Ohlsson, M., & Rydén, L. (2023) · JMIR Cancer · doi:10.2196/46474 · Cited by 4 · see citing works
Vasey, B., & Collins, P. G. S. (2023) · Surgery · doi:10.1016/j.surg.2023.04.037 · Cited by 5 · see citing works
van Maurik, I. S., Doodeman, H. J., Nuijens, B. W., Möhringer, R. P. M., Sudiono, D. R., Jongbloed, W., & van Soelen, E. (2024) · JMIR Medical Informatics · doi:10.2196/57035 · Cited by 3 · see citing works
Lazaridou, A., Sivakumar, S., Biefer, H. R. C., Weilenmann, S., Princip, M., Zuccarella‐Hackl, C., Petzschner, F. H., Heinzle, J., Stephan, K. E., Dzemali, O., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-108061 · Cited by 5 · see citing works
Liu, W., Feng, Z., Zhang, M., Mao, R., & Li, J. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106345 · Cited by 3 · see citing works
Duquenne, P., Liposits, G. I., Vonnes, C., Navarrete, E., Serrano, A., Canouï‐Poitrine, F., Marinho, J. C. L., Akagündüz, B., Haase, K. R., Verduzco-Aguirre, H. C., et al. (2026) · The Lancet Healthy Longevity · doi:10.1016/j.lanhl.2026.100829 · Cited by 3 · see citing works
Clift, A. K., Mahon, H., Khan, G., Boardman-Pretty, F., Worker, A., Marchini, E., Buendia, O., Fish, P., & Khan, M. S. (2024) · British Journal of Cancer · doi:10.1038/s41416-024-02736-1 · Cited by 3 · see citing works
Su, J., Chen, J., Wang, T., & Lin, T. (2025) · Clinical Cosmetic and Investigational Dermatology · doi:10.2147/ccid.s542866 · Cited by 3 · see citing works
Chu, H. O., Buchan, E., Smith, D. J., & Oppenheimer, P. G. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108014 · Cited by 3 · see citing works
The application of time-to-event analysis in machine learning prognostic models
Peng, Z.-H., Huang, Z., Tian, J., Chong, T., & Li, Z.-L. (2024) · Journal of Translational Medicine · doi:10.1186/s12967-024-04909-1 · Cited by 3 · see citing works
Risk scores in anaesthesia: the future is hard to predict
Drayton, D. J., Ayres, M., Relton, S. D., Sperrin, M., & Hall, M. (2022) · BJA Open · doi:10.1016/j.bjao.2022.100027 · Cited by 3 · see citing works
Using AI in the Economic Evaluation of AI-Based Health Technologies
Ghabri, S. (2025) · PharmacoEconomics (Auckland) · doi:10.1007/s40273-025-01496-x · Cited by 2 · see citing works
The reporting quality and methodological quality of dynamic prediction models for cancer prognosis
Yan, P., Xu, Z., Xu, H., Chu, X., Chen, Y., Yang, C., Xu, S., Cui, H., Zhang, L., Zhang, W., et al. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02516-2 · Cited by 2 · see citing works
Diomatari, C., Martin, G. P., Jenkins, D. A., & Jani, M. (2025) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2025.152728 · Cited by 2 · see citing works
Thomassen, D., Hackmann, T., Goeman, J., Steyerberg, E., & le Cessie, S. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100911 · Cited by 7 · see citing works
Domínguez-Sosa, M. S., Cabrera-Ramírez, M. S., Marrero-Ramos, M. D. C., Cabrera-López, C., Carrillo, T., Benitez-Rosario, J., & Quintana, C. D. D. (2026) · Frontiers in Allergy · doi:10.3389/falgy.2026.1710163 · Cited by 2 · see citing works
Zhang, X., Xiao, N., Du, S., Anipindi, M., Lee, C.-H., Yoshino, T., Anand, A., Xiao, L., & Kumar, A. (2026) · Blood Neoplasia · doi:10.1016/j.bneo.2026.100217 · Cited by 2 · see citing works
Zhang, X., Bustamante, J., Stefani, H., Lee, H., Xiao, N., Xiao, L., Soni, A., & Danpanichkul, P. (2026) · Leukemia Research · doi:10.1016/j.leukres.2026.108199 · Cited by 2 · see citing works
Sample size calculation for training ensemble machine learning models on health data
Mitsakakis, N., Liu, D., Walters, T. D., & Emam, K. E. (2026) · Patterns · doi:10.1016/j.patter.2026.101498 · Cited by 2 · see citing works
Jennings, C., Broad, A., Godson, L., Clarke, E. L., Westhead, D. R., & Treanor, D. E. (2026) · Cancer Informatics · doi:10.1177/11769351261434523 · Cited by 7 · see citing works
Wang, X., Dou, Y., Wang, Y., Sun, K., & Zhou, G. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1798803 · Cited by 3 · see citing works
Riley, R. D., Collins, P. G. S., Archer, L., Whittle, R., Legha, A., Kirton, L., Dhiman, P., Sadatsafavi, M., Adderley, N. J., Alderman, J. E., et al. (2025) · Diagnostic and Prognostic Research · doi:10.1186/s41512-025-00204-9 · Cited by 2 · see citing works
Ye, Z., Zhang, T., Wu, C., Qiao, Y., Su, W., Chen, J., Xie, G., Dong, S., Xu, J., & Zhao, J. (2023) · The American Journal of Sports Medicine · doi:10.1177/03635465231161060 · Cited by 2 · see citing works
Heesen, P., Christ, S. M., Ciobanu-Caraus, O., Kahraman, A., Schelling, G., Studer, G., Bode‐Lesniewska, B., & Fuchs, B. (2025) · Diagnostic and Prognostic Research · doi:10.1186/s41512-025-00186-8 · Cited by 1 · see citing works
Lazare, S., Gu, Z., Earland, N. J., Harmon, M. A., Pacula, M., Rivera, M., Tellis, A., Whitfield, D., Benson, A., Gerndt, S. P., et al. (2025) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-25-1221 · Cited by 1 · see citing works
Zamani, A. S., Motwakel, A., Alluhayb, A., Akhtar, M. M., Ayub, R., Abdelrahim, M. A. A., Mohamed, S. S. I., & Ahmad, N. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106217 · Cited by 1 · see citing works
McGough, S. F., Lyalina, S., Incerti, D., Huang, Y., Tyanova, S., Mace, K., Harbron, C. G., Copping, R., Narasimhan, B., & Tibshirani, R. J. (2026) · PLoS ONE · doi:10.1371/journal.pone.0341355 · Cited by 1 · see citing works
Why we need to maintain a critical view on big data and artificial intelligence predictions
Temiz, S. A., & Tascilar, K. (2026) · Current Opinion in Immunology · doi:10.1016/j.coi.2026.102776 · Cited by 1 · see citing works
Cai, Y., Pan, D., Zhang, J., Chen, Z., Chen, K., Sun, C., Su, Y., Li, J., & Yi, H. (2026) · International Journal of Clinical Practice · doi:10.1155/ijcp/1364442 · Cited by 1 · see citing works
Early Detection of Lung Cancer Using Clinical Data and Machine Learning Methods
Jaysawal, B., Sah, R. R., Dewan, M. S., Kumar, U., Kumar, A., & Parija, S. (2026) · International Conference on Intelligent Computing, Networks, and Security (IC-ICNS) · doi:10.1109/ic-icns68863.2026.11537821 · Cited by 1 · see citing works
Hernández, J. R. H. (2026) · Mathematics · doi:10.3390/math14112002 · Cited by 1 · see citing works
Hughes, J. M., Gao, S. Y., Nguyen, S. A., & Newman, J. G. (2026) · Cancers · doi:10.3390/cancers18142271 · Cited by 1 · see citing works
- Authors
- Dhiman, P., Ma, J., Andaur Navarro, C. L., Speich, B., Bullock, G., Damen, J. A. A., Hooft, L., Kirtley, S., Riley, R. D., Van Calster, B., Moons, K. G. M., & Collins, G. S.
- Affiliations
- University of Oxford (GB)
- University of Basel (CH)
- Utrecht University (NL)
- Keele University (GB)
- Corresponding author
- Dhiman, Paula
- ORCID
- Venue
- BMC Medical Research Methodology
- Published
- 2022-04-08
- Volume
- 22
- Issue
- 1
- Pages
- 101-101
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC8991704
- OpenAlex
- W4223540626
- Cited by
- 140
- References
- 126
- Citations by year
- 2026: 37 · 2025: 34 · 2024: 42 · 2023: 23 · 2022: 4
- FWCI
- 3.72
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- Lirias (KU Leuven) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
- Funders
- Cancer Research UK (27294)
- National Institute for Health and Care Research
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- prognostic model, machine learning, oncology, methodological quality, tripod, PROBAST, charm, regression models, ensemble learning, sample size justification, predictor selection, time-to-event outcomes, internal validation, calibration, cancer prognosis, cancer survival prediction, cancer risk prediction, tumor response prediction, tumor recurrence
- MeSH terms
- Machine Learning, Humans, Medical Oncology, Prognosis, Research Design, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review