Overinterpretation of findings in machine learning prediction model studies in oncology: a systematic review
Open accessAbstract
OBJECTIVES: In biomedical research, spin is the overinterpretation of findings, and it is a growing concern. To date, the presence of spin has not been evaluated in prognostic model research in oncology, including studies developing and validating models for individualized risk prediction. STUDY DESIGN AND SETTING: We conducted a systematic review, searching MEDLINE and EMBASE for oncology-related studies that developed and validated a prognostic model using machine learning published between 1st January, 2019, and 5th September, 2019. We used existing spin frameworks and described areas of highly suggestive spin practices. RESULTS: We included 62 publications (including 152 developed models; 37 validated models). Reporting was inconsistent between methods and the results in 27% of studies due to additional analysis and selective reporting. Thirty-two studies (out of 36 applicable studies) reported comparisons between developed models in their discussion and predominantly used discrimination measures to support their claims (78%). Thirty-five studies (56%) used an overly strong or leading word in their title, abstract, results, discussion, or conclusion. CONCLUSION: The potential for spin needs to be considered when reading, interpreting, and using studies that developed and validated prognostic models in oncology. Researchers should carefully report their prognostic model research using words that reflect their actual results and strength of evidence.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The review found a need for more clinical validation of AI prediction models in oncology, a shortcoming that has been noted in other areas of nursing, medicine, and healthcare.
“The review also found a need for more clinical validation of AI prediction models in oncology, a shortcoming that has been noted in other areas of nursing, medicine, and healthcare (Labarere et al., 2014;Dhiman et al., 2021;Dhiman et al., 2023).”
From The application and use of artificial intelligence in cancer nursing: A systematic review (2024) · cites this paper as Dhiman et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283 · see citing works
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · see citing works
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.08.010 · Cited by 51,384 · see citing works
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092 · see citing works
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147 · see citing works
Self-citationStatistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · see citing works
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., PRISMA-S Group, Blunt, H., Brigham, T., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-020-01542-z · Cited by 3,714 · see citing works
Self-citationRethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., & Koffel, J. (2019) · doi:10.31219/osf.io/sfc38 · Cited by 3,696 · see citing works
Self-citationCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267 · see citing works
Self-citationWynants, L., Van Calster, B., Bonten, M. M., Collins, G. S., Debray, T. P., De Vos, M., Haller, M. C., Heinze, G., Moons, K. G., Riley, R. D., et al. (2020) · doi:10.1101/2020.03.24.20041020 · Cited by 3,070 · see citing works
Self-citationCalibration: the Achilles heel of predictive analytics
On behalf of Topic Group ‘Evaluating diagnostic tests and prediction models’ of the STRATOS initiative, Van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., & Steyerberg, E. W. (2019) · BMC Medicine · doi:10.1186/s12916-019-1466-7 · Cited by 2,650 · see citing works
Self-citationChristodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119 · see citing works
Self-citationHippisley-Cox, J., Coupland, C., & Brindle, P. (2017) · BMJ · doi:10.1136/bmj.j2099 · Cited by 1,703 · see citing works
Alba, A. C., Agoritsas, T., Walsh, M., Hanna, S., Iorio, A., Devereaux, P. J., McGinn, T., & Guyatt, G. (2017) · JAMA · doi:10.1001/jama.2017.12126 · Cited by 1,631 · see citing works
Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators
CALSTER, B. V., Wynants, L., Verbeek, J. F. M., Verbakel, J. Y. J., Christodoulou, E., Vickers, A. J., Roobol, M. J., & Steyerberg, E. W. (2018) · European Urology · doi:10.1016/j.eururo.2018.08.038 · Cited by 1,289 · see citing works
Self-citationMachine Learning and Prediction in Medicine — Beyond the Peak of Inflated Expectations
Chen, J. H., & Asch, S. M. (2017) · New England Journal of Medicine · doi:10.1056/nejmp1702071 · Cited by 1,171 · see citing works
Luo, W., Phung, D., Tran, T., Gupta, S., Rana, S., Karmakar, C., Shilton, A., Yearwood, J., Dimitrova, N., Ho, T. B., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5870 · Cited by 988 · see citing works
A calibration hierarchy for risk models was defined: from utopia to empirical data
Van Calster, B., Nieboer, D., Vergouwe, Y., De Cock, B., Pencina, M. J., & Steyerberg, E. W. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.12.005 · Cited by 910 · see citing works
Collins, G. S., Dhiman, P., Andaur Navarro, C. L., Ma, J., Hooft, L., Reitsma, J. B., Logullo, P., Beam, A. L., Peng, L., Van Calster, B., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048008 · Cited by 852 · see citing works
Self-citationSumner, P., Vivian-Griffiths, S., Boivin, J., Williams, A., Venetis, C. A., Davies, A., Ogden, J., Whelan, L., Hughes, B., Dalton, B., et al. (2014) · BMJ · doi:10.1136/bmj.g7015 · Cited by 489 · cited 1× · see citing works
“The reach of spin also extends to the general public, who might be exposed and more vulnerable to embellished press releases from primary studies with spin [9], resulting in potentially more direct but unnecessary behavior change.”
Wishart, G. C., Azzato, E. M., Greenberg, D. C., Rashbass, J., Kearins, O., Lawrence, G., Caldas, C., & Pharoah, P. D. (2010) · Breast Cancer Research · doi:10.1186/bcr2464 · Cited by 418 · see citing works
Andaur Navarro, C. L., Damen, J. A. A., Takada, T., Nijman, S. W. J., Dhiman, P., Ma, J., Collins, G. S., Bajpai, R., Riley, R. D., Moons, K. G. M., et al. (2021) · BMJ · doi:10.1136/bmj.n2281 · Cited by 415 · see citing works
Self-citationYavchitz, A., Boutron, I., Bafeta, A., Marroun, I., Charles, P., Mantz, J., & Ravaud, P. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001308 · Cited by 293 · cited 1× · see citing works
“Studies evaluating spin in medical research have largely focused on randomized and nonrandomized controlled trials [10,11], systematic reviews [12], prognostic factors, and diagnostic test accuracy studies [12 e 15].”
Misrepresentation and distortion of research in biomedical literature
Boutron, I., & Ravaud, P. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710755115 · Cited by 273 · see citing works
Early Prediction of Sepsis in the ICU Using Machine Learning: A Systematic Review
Moor, M., Rieck, B., Horn, M., Jutzeler, C. R., & Borgwardt, K. M. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.607952 · Cited by 234 · see citing works
Wang, W., Kiik, M., Peek, N. B., Ćurčin, V., Marshall, I. J., Rudd, A. G., Wang, Y., Douiri, A., Wolfe, C. D. A., & Bray, B. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234722 · Cited by 205 · see citing works
Survival outcome prediction in cervical cancer: Cox models vs deep-learning model
Matsuo, K., Purushotham, S., Jiang, B., Mandelbaum, R. S., Takiuchi, T., Liu, Y., & Roman, L. D. (2018) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2018.12.030 · Cited by 179 · see citing works
Karhade, A., Thio, Q., Ogink, P., Bono, C. M., Ferrone, M., Oh, K., Saylor, P., Schoenfeld, A. J., Shin, J. H., Harris, M. B., et al. (2019) · Neurosurgery · doi:10.1093/neuros/nyz070 · Cited by 179 · see citing works
Protecting against researcher bias in secondary data analysis: challenges and potential solutions
Baldwin, J. R., Pingault, J.-B., Schoeler, T., Sallis, H. M., & Munafò, M. R. (2022) · European Journal of Epidemiology · doi:10.1007/s10654-021-00839-0 · Cited by 178 · see citing works
Machine learning for genetic prediction of psychiatric disorders: a systematic review
Bracher‐Smith, M., Crawford, K. L., & Escott‐Price, V. (2020) · Molecular Psychiatry · doi:10.1038/s41380-020-0825-2 · Cited by 159 · see citing works
Overinterpretation and Misreporting of Diagnostic Accuracy Studies: Evidence of “Spin”
Ochodo, E. A., de Haan, M. C., Reitsma, J. B., Hooft, L., Bossuyt, P. M., & Leeflang, M. M. G. (2013) · Radiology · doi:10.1148/radiol.12120527 · Cited by 158 · see citing works
Self-citationPeat, G., Riley, R. D., Croft, P. R., Morley, K. I., Kyzas, P., Moons, K. G. M., Perel, P., Steyerberg, E. W., Schroter, S., Altman, D. G., et al. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001671 · Cited by 152 · see citing works
Self-citationKarhade, A., Thio, Q., Ogink, P., Shah, A. A., Bono, C. M., Oh, K., Saylor, P., Schoenfeld, A. J., Shin, J. H., Harris, M. B., et al. (2018) · Neurosurgery · doi:10.1093/neuros/nyy469 · Cited by 151 · see citing works
Hippisley‐Cox, J., & Coupland, C. A. C. (2012) · British Journal of General Practice · doi:10.3399/bjgp13x660733 · Cited by 143 · see citing works
A Plea for Neutral Comparison Studies in Computational Sciences
Boulesteix, A., Lauer, S., & Eugster, M. J. A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0061562 · Cited by 142 · see citing works
Dhiman, P., Ma, J., Andaur Navarro, C. L., Speich, B., Bullock, G., Damen, J. A. A., Hooft, L., Kirtley, S., Riley, R. D., Van Calster, B., et al. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01577-x · Cited by 140 · see citing works
Self-citationLazarus, C., Haneef, R., Ravaud, P., & Boutron, I. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0079-x · Cited by 137 · see citing works
Taninaga, J., Nishiyama, Y., Fujibayashi, K., Gunji, T., Sasabe, N., Iijima, K., & Naito, T. (2019) · Scientific Reports · doi:10.1038/s41598-019-48769-y · Cited by 120 · see citing works
Tseng, Y., Huang, C.-E., Wen, C.-N., Lai, P.-Y., Wu, M., Sun, Y.-C., Wang, H., & Lu, J. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.05.003 · Cited by 117 · see citing works
Karadaghy, O. A., Shew, M. A., New, J., & Bur, A. M. (2019) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2019.0981 · Cited by 114 · see citing works
Dhiman, P., Ma, J., Navarro, C. L. A., Speich, B., Bullock, G. S., Damen, J. A., Kirtley, S., Hooft, L., Riley, R. D., CALSTER, B. V., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.06.024 · Cited by 109 · see citing works
Self-citationAlabi, R. O., Elmusrati, M. S., Sawazaki‐Calone, Í., Kowalski, L. P., Haglund, C. H., Coletta, R. D., Mäkitie, A. A., Salo, T. A., Leivo, I. V. V., & Almangush, A. (2019) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-019-02642-5 · Cited by 109 · see citing works
Use of machine learning to predict early biochemical recurrence after robot‐assisted prostatectomy
Wong, N. C., Lam, C. J., Patterson, L., & Shayegan, B. (2018) · British Journal of Urology · doi:10.1111/bju.14477 · Cited by 101 · see citing works
Luna, J. M., Chao, H., Diffenderfer, E. S., Valdés, G., Chinniah, C., Grace, X., Cengel, K. A., Solberg, T. D., Berman, A. T., & Simone, C. B. (2019) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2019.01.003 · Cited by 99 · see citing works
Muhlestein, W. E., Akagi, D. S., Davies, J. M., & Chambless, L. B. (2018) · Neurosurgery · doi:10.1093/neuros/nyy343 · Cited by 84 · see citing works
Paik, E. S., Lee, J., Park, J., Kim, J.-H., Kim, M., Kim, T., Choi, C. H., Kim, B., Bae, D., & Seo, S. W. (2019) · Journal of Gynecologic Oncology · doi:10.3802/jgo.2019.30.e65 · Cited by 75 · see citing works
Wang, H.-H., Wang, Y.-H., Liang, C.-W., & Li, Y. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.2335 · Cited by 75 · see citing works
Heus, P., Reitsma, J. B., Collins, G. S., Damen, J. A. A. G., Scholten, R. J. P. M., Altman, D. G., Moons, K. G. M., & Hooft, L. (2020) · Annals of Internal Medicine · doi:10.7326/m20-0193 · Cited by 70 · cited 1× · see citing works
“…in the main text (and those specified in the protocol) An abstract extension to TRIPOD (transparent reporting of multivariable prediction models in journal and conference abstracts: TRIPOD for abstracts) is available to ensure prediction model study abstracts are reported more completely [87].”
Self-citationWang, X., Zhang, Y., Hao, S., Zheng, L., Liao, J., Ye, C., Xia, M., Wang, O., Liu, M., Weng, C. H., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/13260 · Cited by 62 · see citing works
Kareemi, H., Vaillancourt, C., Rosenberg, H., Fournier, K., & Yadav, K. (2020) · Academic Emergency Medicine · doi:10.1111/acem.14190 · Cited by 61 · see citing works
Khalaf, M. H., H, V. S. M. P., Abdelrazek, M. A., Shah, R. P., Khosla, A., Jackson, K., Desai, M., & Kothary, N. (2018) · Radiology · doi:10.1148/radiol.2018180257 · Cited by 58 · see citing works
Nartowt, B. J., Hart, G. R., Roffman, D. A., Llor, X., Ali, I. M., Muhammad, W., Liang, Y., & Deng, J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0221421 · Cited by 50 · see citing works
A novel prediction method for lymph node involvement in endometrial cancer: machine learning
Günakan, E., Suat, A. T. A. N., Haberal, N., Küçükyıldız, İ. A., Gökçe, E., & Ayhan, A. (2019) · International Journal of Gynecological Cancer · doi:10.1136/ijgc-2018-000033 · Cited by 50 · see citing works
Oyaga-Iriarte, E., Insausti, A., Sayar, O., & Aldaz, A. (2019) · Journal of Pharmacological Sciences · doi:10.1016/j.jphs.2019.03.004 · Cited by 49 · see citing works
van Niftrik, C. H. B., van der Wouden, F., Staartjes, V. E., Fierstra, J., Stienen, M. N., Akeret, K., Sebök, M., Fedele, T., Sarnthein, J., Bozinov, O., et al. (2019) · Neurosurgery · doi:10.1093/neuros/nyz145 · Cited by 49 · see citing works
Green, A. R., Soria, D., Stephen, J., Powe, D. G., Nolan, C. C., Kunkler, I. H., Thomas, J. S. J., Kerr, G. R., Jack, W. J. L., Cameron, D., et al. (2015) · The Journal of Pathology Clinical Research · doi:10.1002/cjp2.32 · Cited by 46 · see citing works
Artificial neural network models to predict nodal status in clinically node-negative breast cancer
Dihge, L., Ohlsson, M., Edén, P., Bendahl, P., & Rydén, L. (2019) · BMC Cancer · doi:10.1186/s12885-019-5827-6 · Cited by 44 · see citing works
Machine Learning to Predict Delays in Adjuvant Radiation following Surgery for Head and Neck Cancer
Shew, M. A., New, J., & Bur, A. M. (2019) · Otolaryngology · doi:10.1177/0194599818823200 · Cited by 43 · see citing works
Yan, P., Huang, R., Hu, P., Liu, F., Zhu, X., Hu, P., Yin, H., Zhang, J., Meng, T., & Huang, Z. (2019) · Journal of Neuro-Oncology · doi:10.1007/s11060-019-03181-4 · Cited by 40 · see citing works
Kaviarasi, R., & Gandhi Raj, R. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1297-2 · Cited by 39 · see citing works
Semi-supervised learning to improve generalizability of risk prediction models
Chi, S., Li, X., Tian, Y., Li, J., Kong, X., Ding, K., Weng, C., & Li, J. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103117 · Cited by 38 · see citing works
Elarre, P. S., Oyaga-Iriarte, E., Yu, K. H., Baudin, V., Moreno, L. A., Carranza, O. E., Ortega, A. C., Ponz‐Sarvisé, M., Sosa, L., Rotellar, F., et al. (2019) · Cancers · doi:10.3390/cancers11050606 · Cited by 38 · see citing works
Hype in reports of clinical research: The authors' perspectives
Millar, N., Budgell, B., & Salager‐Meyer, F. (2020) · English for Specific Purposes · doi:10.1016/j.esp.2020.07.001 · Cited by 38 · see citing works
Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review
Kempf, E., de Beyer, J. A., Cook, J. A., Holmes, J., Mohammed, S. B., Nguyen, T., Simera, I., Trivella, M., Altman, D. G., Hopewell, S., et al. (2018) · British Journal of Cancer · doi:10.1038/s41416-018-0305-5 · Cited by 36 · see citing works
Self-citationGhannad, M., Olsen, M., Boutron, I., & Bossuyt, P. M. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.07.011 · Cited by 33 · see citing works
Tian, Z., Yen, A., Zhou, Z., Shen, C., Albuquerque, K., & Hrycushko, B. A. (2019) · Brachytherapy · doi:10.1016/j.brachy.2019.04.004 · Cited by 30 · see citing works
A prospective study examining cachexia predictors in patients with incurable cancer
Vagnildhaug, O. M., Brunelli, C., Hjermstad, M. J., Strasser, F., Baracos, V. E., Wilcock, A., Nabal, M., Kaasa, S., Laird, B. J. A., & Solheim, T. S. (2019) · BMC Palliative Care · doi:10.1186/s12904-019-0429-2 · Cited by 30 · see citing works
Egger, M. E., Stevenson, M. A., Bhutiani, N., Jordan, A. C., Scoggins, C. R., Philips, P., Martin, R. C. G., & McMasters, K. M. (2019) · Annals of Surgical Oncology · doi:10.1245/s10434-019-07690-4 · Cited by 28 · see citing works
Survivability prediction of colon cancer patients using neural networks
Al-Bahrani, R., Agrawal, A., & Choudhary, A. (2017) · Health Informatics Journal · doi:10.1177/1460458217720395 · Cited by 26 · see citing works
Predicting Overall Survival in Patients with Metastatic Rectal Cancer: a Machine Learning Approach
Zhao, B., Gabriel, R. A., Vaida, F., Lopez, N., Eisenstein, S. G., & Clary, B. M. (2019) · Journal of Gastrointestinal Surgery · doi:10.1007/s11605-019-04373-z · Cited by 24 · see citing works
Can machine learning predict resecability of a peritoneal carcinomatosis?
Maubert, A., Birtwisle, L., Bernard, J., Benizri, E. I., & Bereder, J. M. (2019) · Surgical Oncology · doi:10.1016/j.suronc.2019.04.008 · Cited by 24 · see citing works
McGrath, T. A., Bowdridge, J., Prager, R. T., Frank, R. A., Treanor, L. M., Sharifabadi, A. D., Salameh, J., Leeflang, M., Korevaar, D. A., Bossuyt, P. M., et al. (2020) · Clinical Chemistry · doi:10.1093/clinchem/hvaa093 · Cited by 24 · see citing works
Preoperative predictors of the need for allogeneic blood transfusion in lung cancer surgery
Cirasino, L., Barosi, G., Torre, M. D., Crespi, S., Colombo, P., & Belloni, P. A. (2000) · Transfusion · doi:10.1046/j.1537-2995.2000.40101228.x · Cited by 18 · see citing works
Li, M., Zhan, C., Sui, X., Jiang, W., Shi, Y., Yang, X., Feng, M., Wang, J., & Wang, Q. (2019) · Frontiers in Oncology · doi:10.3389/fonc.2019.00771 · Cited by 16 · see citing works
Velázquez, N., Press, B. H., Renson, A., Wysock, J. S., Taneja, S. S., Huang, W. C. W., & Bjurlin, M. A. (2018) · Clinical Genitourinary Cancer · doi:10.1016/j.clgc.2018.09.018 · Cited by 15 · see citing works
Beachler, D. C., de Luise, C., Yin, R., Gangemi, K., Cochetti, P. T., & Lanes, S. (2018) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4681 · Cited by 15 · see citing works
Hammer, J., Geinitz, H., Nieder, C., Track, C., Thames, H. D., Seewald, D. H., Petzer, A. L., Helfgott, R., Spiegl, K. J., Heck, D., et al. (2018) · Clinical Breast Cancer · doi:10.1016/j.clbc.2018.08.001 · Cited by 13 · see citing works
Lindsay, W. D., Ahern, C. A., Tobias, J. S., Berlind, C. G., Chinniah, C., Gabriel, P. E., Gee, J. C., & Simone, C. B. (2018) · Medical Physics · doi:10.1002/mp.13314 · Cited by 11 · see citing works
Yang, X., Wang, F., Feng, J., Hu, Y., Lun, D., Hua, K., Liu, Y.-H., Zhang, H., & Yang, L. (2019) · World Neurosurgery · doi:10.1016/j.wneu.2019.02.183 · Cited by 9 · see citing works
Sim, J.-A., & Yun, Y. H. (2019) · Studies in health technology and informatics · doi:10.3233/shti190548 · Cited by 5 · see citing works
Spin, Bias, and Clinical Utility in Systematic Reviews of Diagnostic Studies
Ioannidis, J. P. A. (2020) · Clinical Chemistry · doi:10.1093/clinchem/hvaa114 · Cited by 5 · see citing works
Xu, Y., Kong, S., Cheung, W. Y., Quan, M. L., Nakoneshny, S. C., & Dort, J. C. (2019) · Head & Neck · doi:10.1002/hed.25682 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [85]Van Calster B. McLernon D.J. van Smeden M. Wynants L. Steyerberg E.W. Bossuyt P. Calibration: the Achilles heel of predictive analytics BMC Med 17 1 2019 230 31842878 PMC6912996
Cited by
Sorted by citations
Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-078378 · Cited by 4,718 · see citing works
“59 60 These deficiencies cast considerable doubt on models’ usefulness and safety, and raises concerns about their potential to create or widen healthcare disparities.”
BackgroundSelf-citationMoons, K. G. M., Damen, J. A. A., Kaul, T., Hooft, L., Andaur Navarro, C., Dhiman, P., Beam, A. L., Van Calster, B., Celi, L. A., Denaxas, S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-082505 · Cited by 1,069 · see citing works
Self-citation(2024) · BMJ · doi:10.1136/bmj.q902 · Cited by 416 · see citing works
“59 60 These deficiencies cast considerable doubt on models’ usefulness and safety, and raises concerns about their potential to create or widen healthcare disparities.”
Artificial Intelligence in Orthodontics: Critical Review
Nordblom, N. F., Büttner, M., & Schwendicke, F. (2024) · Journal of dentistry research · doi:10.1177/00220345241235606 · Cited by 118 · see citing works
Importance of sample size on the quality and utility of AI-based prediction models for healthcare
Riley, R. D., Ensor, J., Snell, K. I. E., Archer, L., Whittle, R., Dhiman, P., Alderman, J. E., Liu, X., Kirton, L., Manson-Whitton, J., et al. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.01.013 · Cited by 93 · see citing works
Self-citationThe application and use of artificial intelligence in cancer nursing: A systematic review
O'Connor, S., Vercell, A., Wong, D., Yorke, J., Fallatah, F. A., Cave, L., & Anny Chen, L.-Y. (2024) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2024.102510 · Cited by 51 · see citing works
Tsegaye, B., Snell, K. I. E., Archer, L., Kirtley, S., Riley, R. D., Sperrin, M., CALSTER, B. V., Collins, P. G. S., & Dhiman, P. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111675 · Cited by 36 · see citing works
Self-citationTwelve practical recommendations for developing and applying clinical predictive models
Feng, G., Xu, H., Wan, S., Wang, H., Chen, X., Magari, R. T., Han, Y. K., Wei, Y., & Gu, H. (2024) · The Innovation Medicine · doi:10.59717/j.xinn-med.2024.100105 · Cited by 27 · see citing works
Clinical prediction models using machine learning in oncology: challenges and recommendations
Collins, G. S., Chester-Jones, M., Gerry, S., Ma, J., Matos, J., Sehjal, J., Tsegaye, B., & Dhiman, P. (2025) · BMJ Oncology · doi:10.1136/bmjonc-2025-000914 · Cited by 27 · see citing works
Self-citationCritelli, B., Hassan, A., Lahooti, I., Noh, L., Park, J. S., Tong, K., Lahooti, A., Matzko, N. E., Adams, J. N., Liß, L., et al. (2025) · PLoS Medicine · doi:10.1371/journal.pmed.1004432 · Cited by 24 · see citing works
SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models
Navarro, C. L. A., Damen, J. A., Ghannad, M., Dhiman, P., van Smeden, M., Reitsma, J. B., Collins, P. G. S., Riley, R. D., Moons, K. G. M., & Hooft, L. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111364 · Cited by 13 · see citing works
Self-citationEthical guidance for reporting and evaluating claims of AI outperforming human doctors
Drogt, J., Milota, M., van den Brink, A., & Jongsma, K. R. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01255-w · Cited by 14 · see citing works
Collins, P. G. S. (2024) · BMJ · doi:10.1136/bmj.q832 · Cited by 9 · see citing works
Self-citationde Mattos, B. P., Mattjie, C., Ravazio, R., Barros, R. C., & Grassi‐Oliveira, R. (2024) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-024-01403-z · Cited by 9 · see citing works
Predictive modeling for metastasis in oncology: current methods and future directions
Abbas, G. H., Khouri, E., Thaher, O., Taha, S. A., Vladimirov, M., Oviedo, R. J., Schmidt, J., Bausch, D., & Pouwels, S. (2025) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000003279 · Cited by 8 · see citing works
Riley, R. D., Collins, P. G. S., Whittle, R., Archer, L., Snell, K. I. E., Dhiman, P., Kirton, L., Legha, A., Liu, X., Denniston, A. K., et al. (2025) · Diagnostic and Prognostic Research · doi:10.1186/s41512-025-00193-9 · Cited by 6 · see citing works
Self-citationLyman, G. H., & Kuderer, N. M. (2024) · Cancer Investigation · doi:10.1080/07357907.2024.2354991 · Cited by 5 · see citing works
Leukel, J., Scheurer, L., & Zimpel, T. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.109892 · Cited by 5 · see citing works
PET/CT deep learning prognosis for treatment decision support in esophageal squamous cell carcinoma
Song, J., Zhang, J., Liu, G., Guo, Z., Liao, H., Feng, W., Lin, W., Li, L., Zhang, Y., Yang, Y., et al. (2024) · Insights into Imaging · doi:10.1186/s13244-024-01737-1 · Cited by 5 · see citing works
Maru, S., Kuwatsuru, R., Matthias, M. D., & Simpson, R. J. (2025) · Journal of Medical Internet Research · doi:10.2196/60148 · Cited by 4 · see citing works
Hassan, A., Critelli, B., Lahooti, I., Lahooti, A., Matzko, N., Adams, J. N., Liß, L., Quion, J. M., Restrepo, D. S., Nikahd, M., et al. (2024) · Diagnostic and Prognostic Research · doi:10.1186/s41512-024-00169-1 · Cited by 4 · see citing works
Clift, A. K., Mahon, H., Khan, G., Boardman-Pretty, F., Worker, A., Marchini, E., Buendia, O., Fish, P., & Khan, M. S. (2024) · British Journal of Cancer · doi:10.1038/s41416-024-02736-1 · Cited by 3 · see citing works
Leukel, J., Liu, Z., & Sugumaran, V. (2026) · Journal of Revenue and Pricing Management · doi:10.1057/s41272-026-00574-9 · Cited by 2 · see citing works
Jin, Z., Ma, F., Chen, H., & Guo, S. (2023) · Scientific Reports · doi:10.1038/s41598-023-49925-1 · Cited by 4 · see citing works
Identifying and Minimizing Incentives for Competing Interests in Sports Medicine Publications
Shrier, I., Impellizzeri, F. M., & Stovitz, S. D. (2024) · Sports Medicine · doi:10.1007/s40279-024-02037-w · Cited by 2 · see citing works
Artificial intelligence in peri‐operative prediction model research: are we there yet?
Shah, A., & Dhiman, P. (2024) · Anaesthesia · doi:10.1111/anae.16315 · Cited by 2 · see citing works
Self-citationRiley, R. D., Collins, P. G. S., Archer, L., Whittle, R., Legha, A., Kirton, L., Dhiman, P., Sadatsafavi, M., Adderley, N. J., Alderman, J. E., et al. (2025) · Diagnostic and Prognostic Research · doi:10.1186/s41512-025-00204-9 · Cited by 2 · see citing works
Self-citationChung, Z., Mbuagbaw, L. C. E., Mokete, L., Bhandari, M., & Thabane, L. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-085441 · Cited by 1 · see citing works
Spin and Hacking in Machine Learning Prediction Model Studies in Dentistry: A Systematic Review
Cheng, L., Pan, Y., Wo, P., Zheng, Y., Yi, Y., Park, J. W., & Xiong, X. (2025) · Oral Diseases · doi:10.1111/odi.70023 · Cited by 1 · see citing works
Li, L. X., Hopkins, A. M., Woodman, R. J., Abuhelwa, A. Y., Gao, Y., Parent, N., Rowland, A., & Sorich, M. J. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.112082 · Cited by 1 · see citing works
Artificial Intelligence in Colorectal Cancer Supportive Care: A Scoping Review
Kantabanlang, Y., Hwang, M., Krauss, J. C., & Jiang, Y. (2025) · Seminars in Oncology Nursing · doi:10.1016/j.soncn.2025.152079 · Cited by 1 · see citing works
Sirago, G., Solarino, B., Dell'Erba, A., & Ferorelli, D. (2026) · Legal Medicine · doi:10.1016/j.legalmed.2026.102931 · Cited by 2 · see citing works
Jones, C. E., Taylor, M., Sperrin, M., & Grant, S. W. (2024) · Perfusion · doi:10.1177/02676591241237758 · Cited by 1 · see citing works
Truchot, A., Raynaud, M., & Loupy, A. (2023) · Kidney International · doi:10.1016/j.kint.2023.07.017 · Cited by 1 · see citing works
Critelli, B., Hassan, A., Lahooti, I., Noh, L., Park, J. S., Tong, K., Lahooti, A., Matzko, N., Adams, J. N., Liß, L., et al. (2024) · medRxiv · doi:10.1101/2024.06.26.24309389 · Cited by 1 · see citing works
Guolla, L., Mbuagbaw, L. C. E., Ma, J., Thabane, L., Gupta, S., & Nathan, P. C. (2025) · Health Open Research · doi:10.12688/healthopenres.13924.1
Self-citationRecent advances in machine learning-enhanced extracellular vesicle omics for oncology
Wang, Z., Lu, W. M., Guo, Z., Liu, H., Yin, Z., Ma, S., Wang, Z., Huang, Z., Sun, X. W., Xu, Y., et al. (2026) · Journal of Nanobiotechnology · doi:10.1186/s12951-026-04342-0 · Cited by 1 · see citing works
Chung, Z., Grant, L.-A., Chen, J., Mbuagbaw, L., Mokete, L., & Thabane, L. (2026) · Pilot and Feasibility Studies · doi:10.1186/s40814-026-01846-2
Ensor, J., CALSTER, B. V., Barreñada, L., Wynants, L., & Perry, B. I. (2026) · Research Square · doi:10.21203/rs.3.rs-10767123/v1 · Cited by 1 · see citing works
Self-citationZhong, X., Wang, Y. F., Chai, W., Lin, Y., Zheng, N., & Huang, J. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1875678
Arend, R. B., Peroni, B. Z., Lima, N. L., Garcia, M. C., dos Santos, R. T. F., Kerpel, D., Jung, G. S., Roman, A., Gago, G., da Silva, M. B. C., et al. (2026) · Life · doi:10.3390/life16091538
Bašković, M., & Pogorelić, Z. (2026) · Healthcare · doi:10.3390/healthcare14193269
- Authors
- Dhiman, P., Ma, J., Andaur Navarro, C. L., Speich, B., Bullock, G., Damen, J. A. A., Hooft, L., Kirtley, S., Riley, R. D., Van Calster, B., Moons, K. G. M., & Collins, G. S.
- Affiliations
- University of Oxford (GB)
- University of Basel (CH)
- Utrecht University (NL)
- Keele University (GB)
- Corresponding author
- Dhiman, Paula
- ORCID
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2023-05
- Volume
- 157
- Pages
- 120-133
- ISSN
- 0895-4356 (ISSN-L)
- PMC
- PMC11913775
- OpenAlex
- W4327748010
- Cited by
- 41
- References
- 81
- Citations by year
- 2026: 8 · 2025: 14 · 2024: 17 · 2023: 2
- FWCI
- 2.91
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Clinical Epidemiology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Lirias (KU Leuven) (submittedVersion, cc-by)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Funders
- Swiss National Science Foundation (P300PB_177933, P4P4PM_194496)
- Cancer Research UK (C49297 / A27294)
- Kom op tegen kanker (KOTK TRANS-IOTA)
- KU Leuven Hospital (C24M/20/064)
- NIHR Oxford Biomedical Research Centre
- KU Leuven
- National Institute for Health and Care Research
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- spin, machine learning, prognostic model, oncology, risk prediction, selective reporting, reporting quality, model validation, discrimination measures, research integrity
- MeSH terms
- Machine Learning, Humans, Medical Oncology, Prognosis, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
