Quality assessment standards in artificial intelligence diagnostic accuracy systematic reviews: a meta-research study
Open accessAbstract
Artificial intelligence (AI) centred diagnostic systems are increasingly recognised as robust solutions in healthcare delivery pathways. In turn, there has been a concurrent rise in secondary research studies regarding these technologies in order to influence key clinical and policymaking decisions. It is therefore essential that these studies accurately appraise methodological quality and risk of bias within shortlisted trials and reports. In order to assess whether this critical step is performed, we undertook a meta-research study evaluating adherence to the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool within AI diagnostic accuracy systematic reviews. A literature search was conducted on all studies published from 2000 to December 2020. Of 50 included reviews, 36 performed the quality assessment, of which 27 utilised the QUADAS-2 tool. Bias was reported across all four domains of QUADAS-2. Two hundred forty-three of 423 studies (57.5%) across all systematic reviews utilising QUADAS-2 reported a high or unclear risk of bias in the patient selection domain, 110 (26%) reported a high or unclear risk of bias in the index test domain, 121 (28.6%) in the reference standard domain and 157 (37.1%) in the flow and timing domain. This study demonstrates the incomplete uptake of quality assessment tools in reviews of AI-based diagnostic accuracy studies and highlights inconsistent reporting across all domains of quality assessment. Poor standards of reporting act as barriers to clinical implementation. The creation of an AI-specific extension for quality assessment tools of diagnostic accuracy AI studies may facilitate the safe translation of AI tools into clinical practice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Reference standard bias affects the way instances are labeled and consequently impacts algorithm development.
“A broader concept than annotation bias is reference standard bias, affecting the way instances are labeled and consequently impacting algorithm development.25”
From Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects (2024) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859 · see citing works
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · see citing works
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636 · see citing works
Santos, A. G., da Rocha, G. O., & de Andrade, J. B. (2019) · Scientific Reports · doi:10.1038/s41598-018-37186-2 · Cited by 8,495 · see citing works
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · see citing works
Radiomics: the bridge between medical imaging and personalized medicine
Lambin, P., Leijenaar, R. T. H., Deist, T. M., Peerlings, J., de Jong, E. E. C., van Timmeren, J., Sanduleanu, S., Larue, R. T. H. M., Even, A. J. G., Jochems, A., et al. (2017) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2017.141 · Cited by 5,688 · see citing works
Whiting, P., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. M., & Kleijnen, J. (2003) · BMC Medical Research Methodology · doi:10.1186/1471-2288-3-25 · Cited by 3,592 · see citing works
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287 · see citing works
Self-citationKey challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136 · see citing works
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639 · see citing works
Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning
Poplin, R., Varadarajan, A. V., Blumer, K., Liu, Y., McConnell, M. V., Corrado, G. S., Peng, L., & Webster, D. R. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0195-0 · Cited by 1,865 · see citing works
Attia, Z. I., Noseworthy, P. A., Lopez-Jimenez, F., Asirvatham, S. J., Deshmukh, A. J., Gersh, B. J., Carter, R. E., Yao, X., Rabinstein, A. A., Erickson, B. J., et al. (2019) · The Lancet · doi:10.1016/s0140-6736(19)31721-0 · Cited by 1,613 · see citing works
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421 · see citing works
Self-citationPreparing Medical Imaging Data for Machine Learning
Willemink, M. J., Koszek, W. A., Hardell, C., Wu, J., Fleischmann, D., Harvey, H., Folio, L. R., Summers, R. M., Rubin, D. L., & Lungren, M. P. (2020) · Radiology · doi:10.1148/radiol.2020192224 · Cited by 1,020 · see citing works
Artificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · see citing works
Evaluating Digital Health Interventions
Murray, E., Hekler, E. B., Andersson, G., Collins, L. M., Doherty, A., Hollis, C., Rivera, D. E., West, R., & Wyatt, J. C. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.008 · Cited by 847 · see citing works
Systematic Analysis of Breast Cancer Morphology Uncovers Stromal Features Associated with Survival
Beck, A. H., Sangoi, A. R., Leung, S. C. Y., Marinelli, R. J., Nielsen, T. O., van de Vijver, M. J., West, R. B., van de Rijn, M., & Koller, D. (2011) · Science Translational Medicine · doi:10.1126/scitranslmed.3002564 · Cited by 765 · see citing works
Cruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Darzi, A., Holmes, C., Yau, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1037-7 · Cited by 748 · see citing works
Self-citationJo, T., Nho, K., & Saykin, A. J. (2019) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2019.00220 · Cited by 724 · see citing works
Reporting of artificial intelligence prediction models
Collins, G. S., & Moons, K. G. M. (2019) · The Lancet · doi:10.1016/s0140-6736(19)30037-6 · Cited by 701 · see citing works
Larrazabal, A. J., Nieto, N., Peterson, V., Milone, D. H., & Ferrante, E. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1919012117 · Cited by 685 · see citing works
Machine Learning and Health Care Disparities in Dermatology
Adamson, A. S., & Smith, A. (2018) · JAMA Dermatology · doi:10.1001/jamadermatol.2018.2348 · Cited by 639 · cited 1× · see citing works
“For example, employing an AI model to detect dermatological abnormalities on dark skin resulted in higher rates of missed diagnoses further increasing the disparity in diagnosis 23,24 .”
MethodHassan, C., Spadaccini, M., Iannone, A., Maselli, R., Jovani, M., Chandrasekar, V. T., Antonelli, G., Yu, H., Areia, M., Dinis-Ribeiro, M., et al. (2021) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2020.06.059 · Cited by 529 · see citing works
Nagpal, K., Foote, D., Liu, Y., Chen, P.-H. C., Wulczyn, E., Tan, F., Olson, N., Smith, J. L., Mohtashamian, A., Wren, J. H., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0112-2 · Cited by 495 · see citing works
Reitsma, J. B., Rutjes, A. W. S., Khan, K. S., Coomarasamy, A., & Bossuyt, P. M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.02.005 · Cited by 405 · see citing works
Randomized controlled trial of a 12-week digital care program in improving low back pain
Shebib, R., Bailey, J. F., Smittenaar, P., Perez, D. A., Mecklenburg, G., & Hunter, S. (2019) · npj Digital Medicine · doi:10.1038/s41746-018-0076-7 · Cited by 341 · see citing works
Guo, C., Ashrafian, H., Ghafur, S., Fontana, G., Gardner, C., & Prime, M. S. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00314-2 · Cited by 322 · see citing works
Self-citationPellegrini, E., Ballerini, L., Hernández, M. D. C. V., Chappell, F. M., González‐Castro, V., Anblagan, D., Danso, S., Maniega, S. M., Job, D. E., Pernet, C. R., et al. (2018) · Alzheimer s & Dementia Diagnosis Assessment & Disease Monitoring · doi:10.1016/j.dadm.2018.07.004 · Cited by 297 · see citing works
Sounderajah, V., Ashrafian, H., Aggarwal, R., De Fauw, J., Denniston, A. K., Greaves, F., Karthikesalingam, A., King, D., Liu, X., Markar, S. R., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0941-1 · Cited by 270 · see citing works
Self-citationNatural and Artificial Intelligence in Neurosurgery: A Systematic Review
Senders, J. T., Arnaout, O. M., Karhade, A. V., Dasenbrock, H. H., Gormley, W. B., Broekman, M. L. D., & Smith, T. R. (2017) · Neurosurgery · doi:10.1093/neuros/nyx384 · Cited by 264 · see citing works
Yamada, M., Saito, Y., Imaoka, H., Saiko, M., Yamada, S., Kondo, H., Takamaru, H., Sakamoto, T., Sese, J., Kuchiba, A., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-50567-5 · Cited by 263 · see citing works
Blockchain vehicles for efficient Medical Record management
Vazirani, A. A., O’Donoghue, O., Brindley, D., & Meinert, E. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0211-0 · Cited by 245 · see citing works
Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis
Soffer, S., Klang, E., Shimon, O., Nachmias, N., Eliakim, R., Ben‐Horin, S., Kopylov, U., & Barash, Y. (2020) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2020.04.039 · Cited by 210 · see citing works
Langerhuizen, D. W. G., Janssen, S. J., Mallee, W. H., van den Bekerom, M. P. J., Ring, D., Kerkhoffs, G. M. M. J., Jaarsma, R. L., & Doornberg, J. N. (2019) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000000848 · Cited by 197 · see citing works
Islam, M. M., Yang, H., Poly, T. N., Jian, W., & Li, Y. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105320 · Cited by 188 · see citing works
Rajpara, S. M., Botello, A. P., Townend, J., & Ormerod, A. D. (2009) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2009.09093.x · Cited by 186 · see citing works
Harris, M. T. H., Qi, A., Jeagal, L., Torabi, N., Menzies, D., Korobitsyn, A., Pai, M., Nathavitharana, R. R., & Khan, F. A. (2019) · PLoS ONE · doi:10.1371/journal.pone.0221339 · Cited by 180 · see citing works
Mahmood, H., Shaban, M., Indave, B. I., Santos‐Silva, A. R., Rajpoot, N., & Khurram, S. A. (2020) · Oral Oncology · doi:10.1016/j.oraloncology.2020.104885 · Cited by 163 · see citing works
Ursprung, S., Beer, L., Bruining, A., Woitek, R., Stewart, G. D., Gallagher, F. A., & Sala, E. (2020) · European Radiology · doi:10.1007/s00330-020-06666-3 · Cited by 159 · see citing works
Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma
Chuchu, N., Takwoingi, Y., Dinnes, J., Matin, R. N., Bassett, O., Moreau, J. F., Bayliss, S. E., Davenport, C., Godfrey, K., O'Connell, S., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013192 · Cited by 144 · see citing works
di Ruffano, L. F., Takwoingi, Y., Dinnes, J., Chuchu, N., Bayliss, S. E., Davenport, C. F., Matin, R., Godfrey, K., O'Sullivan, C., Gulati, A., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013186 · Cited by 143 · see citing works
Li, Y., Zhang, Z., Dai, C., Dong, Q., & Badrigilan, S. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103898 · Cited by 142 · see citing works
Groß, S., Trautwein, C., Behrens, A., Winograd, R., Palm, S., Lutz, H. H., Schirin-Sokhan, R., Hecker, H., Aach, T., & Tischendorf, J. J. W. (2011) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2011.08.001 · Cited by 135 · see citing works
de Filippis, R., Carbone, E. A., Gaetano, R., Bruni, A., Pugliese, V., Segura‐Garcìa, C., & Fazio, P. D. (2019) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s202418 · Cited by 130 · see citing works
Pehrson, L. M., Nielsen, M. B., & Lauridsen, C. A. (2019) · Diagnostics · doi:10.3390/diagnostics9010029 · Cited by 126 · see citing works
Lui, T. K.-L., Guo, C., & Leung, W. K. (2020) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2020.02.033 · Cited by 126 · see citing works
Nielsen, K. B., Lautrup, M. L., Andersen, J., Savarimuthu, T. R., & Grauslund, J. (2018) · Ophthalmology Retina · doi:10.1016/j.oret.2018.10.014 · Cited by 118 · see citing works
Brain metastasis detection using machine learning: a systematic review and meta-analysis
Cho, S. J., Sunwoo, L., Baik, S. H., Bae, Y. J., Choi, B. S., & Kim, J. H. (2020) · Neuro-Oncology · doi:10.1093/neuonc/noaa232 · Cited by 113 · see citing works
Halder, A., Dey, D., & Sadhu, A. K. (2020) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-020-00320-6 · Cited by 112 · see citing works
Kamulegeya, L. H., Okello, M., Bwanika, J. M., Musinguzi, D., Lubega, W., Rusoke, D., Nassiwa, F., & Börve, A. (2019) · doi:10.1101/826057 · Cited by 111 · see citing works
Kamulegeya, L., Bwanika, J., Okello, M., Rusoke, D., Nassiwa, F., Lubega, W., Musinguzi, D., & Börve, A. (2023) · African Health Sciences · doi:10.4314/ahs.v23i2.86 · Cited by 111 · see citing works
Steardo, L., Carbone, E. A., de Filippis, R., Pisanu, C., Segura‐Garcìa, C., Squassina, A., Fazio, P. D., & Steardo, L. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00588 · Cited by 108 · see citing works
Bang, C. S., Lee, J. J., & Baik, G. H. (2020) · Journal of Medical Internet Research · doi:10.2196/21983 · Cited by 103 · see citing works
Lui, K. L., Tsui, V. W. M., & Leung, W. K. (2020) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2020.06.034 · Cited by 96 · see citing works
Azer, S. A. (2019) · World Journal of Gastrointestinal Oncology · doi:10.4251/wjgo.v11.i12.1218 · Cited by 91 · see citing works
Automated detection of nonmelanoma skin cancer using digital images: a systematic review
Marka, A., Carter, J. B., Toto, E., & Hassanpour, S. (2019) · BMC Medical Imaging · doi:10.1186/s12880-019-0307-7 · Cited by 90 · see citing works
Diagnostic concordance and discordance in digital pathology: a systematic review and meta-analysis
Azam, A. S., Miligy, I. M., Kimani, P., Maqbool, H., Hewitt, K. J., Rajpoot, N., & Snead, D. R. J. (2020) · Journal of Clinical Pathology · doi:10.1136/jclinpath-2020-206764 · Cited by 83 · see citing works
Moon, S. J., Hwang, J., Kana, R. K., Torous, J. B., & Kim, J. W. (2019) · JMIR Mental Health · doi:10.2196/14108 · Cited by 74 · see citing works
Ninatti, G., Kirienko, M., Neri, E., Sollini, M., & Chiti, A. (2020) · Diagnostics · doi:10.3390/diagnostics10060359 · Cited by 71 · see citing works
Kunze, K. N., Rossi, D. M., White, G. M., Karhade, A. V., Deng, J., Williams, B. T., & Chahla, J. A. (2020) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2020.09.012 · Cited by 70 · see citing works
Li, D., Vilmun, B. M., Carlsen, J. F., Albrecht‐Beste, E., Lauridsen, C. A., Nielsen, M. B., & Hansen, K. L. (2019) · Diagnostics · doi:10.3390/diagnostics9040207 · Cited by 65 · see citing works
Diagnostic accuracy of deep learning in orthopaedic fractures: a systematic review and meta-analysis
Yang, S., Yin, B., Cao, W., Feng, C., Fan, G., & He, S. (2020) · Clinical Radiology · doi:10.1016/j.crad.2020.05.021 · Cited by 64 · see citing works
Zhao, W., Fu, L.-R., Huang, Z.-M., Zhu, J., & Ma, B. (2019) · Medicine · doi:10.1097/md.0000000000016379 · Cited by 62 · see citing works
Computer-aided diagnosis of liver lesions using CT images: A systematic review
Nayantara, P. V., Kamath, S. R., Manjunath, K. N., & Rajagopal, K. V. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.104035 · Cited by 62 · see citing works
Islam, M. M., Poly, T. N., Walther, B. A., Yang, H., & Li, Y. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9041018 · Cited by 61 · see citing works
Nguyen, A. V. P., Blears, E. E., Ross, E., Lall, R. R., & Ortega-Barnett, J. R. (2018) · Neurosurgical FOCUS · doi:10.3171/2018.8.focus18325 · Cited by 58 · see citing works
Xu, L., Gao, J., Wang, Q., et al. (2019) · European Thyroid Journal · doi:10.1159/000504390 · Cited by 58 · see citing works
Crombé, A., Fadli, D., Italiano, A., Saut, O., Buy, X., & Kind, M. (2020) · European Journal of Radiology · doi:10.1016/j.ejrad.2020.109283 · Cited by 54 · see citing works
Bruin, W. B., Denys, D. A. J. P., & van Wingen, G. A. (2018) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2018.08.005 · Cited by 53 · see citing works
McCarthy, J., Collins, D. L., & Ducharme, S. (2018) · NeuroImage: Clinical · doi:10.1016/j.nicl.2018.08.028 · Cited by 47 · see citing works
Wang, S., Zhang, Y., Lei, S., Zhu, H., Li, J., Wang, Q., Yang, J., Chen, S., & Pan, H. (2020) · European Journal of Endocrinology · doi:10.1530/eje-19-0968 · Cited by 46 · see citing works
Groot, O. Q., Bongers, M. E. R., Ogink, P. T., Senders, J. T., Karhade, A. V., Bramer, J. A. M., Verlaan, J., & Schwab, J. H. (2020) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001360 · Cited by 46 · see citing works
Automatic Neuroimage Processing and Analysis in Stroke—A Systematic Review
Sarmento, R. M., Vasconcelos, F. F. X., Filho, P. P. R., Wu, W., & de Albuquerque, V. H. C. (2019) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2019.2934500 · Cited by 45 · see citing works
Mohan, B. P., Khan, S. R., Kassab, L. L., Ponnada, S., Chandan, S., Ali, T., Dulai, P. S., Adler, D. G., & Kochhar, G. S. (2021) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2020.07.038 · Cited by 44 · see citing works
Iannattone, P. A., Zhao, X., VanHouten, J. P., Garg, A., & Huynh, T. T. (2019) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2019.09.013 · Cited by 41 · see citing works
Murtagh, P. J. (2019) · International Journal of Ophthalmology · doi:10.18240/ijo.2020.01.22 · Cited by 37 · see citing works
Smith, A. D., López‐Solà, M., McMahon, K. L., Pedler, A. R., & Sterling, M. M. (2017) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2017.06.005 · Cited by 20 · see citing works
Jun, L., Sang, T., Yu, W.-H., et al. (2020) · Medical Ultrasonography · doi:10.11152/mu-2402 · Cited by 18 · see citing works
de Filippis, R., Carbone, E. A., Gaetano, R., et al. (2019) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 14 · see citing works
Are Study and Journal Characteristics Reliable Indicators of “Truth” in Imaging Research?
Frank, R. A., McInnes, M. D. F., Levine, D., Kressel, H. Y., Jesurum, J. S., Petrcich, W., McGrath, T. A., & Bossuyt, P. M. (2017) · Radiology · doi:10.1148/radiol.2017170586 · Cited by 13 · see citing works
Díaz, J. J. S., Tejeda, M., Yate, J. A., et al. (2017) · Archivos de cardiología de México · doi:10.1016/j.acmx.2017.03.002 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Koçak, B., Ponsiglione, A., Stanzione, A., Bluethgen, C., Santinha, J., Ugga, L., Huisman, M., Klontzas, M. E., Cannella, R., & Cuocolo, R. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.242854 · Cited by 168 · see citing works
“25 Different reference standards are often available to confirm radiological diagnosis, which may also lead to systematic errors.”
Reporting guidelines in medical artificial intelligence: a systematic review and meta-analysis
Kolbinger, F. R., Veldhuizen, G. P., Zhu, J., Truhn, D., & Kather, J. N. (2024) · Communications Medicine · doi:10.1038/s43856-024-00492-0 · Cited by 160 · see citing works
Drukker, K., Chen, W., Gichoya, J. W., Gruszauskas, N. P., Kalpathy-Cramer, J., Koyejo, S., Myers, K. J., Sá, R. C., Sahiner, B., Whitney, H. M., et al. (2023) · Journal of Medical Imaging · doi:10.1117/1.jmi.10.6.061104 · Cited by 149 · see citing works
Ng, F. Y. C., Thirunavukarasu, A. J., Cheng, H., Tan, T. F., Gutierrez, L., Lan, Y., Ong, J. C. L., Chong, Y. S., Ngiam, K. Y., Ho, D., et al. (2023) · Cell Reports Medicine · doi:10.1016/j.xcrm.2023.101230 · Cited by 113 · see citing works
Escalé-Besa, A., Vidal-Alaball, J., Miró Catalina, Q., Gracia, V. H. G., Marin-Gomez, F. X., & Fuster-Casanovas, A. (2024) · Healthcare · doi:10.3390/healthcare12121192 · Cited by 29 · see citing works
- Authors
- Jayakumar, S., Sounderajah, V., Normahani, P., Harling, L., Markar, S. R., Ashrafian, H., & Darzi, A.
- Affiliations
- Imperial College London (GB)
- Guy's Hospital (GB)
- Corresponding author
- Ashrafian, Hutan
- ORCID
- Venue
- npj Digital Medicine
- Published
- 2022-01-27
- Volume
- 5
- Issue
- 1
- Pages
- 11-11
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC8795185
- OpenAlex
- W4210788416
- Cited by
- 110
- References
- 81
- Citations by year
- 2026: 21 · 2025: 35 · 2024: 30 · 2023: 16 · 2022: 8
- FWCI
- 7.31
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Spiral (Imperial College London) (submittedVersion, other-oa)
- Funders
- National Institute for Health and Care Research (NF-SI-0510-10186, NF-SI-0617-10038, CL-2021-21-001)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- artificial intelligence, diagnostic accuracy, QUADAS-2, quality assessment, risk of bias, systematic reviews, reporting guidelines, patient selection bias, clinical implementation of AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
