The DOI matches its Crossref record; 109 of the 109 references it cites were found in a registry or an index.
A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis
Open accessAbstract
BACKGROUND: Deep learning offers considerable promise for medical diagnostics. We aimed to evaluate the diagnostic accuracy of deep learning algorithms versus health-care professionals in classifying diseases using medical imaging. METHODS: In this systematic review and meta-analysis, we searched Ovid-MEDLINE, Embase, Science Citation Index, and Conference Proceedings Citation Index for studies published from Jan 1, 2012, to June 6, 2019. Studies comparing the diagnostic performance of deep learning models and health-care professionals based on medical imaging, for any disease, were included. We excluded studies that used medical waveform data graphics material or investigated the accuracy of image segmentation rather than disease classification. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. Studies undertaking an out-of-sample external validation were included in a meta-analysis, using a unified hierarchical model. This study is registered with PROSPERO, CRD42018091176. FINDINGS: Our search identified 31 587 studies, of which 82 (describing 147 patient cohorts) were included. 69 studies provided enough data to construct contingency tables, enabling calculation of test accuracy, with sensitivity ranging from 9·7% to 100·0% (mean 79·1%, SD 0·2) and specificity ranging from 38·9% to 100·0% (mean 88·3%, SD 0·1). An out-of-sample external validation was done in 25 studies, of which 14 made the comparison between deep learning models and health-care professionals in the same sample. Comparison of the performance between health-care professionals in these 14 studies, when restricting the analysis to the contingency table for each study reporting the highest accuracy, found a pooled sensitivity of 87·0% (95% CI 83·0-90·2) for deep learning models and 86·4% (79·9-91·0) for health-care professionals, and a pooled specificity of 92·5% (95% CI 85·1-96·4) for deep learning models and 90·5% (80·6-95·7) for health-care professionals. INTERPRETATION: Our review found the diagnostic performance of deep learning models to be equivalent to that of health-care professionals. However, a major finding of the review is that few studies presented externally validated results or compared the performance of deep learning models and health-care professionals using the same sample. Additionally, poor reporting is prevalent in deep learning studies, which limits reliable interpretation of the reported diagnostic accuracy. New reporting standards that address specific challenges of deep learning could improve future studies, enabling greater confidence in the results of future evaluations of this promising technology. FUNDING: None.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The use of artificial intelligence (AI) in health care has seen remarkable growth in the past 5 years, with several publications reporting that medical AI models can perform as well as or better than clinicians across a number of tasks and specialties;1–3”
From Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review (2024) · cites this paper as [2]
In the citing paper
“The use of artificial intelligence (AI) in health care has seen remarkable growth in the past 5 years, with several publications reporting that medical AI models can perform as well as or better than clinicians across a number of tasks and specialties;1–3”
From Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review (2024) · cites this paper as [2]
In the citing paper
“Crucial to this is the process for handling and selecting input data, which is sometimes underspecified and poorly reported.1”
From The medical algorithmic audit (2022) · cites this paper as [1]
In the citing paper
“This aligns with broader meta-analytic observations that AI performance frequently approaches but does not surpass specialist-level interpretation5,10-12.”
From Structural error asymmetry and harm-weighted analysis of ChatGPT versus ICU Physicians in acid–base interpretation: a prospective observational study (2026) · cites this paper as [5]
In the citing paper
“surpassing the 93.9% specificity reported for disease detection from medical imaging in a prior meta-analysis.67”
From Assessing the performance of artificial intelligence models in evaluating inflammatory skin disease severity: a systematic review and meta-analysis (2025) · cites this paper as [67]
In the citing paper
“First, the majority of studies comparing efficiency of AI vs. clinicians are found to have unreliable design and known to lack primary replication, i.e., the validation of the algorithms developed in samples coming from other sources than the one used to train algorithms (45).”
From Artificial Intelligence in Medicine: Today and Tomorrow (2020) · cites this paper as [45]
In the citing paper
“furthermore, only one thousandth of the papers that were reviewed validated their results by having algorithms diagnose medical imaging coming from other source populations.”
From Artificial Intelligence in Medicine: Today and Tomorrow (2020) · cites this paper as [45]
In the citing paper
“although deep learning seems to be as efficient as radiologist for diagnosis, the authors pointed that 99% of studies were found not to have a reliable design”
From Artificial Intelligence in Medicine: Today and Tomorrow (2020) · cites this paper as [45]
In the citing paper
“As recent studies have indicated (45), however, comparisons frequently occur between artificial intelligence solutions and physicians, as if the two counterparts were in competition.”
From Artificial Intelligence in Medicine: Today and Tomorrow (2020) · cites this paper as [45]
In the citing paper
“although deep learning seems to be as efficient as radiologist for diagnosis, the authors pointed that 99% of studies were found not to have a reliable design”
From Artificial Intelligence in Medicine: Today and Tomorrow (2020) · cites this paper as [45]
In the citing paper
“The candidate items were also informed by findings from a previous systematic review that evaluated the diagnostic accuracy of deep-learning systems for medical imaging25.”
From Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension (2020) · cites this paper as [25]
In the citing paper
“It has been recognized that most recent AI studies are inadequately reported and existing reporting guidelines do not fully cover potential sources of bias specific to AI systems25.”
From Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension (2020) · cites this paper as [25]
In the citing paper
“The welcome emergence of RCTs seeking to evaluate newer interventions based on, or including, an AI component (called ‘AI interventions’ here)23,26–31 has similarly been met with concerns about the design and reporting25,32–34.”
From Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension (2020) · cites this paper as [25]
In the citing paper
“The candidate items were also informed by findings from a previous systematic review that evaluated the diagnostic accuracy of deep-learning systems for medical imaging17.”
From Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension (2020) · cites this paper as [17]
In the citing paper
“It has been recognized that most recent AI studies are inadequately reported and existing reporting guidelines do not fully cover potential sources of bias specific to AI systems17.”
From Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension (2020) · cites this paper as [17]
In the citing paper
“It has been recognized that most recent AI studies are inadequately reported and existing reporting guidelines do not fully cover potential sources of bias specific to AI systems17.”
From Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension (2020) · cites this paper as [17]
In the citing paper
“of randomized controlled trials seeking to evaluate the clinical efficacy of newer interventions based on, or including, an AI component (called ‘AI interventions’ here)15,18–23 has similarly been met with concerns about design and reporting17,24–26.”
From Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension (2020) · cites this paper as [17]
In the citing paper
“AI introduces several additional potential sources of bias that are currently not always reported by study authors or accounted for by existing guidelines 27 .”
From The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence (2025) · cites this paper as [27]
In the citing paper
“The diagnostic AI literature illustrates this gap: systematic reviews have documented limitations in study design, reporting standards, risk of bias, and claims about performance compared with clinicians [5,6].”
From The AI Hospital Formulary: A Practical Governance Framework for Prescribing, Monitoring, and Deprescribing Artificial Intelligence in Hospitals (2026) · cites this paper as [6]
In the citing paper
“However, the diagnostic accuracy of DL is uncertain.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Moreover, it is noted that there has been no overarching medical specialty-specific meta-analysis assessing diagnostic accuracy of DL performance, particularly in ophthalmology, respiratory medicine and breast surgery, which have the most diagnostic studies to date13.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Even within seminal studies in the field, there remains wide variation in design, methodology and reporting that limits the generalisability and applicability of their findings13.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Furthermore, the inconsistent terminology led to difficulties understanding whether an independent external test set was used to test diagnostic performance13.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Moreover, some studies only reported metrics, such as dice coefficient, F1 score, competition performance metric and Top-1 accuracy that are often used in computer science, but may be unfamiliar to clinicians13.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Furthermore, application of the same test dataset for diagnostic performance assessment of DL algorithms versus healthcare professionals was identified in only select studies13.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Therefore, although some of these methods achieved good performance using the intra-patient scheme, their results are unreliable because their evaluation process was biased [17].”
From Deep Learning for Improving the Effectiveness of Routine Prenatal Screening for Major Congenital Heart Diseases (2022) · cites this paper as [17]
In the citing paper
“The results obtained following previous research using the DL method from fetal US images can help increase the detection of fetal heart abnormalities compared to human experts [10,17,18,24].”
From Deep Learning for Improving the Effectiveness of Routine Prenatal Screening for Major Congenital Heart Diseases (2022) · cites this paper as [17]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2010) · International Journal of Surgery · doi:10.1016/j.ijsu.2010.02.007 · Cited by 12,156
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Self-citationHinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,923
Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning
Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C. S., Liang, H., Baxter, S. L., McKeown, A., Yang, G., Wu, X., Yan, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.02.010 · Cited by 4,842
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293
Coudray, N., Ocampo, P. S., Sakellaropoulos, T., Narula, N., Snuderl, M., Fenyö, D., Moreira, A. L., Razavian, N., & Tsirigos, A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0177-5 · Cited by 2,930
Research works printed but not linked: 4
- Hadsell R, Erkan A, Sermanet P, Scoffier M, Muller U, LeCun Y. Deep belief net learning in a long-range vision system for autonomous off-road driving. 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems; Nice, France; Sept 22–26, 2008: 628–33. · DOI
- Ayed NGB, Masmoudi AD, Sellami D, Abid R. New developments in the diagnostic procedures to reduce prospective biopsies breast. 2015 International Conference on Advances in Biomedical Engineering (ICABME); Beirut, Lebanon; Sept 16–18, 2015: 205–08. · DOI
- Grewal M, Srivastava MM, Kumar P, Varadarajan S. RADnet: radiologist level accuracy using deep learning for hemorrhage detection in CT scans. 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018); Washington, DC, USA; April 4–7, 2018: 281–84. · DOI
- Liu X, Faes, L, Calvert MJ, Denniston AK, CONSORT-AI/SPIRIT-AI Extension Group. Extension of the CONSORT and SPIRIT statements. Lancet (in press).
Cited by
Sorted by citations
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212
Self-citationLiu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421
Self-citationMedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification
Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., & Ni, B. (2023) · Scientific Data · doi:10.1038/s41597-022-01721-8 · Cited by 1,043
“32 Chest CT MC (3) 1,370 1,027 / 103 / 240 VesselMNIST3D Yang et al.”
Preparing Medical Imaging Data for Machine Learning
Willemink, M. J., Koszek, W. A., Hardell, C., Wu, J., Fleischmann, D., Harvey, H., Folio, L. R., Summers, R. M., Rubin, D. L., & Lungren, M. P. (2020) · Radiology · doi:10.1148/radiol.2020192224 · Cited by 1,020
A foundation model for generalizable disease detection from retinal images
Zhou, Y., Chia, M. A., Wagner, S. K., Ayhan, M. S., Williamson, D. J., Struyven, R. R., Liu, T., Xu, M., Lozano, M. G., Woodward-Court, P., et al. (2023) · Nature · doi:10.1038/s41586-023-06555-x · Cited by 1,014
Self-citationDiagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946
Vasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01772-9 · Cited by 1,039
Self-citationCollins, G. S., Dhiman, P., Andaur Navarro, C. L., Ma, J., Hooft, L., Reitsma, J. B., Logullo, P., Beam, A. L., Peng, L., Van Calster, B., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048008 · Cited by 852
Fleuren, L. M., Klausch, T. L. T., Zwager, C. L., Schoonmade, L. J., Guo, T., Roggeveen, L. F., Swart, E. L., Girbes, A. R. J., Thoral, P., Ercole, A., et al. (2020) · Intensive Care Medicine · doi:10.1007/s00134-019-05872-y · Cited by 755
Cruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Darzi, A., Holmes, C., Yau, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1037-7 · Cited by 933
“The candidate items were also informed by findings from a previous systematic review that evaluated the diagnostic accuracy of deep-learning systems for medical imaging 17 .”
Self-citationSalameh, J.-P., Bossuyt, P. M., McGrath, T. A., Thombs, B. D., Hyde, C. J., Macaskill, P., Deeks, J. J., Leeflang, M., Korevaar, D. A., Whiting, P., et al. (2020) · BMJ · doi:10.1136/bmj.m2632 · Cited by 719
Artificial Intelligence in Medicine: Today and Tomorrow
Briganti, G., & Le Moine, O. (2020) · Frontiers in Medicine · doi:10.3389/fmed.2020.00027 · Cited by 743
Liu, X., Rivera, S. C., Moher, D., Calvert, M. J., & Denniston, A. K. (2020) · BMJ · doi:10.1136/bmj.m3164 · Cited by 702
Self-citationVasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · BMJ · doi:10.1136/bmj-2022-070904 · Cited by 665
Self-citationLiu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., Ashrafian, H., Beam, A. L., Chan, A.-W., Collins, G. S., Deeks, A. D. J., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30218-1 · Cited by 555
Self-citationSignaling pathways involved in colorectal cancer: pathogenesis and targeted therapy
Li, Q., Geng, S., Luo, H., Wang, W., Mo, Y.-Q., Luo, Q., Wang, L., Song, G.-B., Sheng, J.-P., & Xu, B. (2024) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-024-01953-7 · Cited by 546
Aung, Y. Y. M., Wong, D. C. S., & Ting, D. S. W. (2021) · British Medical Bulletin · doi:10.1093/bmb/ldab016 · Cited by 511
Cruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., Ashrafian, H., Beam, A. L., Collins, G. S., Darzi, A., Deeks, J. J., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30219-3 · Cited by 486
Self-citationRivera, S. C., Liu, X., Chan, A.-W., Denniston, A. K., & Calvert, M. J. (2020) · BMJ · doi:10.1136/bmj.m3210 · Cited by 463
Self-citationRole of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review
Yin, J., Ngiam, K. Y., & Teo, H. H. (2021) · Journal of Medical Internet Research · doi:10.2196/25759 · Cited by 419
Andaur Navarro, C. L., Damen, J. A. A., Takada, T., Nijman, S. W. J., Dhiman, P., Ma, J., Collins, G. S., Bajpai, R., Riley, R. D., Moons, K. G. M., et al. (2021) · BMJ · doi:10.1136/bmj.n2281 · Cited by 415
The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence
Sounderajah, V., Guni, A., Liu, X., Collins, G. S., Karthikesalingam, A., Markar, S. R., Golub, R. M., Denniston, A. K., Shetty, S., Moher, D., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03953-8 · Cited by 352
Self-citationHan, R., Acosta, J. N., Shakeri, Z., Ioannidis, J. P. A., Topol, E. J., & Rajpurkar, P. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00047-5 · Cited by 336
Self-citationArtificial Intelligence and Patient-Centered Decision-Making
Bjerring, J. C., & Busch, J. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00391-6 · Cited by 274
McGenity, C., Clarke, E. L., Jennings, C., Matthews, G., Cartlidge, C., Freduah-Agyemang, H., Stocken, D. D., & Treanor, D. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01106-8 · Cited by 290
Chen, M., Zhang, B., Cai, Z., Seery, S., Gonzalez, M. J., Ali, N. M., Ren, R., Qiao, Y., Xue, P., & Jiang, Y. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.990604 · Cited by 258
Liu, X., Glocker, B., McCradden, M. M., Ghassemi, M., Denniston, A. K., & Oakden-Rayner, L. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00003-6 · Cited by 266
Self-citationArtificial Intelligence in Dermatology: A Primer
Young, A. T., Xiong, M., Pfau, J., Keiser, M. J., & Wei, M. L. (2020) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.02.026 · Cited by 249
Amann, J., Vetter, D., Blomberg, S. N., Christensen, H. C., Coffee, M., Gerke, S., Gilbert, T. K., Hagendorff, T., Holm, S., Livne, M., et al. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000016 · Cited by 251
“In laboratory settings, proof-of-concept AI-based CDSSs show promising performance [2,3].”
BackgroundTime to reality check the promises of machine learning-powered precision medicine.
Wilkinson, J., Arnold, K. F., Murray, E. J., van Smeden, M., Carr, K., Sippy, R., de Kamps, M., Beam, A., Konigorski, S., Lippert, C., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30200-4 · Cited by 233
Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology
Drukker, L., Noble, J. A., & Papageorghiou, A. T. (2020) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.22122 · Cited by 240
Papadimitroulas, P., Brocki, L., Christopher Chung, N., Marchadour, W., Vermet, F., Gaubert, L., Eleftheriadis, V., Plachouris, D., Visvikis, D., Kagadis, G. C., et al. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.03.009 · Cited by 226
Gao, Y., Zeng, S., Xu, X., Li, H., Yao, S., Song, K., Li, X., Chen, L., Tang, J., Xing, H., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00278-8 · Cited by 185
Self-citationBickman, L. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01065-8 · Cited by 184
The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review
Schwabe, D., Becker, K., Seyferth, M., Klaß, A., & Schaeffter, T. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01196-4 · Cited by 169
Toward an AI Era: Advances in Electronic Skins
Fu, X., Cheng, W., Wan, G., Yang, Z., & Tee, B. C. K. (2024) · Chemical Reviews · doi:10.1021/acs.chemrev.4c00049 · Cited by 172
A Clinician's Guide to Artificial Intelligence: How to Critically Appraise Machine Learning Studies
Faes, L., Liu, X., Wagner, S. K., Fu, D. J., Balaskas, K., Sim, D. A., Bachmann, L. M., Keane, P. A., & Denniston, A. K. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.2.7 · Cited by 162
Self-citationCombining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology
Agarwal, N., Moehring, A., Rajpurkar, P., & Salz, T. (2023) · doi:10.2139/ssrn.4505053 · Cited by 161
Vasey, B., Ursprung, S., Beddoe, B., Taylor, E. H., Marlow, N., Bilbro, N., Watkinson, P., & McCulloch, P. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.1276 · Cited by 154
Generative models improve fairness of medical classifiers under distribution shifts
Ktena, I., Wiles, O., Albuquerque, I., Rebuffi, S.-A., Tanno, R., Roy, A. G., Azizi, S., Belgrave, D., Kohli, P., Cemgil, T., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02838-6 · Cited by 165
Artificial intelligence in healthcare: A bibliometric analysis
Jimma, B. L. (2023) · Telematics and Informatics Reports · doi:10.1016/j.teler.2023.100041 · Cited by 133
Interactive explainable deep learning model informs prostate cancer diagnosis at MRI
Hamm, C. A., Baumgärtner, G. L., Biessmann, F., Beetz, N. L., Hartenstein, A., Savic, L. J., Froböse, K., Dräger, F., Schallenberg, S., Rudolph, M., et al. (2023) · Radiology · doi:10.1148/radiol.222276 · Cited by 133
Bridging the “last mile” gap between AI implementation and operation: “data awareness” that matters
Cabitza, F., Campagner, A., & Balsano, C. (2020) · Annals of Translational Medicine · doi:10.21037/atm.2020.03.63 · Cited by 113
Hickman, S. E., Baxter, G. C., & Gilbert, F. J. (2021) · British Journal of Cancer · doi:10.1038/s41416-021-01333-w · Cited by 110
Systematic Review of Deep Learning Image Analyses for the Diagnosis and Monitoring of Skin Disease
Choy, S. P., Kim, B. J., Paolino, A., Tan, W. R., Lim, S. M. L., Seo, J., Tan, S. P., Francis, L., Tsakok, T., Simpson, M., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00914-8 · Cited by 105
González-Gonzalo, C., Thee, E. F., Klaver, C. C. W., Lee, A. Y., Schlingemann, R. O., Tufail, A., Verbraak, F., & Sánchez, C. I. (2022) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2021.101034 · Cited by 103
Use of artificial intelligence and deep learning in fetal ultrasound imaging
Ramirez Zegarra, R., & Ghi, T. (2023) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.26130 · Cited by 102
Foundational Considerations for Artificial Intelligence Using Ophthalmic Images
Abràmoff, M. D., Cunningham, B., Patel, B., Eydelman, M. B., Leng, T., Sakamoto, T., Blodi, B., Grenon, S. M., Wolf, R. M., Manrai, A. K., et al. (2022) · Ophthalmology · doi:10.1016/j.ophtha.2021.08.023 · Cited by 96
Deep learning for imaging and detection of microorganisms
Zhang, Y., Jiang, H., Ye, T., & Juhas, M. (2021) · Trends in Microbiology · doi:10.1016/j.tim.2021.01.006 · Cited by 96
Explainability and causability in digital pathology
Plass, M., Kargl, M., Kiehl, T., Regitnig, P., Geißler, C., Evans, T., Zerbe, N., Carvalho, R., Holzinger, A., & Müller, H. (2023) · The Journal of Pathology Clinical Research · doi:10.1002/cjp2.322 · Cited by 99
Dhombres, F., Bonnard, J., Bailly, K., Maurice, P., Papageorghiou, A. T., & Jouannic, J.-M. (2022) · Journal of Medical Internet Research · doi:10.2196/35465 · Cited by 93
Lyell, D., Coiera, E., Chen, J., Shah, P., & Magrabi, F. (2021) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2020-100301 · Cited by 95
Big Data and Digitalization in Dentistry: A Systematic Review of the Ethical Issues
Favaretto, M., Shaw, D., De Clercq, E., Joda, T., & Elger, B. S. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17072495 · Cited by 94
Responsible AI in Africa: Challenges and Opportunities
(2023) · Social and Cultural Studies of Robots and AI · doi:10.1007/978-3-031-08215-3 · Cited by 82
Botwe, B. O., Antwi, W. K., Arkoh, S., & Akudjedu, T. N. (2021) · Journal of Medical Radiation Sciences · doi:10.1002/jmrs.460 · Cited by 73
Guha Roy, A., Ren, J., Azizi, S., Loh, A., Natarajan, V., Mustafa, B., Pawlowski, N., Freyberg, J., Liu, Y., Beaver, Z., et al. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102274 · Cited by 77
Could machine learning fuel a reproducibility crisis in science?
Gibney, E. (2022) · Nature · doi:10.1038/d41586-022-02035-w · Cited by 74
Ghasemzadeh, H., Hillman, R. E., & Mehta, D. D. (2024) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2023_jslhr-23-00273 · Cited by 72
Bernal, J., & Mazo, C. (2022) · Applied Sciences · doi:10.3390/app122010228 · Cited by 70
AI in healthcare: A narrative review
Väänänen, A., Haataja, K., Vehviläinen-Julkunen, K., & Toivanen, P. (2021) · F1000Research · doi:10.12688/f1000research.26997.2 · Cited by 69
Ethics of artificial intelligence in medicine
Savulescu, J., Giubilini, A., Vandersluis, R., & Mishra, A. (2024) · Singapore Medical Journal · doi:10.4103/singaporemedj.smj-2023-279 · Cited by 68
Bourcier, S., Klug, J., & Nguyen, L. S. (2021) · World Journal of Gastroenterology · doi:10.3748/wjg.v27.i26.4088 · Cited by 65
AI support for ethical decision-making around resuscitation: proceed with care
Biller-Andorno, N., Ferrario, A., Joebges, S., Krones, T., Massini, F., Barth, P., Arampatzis, G., & Krauthammer, M. (2022) · Journal of Medical Ethics · doi:10.1136/medethics-2020-106786 · Cited by 67
Ferrario, A., Facchini, A., & Termine, A. (2024) · Minds and Machines · doi:10.1007/s11023-024-09681-1 · Cited by 64
“Fueled by an increasing amount of high quality data and recent advances in ML methodologies (namely, deep learning), medical AI has demonstrated performance that often matches or exceeds that of human experts in various tasks (Bjerring & Busch, 2021; Esteva et al., 2017; Liu et al., 2019).”
The ethical requirement of explainability for AI-DSS in healthcare: a systematic review of reasons
Freyer, N., Groß, D., & Lipprandt, M. (2024) · BMC Medical Ethics · doi:10.1186/s12910-024-01103-2 · Cited by 67
Improving the quality of machine learning in health applications and clinical research
Mateen, B. A., Liley, J., Denniston, A. K., Holmes, C. C., & Vollmer, S. J. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00239-1 · Cited by 61
Self-citationNurmaini, S., Partan, R. U., Bernolian, N., Sapitri, A. I., Tutuko, B., Rachmatullah, M. N., Darmawahyuni, A., Firdaus, F., & Mose, J. C. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11216454 · Cited by 61
A survey on deep learning in medical ultrasound imaging
Song, K., Feng, J., & Chen, D. (2024) · Frontiers in Physics · doi:10.3389/fphy.2024.1398393 · Cited by 59
Lekadir, K., Osuala, R., Gallin, C., Lazrak, N., Kushibar, K., Tsakou, G., Aussó, S., Alberich, L. C., Marias, K., Tsiknakis, M., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.09658 · Cited by 58
Preti, L. M., Ardito, V., Compagni, A., Petracca, F., & Cappellaro, G. (2024) · Journal of Medical Internet Research · doi:10.2196/55897 · Cited by 62
Choudhury, A. (2022) · JMIR Human Factors · doi:10.2196/35421 · Cited by 56
Sharing Biomedical Data: Strengthening AI Development in Healthcare
Pereira, T., Morgado, J., Silva, F., Pelter, M. M., Dias, V. R., Barros, R., Freitas, C., Negrão, E., Flor de Lima, B., Correia da Silva, M., et al. (2021) · Healthcare · doi:10.3390/healthcare9070827 · Cited by 54
“AI applied to medical diagnosis has been shown to achieve, for the detection of some clinical conditions, better performance than clinicians [21].”
BackgroundChristou, C. D., & Tsoulfas, G. (2021) · World Journal of Gastroenterology · doi:10.3748/wjg.v27.i37.6191 · Cited by 54
Koch, L. M., Baumgartner, C. F., & Berens, P. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01085-w · Cited by 53
Martindale, A. P. L., Llewellyn, C. D., de Visser, R. O., Ng, B., Ngai, V., Kale, A. U., di Ruffano, L. F., Golub, R. M., Collins, G. S., Moher, D., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-45355-3 · Cited by 58
Self-citationMcLennan, S., Meyer, A., Schreyer, K., & Buyx, A. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000114 · Cited by 50
Mohan, B. P., Khan, S. R., Kassab, L. L., Ponnada, S., Chandan, S., Ali, T., Dulai, P. S., Adler, D. G., & Kochhar, G. S. (2021) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2020.07.038 · Cited by 44
Shelmerdine, S. C., Rosendahl, K., & Arthurs, O. J. (2022) · Pediatric Radiology · doi:10.1007/s00247-021-05195-5 · Cited by 41
Clay, B., Bergman, H. I., Salim, S., Pergola, G., Shalhoub, J., & Davies, A. H. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.109808 · Cited by 41
Joel, M. Z., Umrao, S., Chang, E., Choi, R., Yang, D. X., Duncan, J. S., Omuro, A., Herbst, R., Krumholz, H. M., & Aneja, S. (2022) · JCO Clinical Cancer Informatics · doi:10.1200/cci.21.00170 · Cited by 51
Exploring a new paradigm for the fetal anomaly ultrasound scan: Artificial intelligence in real time
Matthew, J., Skelton, E., Day, T. G., Zimmer, V. A., Gomez, A., Wheeler, G., Toussaint, N., Liu, T., Budd, S., Lloyd, K., et al. (2022) · Prenatal Diagnosis · doi:10.1002/pd.6059 · Cited by 41
Galsgaard, A., Doorschodt, T., Holten, A.-L., Müller, F. C., Ploug Boesen, M., & Maas, M. (2022) · European Journal of Radiology · doi:10.1016/j.ejrad.2022.110231 · Cited by 38
Artificial Intelligence in Dermatopathology: New Insights and Perspectives
Cazzato, G., Colagrande, A., Cimmino, A., Arezzo, F., Loizzi, V., Caporusso, C., Marangio, M., Foti, C., Romita, P., Lospalluti, L., et al. (2021) · Dermatopathology · doi:10.3390/dermatopathology8030044 · Cited by 37
“For example, in some studies [9,10], artificial intelligence algorithms match or outperform doctors in disease detection related to medical imaging.”
BackgroundWalker, M. C., Willner, I., Miguel, O. X., Murphy, M. S. Q., El-Chaâr, D., Moretti, F., Dingwall Harvey, A. L. J., Rennicks White, R., Muldoon, K. A., Carrington, A. M., et al. (2022) · PLOS ONE · doi:10.1371/journal.pone.0269323 · Cited by 31
Holm, S., Stanton, C., & Bartlett, B. (2021) · Health Care Analysis · doi:10.1007/s10728-021-00430-4 · Cited by 32
Differences between human and machine perception in medical diagnosis
Makino, T., Jastrzębski, S., Oleszkiewicz, W., Chacko, C., Ehrenpreis, R., Samreen, N., Chhor, C., Kim, E., Lee, J., Pysarenko, K., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-10526-z · Cited by 30
Pfob, A., Sidey-Gibbons, C., Barr, R. G., Duda, V., Alwafai, Z., Balleyguier, C., Clevert, D.-A., Fastner, S., Gomez, C., Goncalo, M., et al. (2022) · European Radiology · doi:10.1007/s00330-021-08519-z · Cited by 29
Lo, B., Liu, Z., Bendtsen, F., Igel, C., Vind, I., & Burisch, J. (2022) · American Journal of Gastroenterology · doi:10.14309/ajg.0000000000001904 · Cited by 29
Wang, T.-W., Shiao, Y.-C., Hong, J.-S., Lee, W.-K., Hsu, M.-S., Cheng, H.-M., Yang, H.-C., Lee, C.-C., Pan, H.-C., You, W. C., et al. (2024) · Mayo Clinic Proceedings: Digital Health · doi:10.1016/j.mcpdig.2024.01.002 · Cited by 23
Patient-Generated Health Photos and Videos Across Health and Well-being Contexts: Scoping Review
Ploderer, B., Rezaei Aghdam, A., & Burns, K. (2022) · Journal of Medical Internet Research · doi:10.2196/28867 · Cited by 18
AI support for ethical decision-making around resuscitation: proceed with care
Biller-Andorno, N., Ferrario, A., Joebges, S., Krones, T., Massini, F., Barth, P., Arampatzis, G., & Krauthammer, M. (2020) · doi:10.1101/2020.08.17.20171769 · Cited by 67
Koçak, B., Keleş, A., & Köse, F. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.232604 · Cited by 12
Cai, Z. R., Kim, J., Rezaei, S. J., Chen, M. L., Touma, F., Zhu, C., Onyeka, S., Fonjungo, F., Omiye, J. A., Krakowski, I., et al. (2025) · British Journal of Dermatology · doi:10.1093/bjd/ljaf250 · Cited by 8
“67 This underscores the strong performance of AI, not just in identifying diseases, but also in evaluating their severity.”
Gulen, D., Gözden, H. E., Ekin, S., & Ceylan, I. (2026) · Scientific Reports · doi:10.1038/s41598-026-44576-4 · Cited by 1
“This aligns with broader meta-analytic observations that AI performance frequently approaches but does not surpass specialist-level interpretation 5,10-12 .”
The pitfalls of patient-led AI use: a case study in clinical neuropsychology
Moore, I. J., Embry, E., & Padala, P. R. (2026) · The American Journal of Geriatric Psychiatry: Open Science, Education, and Practice · doi:10.1016/j.osep.2025.11.001 · Cited by 1
A quantitative analysis of global AI medical studies: gaps in randomized controlled trials
Wang, Q., Yang, Y., Qiu, L., Xu, Y., Sun, R., Xue, P., & Qiao, Y. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02698-z · Cited by 4
Metadata
- Authors
- Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., Ledsam, J. R., Schmid, M. K., Balaskas, K., Topol, E. J., Bachmann, L. M., Keane, P. A., & Denniston, A. K.
- Affiliations
- University Hospitals Birmingham NHS Foundation Trust (GB)
- Moorfields Eye Hospital NHS Foundation Trust (GB)
- University of Birmingham (GB)
- Google DeepMind (United Kingdom) (GB)
- Luzerner Kantonsspital (CH)
- Scripps Research Institute (US)
- Medignition (Switzerland) (CH)
- Corresponding author
- Denniston, Alastair K
- ORCID
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2019-10
- Volume
- 1
- Issue
- 6
- Pages
- e271-e297
- ISSN
- 2589-7500 (ISSN-L)
- OpenAlex
- W2976398475
- Cited by
- 1,883
- References
- 109
- Citations by year
- 2026: 287 · 2025: 273 · 2024: 305 · 2023: 287 · 2022: 258 · 2021: 269 · 2020: 184 · 2019: 20
- FWCI
- 8.73
- Type
- journal article
- Language
- en
- Update notices
- erratum (2019-11-01)
- Open access
- gold
- doi.org
- Licence
- Locations
- The Lancet Digital Health (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
- Funders
- National Institute for Health Research
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- deep learning, healthcare workers, diagnostic accuracy, diagnostic imaging, disease classification, sensitivity and specificity, external validation, artificial intelligence in healthcare, diagnostic performance comparison
- MeSH terms
- Deep Learning, Diagnostic Imaging, Health Personnel, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
