The DOI matches its Crossref record; 109 of the 109 references it cites were found in a registry or an index.
A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis
Open accessAbstract
BACKGROUND: Deep learning offers considerable promise for medical diagnostics. We aimed to evaluate the diagnostic accuracy of deep learning algorithms versus health-care professionals in classifying diseases using medical imaging. METHODS: In this systematic review and meta-analysis, we searched Ovid-MEDLINE, Embase, Science Citation Index, and Conference Proceedings Citation Index for studies published from Jan 1, 2012, to June 6, 2019. Studies comparing the diagnostic performance of deep learning models and health-care professionals based on medical imaging, for any disease, were included. We excluded studies that used medical waveform data graphics material or investigated the accuracy of image segmentation rather than disease classification. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. Studies undertaking an out-of-sample external validation were included in a meta-analysis, using a unified hierarchical model. This study is registered with PROSPERO, CRD42018091176. FINDINGS: Our search identified 31 587 studies, of which 82 (describing 147 patient cohorts) were included. 69 studies provided enough data to construct contingency tables, enabling calculation of test accuracy, with sensitivity ranging from 9·7% to 100·0% (mean 79·1%, SD 0·2) and specificity ranging from 38·9% to 100·0% (mean 88·3%, SD 0·1). An out-of-sample external validation was done in 25 studies, of which 14 made the comparison between deep learning models and health-care professionals in the same sample. Comparison of the performance between health-care professionals in these 14 studies, when restricting the analysis to the contingency table for each study reporting the highest accuracy, found a pooled sensitivity of 87·0% (95% CI 83·0-90·2) for deep learning models and 86·4% (79·9-91·0) for health-care professionals, and a pooled specificity of 92·5% (95% CI 85·1-96·4) for deep learning models and 90·5% (80·6-95·7) for health-care professionals. INTERPRETATION: Our review found the diagnostic performance of deep learning models to be equivalent to that of health-care professionals. However, a major finding of the review is that few studies presented externally validated results or compared the performance of deep learning models and health-care professionals using the same sample. Additionally, poor reporting is prevalent in deep learning studies, which limits reliable interpretation of the reported diagnostic accuracy. New reporting standards that address specific challenges of deep learning could improve future studies, enabling greater confidence in the results of future evaluations of this promising technology. FUNDING: None.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The use of artificial intelligence (AI) in health care has seen remarkable growth in the past 5 years, with several publications reporting that medical AI models can perform as well as or better than clinicians across a number of tasks and specialties;1–3”
From Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review (2024) · cites this paper as [2]
In the citing paper
“The use of artificial intelligence (AI) in health care has seen remarkable growth in the past 5 years, with several publications reporting that medical AI models can perform as well as or better than clinicians across a number of tasks and specialties;1–3”
From Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review (2024) · cites this paper as [2]
In the citing paper
“Crucial to this is the process for handling and selecting input data, which is sometimes underspecified and poorly reported.1”
From The medical algorithmic audit (2022) · cites this paper as [1]
In the citing paper
“This aligns with broader meta-analytic observations that AI performance frequently approaches but does not surpass specialist-level interpretation5,10-12.”
From Structural error asymmetry and harm-weighted analysis of ChatGPT versus ICU Physicians in acid–base interpretation: a prospective observational study (2026) · cites this paper as [5]
In the citing paper
“surpassing the 93.9% specificity reported for disease detection from medical imaging in a prior meta-analysis.67”
From Assessing the performance of artificial intelligence models in evaluating inflammatory skin disease severity: a systematic review and meta-analysis (2025) · cites this paper as [67]
In the citing paper
“First, the majority of studies comparing efficiency of AI vs. clinicians are found to have unreliable design and known to lack primary replication, i.e., the validation of the algorithms developed in samples coming from other sources than the one used to train algorithms (45).”
From Artificial Intelligence in Medicine: Today and Tomorrow (2020) · cites this paper as [45]
In the citing paper
“furthermore, only one thousandth of the papers that were reviewed validated their results by having algorithms diagnose medical imaging coming from other source populations.”
From Artificial Intelligence in Medicine: Today and Tomorrow (2020) · cites this paper as [45]
In the citing paper
“although deep learning seems to be as efficient as radiologist for diagnosis, the authors pointed that 99% of studies were found not to have a reliable design”
From Artificial Intelligence in Medicine: Today and Tomorrow (2020) · cites this paper as [45]
In the citing paper
“As recent studies have indicated (45), however, comparisons frequently occur between artificial intelligence solutions and physicians, as if the two counterparts were in competition.”
From Artificial Intelligence in Medicine: Today and Tomorrow (2020) · cites this paper as [45]
In the citing paper
“although deep learning seems to be as efficient as radiologist for diagnosis, the authors pointed that 99% of studies were found not to have a reliable design”
From Artificial Intelligence in Medicine: Today and Tomorrow (2020) · cites this paper as [45]
In the citing paper
“The candidate items were also informed by findings from a previous systematic review that evaluated the diagnostic accuracy of deep-learning systems for medical imaging25.”
From Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension (2020) · cites this paper as [25]
In the citing paper
“It has been recognized that most recent AI studies are inadequately reported and existing reporting guidelines do not fully cover potential sources of bias specific to AI systems25.”
From Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension (2020) · cites this paper as [25]
In the citing paper
“The welcome emergence of RCTs seeking to evaluate newer interventions based on, or including, an AI component (called ‘AI interventions’ here)23,26–31 has similarly been met with concerns about the design and reporting25,32–34.”
From Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension (2020) · cites this paper as [25]
In the citing paper
“The candidate items were also informed by findings from a previous systematic review that evaluated the diagnostic accuracy of deep-learning systems for medical imaging17.”
From Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension (2020) · cites this paper as [17]
In the citing paper
“It has been recognized that most recent AI studies are inadequately reported and existing reporting guidelines do not fully cover potential sources of bias specific to AI systems17.”
From Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension (2020) · cites this paper as [17]
In the citing paper
“It has been recognized that most recent AI studies are inadequately reported and existing reporting guidelines do not fully cover potential sources of bias specific to AI systems17.”
From Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension (2020) · cites this paper as [17]
In the citing paper
“of randomized controlled trials seeking to evaluate the clinical efficacy of newer interventions based on, or including, an AI component (called ‘AI interventions’ here)15,18–23 has similarly been met with concerns about design and reporting17,24–26.”
From Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension (2020) · cites this paper as [17]
In the citing paper
“AI introduces several additional potential sources of bias that are currently not always reported by study authors or accounted for by existing guidelines 27 .”
From The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence (2025) · cites this paper as [27]
In the citing paper
“The diagnostic AI literature illustrates this gap: systematic reviews have documented limitations in study design, reporting standards, risk of bias, and claims about performance compared with clinicians [5,6].”
From The AI Hospital Formulary: A Practical Governance Framework for Prescribing, Monitoring, and Deprescribing Artificial Intelligence in Hospitals (2026) · cites this paper as [6]
In the citing paper
“However, the diagnostic accuracy of DL is uncertain.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Moreover, it is noted that there has been no overarching medical specialty-specific meta-analysis assessing diagnostic accuracy of DL performance, particularly in ophthalmology, respiratory medicine and breast surgery, which have the most diagnostic studies to date13.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Even within seminal studies in the field, there remains wide variation in design, methodology and reporting that limits the generalisability and applicability of their findings13.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Furthermore, the inconsistent terminology led to difficulties understanding whether an independent external test set was used to test diagnostic performance13.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Moreover, some studies only reported metrics, such as dice coefficient, F1 score, competition performance metric and Top-1 accuracy that are often used in computer science, but may be unfamiliar to clinicians13.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Furthermore, application of the same test dataset for diagnostic performance assessment of DL algorithms versus healthcare professionals was identified in only select studies13.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [13]
In the citing paper
“Therefore, although some of these methods achieved good performance using the intra-patient scheme, their results are unreliable because their evaluation process was biased [17].”
From Deep Learning for Improving the Effectiveness of Routine Prenatal Screening for Major Congenital Heart Diseases (2022) · cites this paper as [17]
In the citing paper
“The results obtained following previous research using the DL method from fetal US images can help increase the detection of fetal heart abnormalities compared to human experts [10,17,18,24].”
From Deep Learning for Improving the Effectiveness of Routine Prenatal Screening for Major Congenital Heart Diseases (2022) · cites this paper as [17]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2010) · International Journal of Surgery · doi:10.1016/j.ijsu.2010.02.007 · Cited by 12,156
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Self-citationHinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,923
Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning
Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C. S., Liang, H., Baxter, S. L., McKeown, A., Yang, G., Wu, X., Yan, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.02.010 · Cited by 4,842
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293
Coudray, N., Ocampo, P. S., Sakellaropoulos, T., Narula, N., Snuderl, M., Fenyö, D., Moreira, A. L., Razavian, N., & Tsirigos, A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0177-5 · Cited by 2,930
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639
Self-citationArdila, D., Kiraly, A. P., Bharadwaj, S., Choi, B., Reicher, J. J., Peng, L., Tse, D., Etemadi, M., Ye, W., Corrado, G., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0447-x · Cited by 2,167
Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
Towards better clinical prediction models: seven steps for development and an ABCD for validation
Steyerberg, E. W., & Vergouwe, Y. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu207 · Cited by 1,928
Regression modelling strategies for improved prognostic prediction
Harrell, F. E., Lee, K. L., Califf, R. M., Pryor, D. B., & Rosati, R. A. (1984) · Statistics in Medicine · doi:10.1002/sim.4780030207 · Cited by 1,804
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719
Haenssle, H. A., Fink, C., Schneiderbauer, R., Toberer, F., Buhl, T., Blum, A., Kalloo, A., Hassen, A. B. H., Thomas, L., Enk, A., et al. (2018) · Annals of Oncology · doi:10.1093/annonc/mdy166 · Cited by 1,571
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514
Rajpurkar, P., Irvin, J., Ball, R. L., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C. P., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002686 · Cited by 1,449
Luo, W., Phung, D., Tran, T., Gupta, S., Rana, S., Karmakar, C., Shilton, A., Yearwood, J., Dimitrova, N., Ho, T. B., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5870 · Cited by 988
Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2587640 · Cited by 875
Han, S. S., Kim, M. S., Lim, W., Park, G., Park, I., & Chang, S. E. (2018) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2018.01.028 · Cited by 740
Bien, N., Rajpurkar, P., Ball, R. L., Irvin, J., Park, A., Jones, E., Bereket, M., Patel, B. N., Yeom, K. W., Shpanskaya, K., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002699 · Cited by 734
Reporting of artificial intelligence prediction models
Collins, G. S., & Moons, K. G. M. (2019) · The Lancet · doi:10.1016/s0140-6736(19)30037-6 · Cited by 701
Deep neural network improves fracture detection by clinicians
Lindsey, R., Daluiski, A., Chopra, S., Lachapelle, A. S., Mozer, M. C., Sicular, S., Hanel, D. P., Gardner, M. J., Gupta, A., Hotchkiss, R., et al. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1806905115 · Cited by 667
Deep learning ensembles for melanoma recognition in dermoscopy images
Codella, N., Nguyen, Q.-B., Pankanti, S., Gutman, D. A., Helba, B., Halpern, A. C., & Smith, J. R. (2017) · IBM Journal of Research and Development · doi:10.1147/jrd.2017.2708299 · Cited by 665
Brown, J. M., Campbell, J. P., Beers, A., Chang, K., Ostmo, S., Chan, R. V. P., Dy, J., Erdogmus, D., Ioannidis, S., Kalpathy-Cramer, J., et al. (2018) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2018.1934 · Cited by 657
Adapting to Artificial Intelligence: Radiologists and Pathologists as Information Specialists
Jha, S., & Topol, E. J. (2016) · JAMA · doi:10.1001/jama.2016.17438 · Cited by 622
Self-citationFully Automated Detection and Quantification of Macular Fluid in OCT Using Deep Learning
Schlegl, T., Waldstein, S. M., Bogunović, H., Endstraßer, F., Sadeghipour, A., Philip, A.-M., Podkowinski, D., Gerendas, B. S., Langs, G., & Schmidt-Erfurth, U. M. (2017) · Ophthalmology · doi:10.1016/j.ophtha.2017.10.031 · Cited by 601
A Deep Learning Model to Predict a Diagnosis of Alzheimer Disease by Using 18 F-FDG PET of the Brain
Ding, Y., Sohn, J. H., Kawczynski, M. G., Trivedi, H., Harnish, R., Jenkins, N. W., Lituiev, D., Copeland, T. P., Aboian, M. S., Mari Aparici, C., et al. (2019) · Radiology · doi:10.1148/radiol.2018180958 · Cited by 577
Nam, J. G., Park, S., Hwang, E. J., Lee, J. H., Jin, K.-N., Lim, K. Y., Vu, T. H., Sohn, J. H., Hwang, S., Goo, J. M., et al. (2019) · Radiology · doi:10.1148/radiol.2018180237 · Cited by 539
Artificial intelligence for analyzing orthopedic trauma radiographs
Olczak, J., Fahlberg, N., Maki, A., Razavian, A. S., Jilert, A., Stark, A., Sköldenberg, O. G., & Gordon, M. (2017) · Acta Orthopaedica · doi:10.1080/17453674.2017.1344459 · Cited by 502
Machine Learning and the Profession of Medicine
Darcy, A. M., Louie, A. K., & Roberts, L. W. (2016) · JAMA · doi:10.1001/jama.2015.18421 · Cited by 448
Hwang, E. J., Park, S., Jin, K.-N., Kim, J. I., Choi, S. Y., Lee, J. H., Goo, J. M., Aum, J., Yim, J.-J., Cohen, J. G., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.1095 · Cited by 447
Lee, H., Yune, S., Mansouri, M., Kim, M., Tajmir, S. H., Guerrier, C. E., Ebert, S. A., Pomerantz, S. R., Romero, J. M., Kamalian, S., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0324-9 · Cited by 441
Li, X., Zhang, S., Zhang, Q., Wei, X., Pan, Y., Zhao, J., Xin, X., Qin, C., Wang, X., Li, J., et al. (2018) · The Lancet Oncology · doi:10.1016/s1470-2045(18)30762-9 · Cited by 439
van Grinsven, M. J. J. P., van Ginneken, B., Hoyng, C. B., Theelen, T., & Sanchez, C. I. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2526689 · Cited by 426
Sayres, R., Taly, A., Rahimy, E., Blumer, K., Coz, D., Hammel, N., Krause, J., Narayanaswamy, A., Rastegar, Z., Wu, D., et al. (2019) · Ophthalmology · doi:10.1016/j.ophtha.2018.11.016 · Cited by 418
Walsh, S. L. F., Calandriello, L., Silva, M., & Sverzellati, N. (2018) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(18)30286-8 · Cited by 390
Peng, Y., Dharssi, S., Chen, Q., Keenan, T. D. L., Agrón, E., Wong, W. T., Chew, E. Y., & Lu, Z. (2018) · Ophthalmology · doi:10.1016/j.ophtha.2018.11.015 · Cited by 363
Long, E., Lin, H., Liu, Z., Wu, X., Wang, L., Jiang, J., An, Y., Lin, Z., Li, X., Chen, J., et al. (2017) · Nature Biomedical Engineering · doi:10.1038/s41551-016-0024 · Cited by 358
Self-citationBecker, A. S., Marcon, M., Ghafoor, S., Wurnig, M. C., Frauenfelder, T., & Boss, A. (2017) · Investigative Radiology · doi:10.1097/rli.0000000000000358 · Cited by 352
Hamm, C. A., Wang, C. J., Savic, L. J., Ferrante, M., Schobert, I. T., Schlachter, T. R., Lin, M., Duncan, J. S., Weinreb, J. C., Chapiro, J., et al. (2019) · European Radiology · doi:10.1007/s00330-019-06205-9 · Cited by 341
Assessing the value of diagnostic tests: a framework for designing and evaluating trials
di Ruffano, L. F., Hyde, C., McCaffery, K. J., Bossuyt, P. M., & Deeks, J. J. (2012) · BMJ · doi:10.1136/bmj.e686 · Cited by 337
Fujisawa, Y., Otomo, Y., Ogata, Y., Nakamura, Y., Fujita, R., Ishitsuka, Y., Watanabe, R., Okiyama, N., Ohara, K., & Fujimoto, M. (2018) · British Journal of Dermatology · doi:10.1111/bjd.16924 · Cited by 324
Learning long‐range vision for autonomous off‐road driving
Hadsell, R. T., Sermanet, P., Ben, J., Erkan, A., Scoffier, M., Kavukcuoglu, K., Müller, U., & LeCun, Y. (2009) · Journal of Field Robotics · doi:10.1002/rob.20276 · Cited by 314
Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks
Tschandl, P., Rosendahl, C., Akay, B., Argenziano, G., Blum, A., Braun, R., Cabo, H., Gourhant, J., Kreusch, J., Lallas, A., et al. (2019) · JAMA dermatology · doi:10.1001/jamadermatol.2018.4378 · Cited by 313
Han, S. S., Park, G. H., Lim, W., Kim, M. S., Na, J. I., Park, I., & Chang, S. E. (2018) · PLoS ONE · doi:10.1371/journal.pone.0191493 · Cited by 305
Brinker, T. J., Hekler, A., Enk, A. H., Klode, J., Hauschild, A., Berking, C., Schilling, B., Haferkamp, S., Schadendorf, D., Fröhling, S., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.02.005 · Cited by 304
Shichijo, S., Nomura, S., Aoyama, K., Nishikawa, Y., Miura, M., Shinagawa, T., Takiyama, H., Tanimoto, T. E., Ishihara, S., Matsuo, K., et al. (2017) · EBioMedicine · doi:10.1016/j.ebiom.2017.10.014 · Cited by 301
Deep learning predicts hip fracture using confounding patient and healthcare variables
Badgeley, M. A., Zech, J. R., Oakden-Rayner, L., Glicksberg, B. S., Liu, M., Gale, W., McConnell, M. V., Percha, B., Snyder, T. M., & Dudley, J. T. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0105-1 · Cited by 290
(2017) · Radboud Repository (Radboud University) · Cited by 284
Wang, H., Zhou, Z., Li, Y., Chen, Z., Lu, P., Wang, W., Liu, W., & Yu, L. (2017) · EJNMMI Research · doi:10.1186/s13550-017-0260-9 · Cited by 276
Urakawa, T., Tanaka, Y., Goto, S., Matsuzawa, H., Watanabe, K., & Endo, N. (2018) · Skeletal Radiology · doi:10.1007/s00256-018-3016-3 · Cited by 271
Gómez-Valverde, J. J., Antón, A., Fatti, G., Liefers, B. J., Cabarcos, A. H., Santos, A., Sánchez, C. I., & Ledesma‐Carbayo, M. J. (2019) · Biomedical Optics Express · doi:10.1364/boe.10.000892 · Cited by 270
A deep neural network improves endoscopic detection of early gastric cancer without blind spots
Wu, L., Zhou, W., Wan, X., Zhang, J., Shen, L., Hu, S., Ding, Q., Mu, G., Yin, A., Huang, X., et al. (2019) · Endoscopy · doi:10.1055/a-0855-3532 · Cited by 261
Ye, H., Gao, F., Yin, Y., Guo, D., Zhao, P., Lü, Y., Wang, X., Bai, J., Cao, K., Song, Q., et al. (2019) · European Radiology · doi:10.1007/s00330-019-06163-2 · Cited by 255
Byra, M., Galperin, M., Ojeda‐Fournier, H., Olson, L., O'Boyle, M., Comstock, C., & Andre, M. (2019) · Medical Physics · doi:10.1002/mp.13361 · Cited by 253
Hwang, E. J., Park, S., Jin, K.-N., Kim, J. I., Choi, S. Y., Lee, J. H., Goo, J. M., Aum, J., Yim, J.-J., Park, C. M., et al. (2019) · Clinical Infectious Diseases · doi:10.1093/cid/ciy967 · Cited by 241
Ruamviboonsuk, P., Krause, J., Chotcomwongse, P., Sayres, R., Raman, R., Widner, K., Campana, B. J. L., Phene, S., Hemarat, K., Tadarati, M., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0099-8 · Cited by 233
Choi, K. J., Jang, J. K., Lee, S. S., Sung, Y. S., Shim, W. H., Kim, H. S., Yun, J., Choi, J., Lee, Y., Kang, B.-K., et al. (2018) · Radiology · doi:10.1148/radiol.2018180763 · Cited by 226
Zhao, W., Yang, J., Sun, Y., Li, C., Wu, W., Jin, L., Yang, Z., Ni, B., Gao, P., Wang, P., et al. (2018) · Cancer Research · doi:10.1158/0008-5472.can-18-0696 · Cited by 226
Joint Prostate Cancer Detection and Gleason Score Prediction in mp-MRI via FocalNet
Cao, R., Bajgiran, A. M., Mirak, S. A., Shakeri, S., Zhong, X., Enzmann, D. R., Raman, S. S., & Sung, K. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2901928 · Cited by 224
Burlina, P., Pacheco, K. D., Joshi, N., Freund, D., & Bressler, N. M. (2017) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2017.01.018 · Cited by 219
Harbord, R. M., Whiting, P. F., Sterne, J. A. C., Egger, M., Deeks, J. J., Shang, A., & Bachmann, L. M. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.09.013 · Cited by 215
Self-citationApplication of Convolutional Neural Network in the Diagnosis of Jaw Tumors
Poedjiastoeti, W., & Suebnukarn, S. (2018) · Healthcare Informatics Research · doi:10.4258/hir.2018.24.3.236 · Cited by 208
Assessment of Convolutional Neural Networks for Automated Classification of Chest Radiographs
Dunnmon, J. A., Yi, D., Langlotz, C. P., Ré, C., Rubin, D. L., & Lungren, M. P. (2019) · Radiology · doi:10.1148/radiol.2018181422 · Cited by 206
Deep learning in chest radiography: Detection of findings and presence of change
Singh, R., Kalra, M. K., Nitiwarangkul, C., Patti, J. A., Homayounieh, F., Padole, A., Rao, P., Putha, P., Muse, V. V., Sharma, A., et al. (2018) · PLOS ONE · doi:10.1371/journal.pone.0204155 · Cited by 195
Artificial intelligence-based decision-making for age-related macular degeneration
Hwang, D.-K., Hsu, C.-C., Chang, K.-J., Chao, D., Sun, C.-H., Jheng, Y.-C., Yarmishyn, A. A., Wu, J.-C., Tsai, C.-Y., Wang, M.-L., et al. (2019) · Theranostics · doi:10.7150/thno.28447 · Cited by 189
Ariji, Y., Fukuda, M., Kise, Y., Nozawa, M., Yanashita, Y., Fujita, H., Katsumata, A., & Ariji, E. (2018) · Oral Surgery Oral Medicine Oral Pathology and Oral Radiology · doi:10.1016/j.oooo.2018.10.002 · Cited by 180
Fujioka, T., Kubota, K., Mori, M., Kikuchi, Y., Katsuta, L., Kasahara, M., Oda, G., Ishiba, T., Nakagawa, T., & Tateishi, U. (2019) · Japanese Journal of Radiology · doi:10.1007/s11604-019-00831-5 · Cited by 177
Lu, W. P., Tong, Y., Yu, Y., Xing, Y., Chen, C., & Shen, Y. (2018) · Translational Vision Science & Technology · doi:10.1167/tvst.7.6.41 · Cited by 176
Burlina, P., Joshi, N., Pacheco, K. D., et al. (2018) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2018.4118 · Cited by 168
Zhang, C., Sun, X., Dang, K., Li, K., Guo, X.-W., Chang, J., Yu, Z.-Q., Huang, F.-Y., Wu, Y.-S., Liang, Z., et al. (2019) · The Oncologist · doi:10.1634/theoncologist.2018-0908 · Cited by 168
Wang, L., Yang, S., Yang, S., Zhao, C., Tian, G., Gao, Y., Chen, Y., & Lu, Y. (2019) · World Journal of Surgical Oncology · doi:10.1186/s12957-019-1558-z · Cited by 167
Multitask Cascade Convolution Neural Networks for Automatic Thyroid Nodule Detection and Recognition
Song, W., Li, S., Liu, J. Y., Qin, H., Zhang, B., Zhang, S., & Hao, A. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2852718 · Cited by 165
Nakagawa, K., Ishihara, R., Aoyama, K., Ohmori, M., Nakahira, H., Matsuura, N., Shichijo, S., Nishida, T., Yamada, T., Yamaguchi, S., et al. (2019) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2019.04.245 · Cited by 158
Becker, A. S., Mueller, M., Stoffel, E., Marcon, M., Ghafoor, S., & Boss, A. (2018) · The British Journal of Radiology · doi:10.1259/bjr.20170576 · Cited by 157
Big data and medical research in China
Zhang, L., Wang, H., Li, Q., Zhao, M.-H., & Zhan, Q.-M. (2018) · BMJ · doi:10.1136/bmj.j5910 · Cited by 148
Adams, M., Chen, W., Holcdorf, D., McCusker, M. W., Howe, P. D. L., & Gaillard, F. (2018) · Journal of Medical Imaging and Radiation Oncology · doi:10.1111/1754-9485.12828 · Cited by 148
Reporting guidelines for clinical trials evaluating artificial intelligence interventions are needed
The CONSORT-AI and SPIRIT-AI Steering Group (2019) · Nature Medicine · doi:10.1038/s41591-019-0603-3 · Cited by 143
Acral melanoma detection using a convolutional neural network for dermoscopy images
Yu, C., Yang, S., Kim, W. O., Jung, J. W., Chung, K. Y., Lee, S. W., & Oh, B. H. (2018) · PLoS ONE · doi:10.1371/journal.pone.0193321 · Cited by 130
Deep Learning in Diagnosis of Maxillary Sinusitis Using Conventional Radiography
Kim, Y., Lee, K. J., Sunwoo, L., Choi, D., Nam, C.-M., Cho, J., Kim, J., Bae, Y. J., Yoo, R., Choi, B. S., et al. (2018) · Investigative Radiology · doi:10.1097/rli.0000000000000503 · Cited by 111
Deep convolutional neural network for the diagnosis of thyroid nodules on ultrasound
Ko, S. Y., Lee, J. H., Yoon, J. H., Na, H., Hong, E., Han, K., Jung, I., Kim, E., Moon, H. J., Park, V. Y., et al. (2019) · Head & Neck · doi:10.1002/hed.25415 · Cited by 104
Choi, J. S., Han, B.-K., Ko, E. S., Bae, J. M., Ko, E. Y., Song, S. H., Kwon, M.-R., Shin, J. H., & Hahn, S. Y. (2019) · Korean Journal of Radiology · doi:10.3348/kjr.2018.0530 · Cited by 104
The fate of medicine in the time of AI
Coiera, E. W. (2018) · The Lancet · doi:10.1016/s0140-6736(18)31925-1 · Cited by 96
Kumagai, Y., Takubo, K., Kawada, K., et al. (2018) · Esophagus · doi:10.1007/s10388-018-0651-7 · Cited by 95
Matsuba, S., Tabuchi, H., Ohsugi, H., Enno, H., Ishitobi, N., Masumoto, H., & Kiuchi, Y. (2018) · International Ophthalmology · doi:10.1007/s10792-018-0940-0 · Cited by 91
Li, C., Jing, B., Ke, L., Li, B., Xia, W., He, C., Qian, C., Zhao, C., Mai, H., Chen, M., et al. (2018) · 癌症：英文版 · doi:10.1186/s40880-018-0325-9 · Cited by 85
Zhang, Y., Wang, L., Wu, Z., Zeng, J., Chen, Y., Tian, R., Zhao, J., & Zhang, G. (2018) · IEEE Access · doi:10.1109/access.2018.2881042 · Cited by 76
Kise, Y., Ikeda, H., Fujii, T., Fukuda, M., Ariji, Y., Fujita, H., Katsumata, A., & Ariji, E. (2019) · Dentomaxillofacial Radiology · doi:10.1259/dmfr.20190019 · Cited by 75
Kim, K.-W., Kim, S., Lee, S. H., Lee, S. H., Lee, H. S., & Kim, S. J. (2018) · Scientific Reports · doi:10.1038/s41598-018-31486-3 · Cited by 68
Contrast-enhanced ultrasonography parameters in neural network diagnosis of liver tumors
Streba, C. T. (2012) · World Journal of Gastroenterology · doi:10.3748/wjg.v18.i32.4427 · Cited by 66
Global Challenges for Cancer Imaging
Schlemmer, H., Bittencourt, L. K., D’Anastasi, M., Domingues, R. C., Khong, P., Lockhat, Z. I., Muellner, A., Reiser, M. F., Schilsky, R. L., & Hricak, H. (2017) · Journal of Global Oncology · doi:10.1200/jgo.17.00036 · Cited by 66
Artificial Intelligence and Radiology: What Will the Future Hold?
King, B. F. (2018) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2017.11.017 · Cited by 62
Wang, S., Wang, R., Zhang, S., et al. (2018) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims.2018.06.03 · Cited by 57
Breast Nodules Computer-Aided Diagnostic System Design Using Fuzzy Cerebellar Model Neural Networks
Lin, C., Hou, Y.-L., Chen, T.-Y., et al. (2013) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2013.2269149 · Cited by 56
Chee, C. G., Kim, Y., Kang, Y., Lee, K. J., Chae, H.-D., Cho, J., Nam, C.-M., Choi, D., Lee, E., Lee, J. W., et al. (2019) · American Journal of Roentgenology · doi:10.2214/ajr.18.20817 · Cited by 56
Utility of Deep Learning Methods for Referability Classification of Age-Related Macular Degeneration
Burlina, P., Joshi, N., Pacheco, K. D., et al. (2018) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2018.3799 · Cited by 54
Stoffel, E., Becker, A. S., Wurnig, M. C., Marcon, M., Ghafoor, S., Berger, N., & Boss, A. (2018) · European Journal of Radiology Open · doi:10.1016/j.ejro.2018.09.002 · Cited by 47
Multivariable adjustments counteract spectrum and test review bias in accuracy studies
Bachmann, L. M., ter Riet, G., Weber, W. E. J., et al. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.02.007 · Cited by 28
Self-citationShibutani, T., Nakajima, K., Wakabayashi, H., Mori, H., Matsuo, S., Yoneyama, H., Konishi, T., Okuda, K., Onoguchi, M., & Kinuya, S. (2018) · Annals of Nuclear Medicine · doi:10.1007/s12149-018-1306-4 · Cited by 20
Exploratory Study on Direct Prediction of Diabetes Using Deep Residual Networks
Abbasi-Sureshjani, S., Dashtbozorg, B., Romeny, B. M. T. H., & Fleuret, F. (2017) · Lecture notes in computational vision and biomechanics · doi:10.1007/978-3-319-68195-5_86 · Cited by 18
Sun, L., Yang, L., Zhang, Y. T., Shen, J., Xue, F.-H., Cheng, H. D., & Qu, X. F. (2013) · Journal of Ultrasound in Medicine · doi:10.7863/ultra.33.1.83 · Cited by 16
Kim, S. M., Han, H. S., Park, J. M., Choi, Y. J., Yoon, H. S., Sohn, J. H., Baek, M. H., Kim, Y. N., Chae, Y. M., June, J. J., et al. (2012) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-012-9457-7 · Cited by 16
Matter with a mind; a neurological research robot.
Kh, F. (1951) · PubMed · Cited by 13
Individual patient data meta-analysis of diagnostic studies: opportunities and challenges
ter Riet, G., Bachmann, L. M., Kessels, A. G. H., & Khan, K. S. (2013) · BMJ Evidence-Based Medicine · doi:10.1136/eb-2012-101145 · Cited by 11
Self-citationAutomatic classification of DMSA scans using an artificial neural network
Wright, J., Duguid, R., McKiddie, F. I., et al. (2014) · Physics in Medicine and Biology · doi:10.1088/0031-9155/59/7/1789 · Cited by 7
Research works printed but not linked: 4
- Hadsell R, Erkan A, Sermanet P, Scoffier M, Muller U, LeCun Y. Deep belief net learning in a long-range vision system for autonomous off-road driving. 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems; Nice, France; Sept 22–26, 2008: 628–33. · DOI
- Ayed NGB, Masmoudi AD, Sellami D, Abid R. New developments in the diagnostic procedures to reduce prospective biopsies breast. 2015 International Conference on Advances in Biomedical Engineering (ICABME); Beirut, Lebanon; Sept 16–18, 2015: 205–08. · DOI
- Grewal M, Srivastava MM, Kumar P, Varadarajan S. RADnet: radiologist level accuracy using deep learning for hemorrhage detection in CT scans. 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018); Washington, DC, USA; April 4–7, 2018: 281–84. · DOI
- Liu X, Faes, L, Calvert MJ, Denniston AK, CONSORT-AI/SPIRIT-AI Extension Group. Extension of the CONSORT and SPIRIT statements. Lancet (in press).
Cited by
Sorted by citations
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212
Self-citationLiu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421
Self-citationMedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification
Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., & Ni, B. (2023) · Scientific Data · doi:10.1038/s41597-022-01721-8 · Cited by 1,043
“32 Chest CT MC (3) 1,370 1,027 / 103 / 240 VesselMNIST3D Yang et al.”
Preparing Medical Imaging Data for Machine Learning
Willemink, M. J., Koszek, W. A., Hardell, C., Wu, J., Fleischmann, D., Harvey, H., Folio, L. R., Summers, R. M., Rubin, D. L., & Lungren, M. P. (2020) · Radiology · doi:10.1148/radiol.2020192224 · Cited by 1,020
A foundation model for generalizable disease detection from retinal images
Zhou, Y., Chia, M. A., Wagner, S. K., Ayhan, M. S., Williamson, D. J., Struyven, R. R., Liu, T., Xu, M., Lozano, M. G., Woodward-Court, P., et al. (2023) · Nature · doi:10.1038/s41586-023-06555-x · Cited by 1,014
Self-citation
Metadata
- Authors
- Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., Ledsam, J. R., Schmid, M. K., Balaskas, K., Topol, E. J., Bachmann, L. M., Keane, P. A., & Denniston, A. K.
- Affiliations
- University Hospitals Birmingham NHS Foundation Trust (GB)
- Moorfields Eye Hospital NHS Foundation Trust (GB)
- University of Birmingham (GB)
- Google DeepMind (United Kingdom) (GB)
- Luzerner Kantonsspital (CH)
- Scripps Research Institute (US)
- Medignition (Switzerland) (CH)
- Corresponding author
- Denniston, Alastair K
- ORCID
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2019-10
- Volume
- 1
- Issue
- 6
- Pages
- e271-e297
- ISSN
- 2589-7500 (ISSN-L)
- OpenAlex
- W2976398475
- Cited by
- 1,883
- References
- 109
- Citations by year
- 2026: 287 · 2025: 273 · 2024: 305 · 2023: 287 · 2022: 258 · 2021: 269 · 2020: 184 · 2019: 20
- FWCI
- 8.73
- Type
- journal article
- Language
- en
- Update notices
- erratum (2019-11-01)
- Open access
- gold
- doi.org
- Licence
- Locations
- The Lancet Digital Health (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
- Funders
- National Institute for Health Research
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- deep learning, healthcare workers, diagnostic accuracy, diagnostic imaging, disease classification, sensitivity and specificity, external validation, artificial intelligence in healthcare, diagnostic performance comparison
- MeSH terms
- Deep Learning, Diagnostic Imaging, Health Personnel, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
