Artificial Intelligence Versus Clinicians in Disease Diagnosis: Systematic Review
Open accessPublished version of a preprint
Abstract
Background: Artificial intelligence (AI) has been extensively used in a range of medical fields to promote therapeutic development. The development of diverse AI techniques has also contributed to early detections, disease diagnoses, and referral management. However, concerns about the value of advanced AI in disease diagnosis have been raised by health care professionals, medical service providers, and health policy decision makers. Objective: This review aimed to systematically examine the literature, in particular, focusing on the performance comparison between advanced AI and human clinicians to provide an up-to-date summary regarding the extent of the application of AI to disease diagnoses. By doing so, this review discussed the relationship between the current advanced AI development and clinicians with respect to disease diagnosis and thus therapeutic development in the long run. Methods: We systematically searched articles published between January 2000 and March 2019 following the Preferred Reporting Items for Systematic reviews and Meta-Analysis in the following databases: Scopus, PubMed, CINAHL, Web of Science, and the Cochrane Library. According to the preset inclusion and exclusion criteria, only articles comparing the medical performance between advanced AI and human experts were considered. Results: A total of 9 articles were identified. A convolutional neural network was the commonly applied advanced AI technology. Owing to the variation in medical fields, there is a distinction between individual studies in terms of classification, labeling, training process, dataset size, and algorithm validation of AI. Performance indices reported in articles included diagnostic accuracy, weighted errors, false-positive rate, sensitivity, specificity, and the area under the receiver operating characteristic curve. The results showed that the performance of AI was at par with that of clinicians and exceeded that of clinicians with less experience. Conclusions: Current AI development has a diagnostic performance that is comparable with medical experts, especially in image recognition-related fields. Further studies can be extended to other types of medical imaging such as magnetic resonance imaging and other medical practices unrelated to images. With the continued development of AI-assisted technologies, the clinical implications underpinned by clinicians’ experience and guided by patient-centered health care principle should be constantly considered in future AI-related and other technology-based medical research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The clinical application of AI remains limited.
“Despite some promising results [27, 28], the clinical application of AI remains limited [29–31].”
From Implementing Artificial Intelligence in Critical Care Medicine: a consensus of 22. (2025) · cites this paper as [30]
The Introduction (Background; page not provided) says AI has been extensively used across medical fields, including to assist clinical diagnoses. Although the Discussion (Principal Findings; page not provided) identifies unresolved challenges, it does not characterize clinical application as limited; it describes AI as improving efficiency in clinical practice. (LLM, conf=0.78)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredNurse leaders face challenges integrating AI into existing workflows and ensuring they have the skills and knowledge to effectively utilize these new tools.
“Nurse leaders also face challenges integrating AI into existing workflows and ensuring they have the skills and knowledge to effectively utilize these new tools [2, 3].”
From Leading with AI in critical care nursing: challenges, opportunities, and the human factor (2024) · cites this paper as [2]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · see citing works
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697 · see citing works
Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning
Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C. S., Liang, H., Baxter, S. L., McKeown, A., Yang, G., Wu, X., Yan, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.02.010 · Cited by 4,842 · see citing works
Self-citationClinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639 · see citing works
Rajpurkar, P., Irvin, J., Ball, R. L., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C. P., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002686 · Cited by 1,449 · see citing works
Predicting breast cancer survivability: a comparison of three data mining methods
Delen, D., Walker, G., & Kadam, A. (2004) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2004.07.002 · Cited by 1,215 · see citing works
Artificial Intelligence in Precision Cardiovascular Medicine
Krittanawong, C., Zhang, H., Wang, Z., Aydar, M., & Kitai, T. (2017) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2017.03.571 · Cited by 1,126 · see citing works
Artificial neural networks in medical diagnosis
Amato, F., López, A., Peña-Méndez, E. M., Vaňhara, P., Hampl, A., & Havel, J. (2013) · Journal of Applied Biomedicine · doi:10.2478/v10136-012-0031-x · Cited by 898 · see citing works
Artificial intelligence in medical practice: the question to the answer?
Miller, D. D., & Brown, E. W. (2018) · The American Journal of Medicine · doi:10.1016/j.amjmed.2017.10.035 · Cited by 774 · see citing works
Orringer, D. A., Pandian, B., Niknafs, Y. S., Hollon, T. C., Szczepanski, J. M., Lewis, S., Garrard, M. R., Hervey‐Jumper, S. L., Garton, H. J. L., Maher, C. O., et al. (2017) · Nature Biomedical Engineering · doi:10.1038/s41551-016-0027 · Cited by 570 · see citing works
Nam, J. G., Park, S., Hwang, E. J., Lee, J. H., Jin, K.-N., Lim, K. Y., Vu, T. H., Sohn, J. H., Hwang, S., Goo, J. M., et al. (2019) · Radiology · doi:10.1148/radiol.2018180237 · Cited by 539 · see citing works
Skin Cancer Classification Using Convolutional Neural Networks: Systematic Review
Brinker, T. J., Hekler, A., Utikal, J. S., Grabe, N., Schadendorf, D., Klode, J., Berking, C., Steeb, T., Enk, A. H., & Kalle, C. V. (2018) · Journal of Medical Internet Research · doi:10.2196/11936 · Cited by 504 · see citing works
Long, E., Lin, H., Liu, Z., Wu, X., Wang, L., Jiang, J., An, Y., Lin, Z., Li, X., Chen, J., et al. (2017) · Nature Biomedical Engineering · doi:10.1038/s41551-016-0024 · Cited by 358 · see citing works
Self-citationHan, S. S., Park, G. H., Lim, W., Kim, M. S., Na, J. I., Park, I., & Chang, S. E. (2018) · PLoS ONE · doi:10.1371/journal.pone.0191493 · Cited by 305 · see citing works
Brinker, T. J., Hekler, A., Enk, A. H., Klode, J., Hauschild, A., Berking, C., Schilling, B., Haferkamp, S., Schadendorf, D., Fröhling, S., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.02.005 · Cited by 304 · see citing works
Evidence-based Patient Choice and Consumer health informatics in the Internet age
Eysenbach, G., & Jadad, A. R. (2001) · Journal of Medical Internet Research · doi:10.2196/jmir.3.2.e19 · Cited by 295 · see citing works
Natural and Artificial Intelligence in Neurosurgery: A Systematic Review
Senders, J. T., Arnaout, O. M., Karhade, A. V., Dasenbrock, H. H., Gormley, W. B., Broekman, M. L. D., & Smith, T. R. (2017) · Neurosurgery · doi:10.1093/neuros/nyx384 · Cited by 264 · see citing works
Rajpara, S. M., Botello, A. P., Townend, J., & Ormerod, A. D. (2009) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2009.09093.x · Cited by 186 · see citing works
Early expert systems: Where are they now?
Gill, T. G. (1995) · MIS Quarterly · doi:10.2307/249711 · Cited by 181 · see citing works
Balayla, J., & Shrem, G. (2019) · Archives of Gynecology and Obstetrics · doi:10.1007/s00404-019-05151-7 · Cited by 90 · see citing works
Awwalu, J., Garba, A. G., Ghazvini, A., & Atuah, R. (2014) · International Journal of Computer Theory and Engineering · doi:10.7763/ijcte.2015.v7.999 · Cited by 86 · see citing works
de Langavant, L. C., Bayen, É., & Yaffe, K. S. (2018) · Journal of Medical Internet Research · doi:10.2196/10493 · Cited by 76 · see citing works
Smartphone-Based Real-time Assessment of Swallowing Ability From the Swallowing Sound
Jayatilake, D., Ueno, T., Teramoto, Y., Nakai, K., Hidaka, K., Ayuzawa, S., Eguchi, K., Matsumura, A., & Suzuki, K. (2015) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2015.2500562 · Cited by 71 · see citing works
Smith, W. P., Doctor, J. N., Meyer, J., et al. (2009) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2008.12.002 · Cited by 57 · see citing works
Sidey-Gibbons, C. J., Richards, S. H., Valderas, J. M., & Campbell, J. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6533 · Cited by 55 · see citing works
A decision support system for cost-effective diagnosis
Chi, C., Street, W. N., & Katz, D. A. (2010) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2010.08.001 · Cited by 39 · see citing works
Documenting rare disease data in China
Lin, H., Long, E., Chen, W., et al. (2015) · Science · doi:10.1126/science.349.6252.1064-b · Cited by 38 · see citing works
González‐Castro, V., Hernández, M. D. C. V., Chappell, F. M., Armitage, P. A., Makin, S., & Wardlaw, J. M. (2017) · Clinical Science · doi:10.1042/cs20170051 · Cited by 34 · see citing works
Cascianelli, S., Scialpi, M., Amici, S., et al. (2016) · Current Alzheimer Research · doi:10.2174/1567205013666160620122926 · Cited by 30 · see citing works
Sarwar, A., Suri, J., Ali, M., et al. (2016) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-016-0353-8 · Cited by 22 · see citing works
CANFIS—a computer aided diagnostic tool for cancer detection
Parthiban, L., & Subramanian, R. (2009) · Journal of Biomedical Science and Engineering · doi:10.4236/jbise.2009.25048 · Cited by 10 · see citing works
Park, S. Y., Seo, J. S., Lee, S. C., et al. (2013) · Lecture notes in electrical engineering · doi:10.1007/978-3-642-40861-8_13 · Cited by 9 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [5]Setlak G Dąbrowski M Szajnar W Piróg-Mazur M Kożak T Semantic Scholar 2009 Artificial intelligence approach to diabetes diagnostics https://pdfs.semanticscholar.org/40f3/e4017d497bffe556f882d4f1389462296b59.pdf
- [10]Higgins JP Green S Cochrane Handbook for Systematic Reviews of Interventions. Edition 5.1 2011 London, England The Cochrane Collaboration
Cited by
Sorted by citations
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886 · see citing works
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957 · see citing works
Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review
Yin, J., Ngiam, K. Y., & Teo, H. H. (2021) · Journal of Medical Internet Research · doi:10.2196/25759 · Cited by 419 · see citing works
Evaluation and Mitigation of Racial Bias in Clinical Machine Learning Models: Scoping Review
Huang, J., Galal, G., Etemadi, M., & Vaidyanathan, M. (2022) · JMIR Medical Informatics · doi:10.2196/36388 · Cited by 186 · see citing works
A bird's-eye view of deep learning in bioimage analysis
Meijering, E. (2020) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2020.08.003 · Cited by 126 · see citing works
Jussupow, E., Spohrer, K., & Heinzl, A. (2022) · JMIR Formative Research · doi:10.2196/28750 · Cited by 138 · see citing works
Trust me, i'm a chatbot: how artificial intelligence in health care fails the Turing Test
Powell, J. (2019) · Journal of Medical Internet Research · doi:10.2196/16222 · Cited by 115 · see citing works
Hah, H., & Goldin, D. S. (2021) · Journal of Medical Internet Research · doi:10.2196/33540 · Cited by 81 · see citing works
Implementing Artificial Intelligence in Critical Care Medicine: a consensus of 22.
Cecconi, M., Greco, M., Shickel, B., Angus, D. C., Bailey, H., Bignami, E., Calandra, T., Celi, L. A., Einav, S., Elbers, P., et al. (2025) · Critical Care · doi:10.1186/s13054-025-05532-2 · Cited by 80 · see citing works
Leading with AI in critical care nursing: challenges, opportunities, and the human factor
Hassan, E. A., & El-Ashry, A. M. (2024) · BMC Nursing · doi:10.1186/s12912-024-02363-4 · Cited by 72 · see citing works
Hashmani, M. A., Jameel, S. M., Rizvi, S. S. H., & Shukla, S. (2021) · Applied Sciences · doi:10.3390/app11052145 · Cited by 53 · see citing works
Application of Deep Learning Model in the Sonographic Diagnosis of Uterine Adenomyosis
Raimondo, D., Raffone, A., Aru, A. C., Giorgi, M., Giaquinto, I., Spagnolo, E., Travaglino, A., Galatolo, F. A., Cimino, M. G. C. A., Lenzi, J., et al. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20031724 · Cited by 44 · see citing works
Transforming Maritime Health with ChatGPT-Powered Healthcare Services for Mariners
Sharma, M., & Sharma, S. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03195-0 · Cited by 27 · see citing works
Generative AI and Learning Analytics
Khosravi, H., Viberg, O., Kovanovic, V., & Ferguson, R. (2023) · Journal of Learning Analytics · doi:10.18608/jla.2023.8333 · Cited by 25 · see citing works
Enhancing skin lesion classification: a CNN approach with human baseline comparison
Ajabani, D., Shaikh, Z. A., Yousef, A., Ali, K., & Albahar, M. A. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2795 · Cited by 18 · see citing works
Dallora, A. L., Kvist, O., Berglund, J. S., Ruiz, S. D., Boldt, M., Flodmark, C.-E., & Anderberg, P. (2020) · JMIR Medical Informatics · doi:10.2196/18846 · Cited by 13 · see citing works
Akkus Yildirim, B., Tutun, B., Durak, G., Yildirim, E. B., Uysal, E., Erturk, S. M., & Bagci, U. (2025) · Communications Medicine · doi:10.1038/s43856-025-01315-6 · Cited by 2 · see citing works
- Authors
- Shen, J., Zhang, C. J. P., Jiang, B., Chen, J., Song, J., Liu, Z., He, Z., Wong, S. Y., Fang, P.-H., & Ming, W.-K.
- Affiliations
- Jinan University (CN)
- University of Hong Kong (HK)
- Sun Yat-sen University (CN)
- Brigham and Women's Hospital (US)
- Corresponding author
- Ming, Wai-Kit
- ORCID
- Venue
- JMIR Medical Informatics
- Publisher
- JMIR Publications Inc.
- Published
- 2019-08-16
- Volume
- 7
- Issue
- 3
- Pages
- e10010
- ISSN
- 2291-9694 (ISSN-L)
- PMC
- PMC6716335
- OpenAlex
- W2963852062
- Cited by
- 373
- References
- 33
- Citations by year
- 2026: 52 · 2025: 90 · 2024: 69 · 2023: 54 · 2022: 36 · 2021: 38 · 2020: 28 · 2019: 6
- FWCI
- 1.32
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Medical Informatics (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Keywords
- artificial intelligence, disease diagnosis, convolutional neural networks, diagnostic accuracy, medical image recognition, AI-assisted diagnosis, clinician experience, patient-centered care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
