The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Efficacy of Artificial-Intelligence-Driven Differential-Diagnosis List on the Diagnostic Accuracy of Physicians: An Open-Label Randomized Controlled Study
Open accessAbstract
Background: The efficacy of artificial intelligence (AI)-driven automated medical-history-taking systems with AI-driven differential-diagnosis lists on physicians’ diagnostic accuracy was shown. However, considering the negative effects of AI-driven differential-diagnosis lists such as omission (physicians reject a correct diagnosis suggested by AI) and commission (physicians accept an incorrect diagnosis suggested by AI) errors, the efficacy of AI-driven automated medical-history-taking systems without AI-driven differential-diagnosis lists on physicians’ diagnostic accuracy should be evaluated. Objective: The present study was conducted to evaluate the efficacy of AI-driven automated medical-history-taking systems with or without AI-driven differential-diagnosis lists on physicians’ diagnostic accuracy. Methods: This randomized controlled study was conducted in January 2021 and included 22 physicians working at a university hospital. Participants were required to read 16 clinical vignettes in which the AI-driven medical history of real patients generated up to three differential diagnoses per case. Participants were divided into two groups: with and without an AI-driven differential-diagnosis list. Results: There was no significant difference in diagnostic accuracy between the two groups (57.4% vs. 56.3%, respectively; p = 0.91). Vignettes that included a correct diagnosis in the AI-generated list showed the greatest positive effect on physicians’ diagnostic accuracy (adjusted odds ratio 7.68; 95% CI 4.68–12.58; p < 0.001). In the group with AI-driven differential-diagnosis lists, 15.9% of diagnoses were omission errors and 14.8% were commission errors. Conclusions: Physicians’ diagnostic accuracy using AI-driven automated medical history did not differ between the groups with and without AI-driven differential-diagnosis lists.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Harada et al. (2021) demonstrated that physicians using AI differential diagnosis lists committed 14.8% fewer commission errors when maintaining professional scepticism (particularly experienced physicians).”
From Multimodal neuroimaging and AI integration in cognitive disorders: advances, challenges, and future directions for precision medicine (2026) · cites this paper as Harada et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“On the practitioner side (Harada et al., 2021), observed that clinicians erred more often due to under-reliance than over-reliance when utilising AI decision support.”
From Investigating appropriate reliance on AI-Based decision support systems: the role of expertise, trust, and self-confidence (2025) · cites this paper as Harada et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Singh, H., Meyer, A. N. D., & Thomas, E. J. (2014) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-002627 · Cited by 677
Types and Origins of Diagnostic Errors in Primary Care Settings
Singh, H., Giardina, T. D., Meyer, A. N. D., Forjuoh, S. N., Reis, M. D., & Thomas, E. J. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.2777 · Cited by 559 · cited 1×
“Meanwhile, many diagnostic errors involved missing common diseases such as pneumonia and congestive heart failure [25].”
BackgroundEvaluation of symptom checkers for self diagnosis and triage: audit study
Semigran, H. L., Linder, J. A., Gidengil, C., & Mehrotra, A. (2015) · BMJ · doi:10.1136/bmj.h3480 · Cited by 489
Saber Tehrani, A. S., Lee, H., Mathews, S. C., Shore, A., Makary, M. A., Pronovost, P. J., & Newman-Toker, D. E. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001550 · Cited by 328
Friedman, C. P., Elstein, A. S., Wolf, F. M., Murphy, G. C., Franz, T. M., Heckerling, P. S., Fine, P. L., Miller, T. M., & Abraham, V. (1999) · JAMA · doi:10.1001/jama.282.19.1851 · Cited by 217 · cited 3×
“These findings are consistent with those of previous studies [16,19].”
BackgroundResultGilbert, S., Mehl, A., Baluch, A., Cawley, C., Challiner, J., Fraser, H., Millen, E., Montazeri, M., Multmeier, J., Pick, F., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-040269 · Cited by 183
Comparison of Physician and Computer Diagnostic Accuracy
Semigran, H. L., Levine, D. M., Nundy, S., & Mehrotra, A. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.6001 · Cited by 145
Human Factors Challenges for the Safe Use of Artificial Intelligence in Patient Care
Sujan, M., Furniss, D., Grundy, K., Grundy, H., Nelson, D., Elliott, M., White, S., Habli, I., & Reynolds, N. (2019) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2019-100081 · Cited by 144
Kostopoulou, O., Rosen, A., Round, T., Wright, E., Douiri, A., & Delaney, B. C. (2014) · British Journal of General Practice · doi:10.3399/bjgp15x683161 · Cited by 82
Mamede, S., Filho, C., de Faria, R. M. D., Franci, D., Nunes, M. D. P. T., Ribeiro, L. M. C., Biegelmeyer, J., Zwaan, L., & Schmidt, H. G. (2020) · BMJ Quality & Safety · doi:10.1136/bmjqs-2019-010079 · Cited by 71 · cited 1×
“We used eight cases that included the confirmed diagnosis in the AI-driven top 10 differential-diagnosis list and eight cases that did not include the confirmed diagnosis to allow for automation bias [20].”
Method
Cited by
Durlik, I., Miller, T. I., Kostecka, E., & Tuński, T. (2024) · Applied Sciences · doi:10.3390/app14188420 · Cited by 144
“Extensive modifications and upgrades are often necessary, which can be costly and time-consuming [49,50].”
Hirosawa, T., Kawamura, R., Harada, Y., Mizuta, K., Tokumasu, K., Kaji, Y., Suzuki, T., & Shimizu, T. (2023) · JMIR Medical Informatics · doi:10.2196/48808 · Cited by 130
Self-citationClinical Decision Support Systems for Diagnosis in Primary Care: A Scoping Review
Harada, T., Miyagami, T., Kunitomo, K., & Shimizu, T. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168435 · Cited by 89
“Moreover, there are concerns at present about the validity of the quality of medical history taking by artificial intelligence (AI) [68]; thus, the role of medical history taking in medicine practice is critical.”
BackgroundSelf-citationArtificial Intelligence in Endodontic Education
Aminoshariae, A., Nosrat, A., Nagendrababu, V., Dianat, O., Mohammad‐Rahimi, H., O'Keefe, A. W., & Setzer, F. C. (2024) · Journal of Endodontics · doi:10.1016/j.joen.2024.02.011 · Cited by 96
Martindale, A. P. L., Llewellyn, C. D., de Visser, R. O., Ng, B., Ngai, V., Kale, A. U., di Ruffano, L. F., Golub, R. M., Collins, G. S., Moher, D., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-45355-3 · Cited by 58
Mirkin, S., & Albensi, B. C. (2023) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2023.1094233 · Cited by 46
Sakamoto, T., Harada, Y., & Shimizu, T. (2024) · JMIR Formative Research · doi:10.2196/58666 · Cited by 45
Self-citationHarada, Y., Katsukura, S., Kawamura, R., & Shimizu, T. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18115562 · Cited by 35
“The primary analysis results of our randomized controlled study were previously reported [17].”
BackgroundResultInfluentialSelf-citationKawamura, R., Harada, Y., Sugimoto, S., Nagase, Y., Katsukura, S., & Shimizu, T. (2022) · JMIR Medical Informatics · doi:10.2196/35225 · Cited by 36
“Yet another study showed high reliability of documentation regarding clinical history to assist the diagnostic accuracy of physicians [26]; however, this was not conducted in a clinical practice setting.”
InfluentialSelf-citationMiyachi, Y., Ishii, O., & Torigoe, K. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02123-5 · Cited by 30
“Case selection criteria for evaluation of differential diagnostic performance In previous articles, cases for evaluation of DDX performance are often actual cases [23].”
BackgroundSiira, E., Johansson, H., & Nygren, J. (2024) · Journal of Medical Internet Research · doi:10.2196/53741 · Cited by 27
“However, these studies focused solely on health are professionals’decision-making performance in relation to triage with AI systems, without providing insights into their perspectives [40,44,49,59,77,80,95].”
Advancements, Applications, and Future Directions of Artificial Intelligence in Healthcare
Ifty, S. M. H., Irin, F., Shovon, M. S. S., Amjad, M. H. H., Bhowmik, P. K., Ahmed, R., Ashakin, M. R., Hossain, B., Mushfiq, Sattar, A., et al. (2024) · Journal of Angiotherapy · doi:10.25163/angiotherapy.889843 · Cited by 29
Park, C., An, M. H., Hwang, G., Park, R. W., & An, J. (2025) · JMIR Medical Informatics · doi:10.2196/68409 · Cited by 22
Tong, W., Zhang, X., Zeng, H., Pan, J., Gong, C., & Zhang, H. (2024) · JMIR Medical Education · doi:10.2196/48594 · Cited by 23
“To ensure that students can fully harness and address these technological revolutions, educators need to reconsider curriculum design, integrating these cutting-edge technologies and preparing students for future medical innovations [11-13].”
Harada, Y., Sakamoto, T., Sugimoto, S., & Shimizu, T. (2024) · JMIR Formative Research · doi:10.2196/53985 · Cited by 23
“Indeed, our previous study showed that physicians’ diagnostic accuracy was only 56% when reading the information taken by the same AI-based symptom checker used in this study [34].”
Self-citationDogru, E. Ö., & Krämer, N. C. (2025) · Journal of Decision Systems · doi:10.1080/12460125.2025.2593251 · Cited by 15
“This could be particularly relevant for high-stakes fields, where over-reliance and under-reliance carry the risk of diminishing task performance (Harada et al., 2021; Micocci et al., 2021).”
Diagnostic performance of Large Language Models (LLMs) compared with physicians in sleep medicine
Patel, A., Ruoff, C. M., Helgeson, S. A., Carvalho, D. Z., Castillo, P. R., & Cheung, J. (2025) · Sleep Medicine · doi:10.1016/j.sleep.2025.106677 · Cited by 18
Artificial intelligence in sleep medicine: assessing the diagnostic precision of ChatGPT-4
Patel, A., & Cheung, J. (2025) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.11732 · Cited by 12
“17 Similarly, Harada et al (2021) demonstrated that AI-generated differential diagnosis lists significantly enhanced clinicians ’ diagnostic confidence, particularly when correct suggestions were prominently ranked, supporting the utility of AI as a supplemental tool.”
InfluentialDang, M., Liu, B., Chen, Y., & Zhang, Z. (2026) · Psychoradiology · doi:10.1093/psyrad/kkag007 · Cited by 10
Transforming paperwork with AI: applications across healthcare and other industries
Jnainati, M. A., Jnainati, J. A., Rath, S., Aujla, S., Nasir, E., Govindarajan, A., Samantaray, Y., & Semy, M. (2025) · AI & Society · doi:10.1007/s00146-025-02380-0 · Cited by 14
“For example, an AI algorithm designed for early sepsis identification uses electronic medical information to accurately predict sepsis, beating traditional diagnostic approaches (Li et al. 2023; Harada et al. 2021; Tarumi et al. 2021; Yuan et al. 2020).”
Mitigating cognitive bias with clinical decision support systems: an experimental study
Küper, A., Lodde, G. C., Livingstone, E., Schadendorf, D., & Krämer, N. C. (2023) · Journal of Decision System · doi:10.1080/12460125.2023.2245215 · Cited by 11
“This might also be due to more experienced physicians being more likely to disregard decision support advice, even if it is correct (Jussupow et al., 2021) and inexperienced physicians, on the other hand, being more likely to accept incorrect system advice (Harada et al., 2021b).”
Opportunities & Challenges of Artificial Intelligent-Powered Technology in Healthcare
Anwer, M. S. (2024) · Medical Research Archives · doi:10.18103/mra.v12i3.5141 · Cited by 10
“However, a recent study found that physicians’ diagnostic accuracy using AI-driven automated medical history did not differ between the groups with and without AI-driven differential-diagnosis lists 16 .”
ResultA Systematic Review of Predictive Analytics Applications in Early Disease Detection and Diagnosis
Independent Researcher, R., Kolawole, T. O., Mustapha, A. Y., Kwara State Ministry of Health, N., Mbata, A. O., Kaybat Pharmacy and Stores, B., Tomoh, B. O., Independent Researcher, C., Forkuo, A. Y., Independent Researcher, U., et al. (2025) · Engineering and Technology Journal · doi:10.47191/etj/v10i03.35 · Cited by 7
Current trends in digital camera-based bioassays for point-of-care tests
Pohanka, M. (2023) · Clinica Chimica Acta · doi:10.1016/j.cca.2023.117677 · Cited by 7
Ejdys, J., Czerwińska, M., & Ginevičius, R. (2024) · Human Technology · doi:10.14254/1795-6889.2024.20-1.8 · Cited by 7
“AI has the potential to enhance diagnostic accuracy, improve efficiency, and support healthcare professionals in decision-making processes (Harada, Katsukura, Kawamura, & Shimizu, 2021; Erickson et al., 2017, Jiang et al., 2017).”
BackgroundUsing conversant artificial intelligence to improve diagnostic reasoning: ready for prime time?
Scott, I. A., Miller, T., & Crock, C. (2024) · The Medical Journal of Australia · doi:10.5694/mja2.52401 · Cited by 6
Pereira, R. A. D. C., Lima, R. R., Gomes, A. C. A., Saldanha, F. A. S., Schwingel, D., Schwingel, P. A., & Gambassi, B. B. (2025) · Scientifica · doi:10.1155/sci5/2632410 · Cited by 4
Jeong, M.-A., & Kim, S.-D. (2026) · Applied Sciences · doi:10.3390/app16105146 · Cited by 5
“%) were rated as low overall risk of bias (Yun et al., 2023 [46]; Harada et al., 2021 [49]; Homayounieh et al., 2021 [50]), and two (40”
Boudia, A. O. (2025) · Plasmonics · doi:10.1007/s11468-025-02810-1 · Cited by 3
“However, Harada et al. [15] do not directly address spectral normalization in SERS analysis, making it less relevant to this specific context.”
Harada, Y., Tomiyama, S., Sakamoto, T., Sugimoto, S., Kawamura, R., Yokose, M., Hayashi, A., & Shimizu, T. (2023) · JMIR Formative Research · doi:10.2196/49034 · Cited by 3
“This low accuracy is problematic because the DDx list and medical history generated by the automated medical history–taking system may reduce the diagnostic accuracy of physicians in cases where the automated medical history–taking system does not include the correct diagnosis in the DDx list [27,28].”
BackgroundSelf-citationUrinary Tract Infection Bacteria Classification: Artificial Intelligence-based Medical Application
Fadlil, A., Fathurrahman, H. I. K., Lin, Y., Kamilah, F., & Sunardi, S. (2023) · Journal of Robotics and Control (JRC) · doi:10.18196/jrc.v4i5.18879 · Cited by 3
“…can use artificial intelligence to identify proper squat form for physical therapists [31], improve the accuracy of skeletal age assessments, speed up radiologists' interpretation times [32], pre-assess priority care digitally [33], and provide recommendations in physician diagnosis [34], [35].”
BackgroundKumar, P., Alnaimi, N. A. A., Soman, S., Suansing, L., Arriola, D. R., & Jamea, L. A. (2026) · Sci · doi:10.3390/sci8040073 · Cited by 2
“…Yamamura 2025 [31] 70% 65% Graf 2022 [26] 70% 54% David M. Levine 2024 [14] 88% 96% Delshad 2021 [16] 92% 80% Fonseca 2024 [34] 71% 69% Harada 2021 [24] 57% 56% Lyons 2024 [42] 93% 95% Camkıran 2025 [35] 63% 93% Keenan 2020 [44] 85% 81% A: artificial intelligence; B: expert healthcare…”
InfluentialWANG, P. S. P., Yuan, Y., & Peng, X. (2025) · AI and Ethics · doi:10.1007/s43681-025-00840-3 · Cited by 1
Ang, C., Ito, S., Yeow, A. Y. K., & Cleland, J. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2598673 · Cited by 1
“…2023; Trejo Ome ~ naca et al., 2025; Zhou et al., 2024; Alsalamah et al., 2025; Bundy et al., 2024; Chausiaux et al., 2022; Gosavi et al., 2025 ; Harada et al., 2021; Katyal & Vaid, 2023; Marri et al., 2024; Pokushalov et al., 2024; Shenoy et al., 2018; Singh et al., 2023; Yu et al., 2024)…”
InfluentialHuman oversight of erroneous clinical AI and decision support a systematic review and meta analysis
Zeng, D., Lei, J., & Li, Z. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03408-5 · Cited by 1
ТЕХНОЛОГІЇ ШТУЧНОГО ІНТЕЛЕКТУ В АВТОМАТИЗАЦІЇ ТРАНСПОРТНИХ ПРОЦЕСІВ
Podolian, V. O. (2025) · Вісник Херсонського національного технічного університету · doi:10.35546/kntu2078-4481.2025.1.1.23 · Cited by 1
High Accuracy Automated Differential Diagnosis using Machine Learning
Cherian, A. M., Tonasalli, N., Gupta, D., Joseph, J., & Kingsly, S. (2023) · International Conference on Inventive Research in Computing Applications (ICIRCA) · doi:10.1109/icirca57980.2023.10220703
Introduction to Artificial Intelligence (AI) and AI-Related Concepts
Brigham, C. R., Buttar, W. A., Bucksbaum, M., & Talmage, J. B. (2023) · AMA guides newsletter · doi:10.1001/amaguidesnewsletters.2023.sepoct03
Artificial Intelligence: Revolutionizing the Medicolegal Field
Brigham, C. R., Buttar, W. A., Bucksbaum, M., & Talmage, J. B. (2023) · AMA guides newsletter · doi:10.1001/amaguidesnewsletters.2023.sepoct01 · Cited by 1
Machine Learning Driven Symptoms Diagnosis System
Doughan, Z. F., Hammoud, R., Ghalayini, M.-D., Darweesh, D., Chehade, O. E. H., & Itani, S. (2025) · International Conference on Advances in Biomedical Engineering (ICABME) · doi:10.1109/icabme66883.2025.11211623
The Role of Artificial Intelligence (AI) in the Healthcare Profession
Afridi, M. I. (2023) · ANNALS OF ABBASI SHAHEED HOSPITAL AND KARACHI MEDICAL & DENTAL COLLEGE · doi:10.58397/ashkmdc.v28i03.736
Miyachi, Y., Ishii, O., & Torigoe, K. (2024) · Figshare · doi:10.6084/m9.figshare.26565908
Miyachi, Y., Ishii, O., & Torigoe, K. (2024) · Figshare · doi:10.6084/m9.figshare.26565911
Alegre, J. C. A., & Mytskevych, J. (2026) · doi:10.2196/preprints.99775
Artificial intelligence-supported health counseling scale (AI-HCS): a reliability and validity study
GÖDE, A. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3937
Advanced Point-of-Care Biosensors Integrated with AI and Bioinformatics
Taha, B. A., Addie, A. J., Haider, A. J., & Arsad, N. (2026) · Springer eBooks · doi:10.1007/978-981-95-9151-0_7
Artificial Intelligence in the Healthcare Market: Opportunities and Threats
Szalonka, K., Blicharz, J. R., Błoński, K., Nowak, W., Wiśniewski, T., Żak, K., & Szalonka, A. (2026) · Productivity Press eBooks · doi:10.4324/9781003743804
Verbalized Uncertainty in Medical AI: Differential Diagnosis in Commercial LLMs
Gawior, Y. H., Groefsema, S., Özyilmaz, Ö. T., & Valdenegro-Toro, M. (2026) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms69103.2026.00012
“Multiple studies on accuracy exist [14]–[17], but the relationship between the correctness of these models and their confidence remains largely unexplored for medical LLMs.”
Metadata
- Authors
- Harada, Y., Katsukura, S., Kawamura, R., & Shimizu, T.
- Affiliations
- Nagano Chuo Hospital (JP)
- Dokkyo Medical University (JP)
- Corresponding author
- Shimizu, Taro
- Venue
- International Journal of Environmental Research and Public Health
- Publisher
- MDPI AG
- Published
- 2021-02-21
- Volume
- 18
- Issue
- 4
- Pages
- 2086
- ISSN
- 1660-4601 (ISSN-L)
- PMC
- PMC7924871
- OpenAlex
- W3133179228
- Cited by
- 44
- References
- 24
- Citations by year
- 2026: 7 · 2025: 10 · 2024: 13 · 2023: 11 · 2022: 1 · 2021: 2
- FWCI
- 1.97
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- International Journal of Environmental Research and Public Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Japan Society for the Promotion of Science (21K10355)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- artificial intelligence, differential diagnosis, omission errors, commission errors, clinical vignettes, clinical decision support
- MeSH terms
- Artificial Intelligence, Diagnosis, Differential, Humans, Intelligence, Medical History Taking, Physicians
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

