How Clinicians Perceive Artificial Intelligence–Assisted Technologies in Diagnostic Decision Making: Mixed Methods Approach
Open accessPublished version of a preprint
Abstract
Background: With the rapid development of artificial intelligence (AI) and related technologies, AI algorithms are being embedded into various health information technologies that assist clinicians in clinical decision making. Objective: This study aimed to explore how clinicians perceive AI assistance in diagnostic decision making and suggest the paths forward for AI-human teaming for clinical decision making in health care. Methods: This study used a mixed methods approach, utilizing hierarchical linear modeling and sentiment analysis through natural language understanding techniques. Results: A total of 114 clinicians participated in online simulation surveys in 2020 and 2021. These clinicians studied family medicine and used AI algorithms to aid in patient diagnosis. Their overall sentiment toward AI-assisted diagnosis was positive and comparable with diagnoses made without the assistance of AI. However, AI-guided decision making was not congruent with the way clinicians typically made decisions in diagnosing illnesses. In a quantitative survey, clinicians reported perceiving current AI assistance as not likely to enhance diagnostic capability and negatively influenced their overall performance (β=–0.421, P=.02). Instead, clinicians’ diagnostic capabilities tended to be associated with well-known parameters, such as education, age, and daily habit of technology use on social media platforms. Conclusions: This study elucidated clinicians’ current perceptions and sentiments toward AI-enabled diagnosis. Although the sentiment was positive, the current form of AI assistance may not be linked with efficient decision making, as AI algorithms are not well aligned with subjective human reasoning in clinical diagnosis. Developers and policy makers in health could gather behavioral data from clinicians in various disciplines to help align AI algorithms with the unique subjective patterns of reasoning that humans employ in clinical diagnosis.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Current advances in AI have significantly influenced diagnostic and clinical decision-making.
“Current advances in AI have significantly influenced diagnostic and clinical decision-making [11, 12]”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Designing and Conducting Mixed Methods Research
Robinson, P. M. (2007) · Australian and New Zealand Journal of Public Health · doi:10.1111/j.1753-6405.2007.00096.x · Cited by 23,297 · see citing works
Venkatesh, V., Thong, J. Y. L., & Xu, X. (2012) · MIS Quarterly · doi:10.2307/41410412 · Cited by 15,756 · see citing works
Computer Self-Efficacy: Development of a Measure and Initial Test
Compeau, D. R., & Higgins, C. A. (1995) · MIS Quarterly · doi:10.2307/249688 · Cited by 6,553 · see citing works
Task-Technology Fit and Individual Performance
Goodhue, D. L., & Thompson, R. L. (1995) · MIS Quarterly · doi:10.2307/249689 · Cited by 6,006 · see citing works
Agarwal, R., & Karahanna, E. (2000) · MIS Quarterly · doi:10.2307/3250951 · Cited by 4,726 · see citing works
A Concise Introduction to Mixed Methods Research
Creswell, J. W. (2021) · Cited by 4,465 · see citing works
The potential for artificial intelligence in healthcare
Davenport, T., & Kalakota, R. (2019) · Future Healthcare Journal · doi:10.7861/futurehosp.6-2-94 · Cited by 3,800 · see citing works
Artificial intelligence in medicine
Hamet, P., & Tremblay, J. (2017) · Metabolism · doi:10.1016/j.metabol.2017.01.011 · Cited by 2,334 · see citing works
Resistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015 · see citing works
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743 · see citing works
Advances in natural language processing
Hirschberg, J., & Manning, C. D. (2015) · Science · doi:10.1126/science.aaa8685 · Cited by 1,712 · see citing works
Knowledge for Theory and Practice
de Ven, A. H. V., & Johnson, P. E. (2006) · Academy of Management Review · doi:10.5465/amr.2006.22527385 · Cited by 1,649 · see citing works
Big data and machine learning algorithms for health-care delivery
Ngiam, K. Y., & Khor, I. W. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30149-4 · Cited by 1,424 · see citing works
Nagendran, M., Chen, Y., Lovejoy, C. A., Gordon, A. C., Komorowski, M., Harvey, H., Topol, E. J., Ioannidis, J. P. A., Collins, G. S., & Maruthappu, M. (2020) · BMJ · doi:10.1136/bmj.m689 · Cited by 1,152 · see citing works
(2019) · Cited by 986 · see citing works
Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence
Anderson, C. J. (2005) · Journal of the American Statistical Association · doi:10.1198/jasa.2005.s7 · Cited by 848 · see citing works
Artificial Intelligence and Human Trust in Healthcare: Focus on Clinicians
Asan, O., Bayrak, A. E., & Choudhury, A. (2020) · Journal of Medical Internet Research · doi:10.2196/15154 · Cited by 825 · see citing works
Nadarzynski, T., Miles, O., Cowie, A., & Ridge, D. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619871808 · Cited by 810 · see citing works
Smart healthcare: making medical care more intelligent
Tian, S., Yang, W., Grange, J. M. L., Wang, P., Huang, W., & Ye, Z. (2019) · Global Health Journal · doi:10.1016/j.glohj.2019.07.001 · Cited by 716 · see citing works
A Methodology for Conducting Integrative Mixed Methods Research and Data Analyses
Castro, F. G., Kellison, J. G., Boyd, S. J., & Kopak, A. M. (2010) · Journal of Mixed Methods Research · doi:10.1177/1558689810382916 · Cited by 640 · see citing works
Artificial Intelligence in Health Care: A Report From the National Academy of Medicine
Matheny, M. E., Whicher, D., & Thadaney Israni, S. (2020) · JAMA · doi:10.1001/jama.2019.21579 · Cited by 628 · see citing works
Clinical reasoning in the health professions
Oehme, F. W. (1996) · Toxicology Letters · doi:10.1016/s0378-4274(96)90055-7 · Cited by 615 · see citing works
Xu, W. M. (2019) · interactions · doi:10.1145/3328485 · Cited by 479 · see citing works
Kim, D. W., Jang, H. Y., Kim, K. W., Shin, Y., & Park, S. H. (2019) · Korean Journal of Radiology · doi:10.3348/kjr.2019.0025 · Cited by 459 · see citing works
Teaching Clinical Reasoning: Case-Based and Coached
Kassirer, J. P. (2010) · Academic Medicine · doi:10.1097/acm.0b013e3181d5dd0d · Cited by 453 · see citing works
Scripts and Medical Diagnostic Knowledge
Charlin, B., Tardif, J., & Boshuizen, H. P. A. (2000) · Academic Medicine · doi:10.1097/00001888-200002000-00020 · Cited by 445 · see citing works
Sociologic Influences on Decision-Making by Clinicians
Eisenberg, J. M. (1979) · Annals of Internal Medicine · doi:10.7326/0003-4819-90-6-957 · Cited by 423 · see citing works
Questions for artificial intelligence in health care
Maddox, T. M., Rumsfeld, J. S., & Payne, P. R. O. (2019) · JAMA · doi:10.1001/jama.2018.18932 · Cited by 385 · see citing works
Artificial Intelligence Versus Clinicians in Disease Diagnosis: Systematic Review
Shen, J., Zhang, C. J. P., Jiang, B., Chen, J., Song, J., Liu, Z., He, Z., Wong, S. Y., Fang, P.-H., & Ming, W.-K. (2019) · JMIR Medical Informatics · doi:10.2196/10010 · Cited by 373 · see citing works
Fan, W., Liu, J., Zhu, S., & Pardalos, P. M. (2020) · Annals of Operations Research · doi:10.1007/s10479-018-2818-y · Cited by 346 · see citing works
Evaluating Natural Language Understanding Services for Conversational Question Answering Systems
Braun, D., Hernandez-Mendez, A., Matthes, F., & Langen, M. (2017) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w17-5522 · Cited by 199 · see citing works
Who is the "Human" in Human-Centered Machine Learning
Chancellor, S., Baumer, E., & de Choudhury, M. (2019) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3359249 · Cited by 187 · see citing works
Inconsistent Performance of Deep Learning Models on Mammogram Classification
Wang, X., Liang, G., Zhang, Y., Blanton, H., Bessinger, Z., & Jacobs, N. (2020) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2020.01.006 · Cited by 176 · see citing works
Stanfill, M. H., & Marc, D. T. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677913 · Cited by 170 · see citing works
A Comparison of Artificial Intelligence and Human Doctors for the Purpose of Triage and Diagnosis
Baker, A., Perov, Y., Middleton, K., Baxter, J., Mullarkey, D., Sangar, D., Butt, M., DoRosario, A., & Johri, S. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.543405 · Cited by 167 · see citing works
Evaluating artificial intelligence in medicine: phases of clinical research
Park, Y., Jackson, G. P., Foreman, M. A., Gruen, D., Hu, J., & Das, A. K. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooaa033 · Cited by 150 · see citing works
Shinners, L., Aggar, C., Grace, S., & Smith, S. (2020) · Health Informatics Journal · doi:10.1177/1460458219874641 · Cited by 148 · see citing works
Young, I. J. B., Luz, S. F., & Lone, N. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.103971 · Cited by 142 · see citing works
Computer aided fuzzy medical diagnosis
Innocent, P. R., & John, R. I. (2004) · Information Sciences · doi:10.1016/j.ins.2004.03.003 · Cited by 131 · see citing works
Kim, J., & Heo, W. (2021) · Information Technology and People · doi:10.1108/itp-04-2019-0173 · Cited by 106 · see citing works
Hobbs, R. (2017) · Wiley eBooks · doi:10.1002/9781394260201 · Cited by 105 · see citing works
The application of decision making concepts by nurse practitioners in general practice
Offredy, M. (1998) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.1998.00823.x · Cited by 102 · see citing works
Artificial intelligence systems for complex decision-making in acute care medicine: a review
Lynn, L. A. (2019) · Patient Safety in Surgery · doi:10.1186/s13037-019-0188-2 · Cited by 100 · see citing works
Healthcare information systems research, revelations and visions
LeRouge, C., Mantzana, V., & Wilson, E. V. (2007) · European Journal of Information Systems · doi:10.1057/palgrave.ejis.3000712 · Cited by 81 · see citing works
Nursing education 2.0: YouTube.
Skiba, D. J. (2007) · PubMed · Cited by 54 · see citing works
Thompson, B., & Skiba, D. J. (2008) · PubMed · Cited by 45 · see citing works
Six ways of experiencing information literacy in nursing: The findings of a phenomenographic study
Forster, M. (2014) · Nurse Education Today · doi:10.1016/j.nedt.2014.06.005 · Cited by 44 · see citing works
How Different Theories of Clinical Reasoning Influence Teaching and Assessment
Young, M. E., Dory, V., Lubarsky, S., et al. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002303 · Cited by 43 · see citing works
Conceptualizing Telehealth in Nursing Practice
Nagel, D. A., & Penner, J. L. (2015) · Journal of Holistic Nursing · doi:10.1177/0898010115580236 · Cited by 41 · see citing works
Is Artificial Intelligence Better Than Human Clinicians in Predicting Patient Outcomes?
Lee, J. (2020) · Journal of Medical Internet Research · doi:10.2196/19918 · Cited by 36 · see citing works
Bryce, D. A., King, N. J. C., Graebner, C. F., & Myers, J. H. (1998) · Journal of Interactive Media in Education · doi:10.5334/1998-1 · Cited by 30 · see citing works
Choudhury, M. D., Lee, M. K., Zhu, H., & Shamma, D. A. (2020) · Human-Computer Interaction · doi:10.1080/07370024.2020.1744146 · Cited by 30 · see citing works
Computer Literacy of Nurses in a Community Hospital: Where Are We Today?
Campbell, C. J., & McDowell, D. (2011) · The Journal of Continuing Education in Nursing · doi:10.3928/00220124-20110215-01 · Cited by 26 · see citing works
Research on Application of Artificial Intelligence in Medical Education
Zhao, H., Li, G., & Feng, W. (2018) · doi:10.1109/esaic.2018.00085 · Cited by 26 · see citing works
RADAR: automated task planning for proactive decision support
Grover, S., Sengupta, S., Chakraborti, T., Mishra, A. P., & Subbarao Kambhampati, B. (2020) · Human-Computer Interaction · doi:10.1080/07370024.2020.1726751 · Cited by 26 · see citing works
A Computer Literacy Scale for Newly Enrolled Nursing College Students
Lin, T. (2011) · Journal of Nursing Research · doi:10.1097/jnr.0b013e318236d03f · Cited by 24 · see citing works
Reconceptualizing media literacy for the digital age
Hobbs, R. (2018) · Facet eBooks · doi:10.29085/9781856049870.011 · Cited by 24 · see citing works
Hah, H., Goldin, D. S., & Ha, S. (2019) · Journal of Medical Internet Research · doi:10.2196/15087 · Cited by 17 · see citing works
Self-citationHealth Literacy Education in Baccalaureate Nursing Programs in the United States
Scott, S. A. (2016) · Nursing Education Perspectives · doi:10.1097/01.nep.0000000000000005 · Cited by 16 · see citing works
LeRouge, C., Hah, H., Deckard, G. J., & Jiang, H. (2020) · JMIR mhealth and uhealth · doi:10.2196/18391 · Cited by 14 · see citing works
Self-citationDong, L., Li, W., Ma, D., et al. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1549/4/042076 · Cited by 5 · see citing works
The Coming Era Of Precision Health
Phillips, K. A. (2021) · Health Affairs · doi:10.1377/hlthaff.2020.02393 · Cited by 5 · see citing works
Saunders, C., & Ives, B. (2000) · Communications of the Association for Information Systems · doi:10.17705/1cais.00305 · Cited by 4 · see citing works
Training the Next Generation of Doctors in Uganda
(2010) · Internal Medicine News · doi:10.1016/s1097-8690(10)70858-1 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [52]Canonico M De Russis L A comparison and critique of natural language understanding tools Cloud Computing 2018 2021-12-10 https://www.semanticscholar.org/paper/A-Comparison-and-Critique-of-Natural-Language-Tools-Canonico-Russis/fdf8bbd6dff6201c1965f81842852cbba59fe79e
- [55]Raudenbush SW Bryk AS Hierarchical Linear Models: Applications and Data Analysis Methods (Advanced Quantitative Techniques in the Social Sciences) 2002 Thousand Oaks, CA Sage Publications, Inc
Cited by
Sorted by citations
Chen, M., Zhang, B., Cai, Z., Seery, S., Gonzalez, M. J., Ali, N. M., Ren, R., Qiao, Y., Xue, P., & Jiang, Y. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.990604 · Cited by 258 · see citing works
Zhang, M., Scandiffio, J., Younus, S., Jeyakumar, T., Karsan, I., Charow, R., Salhia, M., & Wiljer, D. (2023) · JMIR Formative Research · doi:10.2196/47847 · Cited by 67 · see citing works
“This education can be provided in a variety of ways, including integration into the undergraduate curriculum, workshops, and continuing education courses [44,45].”
BackgroundClinical experience and perception of risk affect the acceptance and trust of using AI in medicine
Schulz, P. J., Kee, K. M., Lwin, M. O., Goh, W. W., Chia, K. Y., Cheung, M. F. K., Lam, T. Y. T., & Sung, J. J. Y. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1620127 · Cited by 12 · see citing works
Mastour, H., Mirzaei, S., Sohrabi, S., & Toofaninejad, E. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09630-9 · Cited by 3 · see citing works
“Current advances in AI have significantly influenced diagnostic and clinical decision-making [11, 12], yet the adoption of AI training in UME is hindered by curricular overcrowding and a lack of systematic evidence.”
- Authors
- Hah, H., & Goldin, D. S.
- Affiliations
- Florida International University (US)
- Corresponding author
- Hah, Hyeyoung
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2021-12-16
- Volume
- 23
- Issue
- 12
- Pages
- e33540
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC8726017
- OpenAlex
- W3214309531
- Cited by
- 81
- References
- 64
- Citations by year
- 2026: 27 · 2025: 21 · 2024: 19 · 2023: 10 · 2022: 4
- FWCI
- 2.39
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- artificial intelligence, AI-assisted diagnosis, clinical decision-making, clinician perceptions, diagnostic reasoning, general practice, social media use, diagnostic skills
- MeSH terms
- Sentiment Analysis, Algorithms, Artificial Intelligence, Decision Making, Humans, Technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
