Trust and medical AI: the challenges we face and the expertise needed to overcome them
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Artificial intelligence (AI) is increasingly of tremendous interest in the medical field. How-ever, failures of medical AI could have serious consequences for both clinical outcomes and the patient experience. These consequences could erode public trust in AI, which could in turn undermine trust in our healthcare institutions. This article makes 2 contributions. First, it describes the major conceptual, technical, and humanistic challenges in medical AI. Second, it proposes a solution that hinges on the education and accreditation of new expert groups who specialize in the development, verification, and operation of medical AI technologies. These groups will be required to maintain trust in our healthcare institutions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“a minority of AI chatbot users realized early that these LLMs do not voluntarily cite credible references to authenticate their information [10-12].”
From Reference Hallucination Score for Medical Artificial Intelligence Chatbots: Development and Usability Study (2024) · cites this paper as [11]
In the citing paper
“It has been argued that AI's transformational role in healthcare allows for patients to be treated as commodities and not as individuals”
From SHIFTing artificial intelligence to be responsible in healthcare: a systematic review (2022) · cites this paper as Quinn et al., 2021
Citations
Cites
Principles of Biomedical Ethics
(2012) · Thieves of Virtue · doi:10.7551/mitpress/9079.003.0009 · Cited by 13,453
Shared decision making: A model for clinical practice
Elwyn, G., Frosch, D., Thomson, R., Joseph-Williams, N., Lloyd, A., Kinnersley, P., Cording, E., Tomson, D., Dodd, C., Rollnick, S., et al. (2012) · Journal of General Internal Medicine · doi:10.1007/s11606-012-2077-6 · Cited by 4,405 · cited 1×
“If practitioners fail to incorporate the values and the wishes of a specific patient into their professional decisions, the AI system may paternalistically interfere with shared decisionmaking and informed choice [18, 19].”
BackgroundPrinciples of biomedical ethics
Horton, J. (2002) · Transactions of the Royal Society of Tropical Medicine and Hygiene · doi:10.1016/s0035-9203(02)90265-8 · Cited by 4,377
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Four Models of the Physician-Patient Relationship
Emanuel, E. J. (1992) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1992.03480160079038 · Cited by 2,449
DARPA's Explainable Artificial Intelligence Program
Gunning, D., & Aha, D. W. (2019) · AI Magazine · doi:10.1609/aimag.v40i2.2850 · Cited by 1,360
An investigation of the Therac-25 accidents
Leveson, N. G., & Turner, C. S. (1993) · Computer · doi:10.1109/mc.1993.274940 · Cited by 1,238
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162
Nagendran, M., Chen, Y., Lovejoy, C. A., Gordon, A. C., Komorowski, M., Harvey, H., Topol, E. J., Ioannidis, J. P. A., Collins, G. S., & Maruthappu, M. (2020) · BMJ · doi:10.1136/bmj.m689 · Cited by 1,152
Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions
Evans, D. (2003) · Journal of Clinical Nursing · doi:10.1046/j.1365-2702.2003.00662.x · Cited by 993
Cited by
Items 301–349 of 349
Yang, X., Li, J., Xiang, Y., Ju, M., Liang, K., He, Y., Zhao, Q., & Huang, H. (2026) · doi:10.2196/preprints.99511
Ethical Considerations and Challenges in AI Dermatology
Kaya, G., & Seyyedabbasi, A. (2026) · Studies in computational intelligence · doi:10.1007/978-3-032-17975-3_1
Inteligencia Artificial en Medicina: Percepciones de estudiantes y trabajadores del área de la salud
Espinoza, I. R. G., Hernández, A. D. L., Romero, E. M., Castro-Ponce, A., Juárez-Salazar, G., Gutiérrez, F. A. C., Carmona, D. Y. D., & Cuenca, L. T. C. (2026) · Revista Mexicana de Investigación Médica · doi:10.62954/g1qa3868
Impact of artificial intelligence (AI) on disease diagnosis and detection
Manimaran, S., Shreenidhi, K. S., Nayyaar, A., & Anandaram, H. (2026) · Elsevier eBooks · doi:10.1016/b978-0-44-333496-2.00016-2
Building Toward a Future for Electronic Health Record Systems
Rosenbloom, S. T., Holmgren, A. J., & Steitz, B. D. (2026) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-092724-050949
Gut Microbiome, Machine Learning, and Artificial Intelligence Way Forward with Precision Medicine
Patel, R., Dubey, P., Patel, S., Patel, N. A., & Prajapati, B. G. (2026) · Springer eBooks · doi:10.1007/978-981-95-7494-0_12
Drogt, J., Jongsma, K. R., & Milota, M. (2026) · Synthese Library/Synthese library · doi:10.1007/978-3-032-10073-3_6
Sides, T. (2026) · Open Research Online (The Open University) · doi:10.21954/ou.ro.00109947
Designing Trustworthy and Explainable Clinical Decision Support Systems
Maleki, D. N. M., Zamanifar, A., Farhadi, A., & Sorouri, M. (2026) · Springer eBooks · doi:10.1007/978-3-032-15088-2_5
Chieregato, M., Spatola, G., Morassi, M., Migliorati, K., Cobelli, M., Giudice, L., Redolfi, N., Bassetti, C., D’Arrigo, C., Giorgi, C., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-9640465/v1
Lee, H.-Y., Li, P.-H., Yeh, T., & Chang, C. (2026) · doi:10.2196/preprints.102095
Omar, D. I., Althobaiti, S., Hani, B. M., Ali, S. A. E., & Anwer, M. M. (2026) · Scientific Reports · doi:10.1038/s41598-026-54681-z
Χαλλουμής, C. C. Κ. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8944-8.ch002
Let AI Decide for Me? Decision Autonomy and Emotional Response in Tragic Medical Scenarios
Zhao, L., Wang, Y., Wang, Y., Chu, H., & Hu, F. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2690097
Artificial Intelligence in Healthcare
Devi, K. N., Amaleswari, M. L. N. T., Charan, K. S., Kandapu, V., Keerthana, K., Mahathi, K., & Sanjay, K. (2025) · International Journal of Research in Pharmacology & Pharmacotherapeutics · doi:10.61096/ijrpp.v14.i3.2025.334-341
Obuchowicz, R., MINKOWSKI, J. M., Kamiński, P., & Glinkowski, W. (2026) · doi:10.2196/preprints.106040
Sharma, A., Kaur, H., & Saxena, S. (2026) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2026.105812
Trust typology of medical doctors in artificial intelligence
Velasco, L. C., & Wang, W. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.06.625
Perspective Chapter: The Doctor–AI–Patient Dynamics
Kenig, N., & Vives, A. M. (2026) · Artificial intelligence · doi:10.5772/intechopen.1015263
Mamven, M., Oleribe, O. O., Offiong, U. M., Acquah, G. F., Ogunrinde, O., Abubakar, H., Ansa, V. O., Uzochukwu, B. S. C., & Taylor-Robinson, S. D. (2026) · BMC Medical Education · doi:10.1186/s12909-026-10063-7
Implementation of Artificial Intelligence in Retinoblastoma Diagnosis
Moldavsky, Z. (2026) · The Medicine Forum
Nahm, W. J., Yin, E. S., Milam, E. C., & Weed, J. G. (2026) · JMIR Dermatology · doi:10.2196/94764
Gao, Z., Liu, D., He, Y., Zhao, Y., Zuo, Y., Zhang, J., & Zheng, J. (2026) · JMIR Medical Informatics · doi:10.2196/74339
BioMed-Agent-RL: A Meta Learning, All You Need for Biomedical ApplicationsPreprint
Jabin, M. A., Wu, Z., & Liu, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.21864
Condado, J. G., Cristòbal-Cóppulo, E., Gamundi, M., Martínez-Menéndez, I., Sanz, A. P., Ulizarna, L. M., Navarro, M. D. C. C., Palacios, F. K. M., Valdez, Á. C. A., & Rueda-Etxebarria, M. (2026) · Journal of Scientific Innovation in Medicine · doi:10.29024/jsim.282
Cattanach, P. (2026) · CentAUR (University of Reading) · doi:10.48683/1926.00131518
Chen, W., & Zhang, L. (2021) · Journal of Health Informatics and Digital Systems · doi:10.68159/u314569264
Ethical Design of Programmable Drug Delivery Systems: Autonomy, Safety, and Human Oversight
Ramirez, C., Torres, E., Ortega, P., & Mendes, S. (2025) · Journal of Applied Pharmaceutical Technologies and Systems · doi:10.68159/g260073438
AI in Neurological Health Care: Qualitative Study of Patient and Public Perceptions
Bedenik, T., Fennelly, O., & Bennett, K. (2026) · Journal of Medical Internet Research · doi:10.2196/87806
Artificial Intelligence in Infection Prevention: From Prediction to Protection
Das, M. (2026) · IntechOpen eBooks · doi:10.5772/intechopen.1017578
A Review of Deep Learning Models and Explainable Artificial Intelligence for Pathology Recognition
Toala-Paz, A., Becerra-Andrade, A., Cevallos-Levicek, D., Peláez, E., Domínguez, F., & Loayza, F. R. (2026) · International Conference on eDemocracy & eGovernment · doi:10.1109/icedeg70169.2026.11695396
Perceived Subjectivity as an Interactional Effect in Affective Conversational Systems
Leucas, D., Iasulaitis, S., & Caseli, H. D. M. (2026) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw71585.2026.11712075
The Use of Artificial Intelligence in Nursing Documentation and Decision-Making
Ageeli, A. I. (2024) · Review of Contemporary Philosophy
Preventing Cybercrimes through Use of AI in Healthcare: Prospects and Challenges
Sabriya, A. (2024) · Open Access Digital Management and Governance Review
ARAS, T., AYDIN, T., Aydemir, S. Ç., Alay, R. A., & KORKMAZ, Ş. (2026) · Turkiye Klinikleri Journal of Pediatrics · doi:10.5336/pediatr.2026-118260
Large language models for digital mental health: an HCI-centered scoping review
Zeng, M., Zhan, Z., Yu, K., Zeng, C., Zhang, D., Zhou, S., Hou, Y., Lin, M., & Zhang, R. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03317-7
Yahaya, A., Okafor, S. C., Pantami, A. M., Kaosisochukwu, U., Ohaji, J. C., Nwankwo, C. M.-J., Deborah, O. O., Olaoba, J. O., Gabriels, D. P., Usiosefe, H. I., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-10880027/v1
Mutambara, P. (2026) · DUT Open Scholar (Durban University of Technology) · doi:10.51415/10321/6493
Burti, S., Zotti, A., Coghlan, S., & Banzato, T. (2026) · Frontiers in Veterinary Science · doi:10.3389/fvets.2026.1978054
Self-citationPrior-informed machine learning for clinically interpretable medical diagnosis
Lin, Z., Walker, D., Murdoch, C., Colley, H. E., Anumba, D. O. C., & Lang, Z.-Q. (2026) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2026.1973664
Zhao, J., & Chen, Q. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1962331
Diversität und Gesundheit - Digitalisierungsprozesse als Herausforderung und Chance
Crepaz, K. (2025) · Libera Università di Bolzano
Part 2: the challenges and limitations of artificial intelligence in surgery
Rashid, R., Gashi, D., Arif, A., Sosnowski, K., Nicola, M., & Agha, R. (2026) · International Journal of Surgery · doi:10.1097/js9.0000000000005407
Signature patterns in digital representations of health before acute illness
Post, B. (2024) · Spiral (Imperial College London) · doi:10.25560/149050
Криминальные угрозы информационной безопасности в условиях цифровизации уголовного правосудия
Львовна, Р. Н., & Оглы, Г. В. Э. (2025) · Russian Journal of Criminology
Coll, A. S., & Campoy, A. L. (2026) · Medicina Clínica · doi:10.1016/j.medcli.2026.107669
Huang, C., Song, Y., Dong, J., Yang, F., Guo, J., & Sun, S. (2025) · doi:10.3389/fonc.2025.1519144.s001
Amann, J., Vayena, E., Ormond, K. E., Frey, D., Madai, V. I., & Blasimme, A. (2023) · doi:10.1371/journal.pone.0279088.r003
Metadata
- Authors
- Quinn, T. P., Senadeera, M., Jacobs, S., Coghlan, S., & Le, V.
- Affiliations
- Deakin University (AU)
- The University of Melbourne (AU)
- Corresponding author
- Quinn, Thomas P
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2020-12-19
- Volume
- 28
- Issue
- 4
- Pages
- 890-894
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC7973477
- OpenAlex
- W3117511502
- Cited by
- 287
- References
- 32
- Citations by year
- 2026: 66 · 2025: 83 · 2024: 72 · 2023: 37 · 2022: 20 · 2021: 9
- FWCI
- 4.86
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Minerva Access (University of Melbourne) (submittedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- artificial intelligence in healthcare, public trust in AI, AI failure, clinical outcomes, patient experience, professional accreditation, artificial intelligence in education
- MeSH terms
- Accreditation, Algorithms, Artificial Intelligence, Attitude to Computers, Attitude to Health, Humans, Medical Informatics, Trust
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.