Framing the challenges of artificial intelligence in medicine
Abstract
On a clear January morning in Florida, a Tesla enthusiast and network entrepreneur was driving his new Tesla Model S on US Highway 27A, returning from a family trip. He had posted dozens of widely circulated YouTube tutorial videos on his vehicle and clearly understood many of the technical details of his car. That day, he let the vehicle run autonomously on Autopilot mode for 37 min, before it crashed into the trailer of a truck turning left. The Autopilot did not identify the white side of the trailer as a potential hazard, and the driver was killed, leaving his family and his high-tech business behind.1 This tragedy is not a metaphor for artificial intelligence (AI) applications but an example of a long-recognised challenge in AI: the Frame Problem.2 Although rarely appreciated in the scholarly and lay descriptions of the stunning recent successes of AI in medical applications, the Frame Problem and related AI challenges will have unintended harmful effects to the care of patients if not directly addressed. With the recent advancement in machine learning algorithms, many medical tasks previously thought to require human expertise have been replicated by AI systems at or above the level of accuracy in human experts. These important demonstrations range from evaluating fundus retinography3 and histopathology4 to reading chest radiographs5 and assessment of skin lesions.6 These studies have encompassed very large numbers of patient cases and have been extensively benchmarked against clinicians. However, all these studies are retrospective in that they involve a collection of labelled cases against which the AI systems are trained and another collection against which they are tested or validated. So far, they have not entered into routine prospective use in the clinic where the Frame Problem will manifest itself most pathologically. The Frame …
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Implementation of AI in clinical radiology practice is complex and has so far been slow.
“While technical performance of AI applications is expected to increase continuously, its implementation in clinical radiology practice is rather complex and has so far been slow [6, 12, 13].”
From Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors (2020) · cites this paper as [13]
Cites
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697 · see citing works
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616 · see citing works
Some Philosophical Problems from the Standpoint of Artificial Intelligence
McCarthy, J., & Hayes, P. J. (1981) · Elsevier eBooks · doi:10.1016/b978-0-934613-03-3.50033-7 · Cited by 3,296 · see citing works
Artificial intelligence in healthcare
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0305-z · Cited by 2,817 · see citing works
Self-citation(2016) · Cited by 2,334 · see citing works
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789 · see citing works
Estimating causal effects from epidemiological data
Hernán, M. Á., & Robins, J. M. (2006) · Journal of Epidemiology & Community Health · doi:10.1136/jech.2004.029496 · Cited by 1,029 · see citing works
Diagnostic Accuracy of Digital Screening Mammography With and Without Computer-Aided Detection
Lehman, C. D., Wellman, R. D., Buist, D. S. M., Kerlikowske, K., Tosteson, A. N. A., & Miglioretti, D. L. (2015) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2015.5231 · Cited by 678 · see citing works
The "To Err is Human" report and the patient safety literature
Stelfox, H. T. (2006) · BMJ Quality & Safety · doi:10.1136/qshc.2006.017947 · Cited by 391 · see citing works
Lessons learned from the gene therapy trial for ornithine transcarbamylase deficiency
Wilson, J. M. (2009) · Molecular Genetics and Metabolism · doi:10.1016/j.ymgme.2008.12.016 · Cited by 299 · see citing works
Phansalkar, S., van der Sijs, H., Tucker, A. D., Desai, A. A., Bell, D. S., Teich, J. M., Middleton, B., & Bates, D. W. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001089 · Cited by 239 · see citing works
Access to Health and Health Care: How Race and Ethnicity Matter
Richardson, L. D., & Norris, M. M. (2010) · Mount Sinai Journal of Medicine A Journal of Translational and Personalized Medicine · doi:10.1002/msj.20174 · Cited by 220 · see citing works
Death but one unintended consequence of gene-therapy trial.
Sibbald, B. (2001) · PubMed · Cited by 124 · see citing works
Association of Omics Features with Histopathology Patterns in Lung Adenocarcinoma
Yu, K., Berry, G. J., Rubin, D. L., Ré, C., Altman, R. B., & Snyder, M. P. (2017) · Cell Systems · doi:10.1016/j.cels.2017.10.014 · Cited by 119 · see citing works
Self-citationOmics Profiling in Precision Oncology
Yu, K., & Snyder, M. P. (2016) · Molecular & Cellular Proteomics · doi:10.1074/mcp.o116.059253 · Cited by 108 · see citing works
Self-citationLee, T. T., & Kesselheim, A. S. (2018) · Annals of Internal Medicine · doi:10.7326/m17-2715 · Cited by 62 · see citing works
Ethical Issues Concerning Vaccination Requirements
Amin, A. N. E., Parra, M. T., Kim-Farley, R., & Fielding, J. E. (2012) · Public Health Reviews · doi:10.1007/bf03391666 · Cited by 51 · see citing works
Waltz, E. (2017) · Nature Biotechnology · doi:10.1038/nbt.3761 · Cited by 30 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Artificial intelligence in healthcare
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0305-z · Cited by 2,817 · see citing works
Self-citationIn AI we trust? Perceptions about automated decision-making by artificial intelligence
Araujo, T., Helberger, N., Kruikemeier, S., & de Vreese, C. H. (2020) · AI & SOCIETY · doi:10.1007/s00146-019-00931-w · Cited by 1,038 · see citing works
Artificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · see citing works
Yang, G., Ye, Q., & Xia, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.07.016 · Cited by 717 · see citing works
Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review
Yin, J., Ngiam, K. Y., & Teo, H. H. (2021) · Journal of Medical Internet Research · doi:10.2196/25759 · Cited by 419 · see citing works
Artificial intelligence and the implementation challenge
Shaw, J., Rudzicz, F., Jamieson, T., & Goldfarb, A. (2019) · Journal of Medical Internet Research · doi:10.2196/13659 · Cited by 389 · see citing works
Magrabi, F., Ammenwerth, E., McNair, J. B., De Keizer, N. F., Hyppönen, H., Nykänen, P., Rigby, M., Scott, P. J., Vehko, T., Wong, Z. S.-Y., et al. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677903 · Cited by 347 · see citing works
Blease, C., Kaptchuk, T. J., Bernstein, M. H., Mandl, K. D., Halamka, J. D., & DesRoches, C. M. (2019) · Journal of Medical Internet Research · doi:10.2196/12802 · Cited by 346 · see citing works
Strohm, L., Hehakaya, C., Ranschaert, E. R., Boon, W. P. C., & Moors, E. H. M. (2020) · European Radiology · doi:10.1007/s00330-020-06946-y · Cited by 329 · see citing works
Competencies for the Use of Artificial Intelligence–Based Tools by Health Care Professionals
Russell, R. G., Lovett Novak, L., Patel, M., Garvey, K. V., Craig, K. J. T., Jackson, G. P., Moore, D., & Miller, B. M. (2023) · Academic Medicine · doi:10.1097/acm.0000000000004963 · Cited by 320 · see citing works
Regulation of predictive analytics in medicine
Parikh, R. B., Obermeyer, Z., & Navathe, A. S. (2019) · Science · doi:10.1126/science.aaw0029 · Cited by 223 · see citing works
What should medical students know about artificial intelligence in medicine?
Park, S. H., Do, K.-H., Kim, S., Park, J. H., & Lim, Y.-S. (2019) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2019.16.18 · Cited by 196 · see citing works
Ellahham, S., Ellahham, N., & Simsekler, M. C. E. (2020) · American Journal of Medical Quality · doi:10.1177/1062860619878515 · Cited by 177 · see citing works
Zihni, E., Madai, V. I., Livne, M., Galinovic, I., Khalil, A. A., Fiebach, J. B., & Frey, D. (2020) · PLOS ONE · doi:10.1371/journal.pone.0231166 · Cited by 187 · see citing works
Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers
Solanki, P., Grundy, J., & Hussain, W. (2023) · AI and Ethics · doi:10.1007/s43681-022-00195-z · Cited by 157 · see citing works
“Although models that are less interpretable tend to have higher accuracy [123], interpretable models are generally preferred, as this aids transparency and trust [9, 167].”
BackgroundRomero-Brufau, S., Wyatt, K. D., Boyum, P., Mickelson, M., Moore, M., & Cognetta-Rieke, C. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104072 · Cited by 143 · see citing works
Medical Artificial Intelligence and Human Values
Yu, K.-H., Healey, E., Leong, T.-Y., Kohane, I. S., & Manrai, A. K. (2024) · New England Journal of Medicine · doi:10.1056/nejmra2214183 · Cited by 149 · see citing works
Self-citationHuman Factors Challenges for the Safe Use of Artificial Intelligence in Patient Care
Sujan, M., Furniss, D., Grundy, K., Grundy, H., Nelson, D., Elliott, M., White, S., Habli, I., & Reynolds, N. (2019) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2019-100081 · Cited by 144 · see citing works
Amiri, H., Peiravi, S., rezazadeh shojaee, S. S., Rouhparvarzamin, M., Nateghi, M. N., Etemadi, M. H., ShojaeiBaghini, M., Musaie, F., Anvari, M. H., & Asadi Anar, M. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05406-1 · Cited by 149 · see citing works
“AI has been extensively integrated into decision support systems (DSSs) in data-intensive medical specialties like radiology, pathology, and ophthalmology [10].”
BackgroundBlease, C., Locher, C., Leon-Carlyle, M., & Doraiswamy, M. (2020) · DIGITAL HEALTH · doi:10.1177/2055207620968355 · Cited by 121 · see citing works
Governing the safety of artificial intelligence in healthcare
Macrae, C. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2019-009484 · Cited by 123 · see citing works
Hurtful words: quantifying biases in clinical contextual word embeddings
Zhang, H., Lu, A. X., Abdalla, M., McDermott, M., & Ghassemi, M. (2020) · Proceedings of the ACM Conference on Health, Inference, and Learning · doi:10.1145/3368555.3384448 · Cited by 122 · see citing works
Allen, M. R., Webb, S., Mandvi, A., Frieden, M., Tai-Seale, M., & Kallenberg, G. (2024) · BMC Primary Care · doi:10.1186/s12875-024-02282-y · Cited by 100 · see citing works
AI and professional liability assessment in healthcare. a revolution in legal medicine?
Terranova, C., Cestonaro, C., Fava, L., & Cinquetti, A. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2023.1337335 · Cited by 73 · see citing works
Lekadir, K., Osuala, R., Gallin, C., Lazrak, N., Kushibar, K., Tsakou, G., Aussó, S., Alberich, L. C., Marias, K., Tsiknakis, M., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.09658 · Cited by 58 · see citing works
Artificial intelligence for ultrasonography: unique opportunities and challenges
Park, S. H. (2021) · Ultrasonography · doi:10.14366/usg.20078 · Cited by 37 · see citing works
Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
Zhang, H., Lu, A. X., Abdalla, M., McDermott, M., & Ghassemi, M. (2020) · arXiv · doi:10.48550/arxiv.2003.11515 · Cited by 37 · see citing works
Yoo, J., Hur, S., Hwang, W., & Cha, W. C. (2023) · Healthcare Informatics Research · doi:10.4258/hir.2023.29.1.64 · Cited by 34 · see citing works
- Authors
- Yu, K.-H., & Kohane, I. S.
- Affiliations
- Harvard University (US)
- Corresponding author
- Yu, Kun-Hsing
- Venue
- BMJ Quality & Safety
- Publisher
- BMJ
- Published
- 2018-10-05
- Volume
- 28
- Issue
- 3
- Pages
- 238-241
- ISSN
- 1475-3898 (ISSN-L)
- 2044-5415
- 2044-5423
- OpenAlex
- W2895083984
- Cited by
- 233
- References
- 19
- Citations by year
- 2026: 18 · 2025: 29 · 2024: 34 · 2023: 28 · 2022: 29 · 2021: 36 · 2020: 36 · 2019: 21 · 2018: 2
- FWCI
- 6.44
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- BMJ Quality & Safety (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Funders
- Harvard University (Harvard Data Science Fellowship)
- National Institutes of Health (OT3OD025466)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- artificial intelligence in healthcare, frame problem, clinical implementation of AI, radiographic interpretation, histopathology, skin lesion assessment
- MeSH terms
- Artificial Intelligence, Delivery of Health Care, Humans, Quality Control, Decision Support Systems, Clinical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
