How machine learning is embedded to support clinician decision making: an analysis of FDA-approved medical devices
Open accessAbstract
Objective: To examine how and to what extent medical devices using machine learning (ML) support clinician decision making. Methods: We searched for medical devices that were (1) approved by the US Food and Drug Administration (FDA) up till February 2020; (2) intended for use by clinicians; (3) in clinical tasks or decisions and (4) used ML. Descriptive information about the clinical task, device task, device input and output, and ML method were extracted. The stage of human information processing automated by ML-based devices and level of autonomy were assessed. Results: Of 137 candidates, 59 FDA approvals for 49 unique devices were included. Most approvals (n=51) were since 2018. Devices commonly assisted with diagnostic (n=35) and triage (n=10) tasks. Twenty-three devices were assistive, providing decision support but left clinicians to make important decisions including diagnosis. Twelve automated the provision of information (autonomous information), such as quantification of heart ejection fraction, while 14 automatically provided task decisions like triaging the reading of scans according to suspected findings of stroke (autonomous decisions). Stages of human information processing most automated by devices were information analysis, (n=14) providing information as an input into clinician decision making, and decision selection (n=29), where devices provide a decision. Conclusion: Leveraging the benefits of ML algorithms to support clinicians while mitigating risks, requires a solid relationship between clinician and ML-based devices. Such relationships must be carefully designed, considering how algorithms are embedded in devices, the tasks supported, information provided and clinicians’ interactions with them.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The FDA regulatory approach to AI in healthcare seeks to ensure that AI applications meet the same safety and effectiveness standards as traditional medical devices, holding them accountable.
“The Food and Drug Administration (FDA) regulatory approach to AI in healthcare, for example, seeks to ensure that AI applications meet the same safety and effectiveness standards as traditional medical devices.This approach ensures that AI systems, such as those used for diagnosing diseases or making treatment recommendations, are held accountable for their performance and outcomes [143].”
From Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond (2024) · cites this paper as [143]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 29,639 · see citing works
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450 · see citing works
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · see citing works
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · see citing works
Breast imaging reporting and data system (BI-RADS)
Liberman, L., & Menell, J. H. (2002) · Radiologic Clinics of North America · doi:10.1016/s0033-8389(01)00017-3 · Cited by 2,159 · see citing works
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883 · see citing works
Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0040-6 · Cited by 1,638 · see citing works
Ensuring fairness in machine learning to advance health equity.
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1990 · Cited by 1,341 · see citing works
Breast Imaging Reporting and Data System
Berg, W. A., Campassi, C., LANGENBERG, P. W., & Sexton, M. J. (2000) · American Journal of Roentgenology · doi:10.2214/ajr.174.6.1741769 · Cited by 1,268 · see citing works
Benjamens, S., Dhunnoo, P., & Meskó, B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00324-0 · Cited by 1,215 · see citing works
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162 · see citing works
Artificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · see citing works
Markus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103655 · Cited by 811 · see citing works
The master algorithm: how the quest for the ultimate learning machine will remake our world
(2016) · Choice Reviews Online · doi:10.5860/choice.194685 · Cited by 808 · see citing works
Human Performance Consequences of Stages and Levels of Automation: An Integrated Meta-Analysis
Onnasch, L., Wickens, C. D., Li, H., & Manzey, D. (2014) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720813501549 · Cited by 524 · see citing works
Automation bias and verification complexity: a systematic review
Lyell, D., & Coiera, E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw105 · Cited by 451 · see citing works
Self-citationCoiera, E. W. (2015) · CRC Press eBooks · doi:10.1201/b13617 · Cited by 421 · see citing works
Self-citationAutomation and Human Performance
Bowers, C. A., Oser, R. L., Eduardo Salas, A., & Cannon‐Bowers, J. A. (2018) · Routledge eBooks · doi:10.1201/9781315137957 · Cited by 307 · see citing works
Manzey, D., Reichenbach, J., & Onnasch, L. (2012) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343411433844 · Cited by 277 · see citing works
(1998) · Cited by 267 · see citing works
Kim, M. O., Coiera, E., & Magrabi, F. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw154 · Cited by 263 · see citing works
Self-citationArtificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril
Series, T. L. H. S., & National Academy of Medicine (2019) · doi:10.17226/27111 · Cited by 227 · see citing works
Bailey, N. R., & Scerbo, M. W. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500535301 · Cited by 224 · see citing works
The impact of machine learning on patient care: A systematic review
Ben-Israel, D., Jacobs, W. B., Casha, S., Lang, S., Ryu, W. H. A., de Lotbiniere-Bassett, M., & Cadotte, D. W. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101785 · Cited by 207 · see citing works
Medical Device Regulation: An Introduction for the Practicing Physician
Maisel, W. H. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-140-4-200402170-00012 · Cited by 194 · see citing works
Human Decision Makers and Automated Decision Aids: Made for Each Other?
Mosier, K. L., & Skitka, L. J. (2018) · doi:10.1201/9781315137957-10 · Cited by 188 · see citing works
Houssami, N., Kirkpatrick-Jones, G., Noguchi, N., & Lee, C. I. (2019) · Expert Review of Medical Devices · doi:10.1080/17434440.2019.1610387 · Cited by 185 · see citing works
Human Factors Challenges for the Safe Use of Artificial Intelligence in Patient Care
Sujan, M., Furniss, D., Grundy, K., Grundy, H., Nelson, D., Elliott, M., White, S., Habli, I., & Reynolds, N. (2019) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2019-100081 · Cited by 144 · see citing works
The Last Mile: Where Artificial Intelligence Meets Reality
Coiera, E. (2019) · Journal of Medical Internet Research · doi:10.2196/16323 · Cited by 102 · see citing works
Self-citationThe safety and quality of decision support systems.
Coiera, E. W., Westbrook, J. I., & Wyatt, J. C. (2006) · PubMed · Cited by 92 · see citing works
Self-citationThe effect of cognitive load and task complexity on automation bias in electronic prescribing
Lyell, D., Magrabi, F., & Coiera, E. (2018) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720818781224 · Cited by 67 · see citing works
Self-citationPetersen, C. C., Smith, J. R. L., Freimuth, R. R., Goodman, K. W., Jackson, G. P., Kannry, J. L., Liu, H., Madhavan, S., Sittig, D. F., & Wright, A. T. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa319 · Cited by 66 · see citing works
On algorithms, machines, and medicine
Coiera, E. W. (2018) · The Lancet Oncology · doi:10.1016/s1470-2045(18)30835-0 · Cited by 29 · see citing works
Self-citationReduced Verification of Medication Alerts Increases Prescribing Errors
Lyell, D., Magrabi, F., & Coiera, E. W. (2019) · Applied Clinical Informatics · doi:10.1055/s-0038-1677009 · Cited by 19 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Multimodal machine learning in precision health: A scoping review
Kline, A., Wang, H., Li, Y., Dennis, S., Hutch, M., Xu, Z., Wang, F., Cheng, F., & Luo, Y. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00712-8 · Cited by 528 · see citing works
Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond
Nasir, S., Khan, R. A., & Bai, S. (2024) · IEEE Access · doi:10.1109/access.2024.3369912 · Cited by 152 · see citing works
Formosa, P. (2021) · Minds and Machines · doi:10.1007/s11023-021-09579-2 · Cited by 144 · see citing works
Abdelwanis, M., Alarafati, H. K., Tammam, M. M. S., & Simsekler, M. C. E. (2024) · Journal of Safety Science and Resilience · doi:10.1016/j.jnlssr.2024.06.001 · Cited by 137 · see citing works
Zhu, S., Gilbert, M., Chetty, I., & Siddiqui, F. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104828 · Cited by 131 · see citing works
Laxar, D., Eitenberger, M., Maleczek, M., Kaider, A., Hammerle, F. P., & Kimberger, O. (2023) · BMC Medicine · doi:10.1186/s12916-023-03068-2 · Cited by 19 · see citing works
“In accordance with this rule, the US Food and Drug Administration requires supervision of all machine-determined scorings by a qualified physician [21].”
Background
- Authors
- Lyell, D., Coiera, E., Chen, J., Shah, P., & Magrabi, F.
- Affiliations
- Macquarie University (AU)
- Corresponding author
- Lyell, David
- Venue
- BMJ Health & Care Informatics
- Publisher
- BMJ
- Published
- 2021-04
- Volume
- 28
- Issue
- 1
- Pages
- e100301
- ISSN
- 2632-1009 (ISSN-L)
- PMC
- PMC8054073
- OpenAlex
- W3152710649
- Cited by
- 95
- References
- 34
- Citations by year
- 2026: 11 · 2025: 9 · 2024: 17 · 2023: 27 · 2022: 24 · 2021: 7
- FWCI
- 5.46
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMJ Health & Care Informatics (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Health and Medical Research Council (Centre for Research Excellence (CRE) in Digital He)
- Macquarie University (Safety Net Grant)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- machine learning, medical devices, clinician decision-making, FDA-approved devices, clinical decision support, human information processing, diagnostic support, triage, human-AI interaction
- MeSH terms
- Machine Learning, Decision Making, Computer-Assisted, Equipment and Supplies, Humans, United States, United States Food and Drug Administration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
