CoChat
Sign Up
Cited by 33Artificial Intelligence in Healthcare and EducationHealthcare RegulationRadiomics and Machine Learning in Medical Imaging

Trends in clinical validation and usage of US food and Drug Administration-cleared artificial intelligence Algorithms for medical imaging

Khunte, M., Chae, A., Wang, R., Jain, R., Sun, Y., Sollee, J.R., Jiao, Z., Bai, H.X.·2023·Clinical Radiology
DOI 10.1016/j.crad.2022.09.122
Open this paper in CoChat — free account

Abstract

Aim To examine the current landscape of US Food and Drug Administration (FDA)-approved artificial intelligence (AI) medical imaging devices and identify trends in clinical validation strategy.Materials and methods A retrospective study was conducted that analysed data extracted from the American College of Radiology (ACR) Data Science Institute AI Central database as of November 2021 to identify trends in FDA clearance of AI products related to medical imaging. Product and clinical validation information of each device was gathered from their respective public 510(k) summary or de novo request submission, depending on their type of authorisation.Results Overall, the database included a total of 151 AI algorithms that were cleared by the FDA between 2008 and November 2021. Out of the 151 FDA summaries reviewed, 97 (64.2%) reported the use of clinical data to validate their device, with six (4%) revealing study participant demographics, and eight (5.3%) reporting the specifications of the machines used. A total of 51 (33.8%) AI devices characterised their clinical data as multicentre, three (2%) as single-centre, and the remaining 97 (64.2%) did not specify. The ground truth used for clinical validation was specified in 78 (51.6%) FDA summaries.Conclusion A wide breadth of AI algorithms has been developed for medical imaging. Most of the FDA summaries of the devices mention their use of clinical data and patient cases for device validation; however, few devices revealed the patient demographics or machine specifications used in their clinical studies, which may lead some consumers to question their external validation.

Integrity signals

Registry

Crossref · 10.1016/j.crad.2022.09.122

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Most FDA-cleared imaging AI algorithms, despite regulatory approval, demonstrate bias when deployed to various clinical settings, especially when the target population is different from the population the model was trained on.

    “It is thus not surprising that most of these algorithms, despite regulatory approval, demonstrate bias when deployed to various clinical settings especially when the target population is different from the population the model was trained on.”

    From AI pitfalls and what not to do: mitigating bias in AI (2023) · cites this paper as [56]

  • Only 34% of these FDA-cleared imaging AI products had multi-institutional validation and reported which institution(s) was(were) used for validation.

    “Moreover, only 34% had multi-institutional validation and reported which institution(s) was(were) used for validation. 56”

    From AI pitfalls and what not to do: mitigating bias in AI (2023) · cites this paper as [56]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Khunte, M., Chae, A., Wang, R., Jain, R., Sun, Y., Sollee, J. R., Jiao, Z., & Bai, H. X.
Affiliations
  • Brown University (US)
  • University of Pennsylvania (US)
  • Johns Hopkins University (US)
  • World Bank (US)
Corresponding author
Bai, H.X.
ORCID
Venue
Clinical Radiology
Publisher
Elsevier BV
Published
2023-02
Volume
78
Issue
2
Pages
123-129
ISSN
0009-9260 (ISSN-L)
DOI
10.1016/j.crad.2022.09.122
OpenAlex
W4307725213
Cited by
33
References
10
Citations by year
2026: 7 · 2025: 12 · 2024: 11 · 2023: 3
FWCI
1.96
Type
journal article
Language
en
Open access
closed
Licence
Locations
  • Clinical Radiology (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Healthcare Regulation
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
artificial intelligence, diagnostic imaging, FDA clearance, clinical validation, artificial intelligence algorithms, multicentre validation, ground truth, external validation, patient demographics, machine specifications
MeSH terms
Algorithms, Artificial Intelligence, Diagnostic Imaging, Humans, Retrospective Studies, United States, United States Food and Drug Administration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review