Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
The development of computer-aided detection (CAD) using artificial intelligence (AI) and machine learning (ML) is rapidly evolving. Submission of AI/ML-based CAD devices for regulatory approval requires information about clinical trial design and performance criteria, but the requirements vary between countries. This study compares the requirements for AI/ML-based CAD devices approved by the US Food and Drug Administration (FDA) and the Pharmaceuticals and Medical Devices Agency (PMDA) in Japan. A list of 45 FDA-approved and 12 PMDA-approved AI/ML-based CAD devices was compiled. In the USA, devices classified as computer-aided simple triage were approved based on standalone software testing, whereas devices classified as computer-aided detection/diagnosis were approved based on reader study testing. In Japan, however, there was no clear distinction between evaluation methods according to the category. In the USA, a prospective randomized controlled trial was conducted for AI/ML-based CAD devices used for the detection of colorectal polyps, whereas in Japan, such devices were approved based on standalone software testing. This study indicated that the different viewpoints of AI/ML-based CAD in the two countries influenced the selection of different evaluation methods. This study’s findings may be useful for defining a unified global development and approval standard for AI/ML-based CAD.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“AI research in radiology is an active field in healthcare owing to its affinity for imaging [14], with the number of AI-related publications and medical device certifications increasing annually [15–17].”
From Fairness of artificial intelligence in healthcare: review and recommendations (2024) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lieberman, D. A., Rex, D. K., Winawer, S. J., Giardiello, F. M., Johnson, D. A., & Levin, T. R. (2012) · Gastroenterology · doi:10.1053/j.gastro.2012.06.001 · Cited by 1,892
Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0040-6 · Cited by 1,638
CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D. Y., Bagul, A., Langlotz, C. P., Shpanskaya, K. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05225 · Cited by 1,219
Benjamens, S., Dhunnoo, P., & Meskó, B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00324-0 · Cited by 1,215 · cited 2×
“The main findings are as follows: (1) In the USA, devices classified as CAST were approved based on standalone software testing, and all devices classified as CADe/CADx were approved based on reader study testing.”
BackgroundDiagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946
Muehlematter, U. J., Daniore, P., & Vokinger, K. N. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30292-2 · Cited by 665
Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial
Repici, A., Badalamenti, M., Maselli, R., Correale, L., Radaelli, F., Rondonotti, E., Ferrara, E., Spadaccini, M., Alkandari, A., Fugazza, A., et al. (2020) · Gastroenterology · doi:10.1053/j.gastro.2020.04.062 · Cited by 646
Increased Rate of Adenoma Detection Associates With Reduced Risk of Colorectal Cancer and Death
Kaminski, M. F., Wieszczy, P., Rupinski, M., Wojciechowska, U., Didkowska, J., Kraszewska, E., Kobiela, J., Franczyk, R., Rupinska, M., Kocot, B., et al. (2017) · Gastroenterology · doi:10.1053/j.gastro.2017.04.006 · Cited by 500
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · arXiv · doi:10.48550/arxiv.1901.07031 · Cited by 392 · cited 1×
“Indeed, digital databases such as the Digital Database for Screening Mammography (DDSM) 85 , or ChexPert 86 are known for their large-scale database and are frequently used in studies on image analysis algorithm development.”
BackgroundColonoscopy quality measures: experience from the NHS Bowel Cancer Screening Programme
Lee, T. J. W., Rutter, M. D., Blanks, R. G., Moss, S. M., Goddard, A. F., Chilton, A., Nickerson, C., McNally, R. J. Q., Patnick, J., & Rees, C. J. (2012) · Gut · doi:10.1136/gutjnl-2011-300651 · Cited by 324
Cited by
Fairness of artificial intelligence in healthcare: review and recommendations
Ueda, D., Kakinuma, T., Fujita, S., Kamagata, K., Fushimi, Y., Ito, R., Matsui, Y., Nozaki, T., Nakaura, T., Fujima, N., et al. (2024) · Japanese Journal of Radiology · doi:10.1007/s11604-023-01474-3 · Cited by 607
The current and future state of AI interpretation of medical images
Rajpurkar, P., & Lungren, M. P. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2301725 · Cited by 443
Artificial intelligence in emergency medicine: a primer for the nonexpert
Smith, M. E., Zalesky, C. C., Lee, S., Gottlieb, M., Adhikari, S., Goebel, M., Wegman, M., Garg, N., & Lam, S. H. F. (2025) · JACEP Open · doi:10.1016/j.acepjo.2025.100051 · Cited by 18
Metadata
- Authors
- Yuba, M., & Iwasaki, K.
- Affiliations
- Waseda University (JP)
- Corresponding author
- Iwasaki, Kiyotaka
- ORCID
- Iwasaki, Kiyotaka
- Venue
- Scientific Reports
- Published
- 2022-10-07
- Volume
- 12
- Issue
- 1
- Pages
- 16874-16874
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC9542463
- OpenAlex
- W4303432955
- Cited by
- 25
- References
- 29
- Citations by year
- 2026: 7 · 2025: 7 · 2024: 6 · 2023: 5
- FWCI
- 1.36
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Waseda University
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Healthcare Regulation
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Keywords
- computer-aided detection, regulatory approval, FDA, pyromellitic dianhydride, clinical trial design, colorectal polyp detection, cross-national comparison, evaluation criteria, international standardization
- MeSH terms
- Machine Learning, Artificial Intelligence, Pharmaceutical Preparations, Japan, Prospective Studies, Triage
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan. https://app.cochat.ai/papers/10.1038/s41598-022-21426-7
- The paper
- Yuba, M., & Iwasaki, K. (2022). Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan. Scientific Reports. https://doi.org/10.1038/s41598-022-21426-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

