CoChat
Sign Up
Cited by 368Artificial Intelligence in Healthcare and EducationRadiomics and Machine Learning in Medical ImagingMedical Image Segmentation Techniques

A Roadmap for Foundational Research on Artificial Intelligence in Medical Imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop

Open access
Langlotz, Curtis P., Allen, Bibb, Erickson, Bradley J., Kalpathy-Cramer, Jayashree, Bigelow, Keith, Cook, Tessa S., Flanders, Adam E., Lungren, Matthew P. +4 more·2019·Radiology
DOI 10.1148/radiol.2019190613PMC PMC6542624
Open this paper in CoChat — free account

Abstract

Imaging research laboratories are rapidly creating machine learning systems that achieve expert human performance using open-source methods and tools. These artificial intelligence systems are being developed to improve medical image reconstruction, noise reduction, quality assurance, triage, segmentation, computer-aided detection, computer-aided classification, and radiogenomics. In August 2018, a meeting was held in Bethesda, Maryland, at the National Institutes of Health to discuss the current state of the art and knowledge gaps and to develop a roadmap for future research initiatives. Key research priorities include: 1, new image reconstruction methods that efficiently produce images suitable for human interpretation from source data; 2, automated image labeling and annotation methods, including information extraction from the imaging report, electronic phenotyping, and prospective structured image reporting; 3, new machine learning methods for clinical imaging data, such as tailored, pretrained model architectures, and federated machine learning methods; 4, machine learning methods that can explain the advice they provide to human users (so-called explainable artificial intelligence); and 5, validated methods for image de-identification and data sharing to facilitate wide availability of clinical imaging data sets. This research roadmap is intended to identify and prioritize these needs for academic research laboratories, funding agencies, professional societies, and industry.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1148/radiol.2019190613

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Interobserver and intraobserver variability of CT image interpretations rendered by radiologists can be as high as 37%.

    “The problem is exacerbated by issues related to interobserver and intraobserver variability of CT image interpretations rendered by radiologists, which can be as high as 37%.206”

    From Artificial Intelligence and Early Detection of Pancreatic Cancer: 2020 Summative Review (2021) · cites this paper as [206]

    The Introduction states that inter- and intraobserver variability can reach 37%, depending on imaging modality. It does not establish that this rate applies to CT interpretations specifically; the cited reference (17) concerns mammograms. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • The drop in ImageNet error rate below 5% was largely due to advances in deep learning.

    “This was largely due to advances in deep learning, the branch of AI that learns from large amounts of data.”

    From Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology (2020) · cites this paper as [17]

  • In 2015, the ImageNet annual competition reached a milestone when the error rate of automatic classification of images dropped below 5%, which is the average human error rate.

    “In 2015, the ImageNet annual competition reached a milestone when the error rate of automatic classification of images dropped below 5%, which is the average human error rate (Figure S1) 17 .”

    From Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology (2020) · cites this paper as [17]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 15

  1. [11]Liu Y, Gadepalli K, Norouzi M, et al. Detecting Cancer Metastases on Gigapixel Pathology Images. http://arxiv.org/abs/1703.02442. Published 2017. · arXiv
  2. [13]National Academy of Medicine . Improving Diagnosis in Health Care. Washington, DC: The National Academies Press, 2015. [PubMed] [Google Scholar]
  3. [28]Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM. ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases. http://arxiv.org/abs/1705.02315. Published 2017. Accessed DATE. · arXiv
  4. [30]Rajpurkar P, Irvin J, Bagul A, et al. MURA: Large Dataset for Abnormality Detection in Musculoskeletal Radiographs. arXiv [physics.med-ph]. http://arxiv.org/abs/1712.06957. Published 2017. · arXiv
  5. [31]Zbontar J, Knoll F, Sriram A, et al. fastMRI: An Open Dataset and Benchmarks for Accelerated MRI. arXiv [cs.CV]. http://arxiv.org/abs/1811.08839. Published 2018. · arXiv
  6. [32]Irvin J, Rajpurkar P, Ko M, et al. CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. arXiv [cs.CV]. http://arxiv.org/abs/1901.07031. Published 2019. · arXiv
  7. [48]Rolnick D, Veit A, Belongie S, Shavit N. Deep Learning is Robust to Massive Label Noise. arXiv [cs.LG]. http://arxiv.org/abs/1705.10694. Published 2017. · arXiv
  8. [75]Ratner AJ, Bach SH, Ehrenberg HR, Ré C. Snorkel: Fast Training Set Generation for Information Extraction. Proceedings of the 2017 ACM International Conference on Management of Data. New York, NY: ACM, 2017; 1683–1686. [Google Scholar]
  9. [77]Lipton ZC, Berkowitz J, Elkan C. A Critical Review of Recurrent Neural Networks for Sequence Learning. http://arxiv.org/abs/1506.00019. Published 2015. · arXiv
  10. [79]Mikolov T, Chen K, Corrado G, Dean J. Efficient Estimation of Word Representations in Vector Space. http://arxiv.org/abs/1301.3781. Published 2013. · arXiv
  11. [81]Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C. Neural Architectures for Named Entity Recognition. http://arxiv.org/abs/1603.01360. Published 2016. · arXiv
  12. [82]Dernoncourt F, Lee JY, Szolovits P. NeuroNER: an easy-to-use program for named-entity recognition based on neural networks. http://arxiv.org/abs/1705.05487. Published 2017. · arXiv
  13. [86]Kingma DP, Rezende DJ, Mohamed S, Welling M. Semi-Supervised Learning with Deep Generative Models. arXiv [cs.LG]. http://arxiv.org/abs/1406.5298. Published 2014. · arXiv
  14. [87]Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G. Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery. arXiv [cs.CV]. http://arxiv.org/abs/1703.05921. Published 2017. [DOI] [PubMed] · arXiv
  15. [99]Finlayson SG, Chung HW, Kohane IS, Beam AL. Adversarial Attacks Against Medical Deep Learning Systems. arXiv [cs.CR]. http://arxiv.org/abs/1804.05296. Published 2018. · arXiv

Cited by

Sorted by citations

Related

Authors
Langlotz, C. P., Allen, B., Erickson, B. J., Kalpathy-Cramer, J., Bigelow, K., Cook, T. S., Flanders, A. E., Lungren, M. P., Mendelson, D. S., Rudie, J. D., Wang, G., & Kandarpa, K.
Affiliations
  • Mayo Clinic (US)
  • National Institutes of Health (US)
ORCID
Venue
Radiology
Publisher
Radiological Society of North America (RSNA)
Published
2019-06
Volume
291
Issue
3
Pages
781-791
ISSN
  • 0033-8419 (ISSN-L)
  • 1527-1315
DOI
10.1148/radiol.2019190613
PMC
PMC6542624
OpenAlex
W2939071985
Cited by
368
References
85
Citations by year
2026: 34 · 2025: 33 · 2024: 45 · 2023: 57 · 2022: 56 · 2021: 68 · 2020: 58 · 2019: 17
FWCI
15.77
Type
journal article
Language
en
Open access
Locations
  • Radiology (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
National Institute of Biomedical Imaging and Bioengineering
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
Keywords
artificial intelligence in medical imaging, research priorities, image reconstruction, image segmentation, computer-aided detection, computer-aided classification, radiogenomics, automatic image annotation, electronic phenotyping, federated learning, pre-trained models, explainable AI
MeSH terms
Machine Learning, Algorithms, Artificial Intelligence, Diagnostic Imaging, Humans, Image Interpretation, Computer-Assisted, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review