Cited by 1,164Electronic Health Records SystemsArtificial Intelligence in Healthcare and EducationRadiology practices and education

Preparing a collection of radiology examinations for distribution and retrieval

Open access
Demner-Fushman, Dina, Kohli, Marc D., Rosenman, Marc B., Shooshan, Sonya E., Rodriguez, Laritza, Antani, Sameer, Thoma, George R., McDonald, Clement J.·2016·Journal of the American Medical Informatics Association

Abstract

Objective Clinical documents made available for secondary use play an increasingly important role in discovery of clinical knowledge, development of research methods, and education. An important step in facilitating secondary use of clinical document collections is easy access to descriptions and samples that represent the content of the collections. This paper presents an approach to developing a collection of radiology examinations, including both the images and radiologist narrative reports, and making them publicly available in a searchable database. Materials and Methods The authors collected 3996 radiology reports from the Indiana Network for Patient Care and 8121 associated images from the hospitals’ picture archiving systems. The images and reports were de-identified automatically and then the automatic de-identification was manually verified. The authors coded the key findings of the reports and empirically assessed the benefits of manual coding on retrieval. Results The automatic de-identification of the narrative was aggressive and achieved 100% precision at the cost of rendering a few findings uninterpretable. Automatic de-identification of images was not quite as perfect. Images for two of 3996 patients (0.05%) showed protected health information. Manual encoding of findings improved retrieval precision. Conclusion Stringent de-identification methods can remove all identifiers from text radiology reports. DICOM de-identification of images does not remove all identifying information and needs special attention to images scanned from film. Adding manual coding to the radiologist narrative reports significantly improved relevancy of the retrieved clinical documents. The de-identified Indiana chest X-ray collection is available for searching and downloading from the National Library of Medicine ( http://openi.nlm.nih.gov/ ).

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/jamia/ocv080

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Multiple benchmarks have been proposed to evaluate model summarisation capabilities in the biomedical domain, including over biomedical literature, medical forum conversations, and radiology reports.

    “Multiple benchmarks have been proposed to evaluate model summarisation capabilities in the biomedical domain, including over biomedical literature37–40, medical forum conversations41, and radiology reports22,42.”

    From A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation (2025) · cites this paper as [22]

    The Abstract describes a publicly available collection of radiology images and narrative reports, so the source supports the radiology-report part of the citation. Its Retrieval Experiments and Conclusions evaluate search and retrieval, not model summarisation; our copy does not establish that the collection was proposed as a summarisation benchmark. The indexed passages provide section labels but no page numbers. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • There is increased usage of LLMs for clinical documentation.

    “Given the increased usage of LLMs for clinical documentation22,23”

    From A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation (2025) · cites this paper as [22]

    The Abstract describes preparing radiology reports and images for public retrieval, while the Annotation of Radiology Reports section describes manual coding and use of the Medical Text Indexer (page numbers unavailable). Searches for LLMs, large language models, and their use in writing clinical documentation found no passage addressing increased LLM usage. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Demner-Fushman, D., Kohli, M. D., Rosenman, M. B., Shooshan, S. E., Rodriguez, L., Antani, S., Thoma, G. R., & McDonald, C. J.
Affiliations
  • Staff Scientist, Lister Hill National Center for Biomedical Communications National Library of Medicine, National Institutes of Health Bldg. 38A, Room 10S-1022, 8600 Rockville Pike MSC-3824 Bethesda, MD 20894, USA
  • Indiana University Indianapolis (US)
  • National Institutes of Health (US)
Venue
Journal of the American Medical Informatics Association
Published
2015-07-01
Volume
23
Issue
2
Pages
304-310
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
OpenAlex
W2152772232
Cited by
1,164
References
22
Citations by year
2026: 191 · 2025: 264 · 2024: 194 · 2023: 154 · 2022: 98 · 2021: 97 · 2020: 68 · 2019: 58 · 2018: 23 · 2017: 12 · 2016: 4 · 2015: 1
FWCI
25.70
Type
journal article
Language
en
Open access
Licence
public-domain
Locations
  • Journal of the American Medical Informatics Association (publishedVersion, public-domain)
  • PubMed Central (publishedVersion, public-domain)
  • PubMed (publishedVersion)
  • HighWire Press Open Archive (submittedVersion)
  • IUScholarWorks (Indiana University) (submittedVersion)
  • Purdue University Indianapolis (Indiana University) (submittedVersion)
Funders
  • National Institutes of Health
  • U.S. National Library of Medicine
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
Keywords
radiology reports, de-identification, protected health information, PACS
MeSH terms
Data Anonymization, Humans, Radiology Information Systems, Radiography, Thoracic, Information Storage and Retrieval

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review