Automated radiology report summarization using an open-source natural language processing pipeline
Open accessAbstract
Diagnostic radiologists are expected to review and assimilate findings from prior studies when constructing their overall assessment of the current study. Radiology information systems facilitate this process by presenting the radiologist with a subset of prior studies that are more likely to be relevant to the current study, usually by comparing anatomic coverage of both the current and prior studies. It is incumbent on the radiologist to review the full text report and/or images from those prior studies, a process that is time-consuming and confers substantial risk of overlooking a relevant prior study or finding. This risk is compounded when patients have dozens or even hundreds of prior imaging studies. Our goal is to assess the feasibility of natural language processing techniques to automatically extract asserted and negated disease entities from free-text radiology reports as a step towards automated report summarization. We compared automatically extracted disease mentions to a gold-standard set of manual annotations for 50 radiology reports from CT abdomen and pelvis examinations. The automated report summarization pipeline found perfect or overlapping partial matches for 86% of the manually annotated disease mentions (sensitivity 0.86, precision 0.66, accuracy 0.59, F1 score 0.74). The performance of the automated pipeline was good, and the overall accuracy was similar to the interobserver agreement between the two manual annotators.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Deep learning methods have been used for automated generation of radiology reports, typically incorporating LSTM network models to generate textual paragraphs, and to identify findings in radiology reports.
“Recently, deep learning methods have also been used for automated generation of radiology reports , typically incorporating long-short-term-memory (LSTM) network models to generate the textual paragraphs [314, 315, 316, 317], and also to identify findings in radiology reports [318, 319, 320].”
From An overview of deep learning in medical imaging focusing on MRI (2019) · cites this paper as [320]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967 · see citing works
brat: a Web-based Tool for NLP-Assisted Text Annotation
Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., & Tsujii, J. (2012) · Research Explorer (The University of Manchester) · Cited by 961 · see citing works
Natural Language Processing Technologies in Radiology Research and Clinical Applications
Cai, T. A., Giannopoulos, A. A., Yu, S., Kelil, T., Ripley, B. A., Kumamaru, K. K., Rybicki, F. J., & Mitsouras, D. (2016) · Radiographics · doi:10.1148/rg.2016150080 · Cited by 214 · see citing works
Information extraction from multi-institutional radiology reports
Hassanpour, S., & Langlotz, C. P. (2016) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2015.09.007 · Cited by 182 · see citing works
Ni, Y., Wright, J., Perentesis, J., Lingren, T., Deleger, L., Kaiser, M., Kohane, I., & Solti, I. (2015) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-015-0149-3 · Cited by 142 · see citing works
Towards comprehensive syntactic and semantic annotations of the clinical narrative
Albright, D., Lanfranchi, A., Fredriksen, A., Styler, W. F., Warner, C., Hwang, J. D., Choi, J. D., Dligach, D., Nielsen, R. D., Martin, J., et al. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001317 · Cited by 141 · see citing works
DEEPEN: A negation detection system for clinical text incorporating dependency relation into NegEx
Mehrabi, S., Krishnan, A., Sohn, S., Roch, A. M., Schmidt, H., Kesterson, J., Beesley, C., Dexter, P. R., Schmidt, C. M., Liu, H., et al. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.02.010 · Cited by 132 · see citing works
Pham, A.-D., Névéol, A., Lavergne, T., et al. (2014) · BMC Bioinformatics · doi:10.1186/1471-2105-15-266 · Cited by 110 · see citing works
10th Conference of the European Chapter of the Association for Computational Linguistics
Core, M. G., Moore, J. C., & Zinn, C. (2003) · The Association for Computational Linguistics · Cited by 107 · see citing works
Wu, S., Sohn, S., Ravikumar, K. E., Wagholikar, K. B., Jonnalagadda, S., Liu, H., & Juhn, Y. J. (2013) · Annals of Allergy Asthma & Immunology · doi:10.1016/j.anai.2013.07.022 · Cited by 79 · see citing works
A system for coreference resolution for the clinical narrative
Zheng, J., Chapman, W. W., Miller, T. A., Lin, C., Crowley, R. S., & Savova, G. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000599 · Cited by 38 · see citing works
Automatic abstraction of imaging observations with their characteristics from mammography reports
Bozkurt, S., Lipson, J. A., Şenol, U., & Rubin, D. L. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-003009 · Cited by 36 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.11.002 · Cited by 2,058 · see citing works
Learning to Summarize Radiology Findings
Zhang, Y., Ding, D. Y., Qian, T., Manning, C. D., & Langlotz, C. P. (2018) · Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis · doi:10.18653/v1/w18-5623 · Cited by 126 · see citing works
Li, H., Moon, J. T., Iyer, D., Balthazar, P., Krupinski, E. A., Bercu, Z. L., Newsome, J. M., Banerjee, I., Gichoya, J. W., & Trivedi, H. M. (2023) · Clinical Imaging · doi:10.1016/j.clinimag.2023.06.008 · Cited by 118 · see citing works
Pierre, K., Haneberg, A. G., Kwak, S., Peters, K. R., Hochhegger, B., Sananmuang, T., Tunlayadechanont, P., Tighe, P. J., Mancuso, A., & Forghani, R. (2023) · Seminars in Roentgenology · doi:10.1053/j.ro.2023.02.003 · Cited by 79 · see citing works
- Authors
- Goff, D. J., & Loehfelm, T. W.
- Affiliations
- University of California Davis Medical Center (US)
- Corresponding author
- Loehfelm, Thomas W.
- Venue
- Journal of Digital Imaging
- Published
- 2017-10-30
- Volume
- 31
- Issue
- 2
- Pages
- 185-192
- ISSN
- 0897-1889 (ISSN-L)
- 1618-727X
- PMC
- PMC5873469
- OpenAlex
- W2765729115
- Cited by
- 43
- References
- 12
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 3 · 2023: 8 · 2022: 5 · 2021: 5 · 2020: 8 · 2019: 3 · 2018: 5
- FWCI
- 1.79
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Imaging Informatics in Medicine (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Keywords
- radiology report summarization, natural language processing pipeline, radiology information systems, interobserver agreement
- MeSH terms
- Abdomen, Adolescent, Adult, Aged, Aged, 80 and over, Child, Child, Preschool, Female, Humans, Male, Middle Aged, Natural Language Processing, Pelvis, Radiography, Abdominal, Radiology Information Systems, Retrospective Studies, Tomography, X-Ray Computed, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
