CoChat

Automated detection of moderate and large pneumothorax on frontal chest X-rays using deep convolutional neural networks: A retrospective study

Open access

The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Pneumothorax can precipitate a life-threatening emergency due to lung collapse and respiratory or circulatory distress. Pneumothorax is typically detected on chest X-ray; however, treatment is reliant on timely review of radiographs. Since current imaging volumes may result in long worklists of radiographs awaiting review, an automated method of prioritizing X-rays with pneumothorax may reduce time to treatment. Our objective was to create a large human-annotated dataset of chest X-rays containing pneumothorax and to train deep convolutional networks to screen for potentially emergent moderate or large pneumothorax at the time of image acquisition. METHODS AND FINDINGS: In all, 13,292 frontal chest X-rays (3,107 with pneumothorax) were visually annotated by radiologists. This dataset was used to train and evaluate multiple network architectures. Images showing large- or moderate-sized pneumothorax were considered positive, and those with trace or no pneumothorax were considered negative. Images showing small pneumothorax were excluded from training. Using an internal validation set (n = 1,993), we selected the 2 top-performing models; these models were then evaluated on a held-out internal test set based on area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and positive predictive value (PPV). The final internal test was performed initially on a subset with small pneumothorax excluded (as in training; n = 1,701), then on the full test set (n = 1,990), with small pneumothorax included as positive. External evaluation was performed using the National Institutes of Health (NIH) ChestX-ray14 set, a public dataset labeled for chest pathology based on text reports. All images labeled with pneumothorax were considered positive, because the NIH set does not classify pneumothorax by size. In internal testing, our "high sensitivity model" produced a sensitivity of 0.84 (95% CI 0.78-0.90), specificity of 0.90 (95% CI 0.89-0.92), and AUC of 0.94 for the test subset with small pneumothorax excluded. Our "high specificity model" showed sensitivity of 0.80 (95% CI 0.72-0.86), specificity of 0.97 (95% CI 0.96-0.98), and AUC of 0.96 for this set. PPVs were 0.45 (95% CI 0.39-0.51) and 0.71 (95% CI 0.63-0.77), respectively. Internal testing on the full set showed expected decreased performance (sensitivity 0.55, specificity 0.90, and AUC 0.82 for high sensitivity model and sensitivity 0.45, specificity 0.97, and AUC 0.86 for high specificity model). External testing using the NIH dataset showed some further performance decline (sensitivity 0.28-0.49, specificity 0.85-0.97, and AUC 0.75 for both). Due to labeling differences between internal and external datasets, these findings represent a preliminary step towards external validation. CONCLUSIONS: We trained automated classifiers to detect moderate and large pneumothorax in frontal chest X-rays at high levels of performance on held-out test data. These models may provide a high specificity screening solution to detect moderate or large pneumothorax on images collected when human review might be delayed, such as overnight. They are not intended for unsupervised diagnosis of all pneumothoraces, as many small pneumothoraces (and some larger ones) are not detected by the algorithm. Implementation studies are warranted to develop appropriate, effective clinician alerts for the potentially critical finding of pneumothorax, and to assess their impact on reducing time to treatment.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pmed.1002697

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 Diagnostic performance ROC curve and confusion matrix of most sensitive top model evaluated on test set (small pneumothoraces excluded). Mod, moderate; PTX, pneumothorax; ROC, receiver operating characteristic.
Fig 1 Diagnostic performance ROC curve and confusion matrix of most sensitive top model evaluated on test set (small pneumothoraces excluded). Mod, moderate; PTX, pneumothorax; ROC, receiver operating characteristic.CC BY · PubMed Central
Fig 2 Diagnostic performance ROC curve and confusion matrix of most specific top model evaluated on test set (small pneumothoraces excluded). Mod, moderate; PTX, pneumothorax; ROC, receiver operating characteristic.
Fig 2 Diagnostic performance ROC curve and confusion matrix of most specific top model evaluated on test set (small pneumothoraces excluded). Mod, moderate; PTX, pneumothorax; ROC, receiver operating characteristic.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 30 references

Research works printed but not linked: 1

  1. L Oakden-Rayner, 2017, Exploring the ChestXray14 dataset: problems, Exploring the ChestXray14 dataset: problems

Cited by

See all 216

Metadata

Authors
Taylor, A. G., Mielke, C., & Mongan, J.
Affiliations
University of California, San Francisco (US)
Corresponding author
Taylor, Andrew G.
Venue
PLOS Medicine
Published
2018-11-20
Volume
15
Issue
11
Pages
e1002697
ISSN
  • 1549-1277 (ISSN-L)
  • 1549-1676
OpenAlex
W2901794879
Cited by
212
References
28
Citations by year
2026: 10 · 2025: 12 · 2024: 25 · 2023: 26 · 2022: 32 · 2021: 53 · 2020: 40 · 2019: 11 · 2018: 2
FWCI
18.28
Type
journal article
Language
en
Open access
Locations
  • PLoS Medicine (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • eScholarship (California Digital Library) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
Funders
General Electric (A128218)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
pneumothorax, automated detection, chest radiography, deep convolutional neural networks, emergency department triage, NIH ChestX-ray14, model comparison, sensitivity and specificity trade-off, radiology workflow efficiency, atelectasis
MeSH terms
Deep Learning, Automation, Diagnosis, Computer-Assisted, Humans, Pneumothorax, Predictive Value of Tests, Prognosis, Radiographic Image Interpretation, Computer-Assisted, Retrospective Studies, Radiography, Thoracic, Reproducibility of Results, Databases, Factual

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Automated detection of moderate and large pneumothorax on frontal chest X-rays using deep convolutional neural networks: A retrospective study. https://app.cochat.ai/papers/10.1371/journal.pmed.1002697
The paper
Taylor, A. G., Mielke, C., & Mongan, J. (2018). Automated detection of moderate and large pneumothorax on frontal chest X-rays using deep convolutional neural networks: A retrospective study. PLOS Medicine. https://doi.org/10.1371/journal.pmed.1002697

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.