The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Deep Convolutional Neural Network–based Software Improves Radiologist Detection of Malignant Lung Nodules on Chest Radiographs
Open accessAbstract
Background Multicenter studies are required to validate the added benefit of using deep convolutional neural network (DCNN) software for detecting malignant pulmonary nodules on chest radiographs. Purpose To compare the performance of radiologists in detecting malignant pulmonary nodules on chest radiographs when assisted by deep learning–based DCNN software with that of radiologists or DCNN software alone in a multicenter setting. Materials and Methods Investigators at four medical centers retrospectively identified 600 lung cancer–containing chest radiographs and 200 normal chest radiographs. Each radiograph with a lung cancer had at least one malignant nodule confirmed by CT and pathologic examination. Twelve radiologists from the four centers independently analyzed the chest radiographs and marked regions of interest. Commercially available deep learning–based computer-aided detection software separately trained, tested, and validated with 19 330 radiographs was used to find suspicious nodules. The radiologists then reviewed the images with the assistance of DCNN software. The sensitivity and number of false-positive findings per image of DCNN software, radiologists alone, and radiologists with the use of DCNN software were analyzed by using logistic regression and Poisson regression. Results The average sensitivity of radiologists improved (from 65.1% [1375 of 2112; 95% confidence interval {CI}: 62.0%, 68.1%] to 70.3% [1484 of 2112; 95% CI: 67.2%, 73.1%], P < .001) and the number of false-positive findings per radiograph declined (from 0.2 [488 of 2400; 95% CI: 0.18, 0.22] to 0.18 [422 of 2400; 95% CI: 0.16, 0.2], P < .001) when the radiologists re-reviewed radiographs with the DCNN software. For the 12 radiologists in this study, 104 of 2400 radiographs were positively changed (from false-negative to true-positive or from false-positive to true-negative) using the DCNN, while 56 of 2400 radiographs were changed negatively. Conclusion Radiologists had better performance with deep convolutional network software for the detection of malignant pulmonary nodules on chest radiographs than without. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Jacobson in this issue.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“13 (refs. 63–75) studies validated algorithms on external data”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [72]
In the citing paper
“Twenty-one54,63–67,70,72,76–88 studies compared algorithm performance against healthcare professionals.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [72]
In the citing paper
“Only eight studies in ophthalmology imaging14,21,32,33,43,55,108,109, ten studies in respiratory imaging64,66,70,72,75,79,82,87,89,110 and six studies in breast imaging62,91,97,104,106,111 mentioned adherence to the STARD-2015 guidelines or had a STARD flow diagram in the manuscript.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [72]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening
The National Lung Screening Trial Research Team (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1102873 · Cited by 11,064
Lakhani, P., & Sundaram, B. (2017) · Radiology · doi:10.1148/radiol.2017162326 · Cited by 1,775
Park, S. H., & Han, K. (2018) · Radiology · doi:10.1148/radiol.2017171920 · Cited by 894
Self-citationNam, J. G., Park, S., Hwang, E. J., Lee, J. H., Jin, K.-N., Lim, K. Y., Vu, T. H., Sohn, J. H., Hwang, S., Goo, J. M., et al. (2019) · Radiology · doi:10.1148/radiol.2018180237 · Cited by 539
Austin, J. H., Romney, B. M., & Goldsmith, L. S. (1992) · Radiology · doi:10.1148/radiology.182.1.1727272 · Cited by 266
Shah, P. K., Austin, J. H. M., White, C. S., Patel, P., Haramati, L. B., Pearson, G. D. N., Shiau, M. C., & Berkmen, Y. M. (2003) · Radiology · doi:10.1148/radiol.2261011924 · Cited by 198
Missed lung cancer: when, where, and why?
Ciello, A. D., Franchi, P., Contegiacomo, A., Cicchetti, G., Bonomo, L., & Larici, A. R. (2017) · Diagnostic and Interventional Radiology · doi:10.5152/dir.2016.16187 · Cited by 187
Mechanism of satisfaction of search: eye position recordings in the reading of chest radiographs.
Samuel, S., Kundel, H. L., Nodine, C. F., & Toto, L. C. (1995) · Radiology · doi:10.1148/radiology.194.3.7862998 · Cited by 168
Kakeda, S., Moriya, J., Sato, H., Aoki, T., Watanabe, H., Nakata, H., Oda, N., Katsuragawa, S., Yamamoto, K., & Doi, K. (2004) · American Journal of Roentgenology · doi:10.2214/ajr.182.2.1820505 · Cited by 139
Research works printed but not linked: 1
- Hwang S, 2016, Proceedings of SPIE: medical imaging 2016—computer-aided diagnosis. Vol 9785, 97852W
Cited by
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212
“For example, Sim et al. found that AI-assisted clinical experts surpassed both humans and AI alone when detecting malignant nodules on chest radiographs 85 .”
BackgroundDiagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946
Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021) · Nature Medicine · doi:10.1038/s41591-021-01595-0 · Cited by 928
Deep learning for chest X-ray analysis: A survey
Çallı, E., Sogancioglu, E., van Ginneken, B., van Leeuwen, K. G., & Murphy, K. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102125 · Cited by 506
Fu, T., Dai, L.-J., Wu, S.-Y., Xiao, Y., Ma, D., Jiang, Y.-Z., & Shao, Z.-M. (2021) · Journal of Hematology & Oncology · doi:10.1186/s13045-021-01103-4 · Cited by 480
Metadata
- Authors
- Sim, Y., Chung, M. J., Kotter, E., Yune, S., Kim, M., Do, S., Han, K., Kim, H., Yang, S., Lee, D.-J., & Choi, B. W.
- Affiliations
- Yonsei University (KR)
- Harvard University (US)
- ORCID
- Venue
- Radiology
- Published
- 2020-01
- Volume
- 294
- Issue
- 1
- Pages
- 199-209
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- OpenAlex
- W2984645840
- Cited by
- 248
- References
- 26
- Citations by year
- 2026: 10 · 2025: 32 · 2024: 39 · 2023: 55 · 2022: 39 · 2021: 60 · 2020: 12
- FWCI
- 12.78
- Type
- journal article
- Language
- en
- Open access
- Licence
- cc-by-nc-nd
- Locations
- Radiology (publishedVersion)
- PubMed (publishedVersion)
- YUHSpace (Yonsei University Medical Library) (submittedVersion, cc-by-nc-nd)
- Funders
- Samsung Electronics
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Lung Cancer Diagnosis and Treatment
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- deep convolutional neural networks, computer-aided detection, lung cancer, chest radiograph, radiologist performance, false positives, nodule detection, diagnostic accuracy, multicenter validation
- MeSH terms
- Adult, Aged, Solitary Pulmonary Nodule, Female, Humans, Lung, Lung Neoplasms, Male, Middle Aged, Radiographic Image Interpretation, Computer-Assisted, Retrospective Studies, Sensitivity and Specificity, Radiography, Thoracic, Reproducibility of Results, Neural Networks, Computer, Multiple Pulmonary Nodules, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep Convolutional Neural Network–based Software Improves Radiologist Detection of Malignant Lung Nodules on Chest Radiographs. https://app.cochat.ai/papers/10.1148/radiol.2019182465
- The paper
- Sim, Y., Chung, M. J., Kotter, E., Yune, S., Kim, M., Do, S., Han, K., Kim, H., Yang, S., Lee, D.-J., & Choi, B. W. (2020). Deep Convolutional Neural Network–based Software Improves Radiologist Detection of Malignant Lung Nodules on Chest Radiographs. Radiology. https://doi.org/10.1148/radiol.2019182465
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.