Deep Convolutional Neural Network–based Software Improves Radiologist Detection of Malignant Lung Nodules on Chest Radiographs
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Background Multicenter studies are required to validate the added benefit of using deep convolutional neural network (DCNN) software for detecting malignant pulmonary nodules on chest radiographs. Purpose To compare the performance of radiologists in detecting malignant pulmonary nodules on chest radiographs when assisted by deep learning–based DCNN software with that of radiologists or DCNN software alone in a multicenter setting. Materials and Methods Investigators at four medical centers retrospectively identified 600 lung cancer–containing chest radiographs and 200 normal chest radiographs. Each radiograph with a lung cancer had at least one malignant nodule confirmed by CT and pathologic examination. Twelve radiologists from the four centers independently analyzed the chest radiographs and marked regions of interest. Commercially available deep learning–based computer-aided detection software separately trained, tested, and validated with 19 330 radiographs was used to find suspicious nodules. The radiologists then reviewed the images with the assistance of DCNN software. The sensitivity and number of false-positive findings per image of DCNN software, radiologists alone, and radiologists with the use of DCNN software were analyzed by using logistic regression and Poisson regression. Results The average sensitivity of radiologists improved (from 65.1% [1375 of 2112; 95% confidence interval {CI}: 62.0%, 68.1%] to 70.3% [1484 of 2112; 95% CI: 67.2%, 73.1%], P < .001) and the number of false-positive findings per radiograph declined (from 0.2 [488 of 2400; 95% CI: 0.18, 0.22] to 0.18 [422 of 2400; 95% CI: 0.16, 0.2], P < .001) when the radiologists re-reviewed radiographs with the DCNN software. For the 12 radiologists in this study, 104 of 2400 radiographs were positively changed (from false-negative to true-positive or from false-positive to true-negative) using the DCNN, while 56 of 2400 radiographs were changed negatively. Conclusion Radiologists had better performance with deep convolutional network software for the detection of malignant pulmonary nodules on chest radiographs than without. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Jacobson in this issue.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“13 (refs. 63–75) studies validated algorithms on external data”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [72]
In the citing paper
“Twenty-one54,63–67,70,72,76–88 studies compared algorithm performance against healthcare professionals.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [72]
In the citing paper
“Only eight studies in ophthalmology imaging14,21,32,33,43,55,108,109, ten studies in respiratory imaging64,66,70,72,75,79,82,87,89,110 and six studies in breast imaging62,91,97,104,106,111 mentioned adherence to the STARD-2015 guidelines or had a STARD flow diagram in the manuscript.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [72]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening
The National Lung Screening Trial Research Team (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1102873 · Cited by 11,064
Lakhani, P., & Sundaram, B. (2017) · Radiology · doi:10.1148/radiol.2017162326 · Cited by 1,775
Park, S. H., & Han, K. (2018) · Radiology · doi:10.1148/radiol.2017171920 · Cited by 894
Self-citationNam, J. G., Park, S., Hwang, E. J., Lee, J. H., Jin, K.-N., Lim, K. Y., Vu, T. H., Sohn, J. H., Hwang, S., Goo, J. M., et al. (2019) · Radiology · doi:10.1148/radiol.2018180237 · Cited by 539
Austin, J. H., Romney, B. M., & Goldsmith, L. S. (1992) · Radiology · doi:10.1148/radiology.182.1.1727272 · Cited by 266
Shah, P. K., Austin, J. H. M., White, C. S., Patel, P., Haramati, L. B., Pearson, G. D. N., Shiau, M. C., & Berkmen, Y. M. (2003) · Radiology · doi:10.1148/radiol.2261011924 · Cited by 198
Missed lung cancer: when, where, and why?
Ciello, A. D., Franchi, P., Contegiacomo, A., Cicchetti, G., Bonomo, L., & Larici, A. R. (2017) · Diagnostic and Interventional Radiology · doi:10.5152/dir.2016.16187 · Cited by 187
Mechanism of satisfaction of search: eye position recordings in the reading of chest radiographs.
Samuel, S., Kundel, H. L., Nodine, C. F., & Toto, L. C. (1995) · Radiology · doi:10.1148/radiology.194.3.7862998 · Cited by 168
Kakeda, S., Moriya, J., Sato, H., Aoki, T., Watanabe, H., Nakata, H., Oda, N., Katsuragawa, S., Yamamoto, K., & Doi, K. (2004) · American Journal of Roentgenology · doi:10.2214/ajr.182.2.1820505 · Cited by 141
Research works printed but not linked: 1
- Hwang S, 2016, Proceedings of SPIE: medical imaging 2016—computer-aided diagnosis. Vol 9785, 97852W
Cited by
Items 201–248 of 248
Pushing the Limits of an FCN and A CRF Towards Near-Ideal Vertebrae Labelling
Sekuboyina, A., Irmai, J., Shit, S., Kirschke, J. S., Andres, B., & Menze, B. H. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230520 · Cited by 2
“Benefits of computer-aided diagnosis have been demonstrated in multiple instances [1, 2, 3].”
BackgroundAn Artificial Intelligence Copilot System Helps Reduce Mortality Rates of Hemodialysis PatientsPreprint
Chen, J., Lou, X., Qian, S., Chen, J., Liu, B., Zhang, P., Chen, X., Yao, X., Wang, Y., Gai, P., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-3256363/v1 · Cited by 3
Improving Lung Cancer Detection Using Hybrid Features and Optimized through VGG‐16
Ritu, A., & Eshaan, A. (2025) · Wiley eBooks · doi:10.1002/9781394355358.ch16 · Cited by 1
Yoo, H., Kim, E. Y., Kim, H. J., Choi, Y. R., Cho, Y. J., & Jin, K. N. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3742292
Self-citationOliveira, B. A. S., Ziviani, H. E., Oliveira, J. J., Viegas, A. M., & Calvo, D. (2021) · Congresso Brasileiro de Inteligência Computacional (CBIC) · doi:10.21528/cbic2021-14 · Cited by 1
Research on Online Social Network Information Leakage-Tracking Algorithm Based on Deep Learning
Han, S. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/1926794
“+eir corresponding mechanisms are mainly the confirmation mechanism based on the user’s secret information, on the user’s reliable object, on the user’s key, and on the user’s biological characteristics [9, 10].”
BackgroundDeep Feature Selection for Benign and Malignant Classification appearing as Ground Glass Nodules
Ma, C., Yue, S., & Li, K. (2022) · IEEE International Instrumentation and Measurement Technology Conference · doi:10.1109/i2mtc48687.2022.9806673
“These studies are usually divided into two types, one is traditional machine learning, and the other is deep learning [6].”
BackgroundEvaluation of Lung Cancer Incidence Dynamics in Kyrgyzstan
Bazhanova, A., Akhunbaev, S., Soodonbekov, E., Nurtazinova, G., Telmanova, Z., & Igissinov, N. (2022) · Open Access Macedonian Journal of Medical Sciences · doi:10.3889/oamjms.2022.9965
Application of Deep Learning in Radiation Therapy
Rawat, S., Singh, S., Alam, M. A., & Malviya, R. (2022) · Wiley eBooks · doi:10.1002/9781119857983.ch10
Systematic Review of Functional Pathways and Methods for COVID-19 Modelling
Vaišnorė, R., Mazur, G., Mikštienė, V., & Jakaitienė, A. (2022) · Baltic Journal of Modern Computing · doi:10.22364/bjmc.2022.10.4.01
Han, F., Qiu, P., & Feng, K. (2023) · Contrast Media & Molecular Imaging · doi:10.1155/2023/9303688
Yang, Q., Wang, Z., Fu, Q., Hu, X., Chen, L., Chen, W., Lv, L., Liu, Z., Men, W., & Li, W. (2023) · doi:10.2196/preprints.49112
Liu, Y., Feng, Z., Fan, Z., Zhang, Y., Li, C., Liu, X., Duan, H., Cui, X., Zhang, L., Sheng, C., et al. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1203320
ACCURACY OF PATIENTS AGE ASSESSMENT FROM FRONTAL CHEST RADIOGRAPHS
Hameed, M., Mahar, S., B.Das, Khumaini, A. R., Ullah, A., & M.Jamil (2023) · Pakistan Journal of Science · doi:10.57041/pjs.v75i1.819
Artificial Intelligence-Driven Lung Cancer Detection Model Using Chest Radiography ImagesPreprint
Malebary, S. J., Abulfaraj, A. W., Dutta, A. K., & Sait, A. R. W. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4557886
Global research of artificial intelligence in lung Cancer: A bibliometric and visualized studyPreprint
Yang, Q., Wang, Z., Lv, L., Liu, Z., Men, W., & Li, W. (2023) · Research Square · doi:10.21203/rs.3.rs-3495027/v1
Artificial Intelligence in Imaging Diagnosis of Lung Cancer
Nomura, Y. (2023) · Haigan · doi:10.2482/haigan.63.829
Guo, L. J., Cheng, G., Wang, L., Zheng, B., Jaeger, S., Giger, M. L., Fuhrman, J. D., Li, H., Divekar, A., Xiao, Q., et al. (2024) · SPIE Medical Imaging: Computer-Aided Diagnosis · doi:10.1117/12.3005136
“Though a few prior studies evaluating the reading time on either the chest radiography CAD/AI tool 1, 18, 19 or the reduction in radiologists' reading time with the aid of CAD 6 , however, none of study investigated both the CXR image interpretation and report generation time reduction for the…”
ResultWolf, A. L., Hennrich, J., & Buck, C. (2024) · Springer eBooks · doi:10.1007/978-3-658-43860-9_34 · Cited by 1
Occlusive target recognition method of sorting robot based on anchor-free detection network
Wang, F., & Song, C. (2024) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2024.2383647
“Therefore, to achieve effective automatic sorting of sorting robots, a recognition and positioning algorithm based on a Deep Belief Network (DBN) is proposed (Sim et al., 2020; Rajaraman and Antani, 2020).”
Saravanan, B., Gokul, S., Reddy, G. Y. D., Kumar, A., & Manickam, R. K. (2024) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs60874.2024.10717139
Ajeeshkumar, K. K., Aneesh, P. A., Sahu, A., Peerzada, M. N., & Verma, S. M. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-27574-6.00004-7
Hanaoka, S., Nomura, Y., Hayashi, N., Sato, I., Miki, S., Yoshikawa, T., Shibata, H., Nakao, T., Takenaga, T., Koyama, H., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0315646
Hybrid Deep Learning Model for Prostate Cancer Detection Using LeNet and MobileNet V2 Algorithms
Fatima, S., Ramya, I. M., Ridhi Mayura Shivani, R., Rajesh Kumar, K., Jenitha, T., & Amulya, M. P. (2025) · International Conference on Computing for Sustainability and Intelligent Future (COMP-SIF) · doi:10.1109/comp-sif65618.2025.10969943
“Hence, the proposed model is obtained by integrating these architectures which in turn provide a better feature representation and a stable classification outcome[4].”
Real-Time Disease Diagnosis from Medical Images Using DenseNet
M.Preetha, Mamasiddikovich, S. R., Kumar, S., Al-Attabi, K., Holkar, U., & Dwibedi, R. K. (2024) · International Conference on Advances in Computing, Communication and Materials (ICACCM) · doi:10.1109/icaccm61117.2024.11059165
Contribution of Artificial Intelligence to Training and Diagnosis in Medical Radiology
Méndez, J. A. J., Izquierdo, M. M., Marcos-Pablos, S., D’Amato, R., & Haro, F. B. (2025) · Lecture notes in educational technology · doi:10.1007/978-981-96-5658-5_22
AI-Powered Medical Image Analysis: Enhancing Radiology with Deep Learning-Based Image Segmentation
Dongre, S. P., Sennan, S., Kaleem, A. M., Nigam, C., Preetha, K., & Bhuvaneshwari, S. (2025) · International Conference on Emerging Technologies in Engineering Applications (ICETEA) · doi:10.1109/icetea64585.2025.11099652
“To test the efficacy of the proposed method, the set of experiments is carried out on medical imaging dataset[4].”
Instance-Wise Joint Feature and Expert Decision Acquisition for Classification
Ekanayake, S. P., Zois, D., & Wickramasinghe, N. D. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3614214
State-of-the-art DNN techniques for lung cancer diagnosis using chest CT scans: A review
Sakshiwala, & Singh, M. P. (2025) · ELCVIA Electronic Letters on Computer Vision and Image Analysis · doi:10.5565/rev/elcvia.1597
Ueda, D., Yamamoto, A., Shimazaki, A., Walston, S. L., Matsumoto, T., Izumi, N., Tsukioka, T., Komatsu, H., Inoue, H., Kabata, D., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16829879
Ueda, D., Yamamoto, A., Shimazaki, A., Walston, S. L., Matsumoto, T., Izumi, N., Tsukioka, T., Komatsu, H., Inoue, H., Kabata, D., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16829876
Ueda, D., Yamamoto, A., Shimazaki, A., Walston, S. L., Matsumoto, T., Izumi, N., Tsukioka, T., Komatsu, H., Inoue, H., Kabata, D., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16829873
Ueda, D., Yamamoto, A., Shimazaki, A., Walston, S. L., Matsumoto, T., Izumi, N., Tsukioka, T., Komatsu, H., Inoue, H., Kabata, D., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16829882
Ueda, D., Yamamoto, A., Shimazaki, A., Walston, S. L., Matsumoto, T., Izumi, N., Tsukioka, T., Komatsu, H., Inoue, H., Kabata, D., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16829870
Utilizing AI CAD for early pandemic screening in chest radiographs
Kim, Y. (2026) · Scientific Reports · doi:10.1038/s41598-025-14195-6
213Chapter 10 AI-driven advances in lung cancer care
Malviya, R., Rajput, S., Roy, M., Sridhar, S. B., & Thomas, S. (2025) · De Gruyter eBooks · doi:10.1515/9783111670744-010
Shilpa, P., J.Nandhini, Saravanan, P., Ali, D. S. S., Selvi, S., & Johri, M. (2025) · International Conference on IoT, Communication and Automation Technology (ICICAT) · doi:10.1109/icicat68430.2025.11414480
Deep learning-based computer-aided diagnosis for parotid gland tumors on MRI
Irifune, Y., Mitani, S., Kinoshita, K., Kawaguchi, N., Kitani, T., Sato, E., Hosokawa, Y., Isshiki, M., Yoshida, K., Mukaigawa, T., et al. (2026) · Auris Nasus Larynx · doi:10.1016/j.anl.2026.02.012
Quantitative Assessment of Lung Nodule: Deep Learning and Radiomics
Hwang, E. J., & Goo, J. M. (2026) · Springer eBooks · doi:10.1007/978-3-032-14103-3_20
Li, J., Xing, Y., Gao, X., Liu, Y., Zhang, L., Huang, Y., Zhang, P., Ye, Z., Wang, M., & Song, F. (2026) · MedComm · doi:10.1002/mco2.70730 · Cited by 1
“Chest X-ray/chest radiograph (CXR) is still the most widely available and inexpensive first-line examination; however, it can fail to reveal a substantial fraction of cancers [ 4 ], and its interpretation varies considerably across readers [ 5, 6 ].”
Elsheikh, E. E. A., Ramadan, M. A., Abdelhalim, F. I., & Babiker, E. B. H. (2026) · Cureus · doi:10.7759/cureus.107774
“This complementarity is consistent with Sim et al. [25], who reported that AI-augmented workflows improved diagnostic accuracy and efficiency compared to either alone.”
E-Net: Optimized Lightweight CNN for Accurate Lung Cancer Classification
Hossain, M. A., Rafid, M. E. A., Tahsin, T. K., Raha, B. A., Fahim, S. M. T. A., & Fayaz, S. A. (2025) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit68739.2025.11491346
“…DCNNs have been reported to exceed the performance levels of radiologists at localizing malignant nodules in chest radiographs and CT images [8], and also to assist radiologists as decision-support aides, yielding 70.3% sensitivity (~65.1% sensitivity without a DCNN decision-support aid) [9], [10].”
InfluentialSrour, A., Omar, M., Barash, Y., Litmanovich, D., Srour, M., Klang, E., & Gorenshtein, A. (2026) · Journal of the American College of Radiology · doi:10.21203/rs.3.rs-9705085/v1
Kohzai, M., Yamamoto, S., Matsushita, M., Ueno, Y., & Tanigawa, N. (2026) · Diagnostics · doi:10.3390/diagnostics16111637
“Sim et al. reported that combining a radiologist’s diagnosis with AI software increased the detection sensitivity of lung nodules by 5.2% [13].”
InfluentialShin, H. J., Lee, E. H., Kwak, S. H., Kim, N. Y., Kim, E. J., & Suh, Y. J. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-02039-2
Portincasa, P., Khalil, M., Novielli, P., & Tangaro, S. (2026) · European Journal of Clinical Investigation · doi:10.1111/eci.70254
“Less experienced practitioners appear to benefit most from AI guidance [38–40].”
Buradkar, S., Ratnaparkhi, C. R., Joshi, P., Dhok, A., & Ghare, P. H. (2026) · Current Science · doi:10.18520/cs/v131/i3/273-285
Enhancing Multi-Label retinal Disease diagnosis via Multi-Expert graph neural active learning
Ning, Z., & Yang, J. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111609
Metadata
- Authors
- Sim, Y., Chung, M. J., Kotter, E., Yune, S., Kim, M., Do, S., Han, K., Kim, H., Yang, S., Lee, D.-J., & Choi, B. W.
- Affiliations
- Yonsei University (KR)
- Harvard University (US)
- ORCID
- Venue
- Radiology
- Published
- 2020-01
- Volume
- 294
- Issue
- 1
- Pages
- 199-209
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- OpenAlex
- W2984645840
- Cited by
- 248
- References
- 26
- Citations by year
- 2026: 10 · 2025: 32 · 2024: 39 · 2023: 55 · 2022: 39 · 2021: 60 · 2020: 12
- FWCI
- 12.78
- Type
- journal article
- Language
- en
- Open access
- Licence
- cc-by-nc-nd
- Locations
- Radiology (publishedVersion)
- PubMed (publishedVersion)
- YUHSpace (Yonsei University Medical Library) (submittedVersion, cc-by-nc-nd)
- Funders
- Samsung Electronics
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Lung Cancer Diagnosis and Treatment
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- deep convolutional neural networks, computer-aided detection, lung cancer, chest radiograph, radiologist performance, false positives, nodule detection, diagnostic accuracy, multicenter validation
- MeSH terms
- Adult, Aged, Solitary Pulmonary Nodule, Female, Humans, Lung, Lung Neoplasms, Male, Middle Aged, Radiographic Image Interpretation, Computer-Assisted, Retrospective Studies, Sensitivity and Specificity, Radiography, Thoracic, Reproducibility of Results, Neural Networks, Computer, Multiple Pulmonary Nodules, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.