Identifying pulmonary nodules or masses on chest radiography using deep learning: external validation and strategies to improve clinical practice
Open accessAbstract
Aim To test the diagnostic performance of a deep learning-based system for the detection of clinically significant pulmonary nodules/masses on chest radiographs.Materials and methods Using a retrospective study of 100 patients (47 with clinically significant pulmonary nodules/masses and 53 control subjects without pulmonary nodules), two radiologists verified clinically significantly pulmonary nodules/masses according to chest computed tomography (CT) findings. A computer-aided diagnosis (CAD) software using a deep-learning approach was used to detect pulmonary nodules/masses to determine the diagnostic performance in four algorithms (heat map, abnormal probability, nodule probability, and mass probability).Results A total of 100 cases were included in the analysis. Among the four algorithms, mass algorithm could achieve a 76.6% sensitivity (36/47, 11 false negative) and 88.68% specificity (47/53, six false-positive) in the detection of pulmonary nodules/masses at the optimal probability score cut-off of 0.2884. Compared to the other three algorithms, mass probability algorithm had best predictive ability for pulmonary nodule/mass detection at the optimal probability score cut-off of 0.2884 (AUC Mass: 0.916 versus AUC Heat map: 0.682, p<0.001; AUC Mass: 0.916 versus AUC Abnormal: 0.810, p=0.002; AUC Mass: 0.916 versus AUC Nodule: 0.813, p=0.014).Conclusion In conclusion, the deep-learning based computer-aided diagnosis system will likely play a vital role in the early detection and diagnosis of pulmonary nodules/masses on chest radiographs. In future applications, these algorithms could support triage workflow via double reading to improve sensitivity and specificity during the diagnostic process.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Thirteen respiratory studies validated algorithms on external data.
“13 (refs. 63–75) studies validated algorithms on external data”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [68]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · see citing works
Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening
The National Lung Screening Trial Research Team (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1102873 · Cited by 11,064 · see citing works
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759 · see citing works
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616 · see citing works
Greiner, M., Pfeiffer, D., & Smith, R. D. (2000) · Preventive Veterinary Medicine · doi:10.1016/s0167-5877(00)00115-x · Cited by 1,923 · see citing works
Lakhani, P., & Sundaram, B. (2017) · Radiology · doi:10.1148/radiol.2017162326 · Cited by 1,775 · see citing works
Rajpurkar, P., Irvin, J., Ball, R. L., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C. P., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002686 · Cited by 1,449 · see citing works
Deep Learning in Medical Imaging: General Overview
Lee, J., Jun, S., Cho, Y.-W., Lee, H., Kim, G. B., Seo, J. B., & Kim, N. (2017) · Korean Journal of Radiology · doi:10.3348/kjr.2017.18.4.570 · Cited by 1,287 · see citing works
Deep convolutional neural networks for multi-modality isointense infant brain image segmentation
Zhang, W., Li, R., Deng, H., Wang, L., Lin, W., Ji, S., & Shen, D. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2014.12.061 · Cited by 817 · see citing works
Current Applications and Future Impact of Machine Learning in Radiology
Choy, G., Khalilzadeh, O., Michalski, M., Do, S., Samir, A. E., Pianykh, O. S., Geis, J. R., Pandharipande, P. V., Brink, J. A., & Dreyer, K. J. (2018) · Radiology · doi:10.1148/radiol.2018171820 · Cited by 800 · see citing works
The British Thoracic Society guidelines on the investigation and management of pulmonary nodules
Baldwin, D., & Callister, M. E. J. (2015) · Thorax · doi:10.1136/thoraxjnl-2015-207221 · Cited by 748 · see citing works
Detecting and classifying lesions in mammograms with Deep Learning
Ribli, D., Horváth, A., Unger, Z., Pollner, P., & Csabai, I. (2018) · Scientific Reports · doi:10.1038/s41598-018-22437-z · Cited by 696 · see citing works
Error and discrepancy in radiology: inevitable or avoidable?
Brady, A. (2016) · Insights into Imaging · doi:10.1007/s13244-016-0534-1 · Cited by 567 · see citing works
Chen, Y.-J. Y.-J., Hua, K., Hsu, C.-H., Cheng, W.-H., & Hidayati, S. C. (2015) · OncoTargets and Therapy · doi:10.2147/ott.s80733 · Cited by 557 · see citing works
Nam, J. G., Park, S., Hwang, E. J., Lee, J. H., Jin, K.-N., Lim, K. Y., Vu, T. H., Sohn, J. H., Hwang, S., Goo, J. M., et al. (2019) · Radiology · doi:10.1148/radiol.2018180237 · Cited by 539 · see citing works
Fully Convolutional Architectures for Multiclass Segmentation in Chest Radiographs
Novikov, A. A., Lenis, D., Major, D., Hladůvka, J., Wimmer, M., & Bühler, K. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2806086 · Cited by 264 · see citing works
Cicero, M. D., Bilbily, A., Colak, E., Dowdell, T., Gray, B., Perampaladas, K., & Barfett, J. (2016) · Investigative Radiology · doi:10.1097/rli.0000000000000341 · Cited by 247 · see citing works
Missed lung cancer: when, where, and why?
del Ciello, A., Franchi, P., Contegiacomo, A., Cicchetti, G., Bonomo, L., & Larici, A. R. (2017) · Diagnostic and Interventional Radiology · doi:10.5152/dir.2016.16187 · Cited by 192 · see citing works
Workload of Radiologists in United States in 2006–2007 and Trends Since 1991–1992
Bhargavan, M., Kaye, A. H., Forman, H. P., & Sunshine, J. H. (2009) · Radiology · doi:10.1148/radiol.2522081895 · Cited by 156 · see citing works
Wu, F., Huang, Y., Wu, C. C., Tang, E., Chen, C.-S., Mar, G.-Y., Yen, Y. C., & Wu, M. (2016) · Clinical Lung Cancer · doi:10.1016/j.cllc.2016.03.004 · Cited by 135 · see citing works
Self-citationAdded value of double reading in diagnostic radiology,a systematic review
Geijer, H., & Geijer, M. (2018) · Insights into Imaging · doi:10.1007/s13244-018-0599-0 · Cited by 117 · see citing works
Radiologic errors in patients with lung cancer.
Forrest, J. V., & Friedman, P. J. (1981) · PubMed · Cited by 96 · see citing works
Factors Affecting Perception of Pulmonary Lesions
Brogdon, B. G., Kelsey, C. A., & Moseley, R. D. (1983) · Radiologic Clinics of North America · doi:10.1016/s0033-8389(22)01116-2 · Cited by 85 · see citing works
Ciatto, S., Turco, M. R. D., Burke, P., et al. (2003) · British Journal of Cancer · doi:10.1038/sj.bjc.6601356 · Cited by 61 · see citing works
Hsu, H.-T., Tang, E., Wu, M., et al. (2018) · Academic Radiology · doi:10.1016/j.acra.2018.01.012 · Cited by 48 · see citing works
Self-citationPitfalls in Chest Radiographic Interpretation: Blind Spots
de Groot, P. M., Carter, B. W., Abbott, G. F., & Wu, C. C. (2015) · Seminars in Roentgenology · doi:10.1053/j.ro.2015.01.008 · Cited by 44 · see citing works
Wu, F., Chen, P.-A., Wu, C. C., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-16042-9 · Cited by 38 · see citing works
Self-citation(2017) · Elsevier eBooks · doi:10.1016/c2016-0-01903-4 · Cited by 33 · see citing works
Automated identification of pneumonia in chest radiograph reports in critically ill patients
Liu, V. X., Clark, M. P., Mendoza, M., Saket, R. R., Gardner, M. N., Turk, B. J., & Escobar, G. J. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-90 · Cited by 31 · see citing works
Analysis of Radiologists' Imaging Workload Trends by Place of Service
Levin, D. C., Rao, V. M., Parker, L., et al. (2013) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2013.02.016 · Cited by 27 · see citing works
Deep Learning Techniques on Texture Analysis of Chest and Breast Images
Cheng, J.-Z., Chen, C., & Shen, D. G. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-812133-7.00009-0 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Bhargavan M, Kaye AH, Forman HP, et al. Workload of radiologists in United States in 2006e2007 and trends since 1991e1992. Radiology 2009;252(2):458e67.
Cited by
Sorted by citations
Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946 · see citing works
Homayounieh, F., Digumarthy, S., Ebrahimian, S., Rueckel, J., Hoppe, B. F., Sabel, B. O., Conjeti, S., Ridder, K., Sistermanns, M., Wang, L., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.41096 · Cited by 113 · see citing works
- Authors
- Liang, C.-H., Liu, Y.-C., Wu, M.-T., Garcia-Castro, F., Alberich-Bayarri, A., & Wu, F.-Z.
- Affiliations
- National Yang Ming University (TW)
- Xiamen Chang Gung Hospital (CN)
- Kaohsiung Veterans General Hospital (TW)
- Corresponding author
- Wu, F.-Z.
- Venue
- Clinical Radiology
- Publisher
- Elsevier BV
- Published
- 2020-01
- Volume
- 75
- Issue
- 1
- Pages
- 38-45
- ISSN
- 0009-9260 (ISSN-L)
- OpenAlex
- W2972901606
- Cited by
- 69
- References
- 31
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 8 · 2023: 23 · 2022: 11 · 2021: 18 · 2020: 4
- FWCI
- 3.81
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Clinical Radiology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Ministry of Science and Technology, Taiwan
- Kaohsiung Veterans General Hospital
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Lung Cancer Diagnosis and Treatment
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- pulmonary nodules, lung mass, chest radiography, computer-aided diagnosis, deep learning, nodule detection, diagnostic accuracy, heatmaps, probability score, early detection
- MeSH terms
- Deep Learning, Algorithms, Solitary Pulmonary Nodule, Diagnosis, Computer-Assisted, Female, Humans, Lung Neoplasms, Male, Middle Aged, Radiographic Image Interpretation, Computer-Assisted, Retrospective Studies, Sensitivity and Specificity, Software, Taiwan, Radiography, Thoracic, Tomography, X-Ray Computed, Multiple Pulmonary Nodules, Quality Improvement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
