Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Purpose To evaluate the efficacy of deep convolutional neural networks (DCNNs) for detecting tuberculosis (TB) on chest radiographs. Materials and Methods Four deidentified HIPAA-compliant datasets were used in this study that were exempted from review by the institutional review board, which consisted of 1007 posteroanterior chest radiographs. The datasets were split into training (68.0%), validation (17.1%), and test (14.9%). Two different DCNNs, AlexNet and GoogLeNet, were used to classify the images as having manifestations of pulmonary TB or as healthy. Both untrained and pretrained networks on ImageNet were used, and augmentation with multiple preprocessing techniques. Ensembles were performed on the best-performing algorithms. For cases where the classifiers were in disagreement, an independent board-certified cardiothoracic radiologist blindly interpreted the images to evaluate a potential radiologist-augmented workflow. Receiver operating characteristic curves and areas under the curve (AUCs) were used to assess model performance by using the DeLong method for statistical comparison of receiver operating characteristic curves. Results The best-performing classifier had an AUC of 0.99, which was an ensemble of the AlexNet and GoogLeNet DCNNs. The AUCs of the pretrained models were greater than that of the untrained models (P < .001). Augmenting the dataset further increased accuracy (P values for AlexNet and GoogLeNet were .03 and .02, respectively). The DCNNs had disagreement in 13 of the 150 test cases, which were blindly reviewed by a cardiothoracic radiologist, who correctly interpreted all 13 cases (100%). This radiologist-augmented approach resulted in a sensitivity of 97.3% and specificity 100%. Conclusion Deep learning with DCNNs can accurately classify TB at chest radiography with an AUC of 0.99. A radiologist-augmented approach for cases where there was disagreement among the classifiers further improved accuracy. © RSNA, 2017
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Because AI-enabled diagnostic technology has made significant advances in radiology23,24, we recruited physicians with different levels of task expertise to perform a radiological task.”
From Do as AI say: susceptibility in deployment of clinical decision-aids (2021) · cites this paper as [24]
In the citing paper
“several studies have already been published in areas such as lesion detection [7], classification [8], segmentation [9], image reconstruction [10, 11], and natural language processing [12].”
From Convolutional neural networks: an overview and application in radiology (2018) · cites this paper as [7]
In the citing paper
“According to the results, the best area under the curve of receiver operating characteristic curves for detecting pulmonary tuberculosis from healthy cases was 0.99, which was obtained by ensemble of the AlexNet and GoogLeNet 2D-CNNs.”
From Convolutional neural networks: an overview and application in radiology (2018) · cites this paper as [7]
In the citing paper
“Four patient cohorts from one study90 provided contingency tables with raw diagnostic accuracy.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [90]
In the citing paper
“Reverse Transcription Polymerase chain reaction (RT-PCR) is the most used clinical screening method for the COVID-19 patients, which uses respiratory specimens for testing [5].”
From Can AI help in screening viral and COVID-19 pneumonia? (2020) · cites this paper as [5]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
(2014) · Cited by 33,419
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
The use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
Approximate is Better than “Exact” for Interval Estimation of Binomial Proportions
Agresti, A., & Coull, B. A. (1998) · The American Statistician · doi:10.1080/00031305.1998.10480550 · Cited by 3,223
What's In A Name? Malay Seals As Onomastic Sources
Library, T. B., & Gallop, A. T. (2018) · Malay Literature · doi:10.37052/ml.31(1)no1 · Cited by 1,483
Proceedings of the 26th International Conference on Neural Information Processing Systems
Burges, C. J. C., Bottou, L., Welling, M., Zoubin Ghahramani, A., & Weinberger, K. Q. (2013) · Cited by 1,323
Receiver Operating Characteristic Curves and Their Use in Radiology
Obuchowski, N. A. (2003) · Radiology · doi:10.1148/radiol.2291010898 · Cited by 843
Deep convolutional neural networks for multi-modality isointense infant brain image segmentation
Zhang, W., Li, R., Deng, H., Wang, L., Lin, W., Ji, S., & Shen, D. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2014.12.061 · Cited by 817
Machine learning and radiology
Wang, S., & Summers, R. M. (2012) · Medical Image Analysis · doi:10.1016/j.media.2012.02.005 · Cited by 697
Automatic Tuberculosis Screening Using Chest Radiographs
Jaeger, S., Karargyris, A., Candemir, S., Folio, L., Siegelman, J., Callaghan, F., Xue, Z., Palaniappan, K., Singh, R. K., Antani, S., et al. (2014) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2284099 · Cited by 678
Automatic Tuberculosis Screening using Chest Radiographs
Department of Electronics and Communication Engineering, V. T. M. T. D. R. D. S. E. C., Leopauline, S., Kalpana, R., Department of Electronics and Communication Engineering, V. T. M. T. D. R. D. S. E. C., Sharmila, P., & Department of Electronics and Communication Engineering, V. T. M. T. D. R. D. S. E. C. (2019) · International Journal of Engineering and Advanced Technology · doi:10.35940/ijeat.a1507.129219 · Cited by 273
Model Assessment and Selection
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7_7 · Cited by 265
Chest tuberculosis: Radiological review and imaging recommendations
Bhalla, A. S., Goyal, A., Guleria, R., & Gupta, A. K. (2015) · Indian Journal of Radiology and Imaging · doi:10.4103/0971-3026.161431 · Cited by 195
Melendez, J., Sánchez, C. I., Philipsen, R. H. H. M., Maduskar, P., Dawson, R., Theron, G., Dheda, K., & van Ginneken, B. (2016) · Scientific Reports · doi:10.1038/srep25265 · Cited by 152
Computer-aided detection of pulmonary tuberculosis on digital chest radiographs: a systematic review
Pande, T., Cohen, C., Pai, M., & Khan, F. A. (2016) · The International Journal of Tuberculosis and Lung Disease · doi:10.5588/ijtld.15.0926 · Cited by 115
Maduskar, P., Muyoyeta, M., Ayles, H., Hogeweg, L., Peters-Bax, L., & van Ginneken, B. (2013) · The International Journal of Tuberculosis and Lung Disease · doi:10.5588/ijtld.13.0325 · Cited by 77
Hoog, A. H. V., Meme, H., van Deutekom, H., Mithika, A. M., Olunga, C., Onyino, F., & Borgdorff, M. W. (2011) · The International Journal of Tuberculosis and Lung Disease · doi:10.5588/ijtld.11.0004 · Cited by 77
U.S. national library of medicine
(1998) · Biochemical Education · doi:10.1016/s0307-4412(98)00250-7 · Cited by 60
Research works printed but not linked: 9
- Jaeger S, 2013, Quant Imaging Med Surg, 3, 89
- Bar Y, 2015, Proceedings of SPIE: medical imaging 2015—computer-aided diagnosis, 9414
- Hua KL, 2015, Onco Targets Ther, 8, 2015
- Roth HR, 2015, Proceedings of SPIE: medical imaging 2015—image processing, 9413
- Hwang S, 2016, Proceedings of SPIE: medical imaging 2016—title, 9785
- Jaeger S, 2014, Quant Imaging Med Surg, 4, 475
- Fawcett T, 2004, Mach Learn, 31, 1
- Agresti A, 1998, Am Stat, 52, 119
- Srivastava N, 2014, J Mach Learn Res, 15, 1929
Cited by
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Convolutional neural networks: an overview and application in radiology
Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018) · Insights into Imaging · doi:10.1007/s13244-018-0639-9 · Cited by 4,682
“Needless to say, there has been a surge of interest in the potential of CNN among radiology researchers, and several studies have already been published in areas such as lesion detection [7], classification [8], segmentation [9], image reconstruction [10, 11], and natural language processing [12].”
BackgroundArtificial intelligence in healthcare
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0305-z · Cited by 2,817
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D. Y., Bagul, A., Langlotz, C. P., Shpanskaya, K. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05225 · Cited by 1,219
Metadata
- Authors
- Lakhani, P., & Sundaram, B.
- Affiliations
- Thomas Jefferson University (US)
- Venue
- Radiology
- Published
- 2017-08
- Volume
- 284
- Issue
- 2
- Pages
- 574-582
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- OpenAlex
- W2608231518
- Cited by
- 1,775
- References
- 26
- Citations by year
- 2026: 108 · 2025: 147 · 2024: 180 · 2023: 199 · 2022: 246 · 2021: 272 · 2020: 284 · 2019: 188 · 2018: 127 · 2017: 22
- FWCI
- 130.34
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Radiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Health Sciences › Medicine › Infectious Diseases › Tuberculosis Research and Epidemiology
- Keywords
- pulmonary tuberculosis, chest radiography, automatic classification, convolutional neural networks, tuberculosis detection
- MeSH terms
- Algorithms, Humans, Retrospective Studies, ROC Curve, Sensitivity and Specificity, Radiography, Thoracic, Tuberculosis, Pulmonary, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.