The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis
Open accessAbstract
We undertook a systematic review of the diagnostic accuracy of artificial intelligence-based software for identification of radiologic abnormalities (computer-aided detection, or CAD) compatible with pulmonary tuberculosis on chest x-rays (CXRs). We searched four databases for articles published between January 2005-February 2019. We summarized data on CAD type, study design, and diagnostic accuracy. We assessed risk of bias with QUADAS-2. We included 53 of the 4712 articles reviewed: 40 focused on CAD design methods ("Development" studies) and 13 focused on evaluation of CAD ("Clinical" studies). Meta-analyses were not performed due to methodological differences. Development studies were more likely to use CXR databases with greater potential for bias as compared to Clinical studies. Areas under the receiver operating characteristic curve (median AUC [IQR]) were significantly higher: in Development studies AUC: 0.88 [0.82-0.90]) versus Clinical studies (0.75 [0.66-0.87]; p-value 0.004); and with deep-learning (0.91 [0.88-0.99]) versus machine-learning (0.82 [0.75-0.89]; p = 0.001). We conclude that CAD programs are promising, but the majority of work thus far has been on development rather than clinical evaluation. We provide concrete suggestions on what study design elements should be improved.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Big Data and Machine Learning in Health Care
Beam, A. L., & Kohane, I. S. (2018) · JAMA · doi:10.1001/jama.2017.18391 · Cited by 2,506
Lakhani, P., & Sundaram, B. (2017) · Radiology · doi:10.1148/radiol.2017162326 · Cited by 1,775
Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data
Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.3763 · Cited by 1,395
Two public chest X-ray datasets for computer-aided screening of pulmonary diseases.
(2014) · PubMed Central · doi:10.3978/j.issn.2223-4292.2014.11.20 · Cited by 918
Pre-trained convolutional neural networks as feature extractors for tuberculosis detection
Lopes, U. K., & Valiati, J. F. (2017) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2017.08.001 · Cited by 294
Melendez, J., Sánchez, C. I., Philipsen, R. H. H. M., Maduskar, P., Dawson, R., Theron, G., Dheda, K., & van Ginneken, B. (2016) · Scientific Reports · doi:10.1038/srep25265 · Cited by 152
Muyoyeta, M., Maduskar, P., Moyo, M., Kasese, N., Milimo, D., Spooner, R., Kapata, N., Hogeweg, L., van Ginneken, B., & Ayles, H. (2014) · PLoS ONE · doi:10.1371/journal.pone.0093757 · Cited by 98
Maduskar, P., Muyoyeta, M., Ayles, H., Hogeweg, L., Peters-Bax, L., & van Ginneken, B. (2013) · The International Journal of Tuberculosis and Lung Disease · doi:10.5588/ijtld.13.0325 · Cited by 77
Philipsen, R. H. H. M., Sánchez, C. I., Maduskar, P., Melendez, J., Peters-Bax, L., Peter, J. G., Dawson, R., Theron, G., Dheda, K., & van Ginneken, B. (2015) · Scientific Reports · doi:10.1038/srep12215 · Cited by 77
Research works printed but not linked: 60
- WHO, 2017, Global Tuberculosis Report 2017
- FH WILLIAMS, 1907, The Use of X-Ray Examinations in Pulmonary Tuberculosis, The Boston Medical and Surgical Journal, 157, 850, 10.1056/NEJM190712261572602 · DOI
- T Pande, 2015, Use of chest radiography in the 22 highest tuberculosis burden countries, Eur Respir J, 46, 1816, 10.1183/13993003.01064-2015 · DOI
- B Chunhaswasdikul, 1992, Anti-tuberculosis programs in Thailand: a cost analysis, Southeast Asian J Trop Med Public Health, 23, 195
- T Samandari, 2011, Costs and consequences of additional chest x-ray in a tuberculosis prevention program in Botswana, Am J Respir Crit Care Med, 183, 1103, 10.1164/rccm.201004-0620OC · DOI
- T Pande, 2016, Computer-aided detection of pulmonary tuberculosis on digital chest radiographs: A systematic review, Int J Tuberc Lung Dis, 20, 1226, 10.5588/ijtld.15.0926 · DOI
- D Moher, 2009, Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement, PLoS Med, 6, e1000097, 10.1371/journal.pmed.1000097 · DOI · Paper page
- J Shiraishi, 2000, Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists' detection of pulmonary nodules, AJR Am J Roentgenol, 174, 71, 10.2214/ajr.174.1.1740071 · DOI · Paper page
- I Abubakar, 2010, Diagnostic accuracy of digital chest radiography for pulmonary tuberculosis in a UK urban population, Eur Respir J, 35, 689, 10.1183/09031936.00136609 · DOI
- G Theron, 2014, Feasibility, accuracy, and clinical effect of point-of-care Xpert MTB/RIF testing for tuberculosis in primary-care settings in Africa: a multicentre, randomised, controlled trial, Lancet, 383, 424, 10.1016/S0140-6736(13)62073-5 · DOI · Paper page
- T Frieden, 2004, Toman’s tuberculosis: case detection, treatment, and monitoring. Questions and answers
- M Muyoyeta, 2015, Implementation Research to Inform the Use of Xpert MTB/RIF in Primary Health Care Facilities in High TB and HIV Settings in Resource Constrained Settings, PLoS One, 10, e0126376, 10.1371/journal.pone.0126376 · DOI
- M Breuninger, 2014, Diagnostic accuracy of computer-aided detection of pulmonary tuberculosis in chest radiographs: A validation study from sub-Saharan Africa, PLoS One, 9, 10.1371/journal.pone.0106381 · DOI · Paper page
- A Steiner, 2015, Screening for pulmonary tuberculosis in a Tanzanian prison and computer-aided interpretation of chest X-rays, Public health action, 5, 249, 10.5588/pha.15.0037 · DOI
- J Melendez, 2018, Accuracy of an automated system for tuberculosis detection on chest radiographs in high-risk screening, The international journal of tuberculosis and lung disease: the official journal of the International Union against Tuberculosis and Lung Disease, 22, 567, 10.5588/ijtld.17.0492 · DOI
- SMA Zaidi, 2018, Evaluation of the diagnostic accuracy of Computer-Aided Detection of tuberculosis on Chest radiography among private sector patients in Pakistan, Sci Rep, 8, 12339, 10.1038/s41598-018-30810-1 · DOI
- Fatima A, Akram MU, Akhtar M, Shafique I, editors. Detection of tuberculosis using hybrid features from chest radiographs. SPIE; 2017 2017-1-1. Proceedings of the SPIE, Volume 10225, id. 102252B 5 pp. (2017). SPIE.
- Ding M, Antani S, Jaeger S, Xue Z, Candemir S, Kohli M, et al., editors. Local-global classifier fusion for screening chest radiographs. SPIE Medical Imaging; 2017 2017-1-1: SPIE. · DOI
- E Udayakumar, 2017, TB screening using SVM and CBC techniques, Current Pediatric Research, 21, 338
- Hwang S, Kim HE, Jeong J, Kim HJ, editors. A novel approach for tuberculosis screening based on deep convolutional neural networks. Medical Imaging 2016: Computer-Aided Diagnosis; 2016: SPIE. · DOI · Paper page
- Poornimadevi CS, Helen Sulochana C, editors. Automatic detection of pulmonary tuberculosis using image processing techniques. 2016 IEEE International Conference on Wireless Communications, Signal Processing and Networking, WiSPNET 2016; 2016: Presses Polytechniques Et Universitaires Romandes. · doi:10.1109/wispnet.2016.7566243
- S Jaeger, 2014, Automatic tuberculosis screening using chest radiographs, IEEE Trans Med Imaging, 33, 233, 10.1109/TMI.2013.2284099 · DOI · Paper page
- A Karargyris, 2016, Combination of texture and shape features to detect pulmonary abnormalities in digital chest X-rays, Int J Comput Assist Radiol Surg, 11, 99, 10.1007/s11548-015-1242-x · DOI
- S Jaeger, 2012, Detecting tuberculosis in radiographs using combined lung masks, Conf Proc IEEE Eng Med Biol Soc, 2012, 4978
- KC Santosh, 2016, Edge map analysis in chest X-rays for automatic pulmonary abnormality screening, Int J Comput Assist Radiol Surg, 11, 1637, 10.1007/s11548-016-1359-6 · DOI
- S. Jaeger, 2015, Detecting Disease in Radiographs with Intuitive Confidence, Sci World J, 2015
- JM Seixas, 2013, Artificial neural network models to support the diagnosis of pleural tuberculosis in adult patients, Int J Tuberc Lung Dis, 17, 682, 10.5588/ijtld.12.0829 · doi:10.5588/ijtld.12.0829
- L Hogeweg, 2017, Fast and effective quantification of symmetry in medical images for pathology detection: Application to chest radiography, Medical Physics, 44, 2242, 10.1002/mp.12127 · DOI
- P Maduskar, 2016, Automatic detection of pleural effusion in chest radiographs, Med Image Anal, 28, 22, 10.1016/j.media.2015.09.004 · DOI
- J Melendez, 2016, On Combining Multiple-Instance Learning and Active Learning for Computer-Aided Detection of Tuberculosis, IEEE Trans Med Imaging, 35, 1013, 10.1109/TMI.2015.2505672 · DOI
- J Melendez, 2015, A novel multiple-instance learning-based approach to computer-aided detection of tuberculosis on chest X-rays, IEEE Trans Med Imaging, 34, 179, 10.1109/TMI.2014.2350539 · DOI
- L Hogeweg, 2015, Automatic Detection of Tuberculosis in Chest Radiographs Using a Combination of Textural, Focal, and Shape Abnormality Analysis, IEEE Trans Med Imaging, 34, 2429, 10.1109/TMI.2015.2405761 · DOI
- G Giacomini, 2015, Quantification of Pulmonary Inflammatory Processes Using Chest Radiography: Tuberculosis as the Motivating Application, Medicine, 94, e1044, 10.1097/MD.0000000000001044 · doi:10.1097/md.0000000000001044
- A Requena-Mendez, 2015, Robust and Reproducible Quantification of the Extent of Chest Radiographic Abnormalities (And It's Free!), PLoS One, 10, e0128044, 10.1371/journal.pone.0128044 · doi:10.1371/journal.pone.0128044
- J Melendez, 2014, Medical Imaging 2014: Computer-Aided Diagnosis
- A Chauhan, 2014, Role of gist and PHOG features in computer-aided diagnosis of tuberculosis without segmentation, PLoS One, 9, 10.1371/journal.pone.0112980 · DOI
- KM Sundaram, 2013, An adaptive region growing algorithm with support vector machine classifier for Tuberculosis cavity identification, American Journal of Applied Sciences, 10, 1616, 10.3844/ajassp.2013.1616.1628 · doi:10.3844/ajassp.2013.1616.1628
- T Xu, 2011, Automated cavity detection of infectious pulmonary tuberculosis in chest radiographs, Conf Proc IEEE Eng Med Biol Soc, 2011, 5178
- NM Noor, 2011, Applying a statistical PTB detection procedure to complement the gold standard, Comput Med Imaging Graph, 35, 186, 10.1016/j.compmedimag.2010.10.002 · doi:10.1016/j.compmedimag.2010.10.002
- R Shen, 2010, A hybrid knowledge-guided detection technique for screening of infectious pulmonary tuberculosis from chest radiographs, IEEE transactions on bio-medical engineering, 57
- Mouton A, Pitcher RD, Douglas TS. Computer-aided detection of pulmonary pathology in pediatric chest radiographs. 2010. p. 619–25. · DOI
- Hogeweg LE, Mol C, Jong PAd, Ginneken Bv, editors. Rib suppression in chest radiographs to improve classification of textural abnormalities2010 2010-1-1: SPIE. · doi:10.1117/12.844409
- R Lieberman, 2009, Computer-assisted detection (CAD) methodology for early detection of response to pharmaceutical therapy in tuberculosis patients, Proceedings of SPIE—the International Society for Optical Engineering, 7260, 726030
- Y Arzhaeva, 2009, Global and local multi-valued dissimilarity-based classification: application to computer-aided detection of tuberculosis, Med Image Comput Comput Assist Interv, 12, 724
- N Mohd Noor, 2005, Discrimination between two lung diseases using chest radiographs, Conference Proceedings: Annual International Conference of the IEEE Engineering in Medicine & Biology Society, 3, 3320
- Hogeweg LE, Mol C, Jong PAd, Ginneken Bv, editors. Rib suppression in chest radiographs to improve classification of textural abnormalities. SPIE Medical Imaging; 2010 2010-1-1: SPIE. · doi:10.1117/12.844409
- Alfadhli FHO, Mand AA, Sayeed MS, Sim KS, Al-Shabi M. Classification of tuberculosis with SURF spatial pyramid features. 2017 International Conference on Robotics, Automation and Sciences (ICORAS)2017. p. 1–5.
- Gabriella I, Stella A K, Agung W S. Early Detection of Tuberculosis using Chest X-Ray (CXR) with Computer-Aided Diagnosis2018. 76–9 p. · doi:10.1109/ibiomed.2018.8534784
- SJ Heo, 2019, Deep Learning Algorithms with Demographic Information Help to Detect Tuberculosis in Chest Radiographs in Annual Workers' Health Examination Data, Int J Environ Res Public Health, 16
- EJ Hwang, 2018, Development and Validation of a Deep Learning-Based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs, Clin Infect Dis
- S Rajaraman, 2018, A novel stacked generalization of models for improved TB detection in chest radiographs, Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual Conference, 2018, 718
- KC Santosh, 2018, Automated Chest X-Ray Screening: Can Lung Region Symmetry Help Detect Pulmonary Abnormalities?, IEEE Trans Med Imaging, 37, 1168, 10.1109/TMI.2017.2775636 · DOI
- R Sivaramakrishnan, 2018, Comparing deep learning models for population screening using chest radiography, Medical Imaging
- S Vajda, 2018, Feature Selection for Automatic Tuberculosis Screening in Frontal Chest Radiographs, J Med Syst, 42, 146, 10.1007/s10916-018-0991-9 · DOI · Paper page
- MT Rahman, 2017, An evaluation of automated chest radiography reading software for tuberculosis screening among public- and private-sector patients, Eur Respir J, 49, 10.1183/13993003.02159-2016 · DOI · Paper page
- M Muyoyeta, 2017, Digital CXR with computer aided diagnosis versus symptom screen to define presumptive tuberculosis among household contacts and impact on tuberculosis diagnosis, BMC Infectious Diseases, 17, 301, 10.1186/s12879-017-2388-7 · DOI
- RC Koesoemadinata, 2018, Computer-assisted chest radiography reading for tuberculosis screening in people living with diabetes mellitus, The international journal of tuberculosis and lung disease: the official journal of the International Union against Tuberculosis and Lung Disease, 22, 1088, 10.5588/ijtld.17.0827 · DOI · Paper page
- M Muyoyeta, 2017, Digital CXR with computer aided diagnosis versus symptom screen to define presumptive tuberculosis among household contacts and impact on tuberculosis diagnosis, BMC Infect Dis, 17, 301, 10.1186/s12879-017-2388-7 · DOI
- MT Rahman, 2017, An evaluation of automated chest radiography reading software for tuberculosis screening among public- and private-sector patients, European Respiratory Journal, 49, 10.1183/13993003.02159-2016 · DOI · Paper page
- FE HARRELL, 1996, MULTIVARIABLE PROGNOSTIC MODELS: ISSUES IN DEVELOPING MODELS, EVALUATING ASSUMPTIONS AND ADEQUACY, AND MEASURING AND REDUCING ERRORS, Stat Med, 15, 361, 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4 · doi:10.1002/(sici)1097-0258(19960229)15:4<361::aid-sim168>3.0.co;2-4
Cited by
Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review
Yin, J., Ngiam, K. Y., & Teo, H. H. (2021) · Journal of Medical Internet Research · doi:10.2196/25759 · Cited by 419
Alhamrouni, I., Abdul Kahar, N. H., Salem, M., Swadi, M., Zahroui, Y., Kadhim, D. J., Mohamed, F. A., & Alhuyi Nazari, M. (2024) · Applied Sciences · doi:10.3390/app14146214 · Cited by 217
Khan, F. A., Majidulla, A., Tavaziva, G., Nazish, A., Abidi, S. K., Benedetti, A., Menzies, D., Johnston, J. C., Khan, A. J., & Saeed, S. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30221-1 · Cited by 134
Self-citationJayakumar, S., Sounderajah, V., Normahani, P., Harling, L., Markar, S. R., Ashrafian, H., & Darzi, A. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00544-y · Cited by 110
Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis
Codlin, A. J., Dao, T. P., Vo, L. N. Q., Forse, R. J., Van Truong, V., Dang, H. M., Nguyen, L. H., Nguyen, H. B., Nguyen, N. V., Sidney-Annerstedt, K., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-03265-0 · Cited by 106
Metadata
- Authors
- Harris, M., Qi, A., Jeagal, L., Torabi, N., Menzies, D., Korobitsyn, A., Pai, M., Nathavitharana, R. R., & Ahmad Khan, F.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-09-03
- Volume
- 14
- Issue
- 9
- Pages
- e0221339
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2972112160
- Cited by
- 177
- References
- 11
- FWCI
- 4.91
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis. https://app.cochat.ai/papers/10.1371/journal.pone.0221339
- The paper
- Harris, M., Qi, A., Jeagal, L., Torabi, N., Menzies, D., Korobitsyn, A., Pai, M., Nathavitharana, R. R., & Ahmad Khan, F. (2019). A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis. PLOS ONE. https://doi.org/10.1371/journal.pone.0221339
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.