Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 177

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis

Open access
Harris, Miriam, Qi, Amy, Jeagal, Luke, Torabi, Nazi, Menzies, Dick, Korobitsyn, Alexei, Pai, Madhukar, Nathavitharana, Ruvandhi R., Ahmad Khan, Faiz · 2019 · PLOS ONE

Abstract

We undertook a systematic review of the diagnostic accuracy of artificial intelligence-based software for identification of radiologic abnormalities (computer-aided detection, or CAD) compatible with pulmonary tuberculosis on chest x-rays (CXRs). We searched four databases for articles published between January 2005-February 2019. We summarized data on CAD type, study design, and diagnostic accuracy. We assessed risk of bias with QUADAS-2. We included 53 of the 4712 articles reviewed: 40 focused on CAD design methods ("Development" studies) and 13 focused on evaluation of CAD ("Clinical" studies). Meta-analyses were not performed due to methodological differences. Development studies were more likely to use CXR databases with greater potential for bias as compared to Clinical studies. Areas under the receiver operating characteristic curve (median AUC [IQR]) were significantly higher: in Development studies AUC: 0.88 [0.82-0.90]) versus Clinical studies (0.75 [0.66-0.87]; p-value 0.004); and with deep-learning (0.91 [0.88-0.99]) versus machine-learning (0.82 [0.75-0.89]; p = 0.001). We conclude that CAD programs are promising, but the majority of work thus far has been on development rather than clinical evaluation. We provide concrete suggestions on what study design elements should be improved.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0221339

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 11 references

Research works printed but not linked: 60

  1. WHO, 2017, Global Tuberculosis Report 2017
  2. FH WILLIAMS, 1907, The Use of X-Ray Examinations in Pulmonary Tuberculosis, The Boston Medical and Surgical Journal, 157, 850, 10.1056/NEJM190712261572602 · DOI
  3. T Pande, 2015, Use of chest radiography in the 22 highest tuberculosis burden countries, Eur Respir J, 46, 1816, 10.1183/13993003.01064-2015 · DOI
  4. B Chunhaswasdikul, 1992, Anti-tuberculosis programs in Thailand: a cost analysis, Southeast Asian J Trop Med Public Health, 23, 195
  5. T Samandari, 2011, Costs and consequences of additional chest x-ray in a tuberculosis prevention program in Botswana, Am J Respir Crit Care Med, 183, 1103, 10.1164/rccm.201004-0620OC · DOI
  6. T Pande, 2016, Computer-aided detection of pulmonary tuberculosis on digital chest radiographs: A systematic review, Int J Tuberc Lung Dis, 20, 1226, 10.5588/ijtld.15.0926 · DOI
  7. D Moher, 2009, Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement, PLoS Med, 6, e1000097, 10.1371/journal.pmed.1000097 · DOI · Paper page
  8. J Shiraishi, 2000, Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists' detection of pulmonary nodules, AJR Am J Roentgenol, 174, 71, 10.2214/ajr.174.1.1740071 · DOI · Paper page
  9. I Abubakar, 2010, Diagnostic accuracy of digital chest radiography for pulmonary tuberculosis in a UK urban population, Eur Respir J, 35, 689, 10.1183/09031936.00136609 · DOI
  10. G Theron, 2014, Feasibility, accuracy, and clinical effect of point-of-care Xpert MTB/RIF testing for tuberculosis in primary-care settings in Africa: a multicentre, randomised, controlled trial, Lancet, 383, 424, 10.1016/S0140-6736(13)62073-5 · DOI · Paper page
  11. T Frieden, 2004, Toman’s tuberculosis: case detection, treatment, and monitoring. Questions and answers
  12. M Muyoyeta, 2015, Implementation Research to Inform the Use of Xpert MTB/RIF in Primary Health Care Facilities in High TB and HIV Settings in Resource Constrained Settings, PLoS One, 10, e0126376, 10.1371/journal.pone.0126376 · DOI
  13. M Breuninger, 2014, Diagnostic accuracy of computer-aided detection of pulmonary tuberculosis in chest radiographs: A validation study from sub-Saharan Africa, PLoS One, 9, 10.1371/journal.pone.0106381 · DOI · Paper page
  14. A Steiner, 2015, Screening for pulmonary tuberculosis in a Tanzanian prison and computer-aided interpretation of chest X-rays, Public health action, 5, 249, 10.5588/pha.15.0037 · DOI
  15. J Melendez, 2018, Accuracy of an automated system for tuberculosis detection on chest radiographs in high-risk screening, The international journal of tuberculosis and lung disease: the official journal of the International Union against Tuberculosis and Lung Disease, 22, 567, 10.5588/ijtld.17.0492 · DOI
  16. SMA Zaidi, 2018, Evaluation of the diagnostic accuracy of Computer-Aided Detection of tuberculosis on Chest radiography among private sector patients in Pakistan, Sci Rep, 8, 12339, 10.1038/s41598-018-30810-1 · DOI
  17. Fatima A, Akram MU, Akhtar M, Shafique I, editors. Detection of tuberculosis using hybrid features from chest radiographs. SPIE; 2017 2017-1-1. Proceedings of the SPIE, Volume 10225, id. 102252B 5 pp. (2017). SPIE.
  18. Ding M, Antani S, Jaeger S, Xue Z, Candemir S, Kohli M, et al., editors. Local-global classifier fusion for screening chest radiographs. SPIE Medical Imaging; 2017 2017-1-1: SPIE. · DOI
  19. E Udayakumar, 2017, TB screening using SVM and CBC techniques, Current Pediatric Research, 21, 338
  20. Hwang S, Kim HE, Jeong J, Kim HJ, editors. A novel approach for tuberculosis screening based on deep convolutional neural networks. Medical Imaging 2016: Computer-Aided Diagnosis; 2016: SPIE. · DOI · Paper page
  21. Poornimadevi CS, Helen Sulochana C, editors. Automatic detection of pulmonary tuberculosis using image processing techniques. 2016 IEEE International Conference on Wireless Communications, Signal Processing and Networking, WiSPNET 2016; 2016: Presses Polytechniques Et Universitaires Romandes. · doi:10.1109/wispnet.2016.7566243
  22. S Jaeger, 2014, Automatic tuberculosis screening using chest radiographs, IEEE Trans Med Imaging, 33, 233, 10.1109/TMI.2013.2284099 · DOI · Paper page
  23. A Karargyris, 2016, Combination of texture and shape features to detect pulmonary abnormalities in digital chest X-rays, Int J Comput Assist Radiol Surg, 11, 99, 10.1007/s11548-015-1242-x · DOI
  24. S Jaeger, 2012, Detecting tuberculosis in radiographs using combined lung masks, Conf Proc IEEE Eng Med Biol Soc, 2012, 4978
  25. KC Santosh, 2016, Edge map analysis in chest X-rays for automatic pulmonary abnormality screening, Int J Comput Assist Radiol Surg, 11, 1637, 10.1007/s11548-016-1359-6 · DOI
  26. S. Jaeger, 2015, Detecting Disease in Radiographs with Intuitive Confidence, Sci World J, 2015
  27. JM Seixas, 2013, Artificial neural network models to support the diagnosis of pleural tuberculosis in adult patients, Int J Tuberc Lung Dis, 17, 682, 10.5588/ijtld.12.0829 · doi:10.5588/ijtld.12.0829
  28. L Hogeweg, 2017, Fast and effective quantification of symmetry in medical images for pathology detection: Application to chest radiography, Medical Physics, 44, 2242, 10.1002/mp.12127 · DOI
  29. P Maduskar, 2016, Automatic detection of pleural effusion in chest radiographs, Med Image Anal, 28, 22, 10.1016/j.media.2015.09.004 · DOI
  30. J Melendez, 2016, On Combining Multiple-Instance Learning and Active Learning for Computer-Aided Detection of Tuberculosis, IEEE Trans Med Imaging, 35, 1013, 10.1109/TMI.2015.2505672 · DOI
  31. J Melendez, 2015, A novel multiple-instance learning-based approach to computer-aided detection of tuberculosis on chest X-rays, IEEE Trans Med Imaging, 34, 179, 10.1109/TMI.2014.2350539 · DOI
  32. L Hogeweg, 2015, Automatic Detection of Tuberculosis in Chest Radiographs Using a Combination of Textural, Focal, and Shape Abnormality Analysis, IEEE Trans Med Imaging, 34, 2429, 10.1109/TMI.2015.2405761 · DOI
  33. G Giacomini, 2015, Quantification of Pulmonary Inflammatory Processes Using Chest Radiography: Tuberculosis as the Motivating Application, Medicine, 94, e1044, 10.1097/MD.0000000000001044 · doi:10.1097/md.0000000000001044
  34. A Requena-Mendez, 2015, Robust and Reproducible Quantification of the Extent of Chest Radiographic Abnormalities (And It's Free!), PLoS One, 10, e0128044, 10.1371/journal.pone.0128044 · doi:10.1371/journal.pone.0128044
  35. J Melendez, 2014, Medical Imaging 2014: Computer-Aided Diagnosis
  36. A Chauhan, 2014, Role of gist and PHOG features in computer-aided diagnosis of tuberculosis without segmentation, PLoS One, 9, 10.1371/journal.pone.0112980 · DOI
  37. KM Sundaram, 2013, An adaptive region growing algorithm with support vector machine classifier for Tuberculosis cavity identification, American Journal of Applied Sciences, 10, 1616, 10.3844/ajassp.2013.1616.1628 · doi:10.3844/ajassp.2013.1616.1628
  38. T Xu, 2011, Automated cavity detection of infectious pulmonary tuberculosis in chest radiographs, Conf Proc IEEE Eng Med Biol Soc, 2011, 5178
  39. NM Noor, 2011, Applying a statistical PTB detection procedure to complement the gold standard, Comput Med Imaging Graph, 35, 186, 10.1016/j.compmedimag.2010.10.002 · doi:10.1016/j.compmedimag.2010.10.002
  40. R Shen, 2010, A hybrid knowledge-guided detection technique for screening of infectious pulmonary tuberculosis from chest radiographs, IEEE transactions on bio-medical engineering, 57
  41. Mouton A, Pitcher RD, Douglas TS. Computer-aided detection of pulmonary pathology in pediatric chest radiographs. 2010. p. 619–25. · DOI
  42. Hogeweg LE, Mol C, Jong PAd, Ginneken Bv, editors. Rib suppression in chest radiographs to improve classification of textural abnormalities2010 2010-1-1: SPIE. · doi:10.1117/12.844409
  43. R Lieberman, 2009, Computer-assisted detection (CAD) methodology for early detection of response to pharmaceutical therapy in tuberculosis patients, Proceedings of SPIE—the International Society for Optical Engineering, 7260, 726030
  44. Y Arzhaeva, 2009, Global and local multi-valued dissimilarity-based classification: application to computer-aided detection of tuberculosis, Med Image Comput Comput Assist Interv, 12, 724
  45. N Mohd Noor, 2005, Discrimination between two lung diseases using chest radiographs, Conference Proceedings: Annual International Conference of the IEEE Engineering in Medicine & Biology Society, 3, 3320
  46. Hogeweg LE, Mol C, Jong PAd, Ginneken Bv, editors. Rib suppression in chest radiographs to improve classification of textural abnormalities. SPIE Medical Imaging; 2010 2010-1-1: SPIE. · doi:10.1117/12.844409
  47. Alfadhli FHO, Mand AA, Sayeed MS, Sim KS, Al-Shabi M. Classification of tuberculosis with SURF spatial pyramid features. 2017 International Conference on Robotics, Automation and Sciences (ICORAS)2017. p. 1–5.
  48. Gabriella I, Stella A K, Agung W S. Early Detection of Tuberculosis using Chest X-Ray (CXR) with Computer-Aided Diagnosis2018. 76–9 p. · doi:10.1109/ibiomed.2018.8534784
  49. SJ Heo, 2019, Deep Learning Algorithms with Demographic Information Help to Detect Tuberculosis in Chest Radiographs in Annual Workers' Health Examination Data, Int J Environ Res Public Health, 16
  50. EJ Hwang, 2018, Development and Validation of a Deep Learning-Based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs, Clin Infect Dis
  51. S Rajaraman, 2018, A novel stacked generalization of models for improved TB detection in chest radiographs, Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual Conference, 2018, 718
  52. KC Santosh, 2018, Automated Chest X-Ray Screening: Can Lung Region Symmetry Help Detect Pulmonary Abnormalities?, IEEE Trans Med Imaging, 37, 1168, 10.1109/TMI.2017.2775636 · DOI
  53. R Sivaramakrishnan, 2018, Comparing deep learning models for population screening using chest radiography, Medical Imaging
  54. S Vajda, 2018, Feature Selection for Automatic Tuberculosis Screening in Frontal Chest Radiographs, J Med Syst, 42, 146, 10.1007/s10916-018-0991-9 · DOI · Paper page
  55. MT Rahman, 2017, An evaluation of automated chest radiography reading software for tuberculosis screening among public- and private-sector patients, Eur Respir J, 49, 10.1183/13993003.02159-2016 · DOI · Paper page
  56. M Muyoyeta, 2017, Digital CXR with computer aided diagnosis versus symptom screen to define presumptive tuberculosis among household contacts and impact on tuberculosis diagnosis, BMC Infectious Diseases, 17, 301, 10.1186/s12879-017-2388-7 · DOI
  57. RC Koesoemadinata, 2018, Computer-assisted chest radiography reading for tuberculosis screening in people living with diabetes mellitus, The international journal of tuberculosis and lung disease: the official journal of the International Union against Tuberculosis and Lung Disease, 22, 1088, 10.5588/ijtld.17.0827 · DOI · Paper page
  58. M Muyoyeta, 2017, Digital CXR with computer aided diagnosis versus symptom screen to define presumptive tuberculosis among household contacts and impact on tuberculosis diagnosis, BMC Infect Dis, 17, 301, 10.1186/s12879-017-2388-7 · DOI
  59. MT Rahman, 2017, An evaluation of automated chest radiography reading software for tuberculosis screening among public- and private-sector patients, European Respiratory Journal, 49, 10.1183/13993003.02159-2016 · DOI · Paper page
  60. FE HARRELL, 1996, MULTIVARIABLE PROGNOSTIC MODELS: ISSUES IN DEVELOPING MODELS, EVALUATING ASSUMPTIONS AND ADEQUACY, AND MEASURING AND REDUCING ERRORS, Stat Med, 15, 361, 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4 · doi:10.1002/(sici)1097-0258(19960229)15:4<361::aid-sim168>3.0.co;2-4

Cited by

See all 177

Metadata

Authors
Harris, M., Qi, A., Jeagal, L., Torabi, N., Menzies, D., Korobitsyn, A., Pai, M., Nathavitharana, R. R., & Ahmad Khan, F.
Venue
PLOS ONE
Published
2019-09-03
Volume
14
Issue
9
Pages
e0221339
ISSN
1932-6203 (ISSN-L)
OpenAlex
W2972112160
Cited by
177
References
11
FWCI
4.91
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis. https://app.cochat.ai/papers/10.1371/journal.pone.0221339
The paper
Harris, M., Qi, A., Jeagal, L., Torabi, N., Menzies, D., Korobitsyn, A., Pai, M., Nathavitharana, R. R., & Ahmad Khan, F. (2019). A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis. PLOS ONE. https://doi.org/10.1371/journal.pone.0221339

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.