Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Deep learning (DL) neural networks have only recently been employed to interpret chest radiography (CXR) to screen and triage people for pulmonary tuberculosis (TB). No published studies have compared multiple DL systems and populations. We conducted a retrospective evaluation of three DL systems (CAD4TB, Lunit INSIGHT, and qXR) for detecting TB-associated abnormalities in chest radiographs from outpatients in Nepal and Cameroon. All 1196 individuals received a Xpert MTB/RIF assay and a CXR read by two groups of radiologists and the DL systems. Xpert was used as the reference standard. The area under the curve of the three systems was similar: Lunit (0.94, 95% CI: 0.93–0.96), qXR (0.94, 95% CI: 0.92–0.97) and CAD4TB (0.92, 95% CI: 0.90–0.95). When matching the sensitivity of the radiologists, the specificities of the DL systems were significantly higher except for one. Using DL systems to read CXRs could reduce the number of Xpert MTB/RIF tests needed by 66% while maintaining sensitivity at 95% or better. Using a universal cutoff score resulted different performance in each site, highlighting the need to select scores based on the population screened. These DL systems should be considered by TB programs where human resources are constrained, and automated technology is available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some developers recommend a threshold score for universal use, above which an individual would be classified as having abnormalities suggestive of tuberculosis [6, 7]. Other developers, such as CAD4TB (Delft Imaging Systems, Netherlands), recommend scores tailored to each setting due to variation in CAD abnormality scores between different software and populations [5, 6, 8–13]”
From Computer-aided detection thresholds for digital chest radiography interpretation in tuberculosis diagnostic algorithms. (2024) · cites this paper as [10]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Twenty-one54,63–67,70,72,76–88 studies compared algorithm performance against healthcare professionals.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [70]
The cited source says what the claim attributes to it.
In the citing paper
“13 (refs. 63–75) studies validated algorithms on external data”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [70]
In the citing paper
“Only eight studies in ophthalmology imaging14,21,32,33,43,55,108,109, ten studies in respiratory imaging64,66,70,72,75,79,82,87,89,110 and six studies in breast imaging62,91,97,104,106,111 mentioned adherence to the STARD-2015 guidelines or had a STARD flow diagram in the manuscript.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [70]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
(2016) · Cited by 11,679
Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative.
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00010 · Cited by 3,219
Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · BMJ · doi:10.1136/bmj.326.7379.41 · Cited by 3,215
Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · Radiology · doi:10.1148/radiol.2261021292 · Cited by 3,215
The International Journal of Tuberculosis and Lung Disease
Brown III, J. W. (1998) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.280.13.1200-a · Cited by 1,414
Sample size estimation in diagnostic test studies of biomedical informatics
Hajian-Tilaki, K. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.02.013 · Cited by 995 · cited 1×
“The areas under the curve (AUCs) were calculated as the primary index of accuracy of DL systems 22 .”
MethodMarušić, M., & Marušić, A. (2014) · Palgrave Macmillan UK eBooks · doi:10.1057/9781137351197_12 · Cited by 507
MaruÅ¡iÄ, M., & MaruÅ¡iÄ, A. (2014) · The Semiperiphery of Academic Writing · doi:10.1057/9781137351197.0020 · Cited by 498
Albert, H., Nathavitharana, R. R., Isaacs, C., Pai, M., Denkinger, C. M., & Boehme, C. C. (2016) · European Respiratory Journal · doi:10.1183/13993003.00543-2016 · Cited by 300 · cited 1×
“Implementers considering using DL systems for CXR reading should take into account other factors including service, ease of use, maintenance and price - all important considerations in any new technology implementation 26 .”
Method
Research works printed but not linked: 1
- Krizhevsky, A., Sutskever, I. & Hinton, G. E. In Advances in neural information processing systems. 1097–1105.
Cited by
Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946
Development and evaluation of an artificial intelligence system for COVID-19 diagnosis
Jin, C., Chen, W., Cao, Y., Xu, Z., Tan, Z., Zhang, X., Deng, L., Zheng, C., Zhou, J., Shi, H., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-18685-1 · Cited by 592
Deep learning for chest X-ray analysis: A survey
Çallı, E., Sogancioglu, E., van Ginneken, B., van Leeuwen, K. G., & Murphy, K. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102125 · Cited by 506
Frascella, B., Richards, A. S., Sossen, B., Emery, J. C., Odone, A., Law, I., Onozaki, I., Esmail, H., & Houben, R. M. G. J. (2021) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa1402 · Cited by 315
How to improve access to medical imaging in low- and middle-income countries
Frija, G., Blažić, I., Frush, D. P., Hierath, M., Kawooya, M., Donoso-Bach, L., & Brkljačić, B. (2021) · eClinicalMedicine · doi:10.1016/j.eclinm.2021.101034 · Cited by 310
“Also, free access to AI solutions for high-priority diseases, such as tuberculosis [26,27], should be promoted.”
Metadata
- Authors
- Qin, Z. Z., Sander, M. S., Rai, B., Titahong, C. N., Sudrungrot, S., Laah, S. N., Adhikari, L. M., Carter, E. J., Puri, L., Codlin, A. J., & Creswell, J.
- Affiliations
- University of Bamenda (CM)
- International Organization for Migration (CH)
- Brown University (US)
- Corresponding author
- Creswell, Jacob
- Venue
- Scientific Reports
- Published
- 2019-10-18
- Volume
- 9
- Issue
- 1
- Pages
- 15000-15000
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC6802077
- OpenAlex
- W2980965120
- Cited by
- 289
- References
- 31
- Citations by year
- 2026: 31 · 2025: 35 · 2024: 42 · 2023: 47 · 2022: 41 · 2021: 60 · 2020: 33
- FWCI
- 22.12
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- United States Agency for International Development
- Bill and Melinda Gates Foundation
- National Philanthropic Trust
- Government of Canada
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Medicine › Infectious Diseases › Tuberculosis Research and Epidemiology
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- pulmonary tuberculosis, chest X-ray interpretation, deep learning, computer-aided diagnosis, tuberculosis screening, diagnostic accuracy, CAD4TB, Xpert MTB/RIF, radiologist comparison, automated radiology, resource-limited settings, Nepal, Cameroon
- MeSH terms
- Data Accuracy, Deep Learning, Adult, Cameroon, DNA, Bacterial, Female, Humans, Male, Mass Screening, Middle Aged, Mycobacterium tuberculosis, Nepal, Retrospective Studies, Sensitivity and Specificity, Radiography, Thoracic, Triage, Tuberculosis, Pulmonary, Area Under Curve, Nucleic Acid Amplification Techniques
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems. https://app.cochat.ai/papers/10.1038/s41598-019-51503-3
- The paper
- Qin, Z. Z., Sander, M. S., Rai, B., Titahong, C. N., Sudrungrot, S., Laah, S. N., Adhikari, L. M., Carter, E. J., Puri, L., Codlin, A. J., & Creswell, J. (2019). Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems. Scientific Reports. https://doi.org/10.1038/s41598-019-51503-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.