The DOI matches its Crossref record; the references it cites are not checked yet.
Performance of Qure.ai automatic classifiers against a large annotated database of patients with diverse forms of tuberculosis
Open accessAbstract
Availability of trained radiologists for fast processing of CXRs in regions burdened with tuberculosis always has been a challenge, affecting both timely diagnosis and patient monitoring. The paucity of annotated images of lungs of TB patients hampers attempts to apply data-oriented algorithms for research and clinical practices. The TB Portals Program database (TBPP, https://TBPortals.niaid.nih.gov) is a global collaboration curating a large collection of the most dangerous, hard-to-cure drug-resistant tuberculosis (DR-TB) patient cases. TBPP, with 1,179 (83%) DR-TB patient cases, is a unique collection that is well positioned as a testing ground for deep learning classifiers. As of January 2019, the TBPP database contains 1,538 CXRs, of which 346 (22.5%) are annotated by a radiologist and 104 (6.7%) by a pulmonologist-leaving 1,088 (70.7%) CXRs without annotations. The Qure.ai qXR artificial intelligence automated CXR interpretation tool, was blind-tested on the 346 radiologist-annotated CXRs from the TBPP database. Qure.ai qXR CXR predictions for cavity, nodule, pleural effusion, hilar lymphadenopathy was successfully matching human expert annotations. In addition, we tested the 12 Qure.ai classifiers to find whether they correlate with treatment success (information provided by treating physicians). Ten descriptors were found as significant: abnormal CXR (p = 0.0005), pleural effusion (p = 0.048), nodule (p = 0.0004), hilar lymphadenopathy (p = 0.0038), cavity (p = 0.0002), opacity (p = 0.0006), atelectasis (p = 0.0074), consolidation (p = 0.0004), indicator of TB disease (p = < .0001), and fibrosis (p = < .0001). We conclude that applying fully automated Qure.ai CXR analysis tool is useful for fast, accurate, uniform, large-scale CXR annotation assistance, as it performed well even for DR-TB cases that were not used for initial training. Testing artificial intelligence algorithms (encapsulating both machine learning and deep learning classifiers) on diverse data collections, such as TBPP, is critically important toward progressing to clinically adopted automatic assistants for medical data analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
The citation lists are queued
They are fetched later and appear here then.
References 26 · Cited by 37
Research works printed but not linked: 20
- KR Steingart, 2006, Fluorescence versus conventional sputum smear microscopy for tuberculosis: a systematic review, Lancet Infect Dis, 6, 570, 10.1016/S1473-3099(06)70578-3 · DOI
- E Dorman S, 2017, Xpert MTB/RIF Ultra for detection of Mycobacterium tuberculosis and rifampicin resistance: a prospective multicentre diagnostic accuracy study, Lancet Infect Dis
- Springer, 2012, Chest Imaging: An Algorithmic Approach to Learning
- D Pedrazzoli, 2016, Can tuberculosis patients in resource-constrained settings afford chest radiography?, European Respiratory Journal
- ESR European Society of Radiology (ESR), and American College of Radiology (ACR), 2016, “European Society of Radiology (ESR) and American College of Radiology (ACR) Report of the 2015 Global Summit on Radiological Quality and Safety., Insights into Imaging, 7.4, 481, 10.1007/s13244-016-0493-6 · DOI
- German Henostroza, 2016, Chest Radiograph Reading and Recording System: Evaluation in Frontline Clinicians in Zambia, BMC Infectious Diseases, 16, 136, 10.1186/s12879-016-1460-z · DOI
- DeVries G. “Outreach programme of tuberculosis screening and strengthening care integration in Romania” In: 49th Union World Conference Abstract Book, October 2018, The Hague.
- Zaidi S., Adam A., Azeemi KS., Habib SS., Madhani F., Safdar N., et al. “Identification of potential TB hot-spots through a mobile X-ray supported community-based mass screening program in Karachi, Pakistan” In: 49th Union World Conference Abstract Book, October 2018, The Hague.
- S Jaeger, 2014, Automatic Tuberculosis Screening Using Chest Radiographs, IEEE Transactions on Medical Imaging, 33, 233, 10.1109/TMI.2013.2284099 · DOI
- SJ Heo, 2019, Deep Learning Algorithms with Demographic Information Help to Detect Tuberculosis in Chest Radiographs in Annual Workers' Health Examination Data, Int J Environ Res Public Health
- EJ Hwang, 2018, Development and Validation of a Deep Learning-Based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs, Clin Infect Dis
- R Sivaramakrishnan, 2018, Comparing deep learning models for population screening using chest radiography.” Medical Imaging 2018: Computer-Aided Diagnosis, International Society for Optics and Photonics, 10575
- P Putha, 2018, Efficacy of deep learning for screening pulmonary tuberculosis”, ESR 2018 Book of Abstracts, Insights Imaging, 9, 1
- P Putha, 2018, Can Artificial Intelligence Reliably Report Chest X-Rays?: Radiologist Validation of an Algorithm trained on 1.2 Million X-Rays, arXiv:1807.07455 · arXiv
- X Wang, 2017, ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases, IEEE CVPR
- A Rosenthal, 2017, The TB Portals: An open-access, web-based platform for global drug-resistant tuberculosis data sharing and analysis, J Clin Microbiol
- S Ravimohan, 2018, Tuberculosis and lung damage: from epidemiology to pathophysiology, European Respiratory Review
- Liauchuk V, Kovalev V, Kalinovsky A, Tarasau A, Gabrielian A, Rosenthal A. Examining the ability of convolutional neural networks to detect lesions in lung CT images, Proceedings of International Congress on Computer Assisted Radiology and Surgery. 2017.
- F Pasa, 2019, Efficient Deep Network Architectures for Fast Chest X-Ray Tuberculosis Screening and Visualization, Sci Rep
- M Topalovic, 2019, Artificial intelligence outperforms pulmonologists in the interpretation of pulmonary function tests, Eur Respir J
Metadata
- Authors
- Engle, E., Gabrielian, A., Long, A., Hurt, D. E., & Rosenthal, A.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Engle, Eric
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-01-24
- Volume
- 15
- Issue
- 1
- Pages
- e0224445
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC6980594
- OpenAlex
- W3000977647
- Cited by
- 37
- References
- 26
- Citations by year
- 2026: 5 · 2025: 3 · 2024: 8 · 2023: 7 · 2022: 8 · 2021: 4 · 2020: 2
- FWCI
- 2.56
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- creativecommons.org/publicdomain/zero/1.0/ (vor)
- Locations
- PLoS ONE (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Office of Information and Resource Management (HHSN316201300006W/HHSN27200002)
- Civilian Research and Development Foundation (BOB1-31120-MK-13)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Medicine › Infectious Diseases › Tuberculosis Research and Epidemiology
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- tuberculosis, drug-resistant tuberculosis, chest X-ray interpretation, automatic image annotation, deep learning classifier, pleural effusion, hilar lymphadenopathy, pulmonary nodules, pulmonary cavities, treatment response prediction, atelectasis, fibrosis, lung consolidation, opacity, pulmonary fibrosis
- MeSH terms
- Radiologists, Deep Learning, Data Management, Algorithms, Artificial Intelligence, Female, Humans, Lung, Male, Pleural Effusion, Radiography, Thoracic, Tuberculosis, Databases, Factual, Extensively Drug-Resistant Tuberculosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Performance of Qure.ai automatic classifiers against a large annotated database of patients with diverse forms of tuberculosis. https://app.cochat.ai/papers/10.1371/journal.pone.0224445
- The paper
- Engle, E., Gabrielian, A., Long, A., Hurt, D. E., & Rosenthal, A. (2020). Performance of Qure.ai automatic classifiers against a large annotated database of patients with diverse forms of tuberculosis. PLOS ONE. https://doi.org/10.1371/journal.pone.0224445
