The Added Effect of Artificial Intelligence on Physicians’ Performance in Detecting Thoracic Pathologies on CT and Chest X-ray: A Systematic Review
Open accessThe DOI matches its Crossref record; 60 of the 62 references it cites were found in a registry or an index.
Abstract
Our systematic review investigated the additional effect of artificial intelligence-based devices on human observers when diagnosing and/or detecting thoracic pathologies using different diagnostic imaging modalities, such as chest X-ray and CT. Peer-reviewed, original research articles from EMBASE, PubMed, Cochrane library, SCOPUS, and Web of Science were retrieved. Included articles were published within the last 20 years and used a device based on artificial intelligence (AI) technology to detect or diagnose pulmonary findings. The AI-based device had to be used in an observer test where the performance of human observers with and without addition of the device was measured as sensitivity, specificity, accuracy, AUC, or time spent on image reading. A total of 38 studies were included for final assessment. The quality assessment tool for diagnostic accuracy studies (QUADAS-2) was used for bias assessment. The average sensitivity increased from 67.8% to 74.6%; specificity from 82.2% to 85.4%; accuracy from 75.4% to 81.7%; and Area Under the ROC Curve (AUC) from 0.75 to 0.80. Generally, a faster reading time was reported when radiologists were aided by AI-based devices. Our systematic review showed that performance generally improved for the physicians when assisted by AI-based devices compared to unaided interpretation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Zhang, D., Liu, X., Shao, M., et al. (2021) · Personal and Ubiquitous Computing · doi:10.1007/s00779-021-01522-7 · Cited by 13
RetractedQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Nagendran, M., Chen, Y., Lovejoy, C. A., Gordon, A. C., Komorowski, M., Harvey, H., Topol, E. J., Ioannidis, J. P. A., Collins, G. S., & Maruthappu, M. (2020) · BMJ · doi:10.1136/bmj.m689 · Cited by 1,152 · cited 1×
“No clinical trials have been published using AI-based devices on thoracic CT or chest X-rays, whether it be as a stand-alone diagnostic tool or as an additional reader to humans [61].”
BackgroundPreparing Medical Imaging Data for Machine Learning
Willemink, M. J., Koszek, W. A., Hardell, C., Wu, J., Fleischmann, D., Harvey, H., Folio, L. R., Summers, R. M., Rubin, D. L., & Lungren, M. P. (2020) · Radiology · doi:10.1148/radiol.2020192224 · Cited by 1,020 · cited 9×
“To be able to include a larger number of studies, we allowed the possibility of some inter-study variability in the performance of the AI-based devices because of different AI-algorithms being used.”
BackgroundInfluentialDo as AI say: susceptibility in deployment of clinical decision-aids
Gaube, S., Suresh, H., Raue, M., Merritt, A., Berkowitz, S. J., Lermer, E., Coughlin, J. F., Guttag, J. V., Colak, E., & Ghassemi, M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00385-9 · Cited by 543 · cited 2×
“[51] suggested that physicians with less experience were more likely to accept and deploy the suggested advice given to them by AI.”
BackgroundNam, J. G., Park, S., Hwang, E. J., Lee, J. H., Jin, K.-N., Lim, K. Y., Vu, T. H., Sohn, J. H., Hwang, S., Goo, J. M., et al. (2019) · Radiology · doi:10.1148/radiol.2018180237 · Cited by 539
Hwang, E. J., Park, S., Jin, K.-N., Kim, J. I., Choi, S. Y., Lee, J. H., Goo, J. M., Aum, J., Yim, J.-J., Cohen, J. G., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.1095 · Cited by 447
Wang, P., Liu, X., Berzin, T. M., Glissen Brown, J. R., Liu, P., Zhou, C., Lei, L., Li, L., Guo, Z., Lei, S., et al. (2020) · The Lancet Gastroenterology & Hepatology · doi:10.1016/s2468-1253(19)30411-x · Cited by 415 · cited 1×
“However, few prospective clinical trials have been published where AI-based devices have been used, in a more dynamic and clinically realistic environment [59,60].”
BackgroundStrohm, L., Hehakaya, C., Ranschaert, E. R., Boon, W. P. C., & Moors, E. H. M. (2020) · European Radiology · doi:10.1007/s00330-020-06946-y · Cited by 329
Non-invasive decision support for NSCLC treatment using PET/CT radiomics
Mu, W., Jiang, L., Zhang, J., Shi, Y., Gray, J. E., Tunali, I., Gao, C., Sun, Y., Tian, J., Zhao, X., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-19116-x · Cited by 289
Cited by
Using AI to Improve Radiologist Performance in Detection of Abnormalities on Chest Radiographs
Bennani, S., Regnard, N.-E., Ventre, J., Lassalle, L., Nguyen, T., Ducarouge, A., Dargent, L., Guillo, E., Gouhier, E., Zaimi, S.-H., et al. (2023) · Radiology · doi:10.1148/radiol.230860 · Cited by 120
Non-task expert physicians benefit from correct explainable AI advice when reviewing X-rays
Gaube, S., Suresh, H., Raue, M., Lermer, E., Koch, T. K., Hudecek, M. F. C., Ackery, A. D., Grover, S. C., Coughlin, J. F., Frey, D., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-28633-w · Cited by 92
Plesner, L. L., Müller, F. C., Brejnebøl, M. W., Laustrup, L. C., Rasmussen, F., Nielsen, O. W., Boesen, M., & Andersen, M. B. (2023) · Radiology · doi:10.1148/radiol.231236 · Cited by 87
Geppert, J., Asgharzadeh, A., Brown, A., Stinton, C., Helm, E. J., Jayakody, S., Todkill, D., Gallacher, D., Ghiasvand, H., Patel, M., et al. (2024) · Thorax · doi:10.1136/thorax-2024-221662 · Cited by 76
Chest X-ray in Emergency Radiology: What Artificial Intelligence Applications Are Available?
Irmici, G., Cè, M., Caloro, E., Khenkina, N., Pepa, G. D., Ascenti, V., Martinenghi, C. M. A., Papa, S., Oliva, G., & Cellina, M. I. (2023) · Diagnostics · doi:10.3390/diagnostics13020216 · Cited by 74
Metadata
- Authors
- Li, D., Pehrson, L. M., Lauridsen, C. A., Tøttrup, L., Fraccaro, M., Elliott, D., Zając, H. D., Darkner, S., Carlsen, J. F., & Nielsen, M. B.
- Affiliations
- University of Copenhagen (DK)
- Copenhagen University Hospital (DK)
- Unumed Aps, 1055 Copenhagen, Denmark
- Corresponding author
- Li, Dana
- ORCID
- Venue
- Diagnostics
- Publisher
- MDPI AG
- Published
- 2021-11-26
- Volume
- 11
- Issue
- 12
- Pages
- 2206
- ISSN
- 2075-4418 (ISSN-L)
- PMC
- PMC8700414
- OpenAlex
- W3215796869
- Cited by
- 35
- References
- 62
- Citations by year
- 2026: 2 · 2025: 9 · 2024: 8 · 2023: 12 · 2022: 4
- FWCI
- 0.49
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Diagnostics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research at the University of Copenhagen (University of Copenhagen) (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Innovation Fund Denmark (0176-00013B)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Keywords
- artificial intelligence, chest radiography, computed tomography, radiologist performance, diagnostic accuracy, AI-assisted diagnosis, sensitivity and specificity, lung imaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.