Artificial Intelligence Tool for Detection and Worklist Prioritization Reduces Time to Diagnosis of Incidental Pulmonary Embolism at CT
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Purpose To evaluate the diagnostic efficacy of artificial intelligence (AI) software in detecting incidental pulmonary embolism (IPE) at CT and shorten the time to diagnosis with use of radiologist reading worklist prioritization. Materials and Methods In this study with historical controls and prospective evaluation, regulatory-cleared AI software was evaluated to prioritize IPE on routine chest CT scans with intravenous contrast agent in adult oncology patients. Diagnostic accuracy metrics were calculated, and temporal end points, including detection and notification times (DNTs), were assessed during three time periods (April 2019 to September 2020): routine workflow without AI, human triage without AI, and worklist prioritization with AI. Results In total, 11 736 CT scans in 6447 oncology patients (mean age, 63 years ± 12 [SD]; 3367 men) were included. Prevalence of IPE was 1.3% (51 of 3837 scans), 1.4% (54 of 3920 scans), and 1.0% (38 of 3979 scans) for the respective time periods. The AI software detected 131 true-positive, 12 false-negative, 31 false-positive, and 11 559 true-negative results, achieving 91.6% sensitivity, 99.7% specificity, 99.9% negative predictive value, and 80.9% positive predictive value. During prospective evaluation, AI-based worklist prioritization reduced the median DNT for IPE-positive examinations to 87 minutes (vs routine workflow of 7714 minutes and human triage of 4973 minutes). Radiologists’ missed rate of IPE was significantly reduced from 44.8% (47 of 105 scans) without AI to 2.6% (one of 38 scans) when assisted by the AI tool (P < .001). Conclusion AI-assisted workflow prioritization of IPE on routine CT scans in oncology patients showed high diagnostic accuracy and significantly shortened the time to diagnosis in a setting with a backlog of examinations. Keywords: CT, Computer Applications, Detection, Diagnosis, Embolism, Thorax, Thrombosis Supplemental material is available for this article. © RSNA, 2023 See also the commentary by Elicker in this issue.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Diagnostic Error in Medicine: Analysis of 583 Physician-Reported Errors
Schiff, G. D. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.333 · Cited by 638
McDonald, R. J., Schwartz, K. M., Eckel, L. J., Diehn, F. E., Hunt, C. H., Bartholmai, B. J., Erickson, B. J., & Kallmes, D. F. (2015) · Academic Radiology · doi:10.1016/j.acra.2015.05.007 · Cited by 460
Arbabshirani, M. R., Fornwalt, B. K., Mongelluzzo, G. J., Suever, J. D., Geise, B. D., Patel, A. A., & Moore, G. J. (2018) · npj Digital Medicine · doi:10.1038/s41746-017-0015-z · Cited by 443
Automated Triaging of Adult Chest Radiographs with Deep Artificial Neural Networks
Annarumma, M., Withey, S. J., Bakewell, R. J., Pesce, E., Goh, V., & Montana, G. (2019) · Radiology · doi:10.1148/radiol.2018180921 · Cited by 303
Cited by
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
Mastrodicasa, D., van Assen, M., Huisman, M., Leiner, T., Williamson, E. E., Nicol, E. D., Allen, B. D., Saba, L., Vliegenthart, R., & Hanneman, K. (2025) · Radiology · doi:10.1148/radiol.240516 · Cited by 69
Artificial Intelligence in Radiology: Opportunities and Challenges
Flory, M. N., Napel, S., & Tsai, E. B. (2024) · Seminars in Ultrasound, CT and MRI · doi:10.1053/j.sult.2024.02.004 · Cited by 29
Metadata
- Authors
- Topff, L., Ranschaert, E. R., Bartels-Rutten, A., Negoita, A., Menezes, R., Beets-Tan, R. G. H., & Visser, J. J.
- Venue
- Radiology: Cardiothoracic Imaging
- Published
- 2023-04-01
- Volume
- 5
- Issue
- 2
- Pages
- e220163-e220163
- ISSN
- 2638-6135 (ISSN-L)
- OpenAlex
- W4366599061
- Cited by
- 81
- References
- 4
- FWCI
- 2.80
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.