The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Automated Triaging of Adult Chest Radiographs with Deep Artificial Neural Networks
Open accessAbstract
Purpose To develop and test an artificial intelligence (AI) system, based on deep convolutional neural networks (CNNs), for automated real-time triaging of adult chest radiographs on the basis of the urgency of imaging appearances. Materials and Methods An AI system was developed by using 470 388 fully anonymized institutional adult chest radiographs acquired from 2007 to 2017. The free-text radiology reports were preprocessed by using an in-house natural language processing (NLP) system modeling radiologic language. The NLP system analyzed the free-text report to prioritize each radiograph as critical, urgent, nonurgent, or normal. An AI system for computer vision using an ensemble of two deep CNNs was then trained by using labeled radiographs to predict the clinical priority from radiologic appearances only. The system's performance in radiograph prioritization was tested in a simulation by using an independent set of 15 887 radiographs. Prediction performance was assessed with the area under the receiver operating characteristic curve; sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were also determined. Nonparametric testing of the improvement in time to final report was determined at a nominal significance level of 5%. Results Normal chest radiographs were detected by our AI system with a sensitivity of 71%, specificity of 95%, PPV of 73%, and NPV of 94%. The average reporting delay was reduced from 11.2 to 2.7 days for critical imaging findings (P < .001) and from 7.6 to 4.1 days for urgent imaging findings (P < .001) in the simulation compared with historical data. Conclusion Automated real-time triaging of adult chest radiographs with use of an artificial intelligence system is feasible, with clinically acceptable performance. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Auffermann in this issue.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Only eight studies in ophthalmology imaging14,21,32,33,43,55,108,109, ten studies in respiratory imaging64,66,70,72,75,79,82,87,89,110 and six studies in breast imaging62,91,97,104,106,111 mentioned adherence to the STARD-2015 guidelines or had a STARD flow diagram in the manuscript.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [110]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Lakhani, P., & Sundaram, B. (2017) · Radiology · doi:10.1148/radiol.2017162326 · Cited by 1,775
CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D. Y., Bagul, A., Langlotz, C. P., Shpanskaya, K. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05225 · Cited by 1,219
Thrall, J. H., Li, X., Li, Q., Cruz, C., Do, S., Dreyer, K., & Brink, J. (2018) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2017.12.026 · Cited by 710
Cognitive and System Factors Contributing to Diagnostic Errors in Radiology
Lee, C. S., Nagy, P. G., Weaver, S. J., & Newman-Toker, D. E. (2013) · American Journal of Roentgenology · doi:10.2214/ajr.12.10375 · Cited by 415
Towards automatic pulmonary nodule management in lung cancer screening with deep learning
Ciompi, F., Chung, K., van Riel, S. J., Setio, A. A. A., Gerke, P. K., Jacobs, C., Scholten, E. T., Schaefer-Prokop, C., Wille, M. M. W., Marchianò, A., et al. (2017) · Scientific Reports · doi:10.1038/srep46479 · Cited by 304
Cicero, M., Bilbily, A., Colak, E., Dowdell, T., Gray, B., Perampaladas, K., & Barfett, J. (2017) · Investigative Radiology · doi:10.1097/rli.0000000000000341 · Cited by 247
High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks
Geras, K. J., Wolfson, S., Shen, Y., Wu, N., Kim, S.-G., Kim, E. S., Heacock, L. C., Parikh, U., Moy, L. P., & Cho, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07047 · Cited by 185
Research works printed but not linked: 6
- Royal College of Radiologists, 2017, Clinical radiology UK workforce census 2016 report. Contract no.: BFCR(17)6
- Clinical Excellence Commission, 2014, Final report: recommendations of the Clinical Advisory Committee—plain x-ray image reporting backlog
- Royal College of Radiologists, 2015, Unreported x-rays, computed tomography (CT) and magnetic resonance imaging (MRI) scans: results of a snapshot survey of English National Health Service (NHS) trusts
- World Health Organization, Communicating radiation risks in paediatric imaging: information to support healthcare discussions about benefit and risk
- Cornegruta S, Bakewell R, Withey S, Montana G. Modelling radiological language with bidirectional long short-term memory networks. Cornell University Library. https://arxiv.org/abs/1609.08409. Accessed May 20, 2018. · arXiv
- Pesce E, Ypsilantis PP, Withey S, Bakewell R, Goh V, Montana G. Learning to detect chest radiographs containing lung nodules using visual attention networks. Cornell University Library. https://arxiv.org/abs/1712.00996. Accessed May 20, 2018. · arXiv
Cited by
Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946
Redefining Radiology: A Review of Artificial Intelligence Integration in Medical Imaging
Najjar, R. (2023) · Diagnostics · doi:10.3390/diagnostics13172760 · Cited by 886
Sit, C., Srinivasan, R., Amlani, A., Muthuswamy, K., Azam, A., Monzon, L., & Poon, D. S. (2020) · Insights into Imaging · doi:10.1186/s13244-019-0830-7 · Cited by 507
A deep learning approach to detect Covid-19 coronavirus with X-Ray images
Jain, G., Mittal, D., Thakur, D., & Mittal, M. K. (2020) · Biocybernetics and Biomedical Engineering · doi:10.1016/j.bbe.2020.08.008 · Cited by 324
The Effect of Image Resolution on Deep Learning in Radiography
Sabottke, C. F., & Spieler, B. M. (2020) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2019190015 · Cited by 312
“Figure 2 shows the effects of varying the image resolution on an AUC for six distinct diagnosis labels: emphysema, The Effect of Image Resolution on Deep Learning in Radiography or if artificial intelligence can help prioritize the reading order for a study list in a triage scenario (15).”
Background
Metadata
- Authors
- Annarumma, M., Withey, S. J., Bakewell, R. J., Pesce, E., Goh, V., & Montana, G.
- Affiliations
- King's College London (GB)
- Venue
- Radiology
- Published
- 2019-04
- Volume
- 291
- Issue
- 1
- Pages
- 196-202
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- PMC
- PMC6438359
- OpenAlex
- W2911410812
- Cited by
- 303
- References
- 14
- Citations by year
- 2026: 42 · 2025: 34 · 2024: 32 · 2023: 38 · 2022: 34 · 2021: 47 · 2020: 57 · 2019: 19
- FWCI
- 12.56
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Radiology (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Research Portal (King's College London) (submittedVersion, cc-by-nc-nd)
- Warwick Research Archive Portal (University of Warwick) (submittedVersion)
- Funders
- King’s Health Partners
- Guy's & St Thomas' NHS Foundation Trust
- King's College London
- Cancer Research UK
- Medical Research Council (C1519/A16463)
- Wellcome (WT 203148/Z/16/Z)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- chest radiograph, deep convolutional neural networks, delayed reporting
- MeSH terms
- Deep Learning, Adult, Artificial Intelligence, Humans, Retrospective Studies, ROC Curve, Sensitivity and Specificity, Radiography, Thoracic, Triage, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automated Triaging of Adult Chest Radiographs with Deep Artificial Neural Networks. https://app.cochat.ai/papers/10.1148/radiol.2018180921
- The paper
- Annarumma, M., Withey, S. J., Bakewell, R. J., Pesce, E., Goh, V., & Montana, G. (2019). Automated Triaging of Adult Chest Radiographs with Deep Artificial Neural Networks. Radiology. https://doi.org/10.1148/radiol.2018180921
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.