Deep Learning Algorithm Trained with COVID-19 Pneumonia Also Identifies Immune Checkpoint Inhibitor Therapy-Related Pneumonitis
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Abstract
Background: Coronavirus disease 2019 (COVID-19) pneumonia and immune checkpoint inhibitor (ICI) therapy-related pneumonitis share common features. The aim of this study was to determine on chest computed tomography (CT) images whether a deep convolutional neural network algorithm is able to solve the challenge of differential diagnosis between COVID-19 pneumonia and ICI therapy-related pneumonitis. Methods: We enrolled three groups: a pneumonia-free group (n = 30), a COVID-19 group (n = 34), and a group of patients with ICI therapy-related pneumonitis (n = 21). Computed tomography images were analyzed with an artificial intelligence (AI) algorithm based on a deep convolutional neural network structure. Statistical analysis included the Mann–Whitney U test (significance threshold at p < 0.05) and the receiver operating characteristic curve (ROC curve). Results: The algorithm showed low specificity in distinguishing COVID-19 from ICI therapy-related pneumonitis (sensitivity 97.1%, specificity 14.3%, area under the curve (AUC) = 0.62). ICI therapy-related pneumonitis was identified by the AI when compared to pneumonia-free controls (sensitivity = 85.7%, specificity 100%, AUC = 0.97). Conclusions: The deep learning algorithm is not able to distinguish between COVID-19 pneumonia and ICI therapy-related pneumonitis. Awareness must be increased among clinicians about imaging similarities between COVID-19 and ICI therapy-related pneumonitis. ICI therapy-related pneumonitis can be applied as a challenge population for cross-validation to test the robustness of AI models used to analyze interstitial pneumonias of variable etiology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“CNN has become the primary tool for automatic COVID-19 diagnosis in many hospitals around the world using chest X-ray images [256–260].”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [260]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Coronaviridae Study Group of the International Committee on Taxonomy of Viruses, Gorbalenya, A. E., Baker, S. C., Baric, R. S., de Groot, R. J., Drosten, C., Gulyaeva, A. A., Haagmans, B. L., Lauber, C., Leontovich, A. M., et al. (2020) · Nature Microbiology · doi:10.1038/s41564-020-0695-z · Cited by 8,403
Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning
Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C. S., Liang, H., Baxter, S. L., McKeown, A., Yang, G., Wu, X., Yan, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.02.010 · Cited by 4,842
Coronaviruses: An Overview of Their Replication and Pathogenesis
Fehr, A. R., & Perlman, S. (2015) · Methods in Molecular Biology · doi:10.1007/978-1-4939-2438-7_1 · Cited by 3,787 · cited 1×
“Coronaviruses are composed of four major structural proteins, including the envelope (E) protein, the membrane (M) protein, the nucleocapsid (N) protein, and the spike (S) protein, the latter being of paramount importance since it mediates viral attachment to the host cell membrane receptor [4].”
BackgroundCT Imaging Features of 2019 Novel Coronavirus (2019-nCoV)
Chung, M., Bernheim, A., Mei, X., Zhang, N., Huang, M., Zeng, X., Cui, J., Xu, W., Yang, Y., Fayad, Z. A., et al. (2020) · Radiology · doi:10.1148/radiol.2020200230 · Cited by 2,751
Covid-19 — Navigating the Uncharted
Fauci, A. S., Lane, H. C., & Redfield, R. R. (2020) · New England Journal of Medicine · doi:10.1056/nejme2002387 · Cited by 1,969 · cited 1×
“SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) is a novel coronavirus that was first identified in Wuhan, China, and that is responsible for a highly contagious respiratory disease named coronavirus disease 2019 (COVID-19) [1,2].”
BackgroundLi, L., Qin, L., Xu, Z., Yin, Y., Wang, X., Kong, B., Bai, J., Lu, Y., Fang, Z., Song, Q., et al. (2020) · Radiology · doi:10.1148/radiol.2020200905 · Cited by 1,927 · cited 3×
“Artificial intelligence (AI), using deep learning technology, is highly promising in the medical imaging field due to its capability of feature extraction and analysis [9,21,22].”
BackgroundResultGorbalenya, A. E., Baker, S. C., Baric, R. S., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.02.07.937862 · Cited by 1,408 · cited 1×
“SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) is a novel coronavirus that was first identified in Wuhan, China, and that is responsible for a highly contagious respiratory disease named coronavirus disease 2019 (COVID-19) [1,2].”
BackgroundLung pattern classification for interstitial lung diseases using a deep convolutional neural network
Anthimopoulos, M., Christodoulidis, S., Ebner, L., Christe, A., & Mougiakakou, S. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535865 · Cited by 1,343 · cited 1×
“It has been also applied to detect various imaging features of chest CT [23,24], allowing qualitative and quantitative analysis, which could provide an estimation of the disease burden, facilitating and expediting imaging interpretation [25,26].”
MethodThe Role of Chest Imaging in Patient Management During the COVID-19 Pandemic
Rubin, G. D., Ryerson, C. J., Haramati, L. B., Sverzellati, N., Kanne, J. P., Raoof, S., Schluger, N. W., Volpi, A., Yim, J.-J., Martin, I. B. K., et al. (2020) · Chest · doi:10.1016/j.chest.2020.04.003 · Cited by 1,245 · cited 1×
“Computed tomography (CT) of the chest can detect lung manifestations, which are often associated with COVID-19 [6–8].”
Background
Cited by
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Novel Transfer Learning Approach for Medical Imaging with Limited Labeled Data
Alzubaidi, L., Al-Amidie, M., Al-Asadi, A., Humaidi, A. J., Al-Shamma, O., Fadhel, M. A., Zhang, J., Santamaría, J., & Duan, Y. (2021) · Cancers · doi:10.3390/cancers13071590 · Cited by 292
Radiation-Induced Lung Injury—Current Perspectives and Management
Rahi, M. S., Parekh, J., Pednekar, P., Parmar, G., Abraham, S., Nasir, S., Subramaniyam, R., Jeyashanmugaraja, G. P., & Gunasekaran, K. (2021) · Clinics and Practice · doi:10.3390/clinpract11030056 · Cited by 104
“, a similar deep learning algorithm was unable to distinguish COVID-19 pneumonia and ICI treatment-related pneumonitis [83].”
ResultArtificial intelligence and radiomics: fundamentals, applications, and challenges in immunotherapy
Dercle, L., McGale, J., Sun, S. H., Marabelle, A., Yeh, R., Deutsch, É., Mokrane, F., Farwell, M. D., Ammari, S., Schöder, H. M., et al. (2022) · Journal for ImmunoTherapy of Cancer · doi:10.1136/jitc-2022-005292 · Cited by 109
Mallio, C. A., Sertorio, A. C., Bernetti, C., & Zobel, B. B. (2023) · La radiologia medica · doi:10.1007/s11547-023-01651-4 · Cited by 86
“The organization of information and the standardization of terminology, together with artificial intelligence tools, could allow automated data mining, assist longitudinal data tracking and adjuvate the implementation of imaging biomarkers [4, 5].”
MethodSelf-citationPandya, S., Thakur, A., Saxena, S., Jassal, N., Patel, C. I., Modi, K. J., Shah, P., Joshi, R., Gonge, S. S., Kadam, K., et al. (2021) · Sensors · doi:10.3390/s21237786 · Cited by 68
“[130] COVID-19 Deep Learning deep convolutional neural network sensitivity, specificity, AUC, ROC and Mann–Whitney U test A deep CNN model was applied to CT images of Pneumonia, COVID-19 and ICI pneumonitis to differentiate between the three.”
MethodArtificial intelligence and abdominal adipose tissue analysis: a literature review
Greco, F., & Mallio, C. A. (2021) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims-21-370 · Cited by 55
Self-citationMallio, C. A., Radbruch, A., Deike‐Hofmann, K., van der Molen, A. J., Dekkers, I. A., Zaharchuk, G., Parizel, P. M., Zobel, B. B., & Quattrocchi, C. C. (2023) · Investigative Radiology · doi:10.1097/rli.0000000000000983 · Cited by 53
Self-citationMallio, C. A., Vadalà, G., Russo, F., Bernetti, C., Ambrosio, L., Zobel, B. B., Quattrocchi, C. C., Papalia, R., & Denaro, V. (2022) · Diagnostics · doi:10.3390/diagnostics12020420 · Cited by 42
“Concerning the spine, it was suggested that AI applied to MR images can yield a fast and accurate diagnosis and prognosis prediction regarding spinal diseases [67,68].”
BackgroundSelf-citationGreco, F., Salgado, R. A., Hecke, W. V., BUONO, R. D., Parizel, P. M., & Mallio, C. A. (2022) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims-21-945 · Cited by 48
Self-citationClassification of Alzheimer disease using DenseNet-201 based on deep transfer learning technique
Zia-ur-Rehman, Awang, M. K., Rashid, J., Ali, G., Hamid, M., Mahmoud, S. F., Saleh, D. I., & Ahmad, H. I. (2024) · PLoS ONE · doi:10.1371/journal.pone.0304995 · Cited by 50
Infante, A., Gaudino, S., Orsini, F., del Ciello, A., Gullì, C., Merlino, B., Natale, L., Iezzi, R., & Sala, E. (2023) · Clinical Radiology · doi:10.1016/j.crad.2023.11.011 · Cited by 28
Cheng, M., Lin, R., Bai, N., Zhang, Y., Wang, H., Guo, M.-Z., Duan, X., Zheng, J., Qiu, Z., & Zhao, Y. (2023) · Clinical Radiology · doi:10.1016/j.crad.2022.12.013 · Cited by 31
Adverse Effects of Immune-Checkpoint Inhibitors: A Comprehensive Imaging-Oriented Review
Mallio, C. A., Bernetti, C., Cea, L., Buoso, A., Stiffi, M., Vertulli, D., Greco, F., & Zobel, B. B. (2023) · Current Oncology · doi:10.3390/curroncol30050355 · Cited by 27
Self-citationMallio, C. A., Bernetti, C., Sertorio, A. C., & Zobel, B. B. (2024) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims-23-1300 · Cited by 21
Self-citationThe Role of Network Technologies in the Enhancement of the Health, Education, and Energy Sectors
Adebisi, J. A., & Moses, B. O. (2022) · Network and Communication Technologies · doi:10.5539/nct.v7n1p39 · Cited by 16
“As a practical example, Infervision - AI medical technology company, developed an AI tool can reduce the time taken to diagnose a possible of COVID-19 pneumonia from 15 minutes (by manual check) to 10 seconds- this is enough time to save a patient’s life (Mallio et al., 2021).”
MethodRadiomics and Deep Learning Prediction of Immunotherapy-Induced Pneumonitis From Computed Tomography
Smith, D. S., Lippenszky, L., Lenoue-Newton, M. L., Jain, N. M., Mittendorf, K. F., Micheel, C., Cella, P., Wolber, J., & Osterman, T. J. (2025) · JCO Clinical Cancer Informatics · doi:10.1200/cci-24-00198 · Cited by 16
Giordano, F. M., Ippolito, E., Quattrocchi, C. C., Greco, C., Mallio, C. A., Santo, B., D’Alessio, P., Crucitti, P., Fiore, M., Zobel, B. B., et al. (2021) · Cancers · doi:10.3390/cancers13081960 · Cited by 7
“The model training process is presented elsewhere [25].”
MethodSelf-citationComputed tomography density changes of bone metastases after concomitant denosumab
Mallio, C. A., Greco, F., Gaudino, F., Zobel, B. B., & Quattrocchi, C. C. (2023) · Skeletal Radiology · doi:10.1007/s00256-023-04326-3 · Cited by 5
Self-citationMuneer, A., Showkatian, E., Kitsel, Y., Saad, M. B., Sujit, S. J., Soto, F., Shroff, G. S., Faiz, S. A., Ghanbar, M. I., Ismail, S., et al. (2026) · Journal for ImmunoTherapy of Cancer · doi:10.1136/jitc-2026-015014 · Cited by 3
Yang, F., Li, Z., Yin, D., Yang, J., & Zhao, Y. (2025) · Current Medical Imaging Formerly Current Medical Imaging Reviews · doi:10.2174/0115734056399950251003114150 · Cited by 1
Greco, F., Zobel, B. B., Gennaro, G. D., & Mallio, C. A. (2023) · Applied Sciences · doi:10.3390/app13063779 · Cited by 1
Self-citationGreco, F., Zobel, B. B., & Mallio, C. A. (2023) · Hematology Reports · doi:10.3390/hematolrep15030049 · Cited by 1
Self-citationTamura, Y., Sawada, K., Muro, K., Otake, S., Hayashi, H., Ota, Y., Nakajima, S., Hasebe, T., Aso, K., & Fujiya, M. (2025) · Kanzo · doi:10.2957/kanzo.66.39 · Cited by 1
A Predictive Analysis for Diagnosis of COVID-19, Pneumonia and Lung Cancer Using Deep Learning
Banerjee, A., & Paul, S. (2022) · Springer eBooks · doi:10.1007/978-981-16-8150-9_8 · Cited by 1
Lyu, Q., Yuan, H., Lin, Z., Ponnatapura, J., & Whitlow, C. T. (2024) · medRxiv · doi:10.1101/2024.10.14.24315487 · Cited by 1
Intelligent System of Scientific and Technological Talent Inquiry Based on Deep Learning Algorithm
Zheng, H., Nan, L., Yang, Q., Yang, M., Yang, T., & Suandi, T. (2022) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-19-3632-6_21
Chizhova, Y. S., Дмитриевич, Ф. В., & Zakharova, E. M. (2025) · Journal of clinical practice · doi:10.17816/clinpract655829
Mallio, C. A., Sertorio, A. C., Bernetti, C., Greco, F., Gennaro, G. D., & Zobel, B. B. (2025) · The Egyptian Journal of Radiology and Nuclear Medicine · doi:10.1186/s43055-025-01541-x
“These advanced AI systems offer promising avenues in converting unstructured radiological reports into structured formats, potentially revolutionizing radiological workflow, and inter-physician communication [1, 2].”
Self-citationPredicting Immunotherapy-Induced Pneumonitis Based on Chest CT and Non-Imaging Data
Lyu, Q., Yuan, H., Lin, Z., Ponnatapura, J., & Whitlow, C. T. (2025) · Cancers · doi:10.3390/cancers17182980 · Cited by 2
“Mu [29].”
Metadata
- Authors
- Mallio, C. A., Napolitano, A., Castiello, G., Giordano, F. M., D'Alessio, P., Iozzino, M., Sun, Y., Angeletti, S., Russano, M., Santini, D., Tonini, G., Zobel, B. B., Vincenzi, B., & Quattrocchi, C. C.
- Affiliations
- Università Campus Bio-Medico (IT)
- Departmental Faculty of Medicine and Surgery, Unit of Medical Oncology, 00128 Rome, Italy
- Departmental Faculty of Medicine and Surgery, Unit of Diagnostic Imaging and Interventional Radiology, Università Campus Bio-Medico di Roma, 00128 Rome, Italy
- Ospedale Santa Maria Goretti (IT)
- Infervision Europe GmbH, Mainzer Strasse 75, D-65189 Wiesbaden, Germany
- Corresponding author
- Mallio, Carlo Augusto
- Napolitano, Andrea
- ORCID
- Venue
- Cancers
- Publisher
- MDPI AG
- Published
- 2021-02-06
- Volume
- 13
- Issue
- 4
- Pages
- 652
- ISSN
- 2072-6694 (ISSN-L)
- PMC
- PMC7914551
- OpenAlex
- W3126423192
- Cited by
- 30
- References
- 36
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 3 · 2023: 8 · 2022: 5 · 2021: 7
- FWCI
- 3.04
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Cancers (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- IRIS Research product catalog (Sapienza University of Rome) (submittedVersion, cc-by)
- Iris (University of Trento) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- COVID-19 pneumonia, immune-related pneumonitis, differential diagnosis, chest CT, deep convolutional neural networks, artificial intelligence, interstitial lung disease, model robustness, cross-validation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.