Unveiling COVID-19 from CHEST X-ray with deep learning: A hurdles race with small data
Open accessAbstract
The possibility to use widespread and simple chest X-ray (CXR) imaging for early screening of COVID-19 patients is attracting much interest from both the clinical and the AI community. In this study we provide insights and also raise warnings on what is reasonable to expect by applying deep learning to COVID classification of CXR images. We provide a methodological guide and critical reading of an extensive set of statistical results that can be obtained using currently available datasets. In particular, we take the challenge posed by current small size COVID data and show how significant can be the bias introduced by transfer-learning using larger public non-COVID CXR datasets. We also contribute by providing results on a medium size COVID CXR dataset, just collected by one of the major emergency hospitals in Northern Italy during the peak of the COVID pandemic. These novel data allow us to contribute to validate the generalization capacity of preliminary results circulating in the scientific community. Our conclusions shed some light into the possibility to effectively discriminate COVID using CXR.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Shortcut learning can be mitigated using techniques including feature disentanglement and pre-processing techniques like histogram equalization and lung masking.
“This phenomenon, also called shortcut learning, can be mitigated using various techniques including feature disentanglement (where you reward a model when it learns disease characteristics and penalize the model when it learns features that can discriminate between the original data sets where the images are obtained) 51 ; applying pre-processing techniques like histogram equalization to correct”
From AI pitfalls and what not to do: mitigating bias in AI (2023) · cites this paper as [52]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759 · see citing works
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616 · see citing works
Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: a descriptive study
Shi, H., Han, X., Jiang, N., Cao, Y., Alwalid, O., Gu, J., Fan, Y., & Zheng, C. (2020) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(20)30086-4 · Cited by 3,664 · see citing works
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · see citing works
Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535302 · Cited by 3,277 · see citing works
Wang, L., Lin, Z. Q., & Wong, A. M. (2020) · Scientific Reports · doi:10.1038/s41598-020-76550-z · Cited by 3,151 · see citing works
Apostolopoulos, I. D., & Mpesiana, T. A. (2020) · Physical and Engineering Sciences in Medicine · doi:10.1007/s13246-020-00865-4 · Cited by 2,338 · see citing works
The Essential Physics of Medical Imaging
Bushberg, J. T., Seibert, J. A., Leidholdt, E. M., Boone, J. M., & Goldschmidt, E. J. (2003) · Medical Physics · doi:10.1118/1.1585033 · Cited by 2,088 · see citing works
Coronavirus Disease 2019 (COVID-19): A Perspective from China
Zu, Z. Y., Jiang, M. D., Xu, P. P., Chen, W., Ni, Q. Q., Lu, G. M., & Zhang, L. J. (2020) · Radiology · doi:10.1148/radiol.2020200490 · Cited by 2,070 · see citing works
Narin, A., Kaya, C., & Pamuk, Z. (2021) · Pattern Analysis and Applications · doi:10.1007/s10044-021-00984-y · Cited by 2,054 · see citing works
Prechelt, L. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8_3 · Cited by 1,649 · see citing works
Frequency and Distribution of Chest Radiographic Findings in Patients Positive for COVID-19
Wong, H. Y. F., Lam, H. Y. S., Fong, A. H.-T., Leung, S. T., Chin, T. W.-Y., Lo, C. S.-Y., Lui, M. M., Lee, J. C. Y., Chiu, K. W.-H., Chung, T. W., et al. (2020) · Radiology · doi:10.1148/radiol.2020201160 · Cited by 1,485 · see citing works
Lung pattern classification for interstitial lung diseases using a deep convolutional neural network
Anthimopoulos, M., Christodoulidis, S., Ebner, L., Christe, A., & Mougiakakou, S. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535865 · Cited by 1,343 · see citing works
Measures of Diagnostic Accuracy: Basic Definitions.
Šimundić, A.-M. (2009) · PubMed Central · Cited by 1,266 · see citing works
The Role of Chest Imaging in Patient Management During the COVID-19 Pandemic
Rubin, G. D., Ryerson, C. J., Haramati, L. B., Sverzellati, N., Kanne, J. P., Raoof, S., Schluger, N. W., Volpi, A., Yim, J., Martin, I. B. K., et al. (2020) · CHEST Journal · doi:10.1016/j.chest.2020.04.003 · Cited by 1,249 · see citing works
Measures of diagnostic accuracy: basic definitions
Šimundić, A.-M. (2009) · Medical and Biological Sciences · Cited by 1,144 · see citing works
Rubin, G. D., Ryerson, C. J., Haramati, L. B., Sverzellati, N., Kanne, J. P., Raoof, S., Schluger, N. W., Volpi, A., Yim, J.-J., Martin, I. B. K., et al. (2020) · Radiology · doi:10.1148/radiol.2020201365 · Cited by 1,101 · see citing works
On Early Stopping in Gradient Descent Learning
Yao, Y., Rosasco, L., & Caponnetto, A. (2007) · Constructive Approximation · doi:10.1007/s00365-006-0663-2 · Cited by 990 · see citing works
Prechelt, L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_5 · Cited by 952 · see citing works
Automatic lung segmentation for accurate quantitation of volumetric X-ray CT images
Hu, S., Hoffman, E. A., & Reinhardt, J. M. (2001) · IEEE Transactions on Medical Imaging · doi:10.1109/42.929615 · Cited by 943 · see citing works
Yang, Y., Yang, M., Shen, C., Wang, F., Yuan, J., Li, J., Zhang, M., Wang, Z., Xing, L., Wei, J., et al. (2020) · doi:10.1101/2020.02.11.20021493 · Cited by 836 · see citing works
Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images for Classification
Kermany, D. (2018) · Data Archiving and Networked Services (DANS) · doi:10.17632/rscbjbr9sj.2 · Cited by 758 · see citing works
Detection of Coronavirus Disease (COVID-19) Based on Deep FeaturesPreprint
Sethy, P. K., & Behera, S. K. (2020) · Preprints.org · doi:10.20944/preprints202003.0300.v1 · Cited by 723 · see citing works
COVID-19 Image Data CollectionPreprint
Cohen, J., Morrison, P., & Dao, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.11597 · Cited by 722 · see citing works
Lung Segmentation in Chest Radiographs Using Anatomical Atlases With Nonrigid Registration
Candemir, S., Jaeger, S., Palaniappan, K., Musco, J. P., Singh, R. K., Xue, Z., Karargyris, A., Antani, S., Thoma, G. R., & McDonald, C. J. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2290491 · Cited by 684 · see citing works
Gunraj, H., Wang, L., & Wong, A. M. (2020) · Frontiers in Medicine · doi:10.3389/fmed.2020.608525 · Cited by 336 · see citing works
Pre-trained convolutional neural networks as feature extractors for tuberculosis detection
Lopes, U. K., & Valiati, J. F. (2017) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2017.08.001 · Cited by 294 · see citing works
Wang, L., & Wong, A. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.09871 · Cited by 248 · see citing works
A role for CT in COVID-19? What data really tell us so far
Hope, M. D., Raptis, C. A., Shah, A. B., Hammer, M. M., & Henry, T. S. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30728-5 · Cited by 188 · see citing works
A Generic Approach to Pathological Lung Segmentation
Mansoor, A., Bağcı, U., Xu, Z., et al. (2014) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2337057 · Cited by 164 · see citing works
Chest X-Ray Analysis of Tuberculosis by Deep Learning with Segmentation and Augmentation
Stirenko, S., Kochura, Y., Alienin, O., Rokovyi, O., Gordienko, Y. G., Gang, P., & Zeng, W. (2018) · IEEE International Conference on Electronics and Nanotechnology (ELNANO) · doi:10.1109/elnano.2018.8477564 · Cited by 160 · see citing works
Best Practices for Fine-Tuning Visual Classifiers to New Domains
Chu, B., Madhavan, V., Beijbom, O., Hoffman, J., & Darrell, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_34 · Cited by 124 · see citing works
A clinicopathological study of three cases of severe acute respiratory syndrome (SARS)
Lang, Z.-W., Zhang, L., Zhang, S., Meng, X., Li, J., Chen-zhao, S., Sun, L. V., Zhou, Y., & Dwyer, D. E. (2003) · Pathology · doi:10.1080/00313020310001619118 · Cited by 99 · see citing works
Laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections
Yang, Y., Yang, M., Shen, C., et al. (2020) · Cited by 85 · see citing works
Computer Aided Detection of SARS Based on Radiographs Data Mining
Xuanyang, X., Yu-chang, G., Shouhong, W., et al. (2005) · doi:10.1109/iembs.2005.1616237 · Cited by 24 · see citing works
Bondfale, N., & Bhagwat, D. S. (2018) · doi:10.1109/icicct.2018.8473302 · Cited by 12 · see citing works
Texture classification of sars infected region in radiographic image
Tong, X., Tao, D., & Antonio, G. E. (2005) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2004.1421729 · Cited by 4 · see citing works
Post-synaptic Potential Regularization Has Potential
Tartaglione, E., Perlo, D., & Grangetto, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30484-3_16 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- [31]More A. Survey of resampling techniques for improving classification performance in unbalanced datasets arXiv 2016 1608.06048
- [37]Iandola F. Moskewicz M. Karayev S. Girshick R. Darrell T. Keutzer K. Densenet: Implementing efficient convnet descriptor pyramids arXiv 2014 1404.1869
Cited by
Sorted by citations
Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00307-0 · Cited by 1,044 · see citing works
AI pitfalls and what not to do: mitigating bias in AI
Gichoya, J. W., Thomas, K., Celi, L. A., Safdar, N., Banerjee, I., Banja, J. D., Seyyed-Kalantari, L., Trivedi, H., & Purkayastha, S. (2023) · The British Journal of Radiology · doi:10.1259/bjr.20230023 · Cited by 277 · see citing works
Applications of artificial intelligence in battling against covid-19: A literature review
Tayarani N., M.-H. (2021) · Chaos, Solitons & Fractals · doi:10.1016/j.chaos.2020.110338 · Cited by 210 · see citing works
Trivedi, A., Robinson, C., Blazes, M., Ortiz, A., Desbiens, J., Gupta, S., Dodhia, R., Bhatraju, P. K., Liles, W. C., Kalpathy-Cramer, J., et al. (2022) · PLOS ONE · doi:10.1371/journal.pone.0274098 · Cited by 28 · see citing works
- Authors
- Tartaglione, E., Barbano, C. A., Berzovini, C., Calandri, M., & Grangetto, M.
- Affiliations
- University of Turin (IT)
- Azienda Ospedaliera Citta' della Salute e della Scienza di Torino (IT)
- Ospedale San Luigi Gonzaga (IT)
- Corresponding author
- Tartaglione, Enzo
- Venue
- International Journal of Environmental Research and Public Health
- Publisher
- MDPI AG
- Published
- 2020-09-22
- Volume
- 17
- Issue
- 18
- Pages
- 6933
- ISSN
- 1660-4601 (ISSN-L)
- PMC
- PMC7557723
- OpenAlex
- W3015984951
- Cited by
- 211
- References
- 41
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 22 · 2023: 23 · 2022: 52 · 2021: 79 · 2020: 24
- FWCI
- 21.67
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- International Journal of Environmental Research and Public Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Institutional Research Information System University of Turin (University of Turin) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Horizon 2020 (825111)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- COVID-19 pandemic, chest radiography, COVID-19 classification, transfer learning, limited training data, generalization, early screening, deep learning
- MeSH terms
- Deep Learning, COVID-19, Humans, Italy, Pneumonia, Viral, Radiography, Thoracic, Coronavirus Infections, Pandemics, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
