Achieving Multisite Generalization for CNN-Based Disease Diagnosis Models by Mitigating Shortcut Learning
Open accessAbstract
When a learned model has high accuracy under familiar settings (internal testing) and a big drop in accuracy under slightly different circumstances (external testing) we suspect it is using shortcuts to make decisions. This problem is known as shortcut learning. In medical imaging, shortcuts are undesired and unintended features that the model relies on to perform diagnosis. Shortcut-based solutions using medical images could lead to false diagnoses and have dangerous implications for patients. In the current COVID-19 era, a large set of papers have been published proposing the use of deep convolutional neural networks to perform diagnosis or triage of COVID-19 from chest X-rays (CXRs). These studies are reporting high accuracies which could be misleading and overestimated. To our knowledge, none of the currently published papers with high performance reported testing on samples from truly unseen data sources. Studies which did, have noticed a significant performance drop when testing on unseen sources indicating a failure to generalize. In this paper, we elucidate the generalization challenge of deep learning based models trained for disease diagnosis. We use the example of COVID-19 diagnosis from CXRs. Solutions that mitigate shortcut learning are introduced and experimentally shown to be effective. Our proposed methods enable the models to have a statistically significantly reduced performance drop-off on unseen data sources. Thus, lowering the performance drop to only 9% instead of 20%. The issues with convolutional neural networks addressed here generally apply to other imaging modalities and recognition problems, as shown.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
XAI becomes crucial as interpretability methods can help to find and eliminate spurious correlations and shortcut learning.
“In this situation, XAI becomes crucial as interpretability methods can help to find and eliminate spurious correlations and shortcut learning46–48.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [46]
Cites
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages
McNemar, Q. (1947) · Psychometrika · doi:10.1007/bf02295996 · Cited by 4,754 · see citing works
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616 · see citing works
Shortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224 · see citing works
Can AI help in screening viral and COVID-19 pneumonia?
Chowdhury, M. E. H., Rahman, T., Khandakar, A., Mazhar, R., Kadir, M. A., Mahbub, Z. B., Islam, K. R., Khan, M. S., Iqbal, A., Emadi, N. A., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3010287 · Cited by 1,913 · see citing works
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719 · see citing works
Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges
Hesamian, M. H., Jia, W., He, X., & Kennedy, P. J. (2019) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-019-00227-x · Cited by 1,644 · see citing works
Semi-Supervised Learning (Chapelle, O. et al., Eds.; 2006) [Book reviews]
Chapelle, O., Schölkopf, B., & Zien, E. A. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2015974 · Cited by 1,102 · see citing works
COVID-19 Image Data Collection: Prospective Predictions are the Future
Cohen, J., Morrison, P., Dao, L., Roth, K., Duong, T. Q., & Ghassemi, M. (2020) · The Journal of Machine Learning for Biomedical Imaging · doi:10.59275/j.melba.2020-48g7 · Cited by 1,026 · see citing works
Deep Learning COVID-19 Features on CXR Using Limited Training Data Sets
Oh, Y., Park, S. J., & Ye, J. C. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2993291 · Cited by 897 · see citing works
PadChest: A large chest x-ray image dataset with multi-label annotated reports
Bustos, A., Pertusa, A., Salinas, J.-M., & de la Iglesia-Vayá, M. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101797 · Cited by 691 · see citing works
Recognition in Terra Incognita
Beery, S., Van Horn, G., & Perona, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_28 · Cited by 679 · see citing works
AI for radiographic COVID-19 detection selects shortcuts over signal
DeGrave, A. J., Janizek, J. D., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00338-7 · Cited by 631 · see citing works
Caruana, R. (1998) · Learning to Learn · doi:10.1007/978-1-4615-5529-2_5 · Cited by 577 · see citing works
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392 · see citing works
COVIDGR Dataset and COVID-SDNet Methodology for Predicting COVID-19 Based on Chest X-Ray Images
Tabik, S., Gómez-Ríos, A., Rodríguez, J. L. M., Sevillano-García, I., Rey-Area, M., Charte, D., Guirado, E., Suárez, J. L., Luengo, J., González, M. Á. V., et al. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3037127 · Cited by 376 · see citing works
Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-Rays
Rajaraman, S., Siegelman, J., Alderson, P. O., Folio, L. S., Folio, L. R., & Antani, S. (2020) · PubMed Central · doi:10.1109/access.2020.3003810 · Cited by 366 · see citing works
Facial Expression Recognition in the Wild via Deep Attentive Center Loss
Farzaneh, A. H., & Qi, X. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00245 · Cited by 346 · see citing works
Deep Learning-Based Decision-Tree Classifier for COVID-19 Diagnosis From Chest X-ray Imaging
Yoo, S. H., Geng, H., Chiu, T. L., Yu, S. K., Cho, D. C., Heo, J. U., Choi, M., Choi, I. H., Văn, C. C., Nhung, N. V., et al. (2020) · Frontiers in Medicine · doi:10.3389/fmed.2020.00427 · Cited by 289 · see citing works
BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patientsPreprint
MdlI, V., Saborit-Torres, J. M., Ja, M., Pertusa, A., Bustos, A., Cazorla, M., Galant, J., Barber, X., Orozco‐Beltrán, D., García-García, F., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01174 · Cited by 246 · see citing works
Mali, S. A., Ibrahim, A., Woodruff, H. C., Andrearczyk, V., Müller, H., Primakov, S. P., Salahuddin, Z., Chatterjee, A., & Lambin, P. (2021) · Journal of Personalized Medicine · doi:10.3390/jpm11090842 · Cited by 218 · see citing works
Wang, G., Liu, X., Shen, J., Wang, C., Li, Z., Ye, L., Wu, X., Chen, T., Wang, K., Zhang, X., et al. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-021-00704-1 · Cited by 192 · see citing works
Wehbe, R. M., Sheng, J., Dutta, S., Chai, S., Dravid, A., Barutcu, S., Wu, Y., Cantrell, D. R., Xiao, N., Allen, B. D., et al. (2021) · Radiology · doi:10.1148/radiol.2020203511 · Cited by 190 · see citing works
Deep Learning System for COVID-19 Diagnosis Aid Using X-ray Pulmonary Images
Civit-Masot, J., Luna-Perejón, F., Domínguez-Morales, M., & Civit, A. (2020) · Applied Sciences · doi:10.3390/app10134640 · Cited by 172 · see citing works
Understanding Sources of Variation to Improve the Reproducibility of Radiomics
Zhao, B. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2021.633176 · Cited by 158 · see citing works
Impact of Lung Segmentation on the Diagnosis and Explanation of COVID-19 in Chest X-ray Images
Teixeira, L. O., Pereira, R. M., Bertolini, D. S., Oliveira, L. S., Nanni, L., Cavalcanti, G. D. C., & Costa, Y. M. G. (2021) · Sensors · doi:10.3390/s21217116 · Cited by 139 · see citing works
Pooch, E. H. P., Ballester, P., & Barros, R. C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-62469-9_7 · Cited by 134 · see citing works
Current limitations to identify COVID-19 using artificial intelligence with chest X-ray imaging
López-Cabrera, J. D., Orozco‐Morales, R., Portal-Díaz, J. A., Enríquez, O. A. L., & Pérez-Díaz, M. (2021) · Health and Technology · doi:10.1007/s12553-021-00520-2 · Cited by 96 · see citing works
AANet: Adaptive Attention Network for COVID-19 Detection From Chest X-Ray Images
Lin, Z., He, Z., Xie, S., Wang, X., Tan, J., Lu, J., & Tan, B. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3114747 · Cited by 70 · see citing works
Uncovering and Correcting Shortcut Learning in Machine Learning Models for Skin Cancer Diagnosis
Nauta, M., Walsh, R., Dubowski, A., & Seifert, C. (2021) · Diagnostics · doi:10.3390/diagnostics12010040 · Cited by 67 · see citing works
Chest imaging representing a COVID-19 positive rural U.S. population
Desai, S., Baghal, A., Wongsurawat, T., Jenjaroenpun, P., Powell, T., Al‐Shukri, S., Gates, K., Farmer, P., Rutherford, M., Blake, G., et al. (2020) · Scientific Data · doi:10.1038/s41597-020-00741-6 · Cited by 57 · see citing works
Generalizable Multi-Site Training and Testing Of Deep Neural Networks Using Image Normalization
Onofrey, J. A., Casetti-Dinescu, D. I., Lauritzen, A. D., Sarkar, S., Venkataraman, R., Fan, R. E., Sonn, G. A., Sprenkle, P. C., Staib, L. H., & Papademetris, X. (2019) · 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) · doi:10.1109/isbi.2019.8759295 · Cited by 52 · see citing works
Discovery of a Generalization Gap of Convolutional Neural Networks on COVID-19 X-Rays Classification
Ahmed, K. B., Goldgof, G. M., Paul, R., Goldgof, D. B., & Hall, L. (2021) · IEEE Access · doi:10.1109/access.2021.3079716 · Cited by 41 · see citing works
Self-citationWinther, H. B., Laser, H., Gerbel, S., et al. (2020) · Forschungsdaten-Repositorium der LUH · doi:10.25835/0090041 · Cited by 31 · see citing works
Li, F., Lu, X., & Yuan, J. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3134270 · Cited by 28 · see citing works
Lung Nodule Sizes Are Encoded When Scaling CT Image for CNN's
Cherezov, D., Paul, R., Fetisov, N., Gillies, R. J., Schabath, M. B., Goldgof, D. B., & Hall, L. (2020) · Tomography · doi:10.18383/j.tom.2019.00024 · Cited by 9 · see citing works
Self-citationDebiasing pipeline improves deep learning model generalization for X-ray based lung nodule detectionPreprint
Horry, M. J., Chakraborty, S., Pradhan, B., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.09563 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Pahud de Mortanges, A., Luo, H., Shu, S. Z., Kamath, A., Suter, Y., Shelan, M., Pöllinger, A., & Reyes, M. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01190-w · Cited by 93 · see citing works
“In this situation, XAI becomes crucial as interpretability methods can help to fi nd and eliminate spurious correlations and shortcut learning 46 – 48 .”
- Authors
- Ben Ahmed, K., Hall, L. O., Goldgof, D. B., & Fogarty, R.
- Affiliations
- Department of Computer Science and Engineering, University of South Florida, Tampa, FL, USA
- University of South Florida (US)
- Venue
- IEEE Access
- Published
- 2022
- Volume
- 10
- Pages
- 78726-78738
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4287889976
- Cited by
- 12
- References
- 37
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 5 · 2023: 4
- FWCI
- 1.57
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- shortcut learning, COVID-19 diagnosis, chest radiography, convolutional neural networks, external validation, medical image classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
