On Data Bias and the Usability of Deep Learning Algorithms in Classifying COVID-19 based on Chest X-ray
Open accessAbstract
SARS-COV-2 is a new strain of virus that was first detected in China. It quickly spread across the world affecting millions of people. For this reason, early detection of the virus is mandatory in order to limit the spread of the virus. Real-time reverse transcription polymerase chain reaction (RT-PCR) and the antibody test are the main tests used to detect the virus. Chest X-rays (CXRs) and computerized tomography (CT) scans are also used to detect the virus although the American college of Radiology does not recommend using medical imaging as a diagnostic tool. Like other medical imaging, convolutional neural networks are used to classify the images. We believe that developing a model to detect COVID-19 has no clinical value regardless of the accuracy achieved since 58% of CXRs seem to be normal. During literature review, several papers with suspicious accuracy of 90% and higher were found. We believe that the dataset used to train and validate the network is biased and is not appropriate for deep learning as any model we train using the same dataset has achieved high accuracy. Our experiments on Cohen's Covid dataset, augmented with Wang dataset, shows that any model trained on Cohen dataset can easily achieve high accuracy. This was further validated with two experienced radiologists who participated in this study were only able to classify 60% as being Covid. Our study highlight the importance of addressing bias in data and developing trustworthy and explainable ML models based on well curated data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Bias in AI models can be observed through the type of input data used and data gaps during data collection.
“Primarily, bias in the AI-model can be observed through the type of input data exploited and the data gaps during data collection [41].”
From Artificial intelligence bias in medical system designs: a systematic review (2023) · cites this paper as [41]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
A pneumonia outbreak associated with a new coronavirus of probable bat origin
Zhou, P., Yang, X.-L., Wang, X.-G., Hu, B., Zhang, L., Zhang, W., Si, H.-R., Zhu, Y., Li, B., Huang, C.-L., et al. (2020) · Nature · doi:10.1038/s41586-020-2012-7 · Cited by 21,969 · see citing works
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616 · see citing works
Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR
Fang, Y., Zhang, H., Xie, J., Lin, M., Ying, L., Pang, P., & Ji, W. (2020) · Radiology · doi:10.1148/radiol.2020200432 · Cited by 3,340 · see citing works
Coudray, N., Ocampo, P. S., Sakellaropoulos, T., Narula, N., Snuderl, M., Fenyö, D., Moreira, A. L., Razavian, N., & Tsirigos, A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0177-5 · Cited by 2,930 · see citing works
Automated detection of COVID-19 cases using deep neural networks with X-ray images
Öztürk, T., Talo, M., Yildirim, E. A., Baloğlu, U. B., Yıldırım, Ö., & Acharya, U. R. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103792 · Cited by 2,607 · see citing works
Narin, A., Kaya, C., & Pamuk, Z. (2021) · Pattern Analysis and Applications · doi:10.1007/s10044-021-00984-y · Cited by 2,054 · see citing works
COVIDX-Net: A Framework of Deep Learning Classifiers to Diagnose COVID-19 in X-Ray ImagesPreprint
Hemdan, E. E., Shouman, M. A., & Karar, M. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.11055 · Cited by 1,181 · see citing works
Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network
Abbas, A., Abdelsamea, M. M., & Gaber, M. M. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01829-7 · Cited by 1,142 · see citing works
Deep learning for brain MRI segmentation: state of the art and future directions
Akkus, Z., Galimzianova, A., Hoogi, A., Rubin, D. L., & Erickson, B. J. (2017) · Journal of Digital Imaging · doi:10.1007/s10278-017-9983-4 · Cited by 1,129 · see citing works
Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00307-0 · Cited by 1,044 · see citing works
COVID-19 Image Data Collection: Prospective Predictions are the Future
Cohen, J., Morrison, P., Dao, L., Roth, K., Duong, T. Q., & Ghassemi, M. (2020) · The Journal of Machine Learning for Biomedical Imaging · doi:10.59275/j.melba.2020-48g7 · Cited by 1,026 · see citing works
Roberts, M. T., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Avilés-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.64000 · Cited by 902 · see citing works
Cheng, J.-Z., Ni, D., Chou, Y.-H., Qin, J., Tiu, C.-M., Chang, Y.-C., Huang, C.-S., Shen, D., & Chen, C.-M. (2016) · Scientific Reports · doi:10.1038/srep24454 · Cited by 755 · see citing works
Deep learning approaches for COVID-19 detection based on chest X-ray images
Ismael, A. M., & Şengür, A. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114054 · Cited by 754 · see citing works
Detecting and classifying lesions in mammograms with Deep Learning
Ribli, D., Horváth, A., Unger, Z., Pollner, P., & Csabai, I. (2018) · Scientific Reports · doi:10.1038/s41598-018-22437-z · Cited by 696 · see citing works
Deep learning based detection and analysis of COVID-19 on chest X-ray images
Jain, R., Gupta, M., Taneja, S., & Hemanth, D. J. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01902-1 · Cited by 523 · see citing works
CoroDet: A deep learning based classification for COVID-19 detection using chest X-ray images
Hussain, E., Hasan, M., Rahman, A., Lee, I., Tamanna, T., & Parvez, M. Z. (2020) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2020.110495 · Cited by 462 · see citing works
COVID-19 Screening on Chest X-ray Images Using Deep Learning based Anomaly Detection
Zhang, J., Xie, Y., Li, Y., Shen, C., & Xia, Y. (2020) · Europe PMC (PubMed Central) · Cited by 389 · see citing works
Luz, E. J. D. S., Silva, P., Silva, R., Silva, L., Guimarães, J., Miozzo, G., Moreira, G., & Menotti, D. (2021) · Research on Biomedical Engineering · doi:10.1007/s42600-021-00151-6 · Cited by 357 · see citing works
Gupta, A., Anjum, Gupta, S., & Katarya, R. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106859 · Cited by 236 · see citing works
Carneiro, G., Nascimento, J. C., & Freitas, A. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2169273 · Cited by 202 · see citing works
Lightweight deep learning models for detecting COVID-19 from chest X-ray images
Karakanis, S., & Leontidis, G. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.104181 · Cited by 147 · see citing works
Shankar, K., & Perumal, E. (2020) · Complex & Intelligent Systems · doi:10.1007/s40747-020-00216-6 · Cited by 113 · see citing works
Computer-assisted diagnosis of lung nodule detection using artificial convoultion neural network
Lo, S. B., Lin, J.-S., Freedman, M. T., et al. (1993) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.154572 · Cited by 42 · see citing works
Cruz, B. G. S., Bossa, M. N., Sölter, J., et al. (2021) · medRxiv · doi:10.1101/2021.02.15.21251775 · Cited by 29 · see citing works
A Progressive and Cross-Domain Deep Transfer Learning Framework for Wrist Fracture Detection
Karam, C., Zini, J. E., Awad, M., Saade, C., Naffaa, L. N., & Amine, M. E. (2021) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2022-0007 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Artificial intelligence bias in medical system designs: a systematic review
Kumar, A., Aelgani, V., Vohra, R., Gupta, S. K., Bhagawati, M., Paul, S., Saba, L., Suri, N., Khanna, N. N., Laird, J. R., et al. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16029-x · Cited by 65 · see citing works
- Authors
- Ezzeddine, H., Awad, M., Ghanem, A. S. A., & Mourani, B.
- Affiliations
- American University of Beirut (LB)
- American University of Beirut,Department of Diagnostic Radiology,Beirut,Lebanon
- ORCID
- Awad, Mariette
- Venue
- 2021 IEEE 3rd International Multidisciplinary Conference on Engineering Technology (IMCET)
- Publisher
- IEEE
- Published
- 2021-12-08
- Volume
- 24 10
- Pages
- 136-143
- OpenAlex
- W4206481454
- Cited by
- 2
- References
- 27
- Citations by year
- 2024: 1 · 2023: 1
- FWCI
- 0.63
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- IEEE International Multidisciplinary Conference on Engineering Technology (IMCET) (publishedVersion)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Infectious Diseases › SARS-CoV-2 detection and testing
- Keywords
- COVID-19 pandemic, chest radiography, data bias, convolutional neural networks, medical image classification, data curation, explainable machine learning, diagnostic accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
