The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
A critic evaluation of methods for COVID-19 automatic detection from X-ray images
Open accessAbstract
In this paper, we compare and evaluate different testing protocols used for automatic COVID-19 diagnosis from X-Ray images in the recent literature. We show that similar results can be obtained using X-Ray images that do not contain most of the lungs. We are able to remove the lungs from the images by turning to black the center of the X-Ray scan and training our classifiers only on the outer part of the images. Hence, we deduce that several testing protocols for the recognition are not fair and that the neural networks are learning patterns in the dataset that are not correlated to the presence of COVID-19. Finally, we show that creating a fair testing protocol is a challenging task, and we provide a method to measure how fair a specific testing protocol is. In the future research we suggest to check the fairness of a testing protocol using our tools and we encourage researchers to look for better techniques than the ones that we propose.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616
Contrast Limited Adaptive Histogram Equalization
Zuiderveld, K. (1994) · Graphics Gems · doi:10.1016/b978-0-12-336156-1.50061-6 · Cited by 4,192 · cited 1×
“In the third one we used a 240 sized square, but we also preprocess the image by applying contrast limited adaptive histogram equalization [29] and by cropping the image by cutting its upper and lower part so that the height of the new image is between the 90% and the 100% of its width.”
COVID-19 and Italy: what next?
Remuzzi, A., & Remuzzi, G. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30627-9 · Cited by 3,119
Apostolopoulos, I. D., & Mpesiana, T. A. (2020) · Physical and Engineering Sciences in Medicine · doi:10.1007/s13246-020-00865-4 · Cited by 2,338 · cited 1×
“[11] combined Cohen repository and many other sources to create a larger dataset containing 224 images of COVID-19, 700 of pneumonia and 504 negative.”
Narin, A., Kaya, C., & Pamuk, Z. (2021) · Pattern Analysis and Applications · doi:10.1007/s10044-021-00984-y · Cited by 2,054 · cited 1×
“[10] created a small dataset with 50 COVID-19 cases coming from Cohen repository and 50 heathy cases coming from Kaggle (https://www.”
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719 · cited 1×
“The problem of generalizing to new datasets or of learning statistical features independent on the content of the images is not new in the deep learning community [15–17].”
Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification
Baltruschat, I. M., Nickisch, H., Grass, M., Knopp, T., & Saalbach, A. (2019) · Scientific Reports · doi:10.1038/s41598-019-42294-8 · Cited by 455 · cited 1×
“We are not the first to report potential biases in chest x-ray image classification [17,18].”
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · arXiv · doi:10.48550/arxiv.1901.07031 · Cited by 392 · cited 4×
“Irvin et al. collected and labelled Chexpert [28], a large dataset containing 224,316 chest radiographs of 65,240 patients divided into 14 classes.”
BackgroundInfluentialViral pneumonia screening on chest X-rays using confidence-aware anomaly detection
Zhang, J., Xie, Y., Pang, G., Liao, Z., Verjans, J., Li, W., Sun, Z., He, J., Li, Y., Shen, C., et al. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3040950 · Cited by 389
Unveiling COVID-19 from CHEST X-ray with deep learning: A hurdles race with small data
Tartaglione, E., Barbano, C. A., Berzovini, C., Calandri, M., & Grangetto, M. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17186933 · Cited by 211
Pooch, E. H. P., Ballester, P., & Barros, R. C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-62469-9_7 · Cited by 134
Pooch, E. H. P., Ballester, P. L., & Barros, R. C. (2019) · arXiv · doi:10.48550/arxiv.1909.01940 · Cited by 32
Cited by
Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00307-0 · Cited by 1,044
COVID-19 Image Data Collection: Prospective Predictions are the Future
Cohen, J. P., Morrison, P., Dao, L., Roth, K., Duong, T., & Ghassem, M. (2020) · Machine Learning for Biomedical Imaging · doi:10.59275/j.melba.2020-48g7 · Cited by 927
Roberts, M. T., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Avilés-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.64000 · Cited by 895
AI for radiographic COVID-19 detection selects shortcuts over signal
DeGrave, A. J., Janizek, J. D., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00338-7 · Cited by 631
COVID-19 identification in chest X-ray images on flat and hierarchical classification scenarios
Pereira, R. M., Bertolini, D., Teixeira, L. O., Silla, C. N., & Costa, Y. M. G. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105532 · Cited by 509
Metadata
- Authors
- Maguolo, G., & Nanni, L.
- Venue
- Information Fusion
- Publisher
- Elsevier BV
- Published
- 2021-12
- Volume
- 76
- Pages
- 1-7
- ISSN
- 1566-2535 (ISSN-L)
- OpenAlex
- W3017744567
- Cited by
- 250
- References
- 14
- FWCI
- 25.63
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.