Cited by 6COVID-19 diagnosis using AIArtificial Intelligence in HealthcareSARS-CoV-2 detection and testing

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Deep learning with robustness to missing data: A novel approach to the detection of COVID-19

Open access

Abstract

In the context of the current global pandemic and the limitations of the RT-PCR test, we propose a novel deep learning architecture, DFCN (Denoising Fully Connected Network). Since medical facilities around the world differ enormously in what laboratory tests or chest imaging may be available, DFCN is designed to be robust to missing input data. An ablation study extensively evaluates the performance benefits of the DFCN as well as its robustness to missing inputs. Data from 1088 patients with confirmed RT-PCR results are obtained from two independent medical facilities. The data includes results from 27 laboratory tests and a chest x-ray scored by a deep learning model. Training and test datasets are taken from different medical facilities. Data is made publicly available. The performance of DFCN in predicting the RT-PCR result is compared with 3 related architectures as well as a Random Forest baseline. All models are trained with varying levels of masked input data to encourage robustness to missing inputs. Missing data is simulated at test time by masking inputs randomly. DFCN outperforms all other models with statistical significance using random subsets of input data with 2-27 available inputs. When all 28 inputs are available DFCN obtains an AUC of 0.924, higher than any other model. Furthermore, with clinically meaningful subsets of parameters consisting of just 6 and 7 inputs respectively, DFCN achieves higher AUCs than any other model, with values of 0.909 and 0.919.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0255301

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Studies addressing missing data were not limited to a particular missingness mechanism.

    “Studies addressing missing data were not limited to a particular missingness mechanism (e.g., missing completely at random, missing at random, missing not at random)31,32.”

    From A scoping review of robustness concepts for machine learning in healthcare (2025) · cites this paper as [31]

    The Data and methods > Data section (page unavailable) describes experiments with random subsets of inputs and two clinically motivated subsets, so the study did not examine only randomly selected missing inputs. It does not classify those scenarios as missing completely at random, missing at random, or missing not at random, or establish robustness across those mechanisms. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 29

  1. G Lippi, 2020, Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19), Clinical Chemistry and Laboratory Medicine, 58, 1070, 10.1515/cclm-2020-0285 · DOI
  2. G Lippi, 2020, Laboratory abnormalities in patients with COVID-2019 infection, Clinical Chemistry and Laboratory Medicine (CCLM), 58, 1131, 10.1515/cclm-2020-0198 · DOI
  3. E Terpos, 2020, Hematological findings and complications of COVID-19, American Journal of Hematology, 95, 834, 10.1002/ajh.25829 · DOI
  4. K Murphy, 2020, COVID-19 on the Chest Radiograph: A Multi-Reader Evaluation of an AI System, Radiology, 201874
  5. A Jacobi, 2020, Portable chest X-ray in coronavirus disease-19 (COVID-19): A pictorial review, Clinical Imaging, 64, 35, 10.1016/j.clinimag.2020.04.001 · DOI
  6. HYF Wong, 2020, Frequency and Distribution of Chest Radiographic Findings in Patients Positive for COVID-19, Radiology, 296, E72, 10.1148/radiol.2020201160 · DOI
  7. S Schiaffino, 2020, Diagnostic Performance of Chest X-Ray for COVID-19 Pneumonia During the SARS-CoV-2 Pandemic in Lombardy, Italy, Journal of Thoracic Imaging, 35, W105, 10.1097/RTI.0000000000000533 · DOI
  8. M Bandirali, 2020, Chest Radiograph Findings in Asymptomatic and Minimally Symptomatic Quarantined Patients in Codogno, Italy during COVID-19 Pandemic, Radiology, 295, E7, 10.1148/radiol.2020201102 · DOI
  9. A Borghesi, 2020, COVID-19 outbreak in Italy: experimental chest X-ray scoring system for quantifying and monitoring disease progression, La Radiologia Medica, 125, 509, 10.1007/s11547-020-01200-3 · DOI
  10. S Kurstjens, 2020, Rapid identification of SARS-CoV-2-infected patients at the emergency department using routine testing, Clinical Chemistry and Laboratory Medicine (CCLM), 58, 1587, 10.1515/cclm-2020-0593 · DOI
  11. MM Islam, 2021, A Review on Deep Learning Techniques for the Diagnosis of Novel Coronavirus (COVID-19), IEEE Access, 9, 30551, 10.1109/ACCESS.2021.3058537 · DOI
  12. E Putin, 2016, Deep biomarkers of human aging: Application of deep neural networks to biomarker development, Aging (Albany NY), 8, 1021, 10.18632/aging.100968 · DOI
  13. BA Goldstein, 2017, Moving beyond regression techniques in cardiovascular risk prediction: applying machine learning to address analytic challenges, European Heart Journal, 38, 1805
  14. A Dauvin, 2019, Machine learning can accurately predict pre-admission baseline hemoglobin and creatinine in intensive care patients, npj Digital Medicine, 2, 1, 10.1038/s41746-019-0192-z · DOI
  15. N Schütz, 2019, A comparative study of pattern recognition algorithms for predicting the inpatient mortality risk using routine laboratory measurements, Artificial Intelligence Review, 52, 2559, 10.1007/s10462-018-9625-3 · DOI
  16. P Vincent, 2010, Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion, The Journal of Machine Learning Research, 11, 3371
  17. X Dong, 2019, Pattern Recognition and Computer Vision. Lecture Notes in Computer Science, 517, 10.1007/978-3-030-31654-9_44 · DOI
  18. T Amarbayasgalan, 2019, Heterogeneous Data Management, Polystores, and Analytics for Healthcare. Lecture Notes in Computer Science, 237, 10.1007/978-3-030-33752-0_17 · DOI
  19. Alhassan Z, Budgen D, Alshammari R, Daghstani T, McGough AS, Moubayed NA. Stacked Denoising Autoencoders for Mortality Risk Prediction Using Imbalanced Clinical Data. In: 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA); 2018. p. 541–546. · DOI
  20. Alhassan Z, Budgen D, Alessa A, Alshammari R, Daghstani T, Al Moubayed N. Collaborative denoising autoencoder for high glycated haemoglobin prediction. In: Tetko IV, Kůrková V, Karpov P, Theis F, editors. Artificial neural networks and machine learning–ICANN 2019; 28th International Conference on Artificial Neural Networks, Munich, Germany, September 17–19, 2019; proceedings. Cham: Springer; 2019. p. 338–350. Available from: https://doi.org/10.1007/978-3-030-30493-5_34. · DOI
  21. JG Ibrahim, 2012, Missing Data in Clinical Studies: Issues and Methods, Journal of Clinical Oncology, 30, 3297, 10.1200/JCO.2011.38.7589 · DOI
  22. K Matsuoka, 1992, Noise injection into inputs in back-propagation learning, IEEE Transactions on Systems, Man, and Cybernetics, 22, 436, 10.1109/21.155944 · DOI
  23. RM Zur, 2009, Noise injection for training artificial neural networks: A comparison with weight decay and early stopping, Medical Physics, 36, 4810, 10.1118/1.3213517 · DOI
  24. N Srivastava, 2014, Dropout: a simple way to prevent neural networks from overfitting, The Journal of Machine Learning Research, 15, 1929
  25. MA Kramer, 1991, Nonlinear principal component analysis using autoassociative neural networks, AIChE Journal, 37, 233, 10.1002/aic.690370209 · DOI · Paper page
  26. L Breiman, 2001, Random Forests, Machine Learning, 45, 5, 10.1023/A:1010933404324 · DOI · Paper page
  27. F Pedregosa, 2011, Scikit-learn: Machine Learning in Python, Journal of Machine Learning Research, 12, 2825
  28. ER DeLong, 1988, Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach, Biometrics, 44, 837, 10.2307/2531595 · DOI · Paper page
  29. W Wang, 2020, Detection of SARS-CoV-2 in Different Types of Clinical Specimens, JAMA, 10.1001/jama.2020.3786 · DOI

Cited by

Sorted by citations

Metadata

Authors
Çallı, E., Murphy, K., Kurstjens, S., Samson, T., Herpers, R., Smits, H., Rutten, M., & van Ginneken, B.
Affiliations
  • Radboud University Nijmegen (NL)
  • Jeroen Bosch Ziekenhuis (NL)
  • Ziekenhuis Bernhoven (NL)
Corresponding author
Çallı, Erdi
Venue
PLOS ONE
Published
2021-07-30
Volume
16
Issue
7
Pages
e0255301
ISSN
1932-6203 (ISSN-L)
OpenAlex
W3137346378
Cited by
6
References
3
Citations by year
2026: 1 · 2025: 1 · 2024: 1 · 2023: 2 · 2022: 1
FWCI
0.53
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Radboud Repository (Radboud University) (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion, other-oa)
  • arXiv (Cornell University) (cc-by)
Funders
Fondation Botnar
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Health Sciences › Medicine › Infectious Diseases › SARS-CoV-2 detection and testing
Keywords
COVID-19 detection, deep learning, missing data robustness, laboratory findings, chest radiography, input masking, clinical decision support
MeSH terms
Deep Learning, COVID-19, SARS-CoV-2, COVID-19 Nucleic Acid Testing, Humans, Models, Theoretical, Random Allocation, Databases, Factual

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review