The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Deep learning with robustness to missing data: A novel approach to the detection of COVID-19
Open accessAbstract
In the context of the current global pandemic and the limitations of the RT-PCR test, we propose a novel deep learning architecture, DFCN (Denoising Fully Connected Network). Since medical facilities around the world differ enormously in what laboratory tests or chest imaging may be available, DFCN is designed to be robust to missing input data. An ablation study extensively evaluates the performance benefits of the DFCN as well as its robustness to missing inputs. Data from 1088 patients with confirmed RT-PCR results are obtained from two independent medical facilities. The data includes results from 27 laboratory tests and a chest x-ray scored by a deep learning model. Training and test datasets are taken from different medical facilities. Data is made publicly available. The performance of DFCN in predicting the RT-PCR result is compared with 3 related architectures as well as a Random Forest baseline. All models are trained with varying levels of masked input data to encourage robustness to missing inputs. Missing data is simulated at test time by masking inputs randomly. DFCN outperforms all other models with statistical significance using random subsets of input data with 2-27 available inputs. When all 28 inputs are available DFCN obtains an AUC of 0.924, higher than any other model. Furthermore, with clinically meaningful subsets of parameters consisting of just 6 and 7 inputs respectively, DFCN achieves higher AUCs than any other model, with values of 0.909 and 0.919.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies addressing missing data were not limited to a particular missingness mechanism.
“Studies addressing missing data were not limited to a particular missingness mechanism (e.g., missing completely at random, missing at random, missing not at random)31,32.”
From A scoping review of robustness concepts for machine learning in healthcare (2025) · cites this paper as [31]
The Data and methods > Data section (page unavailable) describes experiments with random subsets of inputs and two clinically motivated subsets, so the study did not examine only randomly selected missing inputs. It does not classify those scenarios as missing completely at random, missing at random, or missing not at random, or establish robustness across those mechanisms. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Henry, B. M., de Oliveira, M. H. S., Benoit, S., Plebani, M., & Lippi, G. (2020) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2020-0369 · Cited by 1,964
Flagged first, then by citations
Research works printed but not linked: 29
- G Lippi, 2020, Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19), Clinical Chemistry and Laboratory Medicine, 58, 1070, 10.1515/cclm-2020-0285 · DOI
- G Lippi, 2020, Laboratory abnormalities in patients with COVID-2019 infection, Clinical Chemistry and Laboratory Medicine (CCLM), 58, 1131, 10.1515/cclm-2020-0198 · DOI
- E Terpos, 2020, Hematological findings and complications of COVID-19, American Journal of Hematology, 95, 834, 10.1002/ajh.25829 · DOI
- K Murphy, 2020, COVID-19 on the Chest Radiograph: A Multi-Reader Evaluation of an AI System, Radiology, 201874
- A Jacobi, 2020, Portable chest X-ray in coronavirus disease-19 (COVID-19): A pictorial review, Clinical Imaging, 64, 35, 10.1016/j.clinimag.2020.04.001 · DOI
- HYF Wong, 2020, Frequency and Distribution of Chest Radiographic Findings in Patients Positive for COVID-19, Radiology, 296, E72, 10.1148/radiol.2020201160 · DOI
- S Schiaffino, 2020, Diagnostic Performance of Chest X-Ray for COVID-19 Pneumonia During the SARS-CoV-2 Pandemic in Lombardy, Italy, Journal of Thoracic Imaging, 35, W105, 10.1097/RTI.0000000000000533 · DOI
- M Bandirali, 2020, Chest Radiograph Findings in Asymptomatic and Minimally Symptomatic Quarantined Patients in Codogno, Italy during COVID-19 Pandemic, Radiology, 295, E7, 10.1148/radiol.2020201102 · DOI
- A Borghesi, 2020, COVID-19 outbreak in Italy: experimental chest X-ray scoring system for quantifying and monitoring disease progression, La Radiologia Medica, 125, 509, 10.1007/s11547-020-01200-3 · DOI
- S Kurstjens, 2020, Rapid identification of SARS-CoV-2-infected patients at the emergency department using routine testing, Clinical Chemistry and Laboratory Medicine (CCLM), 58, 1587, 10.1515/cclm-2020-0593 · DOI
- MM Islam, 2021, A Review on Deep Learning Techniques for the Diagnosis of Novel Coronavirus (COVID-19), IEEE Access, 9, 30551, 10.1109/ACCESS.2021.3058537 · DOI
- E Putin, 2016, Deep biomarkers of human aging: Application of deep neural networks to biomarker development, Aging (Albany NY), 8, 1021, 10.18632/aging.100968 · DOI
- BA Goldstein, 2017, Moving beyond regression techniques in cardiovascular risk prediction: applying machine learning to address analytic challenges, European Heart Journal, 38, 1805
- A Dauvin, 2019, Machine learning can accurately predict pre-admission baseline hemoglobin and creatinine in intensive care patients, npj Digital Medicine, 2, 1, 10.1038/s41746-019-0192-z · DOI
- N Schütz, 2019, A comparative study of pattern recognition algorithms for predicting the inpatient mortality risk using routine laboratory measurements, Artificial Intelligence Review, 52, 2559, 10.1007/s10462-018-9625-3 · DOI
- P Vincent, 2010, Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion, The Journal of Machine Learning Research, 11, 3371
- X Dong, 2019, Pattern Recognition and Computer Vision. Lecture Notes in Computer Science, 517, 10.1007/978-3-030-31654-9_44 · DOI
- T Amarbayasgalan, 2019, Heterogeneous Data Management, Polystores, and Analytics for Healthcare. Lecture Notes in Computer Science, 237, 10.1007/978-3-030-33752-0_17 · DOI
- Alhassan Z, Budgen D, Alshammari R, Daghstani T, McGough AS, Moubayed NA. Stacked Denoising Autoencoders for Mortality Risk Prediction Using Imbalanced Clinical Data. In: 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA); 2018. p. 541–546. · DOI
- Alhassan Z, Budgen D, Alessa A, Alshammari R, Daghstani T, Al Moubayed N. Collaborative denoising autoencoder for high glycated haemoglobin prediction. In: Tetko IV, Kůrková V, Karpov P, Theis F, editors. Artificial neural networks and machine learning–ICANN 2019; 28th International Conference on Artificial Neural Networks, Munich, Germany, September 17–19, 2019; proceedings. Cham: Springer; 2019. p. 338–350. Available from: https://doi.org/10.1007/978-3-030-30493-5_34. · DOI
- JG Ibrahim, 2012, Missing Data in Clinical Studies: Issues and Methods, Journal of Clinical Oncology, 30, 3297, 10.1200/JCO.2011.38.7589 · DOI
- K Matsuoka, 1992, Noise injection into inputs in back-propagation learning, IEEE Transactions on Systems, Man, and Cybernetics, 22, 436, 10.1109/21.155944 · DOI
- RM Zur, 2009, Noise injection for training artificial neural networks: A comparison with weight decay and early stopping, Medical Physics, 36, 4810, 10.1118/1.3213517 · DOI
- N Srivastava, 2014, Dropout: a simple way to prevent neural networks from overfitting, The Journal of Machine Learning Research, 15, 1929
- MA Kramer, 1991, Nonlinear principal component analysis using autoassociative neural networks, AIChE Journal, 37, 233, 10.1002/aic.690370209 · DOI · Paper page
- L Breiman, 2001, Random Forests, Machine Learning, 45, 5, 10.1023/A:1010933404324 · DOI · Paper page
- F Pedregosa, 2011, Scikit-learn: Machine Learning in Python, Journal of Machine Learning Research, 12, 2825
- ER DeLong, 1988, Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach, Biometrics, 44, 837, 10.2307/2531595 · DOI · Paper page
- W Wang, 2020, Detection of SARS-CoV-2 in Different Types of Clinical Specimens, JAMA, 10.1001/jama.2020.3786 · DOI
Cited by
Sorted by citations
A scoping review of robustness concepts for machine learning in healthcare
Balendran, A., Beji, C., Bouvier, F., Khalifa, O., Evgeniou, T., Ravaud, P., & Porcher, R. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01420-1 · Cited by 43
Metadata
- Authors
- Çallı, E., Murphy, K., Kurstjens, S., Samson, T., Herpers, R., Smits, H., Rutten, M., & van Ginneken, B.
- Affiliations
- Radboud University Nijmegen (NL)
- Jeroen Bosch Ziekenhuis (NL)
- Ziekenhuis Bernhoven (NL)
- Corresponding author
- Çallı, Erdi
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2021-07-30
- Volume
- 16
- Issue
- 7
- Pages
- e0255301
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC8323880
- OpenAlex
- W3137346378
- Cited by
- 6
- References
- 3
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 1 · 2023: 2 · 2022: 1
- FWCI
- 0.53
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Funders
- Fondation Botnar
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Infectious Diseases › SARS-CoV-2 detection and testing
- Keywords
- COVID-19 detection, deep learning, missing data robustness, laboratory findings, chest radiography, input masking, clinical decision support
- MeSH terms
- Deep Learning, COVID-19, SARS-CoV-2, COVID-19 Nucleic Acid Testing, Humans, Models, Theoretical, Random Allocation, Databases, Factual
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review