Deep Validation: Toward Detecting Real-World Corner Cases for Deep Neural Networks
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
The exceptional performance of Deep neural networks (DNNs) encourages their deployment in safety-and dependability-critical systems. However, DNNs often demonstrate erroneous behaviors in real-world corner cases. Existing countermeasures center on improving the testing and bug-fixing practice. Unfortunately, building a bug-free DNN-based system is almost impossible currently due to its black-box nature, so anomaly detection is imperative in practice. Motivated by the idea of data validation in a traditional program, we propose and implement Deep Validation, a novel framework for detecting real-world error-inducing corner cases in a DNN-based system during runtime. We model the specifications of DNNs by resorting to their training data and cast checking input validity of DNNs as the problem of discrepancy estimation. Deep Validation achieves excellent detection results against various corner case scenarios across three popular datasets. Consequently, Deep Validation greatly complements existing efforts and is a crucial step toward building safe and dependable DNN-based systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Frossard, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.282 · Cited by 5,317 · cited 1×
“They discover that perturbing clean images with imperceptible noise can fool state-of-the-art DNN classifiers to make wrong predictions with high confidence [7], [16], [38], [44], [45], [54], [55], [65].”
BackgroundThe Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993
Practical Black-Box Attacks against Machine Learning
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017) · Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security · doi:10.1145/3052973.3053009 · Cited by 3,495 · cited 1×
“They discover that perturbing clean images with imperceptible noise can fool state-of-the-art DNN classifiers to make wrong predictions with high confidence [7], [16], [38], [44], [45], [54], [55], [65].”
BackgroundUniversal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680
Cited by
Software Engineering for AI-Based Systems: A Survey
Martínez-Fernández, S., Bogner, J., Franch, X., Oriol, M., Siebert, J., Trendowicz, A., Vollmer, A. M., & Wagner, S. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487043 · Cited by 335
Deep learning library testing via effective model generation
Wang, Z., Yan, M., Chen, J., Liu, S., & Zhang, D. (2020) · Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3368089.3409761 · Cited by 167
“Furthermore, there are a great deal of researches focusing on testing DL models in the literature [10, 20, 31, 45ś47, 55, 62, 65, 66, 69].”
BackgroundMuffin: testing deep learning libraries via neural architecture fuzzing
Gu, J., Luo, X., Zhou, Y., & Wang, X. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510092 · Cited by 71
“quite challenging to find specific inputs that can trigger corner cases [54].”
BackgroundOn failures of RGB cameras and their effects in autonomous driving applications
Secci, F., & Ceccarelli, A. (2020) · 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre5003.2020.00011 · Cited by 42
“W. Wu et al. [36] present a faults model for deep neural networks classifiers, which includes several corner cases based on the alteration of the input image, including amongst the possible causes brightness, camera alignment, and object movements.”
BackgroundMethod
Metadata
- Authors
- Wu, W., Xu, H., Zhong, S., Lyu, M. R., & King, I.
- Venue
- 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 125-137
- OpenAlex
- W2969772318
- Cited by
- 48
- References
- 17
- FWCI
- 4.54
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.