A Reliability Analysis of a Deep Neural Network
The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Deep Learning, and in particular its implementation using Convolutional Neural Networks (CNNs), is currently one of the most intensively and widely used predictive models for safety-critical applications like autonomous driving assistance on pedestrian, objects and structures recognition. Today, ensuring the reliability of these innovations is becoming very important since they involve human lives. One of the peculiarities of the CNNs is the inherent resilience to errors due to the iterative nature of the learning process. In this work we present a methodology to evaluate the impact of permanent faults affecting CNN exploited for automotive applications. Such a characterization is performed through a fault injection environment built upon on the darknet open source DNN framework. Results are shown about fault injection campaigns where permanent faults are affecting the connection weights in the LeNet and Yolo; the behavior of the corrupted CNN is classified according to the criticality of the introduced deviation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Application-Level Fuctional Error Simulation , Fuctional Error Simulation (FES) in short, provides shorter development and deployment times, it can be applied very early during the design process, and it ensures that every experiment produces a corrupted output.”
From Fast and Accurate Error Simulation for CNNs Against Soft Errors (2023) · cites this paper as [19]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“The first one is LeNeT [14] used as classifier for handwritten digit recognition task (we used the MNSIT database of training and validation).”
MethodYou Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 1×
“In most of the cases, CNNs provide very good results for many complex tasks such as object recognition in images/videos, drug discovery, natural language processing up to playing complex games [2]–[4].”
BackgroundDeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
Chen, C., Seff, A., Kornhauser, A., & Xiao, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.312 · Cited by 1,876 · cited 1×
“Thus, these techniques are now deeply used for safety-critical applications like autonomous driving [6].”
BackgroundUnderstanding error propagation in deep learning neural network (DNN) accelerators and applications
Li, G., Hari, S. K. S., Sullivan, M., Tsai, T., Pattabiraman, K., Emer, J., & Keckler, S. W. (2017) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/3126908.3126964 · Cited by 492 · cited 5×
“As already discussed in [12] this is the most critical observed fault;”
BackgroundResultInfluentialStatistical fault injection: Quantified error and confidence
Leveugle, R., Calvez, A., Maistri, P., & Vanhauwaert, P. (2009) · 2009 Design, Automation & Test in Europe Conference & Exhibition · doi:10.1109/date.2009.5090716 · Cited by 401 · cited 1×
“The precise numbers are given in the last column of tables III and IV and they were calculated by using the approach presented in [17].”
MethodMicroprocessor Software-Based Self-Testing
Psarakis, M., Gizopoulos, D., Sanchez, E., & Sonza Reorda, M. (2010) · IEEE Design & Test of Computers · doi:10.1109/mdt.2010.5 · Cited by 258 · cited 1×
“g, Software-based Self-test [8]), or a combination of both.”
BackgroundResiliency of Deep Neural Networks under Quantization
Sung, W., Shin, S., & Hwang, K. (2015) · arXiv · doi:10.48550/arxiv.1511.06488 · Cited by 132
Fernandes dos Santos, F., Draghetti, L., Weigel, L., Carro, L., Navaux, P., & Rech, P. (2017) · 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W) · doi:10.1109/dsn-w.2017.47 · Cited by 72
On-Line Functionally Untestable Fault Identification in Embedded Processor Cores
Bernardi, P., Bonazza, M., Sanchez, E. R., Reorda, M. S., & Ballan, O. (2013) · Design, Automation, and Test in Europe · doi:10.7873/date.2013.298 · Cited by 34 · cited 1×
“Some examples can be found in [9].”
Self-citationCache- and register-aware system reliability evaluation based on data lifetime analysis
Kooli, M., Kaddachi, F., Natale, G. D., & Bosio, A. (2016) · IEEE VLSI Test Symposium (VTS) · doi:10.1109/vts.2016.7477299 · Cited by 14
Self-citationCantoro, R., Firrincieli, A., Piumatti, D., Restifo, M., Sánchez, E., & Reorda, M. S. (2018) · IEEE Latin American Test Symposium (LATS) · doi:10.1109/latw.2018.8349679 · Cited by 14 · cited 2×
“, IEC 61508 for industrial systems, DO-254 for avionics, ISO 26262 for automotive) [7].”
Background
Research works printed but not linked: 1
- 0, Recent Advances In Deep Learning For Speech Research At Microsoft ICASSP
Cited by
Perez-Cerrolaza, J., Abella, J., Borg, M., Donzella, C., Cerquides, J., Cazorla, F. J., Englund, C., Tauber, M., Nikolakopoulos, G., & Flores, J. L. (2024) · ACM Computing Surveys · doi:10.1145/3626314 · Cited by 185
“The analysis and error mitigation in the DL algorithms and software implementation is also an active research ield [Bosio et al. 2019; Geißler et al. 2021; Li et al. 2017; Ruospo et al. 2020].”
BackgroundMethodA survey on modeling and improving reliability of DNN algorithms and accelerators
Mittal, S. (2020) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2019.101689 · Cited by 175
Software Verification and Validation of Safe Autonomous Cars: A Systematic Literature Review
Rajabli, N., Flammini, F., Nardone, R., & Vittorini, V. (2020) · IEEE Access · doi:10.1109/access.2020.3048047 · Cited by 148
Soft errors in DNN accelerators: A comprehensive review
Ibrahim, Y., Wang, H., Liu, J., Wei, J., Chen, L., Rech, P., Adam, K., & Guo, G. (2020) · Microelectronics Reliability · doi:10.1016/j.microrel.2020.113969 · Cited by 102
Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives
Su, F., Liu, C., & Stratigopoulos, H. (2023) · IEEE Design and Test · doi:10.1109/mdat.2023.3241116 · Cited by 97
“The interested reader is referred to [70], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83], [84], and [85] for more software-based fault injection experiments studying the fault impact for different neural network models, data type representations, layer types, network sizes, pruned networ…”
Background
Metadata
- Authors
- Bosio, A., Bernardi, P., Ruospo, A., & Sanchez, E.
- Affiliations
- Politecnico di Torino (IT)
- Venue
- 2019 IEEE Latin American Test Symposium (LATS)
- Publisher
- IEEE
- Published
- 2019-03
- Pages
- 1-6
- OpenAlex
- W2943759410
- Cited by
- 114
- References
- 14
- Citations by year
- 2026: 9 · 2025: 20 · 2024: 15 · 2023: 23 · 2022: 16 · 2021: 19 · 2020: 9 · 2019: 3
- FWCI
- 4.50
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Latin American Test Symposium (LATS) (publishedVersion)
- HAL (submittedVersion)
- PORTO@Iris (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Hardware and Architecture › VLSI and Analog Circuit Testing
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- permanent faults, fault injection, convolutional neural networks, automotive safety-critical systems, autonomous driving, LeNet, YOLO, criticality classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.