Analyzing and Increasing the Reliability of Convolutional Neural Networks on GPUs
The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
Graphics processing units (GPUs) are playing a critical role in convolutional neural networks (CNNs) for image detection. As GPU-enabled CNNs move into safety-critical environments, reliability is becoming a growing concern. In this paper, we evaluate and propose strategies to improve the reliability of object detection algorithms, as run on three NVIDIA GPU architectures. We consider three algorithms: 1) you only look once; 2) a faster region-based CNN (Faster R-CNN); and 3) a residual network, exposing live hardware to neutron beams. We complement our beam experiments with fault injection to better characterize fault propagation in CNNs. We show that a single fault occurring in a GPU tends to propagate to multiple active threads, significantly reducing the reliability of a CNN. Moreover, relying on error correcting codes dramatically reduces the number of silent data corruptions (SDCs), but does not reduce the number of critical errors (i.e., errors that could potentially impact safety-critical applications). Based on observations on how faults propagate on GPU architectures, we propose effective strategies to improve CNN reliability. We also consider the benefits of using an algorithm-based fault-tolerance technique for matrix multiplication, which can correct more than 87% of the critical SDCs in a CNN, while redesigning maxpool layers of the CNN to detect up to 98% of critical SDCs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“as commented in [9], SASSIFI is the only tool working with the NVIDIA proprietary libraries generally employed when implementing CNNs.”
From Fast and Accurate Error Simulation for CNNs Against Soft Errors (2023) · cites this paper as [9]
In the citing paper
“radiation test offers the most adherent solution to cause soft errors, however, apart from the highly expensive equipment and the complex setup, there is little to no controllability on the injection process, and the observability is only on the primary outputs. As a result, it is suitable for final validation and blackbox campaigns, not for in-depth analysis and investigations [9], [36].”
From Fast and Accurate Error Simulation for CNNs Against Soft Errors (2023) · cites this paper as [9]
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“Specifically, neural networks for object detection have the peculiarity of using convolution to reduce the information to be processed [15], a class of computations where reliability has not been studied deeply to date.”
BackgroundYou Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 2×
“5 to compare the corrupted output of a CNN with the radiation-free output, following the T J tradeoff discussion presented in Fawcett et al. [47], which is valid, independent of the source of detection imprecision (intrinsic algorithm detection imprecision or radiation-induced corruption).”
MethodThe pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 1×
“We use the PASCAL VOC 2012 [48] and the Caltech Pedes-trian datasets [49] as input for our study.”
MethodCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · cited 2×
“Convolutional neural networks (CNNs) have, in fact, been deployed in self-driving cars to dynamically identify/classify objects [2], [3].”
BackgroundMethodIdentity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665 · cited 2×
“This process is performed repeatedly for all layers in the network, allowing us to use deeper network structures (e.g., 50 to 1000 layers) without impacting precision, and without increasing the algorithmic complexity [45].”
MethodIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · cited 1×
“As GPUs are highly efficient for accelerating matrix multiplication, they are a great target for CNNs. Specific processors can be designed for DNN, achieving good performance and low power consumption [42], [43].”
Background
Research works printed but not linked: 4
- sridharan, 2009, The effect of input data on program vulnerability, Workshop on Silicon Errors in Logic-System Effects, 1
- 10.1109/TNS.2014.2301768 · DOI
- 2018, Log database for IEEE transaction on reliability, Github
- collobert, 0, BigLearn NIPS Workshop
Cited by
A survey on modeling and improving reliability of DNN algorithms and accelerators
Mittal, S. (2020) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2019.101689 · Cited by 175
Reliable Data Transmission Model for Mobile Ad Hoc Network Using Signcryption Technique
Elhoseny, M., & Shankar, K. (2020) · IEEE Transactions on Reliability · doi:10.1109/tr.2019.2915800 · Cited by 159
“Steering overhead is the principle issue with course disclosure, which causes data loss [19].”
BackgroundA Low-cost Fault Corrector for Deep Neural Networks through Range Restriction
Chen, Z., Li, G., & Pattabiraman, K. (2021) · 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) · doi:10.1109/dsn48987.2021.00018 · Cited by 104
“Techniques based on Algorithm-based Fault Tolerance (ABFT) have also been proposed for detecting faults in particular layers in DNNs (e.g., Conv layer [17], [18], [61]; matrix multiplication [62]).”
Soft errors in DNN accelerators: A comprehensive review
Ibrahim, Y., Wang, H., Liu, J., Wei, J., Chen, L., Rech, P., Adam, K., & Guo, G. (2020) · Microelectronics Reliability · doi:10.1016/j.microrel.2020.113969 · Cited by 102
Self-citationA Systematic Literature Review on Hardware Reliability Assessment Methods for Deep Neural Networks
Ahmadilivani, M. H., Taheri, M., Raik, J., Daneshtalab, M., & Jenihhin, M. (2024) · ACM Computing Surveys · doi:10.1145/3638242 · Cited by 132
“The general model for classifying faults in the mentioned works is as follows: — Masked: Fault does not affect the output, — SDC: Output confidence score differs from that of the golden model, — DUE: The program hangs or the system reboots (also called Crash in References [10, 121]).”
BackgroundMethodInfluentialMaking Convolutions Resilient Via Algorithm-Based Error Detection Techniques
Hari, S. K. S., Sullivan, M. B., Tsai, T., & Keckler, S. W. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3063083 · Cited by 79
Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives
Su, F., Liu, C., & Stratigopoulos, H. (2023) · IEEE Design and Test · doi:10.1109/mdat.2023.3241116 · Cited by 97
“The interested reader is referred to [70], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83], [84], and [85] for more software-based fault injection experiments studying the fault impact for different neural network models, data type representations, layer types, network sizes, pruned networ…”
BackgroundMethodInfluentialRech, P. (2024) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2024.3349956 · Cited by 75
“When adapted to CNNs, ABFT has been shown to outperform ECC or even duplication [11].”
BackgroundMethodResultInfluentialSelf-citationSoft Error Resilience of Deep Residual Networks for Object Recognition
Ibrahim, Y., Wang, H., Bai, M., Liu, Z., Wang, J., Yang, Z., & Chen, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2968129 · Cited by 72
“There have been a number of studies [10], [15]–[18] on the reliability of DNNs.”
BackgroundResultInfluentialChatzidimitriou, A., Bodmann, P. R., Papadimitriou, G., Gizopoulos, D., & Rech, P. (2019) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2019.00018 · Cited by 64
“Some preliminary studies have proposed a comparison or combination of different reliability evaluation techniques [23], [28], [29], [39], [64], [65].”
BackgroundSelf-citationAre CNNs Reliable Enough for Critical Applications? An Exploratory Study
Neggaz, M. A., Alouani, I., Niar, S., & Kurdahi, F. J. (2019) · IEEE Design and Test · doi:10.1109/mdat.2019.2952336 · Cited by 60
“The reliability of object detection networks on GPUs has been studied in [9].”
BackgroundResiliency of automotive object detection networks on GPU architectures
Lotfi, A., Hukerikar, S., Balasubramanian, K., Racunas, P., Saxena, N. R., Bramley, R., & Huang, Y. (2019) · IEEE International Test Conference (ITC) · doi:10.1109/itc44170.2019.9000150 · Cited by 72
“Therefore, while the general trend of the results on the sensitivity of convolutional neural networks to transient faults in [17] match our findings, we are also able to assess the effectiveness and limitations of these chip-level mechanisms for detecting and mitigating faults.”
BackgroundMethodFlexGripPlus: An improved GPGPU model to support reliability analysis
Condia, J. E. R., Du, B., Reorda, M. S., & Sterpone, L. (2020) · Microelectronics Reliability · doi:10.1016/j.microrel.2020.113660 · Cited by 70
SNIFF: Reverse Engineering of Neural Networks With Fault Attacks
Breier, J., Jap, D., Hou, X., Bhasin, S., & Liu, Y. (2021) · IEEE Transactions on Reliability · doi:10.1109/tr.2021.3105697 · Cited by 85
“In some cases, a single fault occurring in a GPU could reduce the reliability of a CNN performance [6].”
BackgroundSoft Error Effects on Arm Microprocessors: Early Estimations versus Chip Measurements
Bodmann, P. R., Papadimitriou, G., Rech, R. L., Gizopoulos, D., & Rech, P. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3128501 · Cited by 56
“7, 8, and 9 show the comparison between FIT rates measured with beam experiments and predicted with GeFIN for SDC, Crashes/AppCrashes, and SysCrashes, respectively.”
BackgroundMethodInfluentialSelf-citationNeuron Fault Tolerance in Spiking Neural Networks
Spyrou, T., El-Sayed, S. A., Afacan, E., Camuñas-Mesa, L. A., Linares-Barranco, B., & Stratigopoulos, H. (2021) · Design, Automation, and Test in Europe · doi:10.23919/date51398.2021.9474081 · Cited by 63
Libano, F., Rech, P., Neuman, B., Leavitt, J. C., Wirthlin, M., & Brunhaver, J. S. (2021) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2021.3050707 · Cited by 52
“The authors have also tested an algorithm-based fault-tolerance (ABFT) technique for matrix multiplication, achieving significant error detection/correction rates [19], [20].”
BackgroundMethodSelf-citationBoosting Bit-Error Resilience of DNN Accelerators Through Median Feature Selection
Ozen, E., & Orailoğlu, A. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3012209 · Cited by 49
“Dos Santos et al. [21] carried out architecture-level error injection as well as neutron beam experiments to assess the reliability of deep learning algorithms on the GPU devices.”
BackgroundGPU Devices for Safety-Critical Systems: A Survey
Perez-Cerrolaza, J., Abella, J., Kosmidis, L., Calderon, A. J., Cazorla, F., & Flores, J. L. (2023) · ACM Computing Surveys · doi:10.1145/3549526 · Cited by 43
“…· Application-dependent: -Detectionandfault-tolerancealgorithmicapproaches[35,49,50,58,76,80,102,110,138–140,156,168–170,195,202,204] -Fault-tolerancebasedonintrinsicapplicationand/orinputdatachar-acteristics[63,78,158,159] Section4”
BackgroundMethodInfluentialRemadna, I., Terrissa, L. S., Masry, Z. A., & Noureddine Zerhouni, A. (2022) · IEEE Transactions on Reliability · doi:10.1109/tr.2022.3190639 · Cited by 45
“Although Convolutional Neural Network (CNN) are one of the most dominant methods for image processing [24], [25], [26], [27], [28] CNNs have also been explored for RUL prediction by [29], [30] on the multi-channel time series.”
MethodImpact of Tensor Cores and Mixed Precision on the Reliability of Matrix Multiplication in GPUs
Basso, P. M., dos Santos, F. F., & Rech, P. (2020) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2020.2977583 · Cited by 41
“An error on the convolution step may lead to an incorrect output detection [10].”
BackgroundSelf-citationHigh Energy and Thermal Neutron Sensitivity of Google Tensor Processing Units
Rech, R. L., Malde, S., Cazzaniga, C., Kastriotou, M., Létiche, M., Frost, C. D., & Rech, P. (2022) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2022.3142092 · Cited by 45
Self-citationAutomated design of error-resilient and hardware-efficient deep neural networks
Schorn, C., Elsken, T., Vogel, S., Runge, A., Guntoro, A., & Ascheid, G. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-04969-6 · Cited by 48
Adam, K., Mohamed, I. I., & Ibrahim, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3076716 · Cited by 49
“First, the authors in [16] studied the only classification model (ResNet) in their paper only by measuring the Precision and Recall of the corrupted output, without getting insights into how errors propagate among ResNet’s layers or kernels.”
BackgroundMethodRevealing GPUs Vulnerabilities by Combining Register-Transfer and Software-Level Fault Injection
dos Santos, F. F., Condia, J. E. R., Carro, L., Reorda, M. S., & Rech, P. (2021) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn48987.2021.00042 · Cited by 51
Self-citationAbich, G., Gava, J., Garibotti, R., Reis, R. A. L., & Ost, L. C. (2021) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2021.3097981 · Cited by 40
FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions
Ghavami, B., Sadati, M., Fang, Z., & Shannon, L. (2022) · Design, Automation, and Test in Europe · doi:10.23919/date54114.2022.9774635 · Cited by 42
Transient-Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead
Cavagnero, N., dos Santos, F. F., Ciccone, M., Averta, G., Tommasi, T., & Rech, P. (2022) · IEEE International On-Line Testing Symposium (IOLTS) · doi:10.1109/iolts56730.2022.9897813 · Cited by 45
“To ensure effective hardening solutions, we inject, the convolution fault models that we have observed with beam experiments or RTL fault injections [2], [30], [31].”
BackgroundMethodInfluentialSelf-citationA Multi-level Approach to Evaluate the Impact of GPU Permanent Faults on CNN's Reliability
Condia, J. E. R., Guerrero-Balaguera, J.-D., dos Santos, F. F., Reorda, M. S., & Rech, P. (2022) · IEEE International Test Conference (ITC) · doi:10.1109/itc50671.2022.00036 · Cited by 41
“The sensitivity of GPUs and parallel applications, including Convolutional Neural Networks (CNNs), to radiation and other sources of transient faults, has been widely studied through beam experiments [1], software fault injection [2], [3], and microarchitectural fault simulation [4], [5].”
BackgroundSelf-citationEfficient Error Detection for Matrix Multiplication With Systolic Arrays on FPGAs
Libano, F., Rech, P., & Brunhaver, J. S. (2023) · IEEE Transactions on Computers · doi:10.1109/tc.2023.3248282 · Cited by 32
Self-citationSNR: S queezing N umerical R ange Defuses Bit Error Vulnerability Surface in Deep Neural Networks
Ozen, E., & Orailoğlu, A. (2021) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3477007 · Cited by 39
“Previous literature has witnessed a flurry of works exploring the error resilience limits of deep neural networks through error simulation [28, 32, 33, 47, 48, 56, 59, 72] and physical error injection experiments [13, 42].”
BackgroundBurel, S., Evans, A., & Anghel, L. (2022) · IEEE Transactions on Device and Materials Reliability · doi:10.1109/tdmr.2022.3159089 · Cited by 35
Estimating Vulnerability of All Model Parameters in DNN with a Small Number of Fault Injections
Zhang, Y., Itsuji, H., Uezono, T., Toba, T., & Hashimoto, M. (2022) · Design, Automation, and Test in Europe · doi:10.23919/date54114.2022.9774569 · Cited by 34
dos Santos, F. F., Hari, S. K. S., Basso, P. M., Carro, L., & Rech, P. (2021) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps49936.2021.00037 · Cited by 32
“On CNNs, some faults that propagate to the output are not considered errors since they do not modify the classification result [33].”
BackgroundSelf-citationAnalyzing DUE Errors on GPUs With Neutron Irradiation Test and Fault Injection to Control Flow
Ito, K., Zhang, Y., Itsuji, H., Uezono, T., Toba, T., & Hashimoto, M. (2021) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2021.3098845 · Cited by 32
“However, the previous works of radiation experiments [1]–[5], which were mentioned in a previous paragraph, and fault injection experiments [2], [7]–[11] focused on the categorization between SDC and DUE and SDC pattern analysis, and detailed categorization of DUE was not investigated.”
BackgroundMethodImpact of Reduced Precision in the Reliability of Deep Neural Networks for Object Detection
dos Santos, F. F., Navaux, P. O. A., Carro, L., & Rech, P. (2019) · IEEE European Test Symposium (ETS) · doi:10.1109/ets.2019.8791554 · Cited by 35
“Object detection DNNs have a much higher probability of experiencing DUEs when compared to arithmetic codes.”
BackgroundResultSelf-citationReliability Analysis of a Spiking Neural Network Hardware Accelerator
Spyrou, T., El-Sayed, S. A., Afacan, E., Camuñas-Mesa, L. A., Linares-Barranco, B., & Stratigopoulos, H. (2022) · Design, Automation, and Test in Europe · doi:10.23919/date54114.2022.9774711 · Cited by 25
“Few works exist demonstrating fault injection in actual hardware, in particular fault injection in FPGA implementations [16] and radiation experiments on ANNs running in GPUs [17], [18] and FPGAs [19].”
BackgroundFireNN: Neural Networks Reliability Evaluation on Hybrid Platforms
Sio, C. D., Azimi, S., & Sterpone, L. (2022) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2022.3152668 · Cited by 27
Characterizing a Neutron-Induced Fault Model for Deep Neural Networks
dos Santos, F. F., Kritikakou, A. S., Condia, J. E. R., Guerrero-Balaguera, J.-D., Reorda, M. S., Sentieys, O., & Rech, P. (2022) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2022.3224538 · Cited by 34
“In recent works, the DNNs fault model on GPUs has been analyzed to provide a more realistic reliability evaluation using fault injection [10], [21], [41].”
InfluentialSelf-citationDigital-twin-assisted meta learning for soft-failure localization in ROADM-based optical networks
Wang, R., Zhang, J., Gu, Z., Ibrahimi, M., Zhang, B., Musumeci, F., Tornatore, M., & Ji, Y. (2024) · Journal of Optical Communications and Networking · doi:10.1364/jocn.517551 · Cited by 25
“There are some research works about fault injection in edge computing systems [28], network applications [29], and ML models [30].”
BackgroundAnalyzing the impact of soft errors in VGG networks implemented on GPUs
Wei, J., Ibrahim, Y., Qian, S., Wang, H., Liu, G., Yu, Q., Qian, R., & Shi, J. (2020) · Microelectronics Reliability · doi:10.1016/j.microrel.2020.113648 · Cited by 31
Wang, S., Wu, T., & Wang, K. (2021) · Wear · doi:10.1016/j.wear.2021.203696 · Cited by 24
Reliability of Google's Tensor Processing Units for Embedded Applications
Rech, R. L., & Rech, P. (2022) · Design, Automation, and Test in Europe · doi:10.23919/date54114.2022.9774600 · Cited by 32
Self-citationDependable DNN Accelerator for Safety-Critical Systems: A Review on the Aging Perspective
Moghaddasi, I., Gorgin, S., & Lee, J. (2023) · IEEE Access · doi:10.1109/access.2023.3300376 · Cited by 30
“In the first one, physical faults are inducted in real hardware devices [33], [34], emulated on hybrid platforms [60], virtually injected in model”
BackgroundInfluentialOzen, E., & Orailoğlu, A. (2020) · doi:10.1145/3400302.3415680 · Cited by 42
“The bit-error resilience of DNNs is examined in the prior literature through error injection experiments on the DNN graph [17, 21, 25] and physical neutron beam experiments on GPUs [4].”
MethodEvaluating and Mitigating Neutrons Effects on COTS EdgeAI Accelerators
Blower, S., Rech, P., Cazzaniga, C., Kastriotou, M., & Frost, C. D. (2021) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2021.3086686 · Cited by 24
Self-citationCompact Functional Testing for Neuromorphic Computing Circuits
El-Sayed, S. A., Spyrou, T., Camuñas-Mesa, L. A., & Stratigopoulos, H. (2022) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2022.3223843 · Cited by 31
“Recent fault injection experiments in AI hardware accelerators [10], [11], [12], [13], [14], [15], [16] have shown that there is some inherent fault tolerance since many faults are benign, i.e., they are masked before their effect propagates to the output layer or they produce an output change that…”
BackgroundMethodGutiérrez‐Zaballa, J., Basterretxea, K., & Echanobe, J. (2024) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2024.103242 · Cited by 29
Soft Error Reliability Assessment of Neural Networks on Resource-constrained IoT Devices
Abich, G., Gava, J., Reis, R. A. L., & Ost, L. C. (2020) · IEEE International Conference on Electronics, Circuits and Systems (ICECS) · doi:10.1109/icecs49266.2020.9294951 · Cited by 24
“In turn, authors in [6] evaluate the reliability of a convolution neural network (CNN) in the presence of soft errors in GPUs.”
BackgroundAssessment of a Hardware-Implemented Machine Learning Technique Under Neutron Irradiation
Trindade, M. G., Coelho, A. A. P., Valadares, C., Viera, R., Rey, S., Cheymol, B., Baylac, M., Velazco, R., & Bastos, R. P. (2019) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2019.2920747 · Cited by 21
“Previous works on other machine learning techniques, such as the Bayesian machines [10] (not evaluated under radiation) and ANNs/CNNs [7]–[9], have also shown that these techniques do have a built-in fault tolerance.”
BackgroundChallenges of Reliability Assessment and Enhancement in Autonomous Systems
Jenihhin, M., Reorda, M. S., Balakrishnan, A., & Alexandrescu, D. (2019) · Proceedings · doi:10.1109/dft.2019.8875379 · Cited by 25
“While several theoretical and experimental analysis provided information about the Architecture Vulnerability Factor [26] of traditional architectures [27], only preliminary results have been provided concerning the new ones and about the main modules they are composed of [28],[29],[30].”
BackgroundExposing Reliability Degradation and Mitigation in Approximate DNNs Under Permanent Faults
Siddique, A., & Hoque, K. A. (2023) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2023.3238907 · Cited by 21
“[21] injected the permanent faults in the register files of GPU to demonstrate their effect on reduced precision AccDNNs.”
BackgroundAnalyzing the Reliability of Convolutional Neural Networks on GPUs: GoogLeNet as a Case Study
Ibrahim, Y., Wang, H., & Adam, K. (2020) · International Conference on Computing and Information Technology (ICCIT) · doi:10.1109/iccit-144147971.2020.9213804 · Cited by 27
“They use filters to extract features of the image, by sliding a filter over the input image, multiplying and accumulating products at every position of the input (i.e., receptive field) with this filter [7].”
MethodCondia, J. E. R., dos Santos, F. F., Reorda, M. S., & Rech, P. (2021) · IEEE VLSI Test Symposium (VTS) · doi:10.1109/vts50974.2021.9441044 · Cited by 26
“It has been shown that in modern CNNs less than 10% of SDCs modify detection and the great majority of them originates from faults in convolutional layers [8], [13].”
BackgroundInfluentialSelf-citationReliability of Google’s Tensor Processing Units for Convolutional Neural Networks
Rech, R. L., & Rech, P. (2022) · IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S) · doi:10.1109/dsn-s54099.2022.00018 · Cited by 20
Self-citationGava, J., Hanneman, A., Abich, G., Garibotti, R., Cuenca-Asensi, S., Bastos, R. P., Reis, R. A. L., & Ost, L. C. (2023) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2023.3262448 · Cited by 22
“On the soft error mitigation side, in addition to this article, only three works have considered hardened inference models in their radiation experiments [6], [11], [16].”
BackgroundAn Uncertainty-Aware Deep Learning Model for Reliable Detection of Steel Wire Rope Defects
Yi, W., Chan, W. K., Lee, H. H., Boles, S. T., & Zhang, X. (2023) · IEEE Transactions on Reliability · doi:10.1109/tr.2023.3335958 · Cited by 23
“…detection, most studies in the literature have rarely considered the quantification of uncertainty in model prediction, the resulting lack of safety and reliability assurance has increasingly emerged as an impending roadblock to the adoption of deep learning in risk-sensitive fields [41], [42], [43].”
BackgroundBrewer, R. M., Reed, R. A., Moran, S. L., Cox, J. A., Sierawski, B. D., McCurdy, M. W., Zhang, E. X., Iyer, S. S., Schrimpf, R. D., & Alles, M. L. (2019) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2019.2957477 · Cited by 20
“It was found in [6] that a single fault in a GPU tends to spread through multiple threads, and, therefore, it is important to be able to detect and correct errors in critical applications.”
BackgroundReliability and Security of AI Hardware
Gnad, D. R. E., Gotthard, M., Krautter, J., Kritikakou, A. S., Meyers, V., Rech, P., Condia, J. E. R., Ruospo, A., Sánchez, E., dos Santos, F. F., et al. (2024) · IEEE European Test Symposium (ETS) · doi:10.1109/ets61313.2024.10567471 · Cited by 25
“Since the floating-point representation range is large [10], [55], [61]–[65], the corrupted values can go to infinity or even become Not a Number (NaN).”
BackgroundMethodInfluentialSelf-citationThe Impact of SoC Integration and OS Deployment on the Reliability of Arm Processors
Bodmann, P. R., Papadimitriou, G., Gizopoulos, D., & Rech, P. (2021) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass51385.2021.00040 · Cited by 18
Self-citationEfficient Soft-Error Detection for Low-precision Deep Learning Recommendation Models
Li, S., Huang, J., Tang, P. T. P., Khudia, D. S., Park, J., Dixit, H. D., & Chen, Z. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020972 · Cited by 19
“Recently, it has been applied and optimized for convolutional neural networks (CNN) [14], [24], [25].”
MethodResilience Assessment of Large Language Models under Transient Hardware Faults
Agarwal, U. K., Chan, A., & Pattabiraman, K. (2023) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre59848.2023.00052 · Cited by 35
“Since SDCs are rare events, for the SDC rate, we used the confidence intervals proposed by Leveugle et al. [55], which is in line with prior work on FI [25, 56].”
ResultUnderstanding the Effects of Permanent Faults in GPU's Parallelism Management and Control Units
Guerrero-Balaguera, J.-D., Condia, J. E. R., dos Santos, F. F., Reorda, M. S., & Rech, P. (2023) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3581784.3607086 · Cited by 32
Self-citationG-SEPM: building an accurate and efficient soft error prediction model for GPGPUs
Yue, H., Wei, X., Li, G., Zhao, J., Jiang, N., & Tan, J. (2021) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3458817.3476170 · Cited by 22
“A large number of studies focus on building an error propagation model to estimate the resilience of both CPU and GPU programs [1, 12, 15, 19, 21, 23, 24, 39, 43].”
BackgroundDeepVigor: VulnerabIlity Value RanGes and FactORs for DNNs’ Reliability Assessment
Ahmadilivani, M. H., Taheri, M., Raik, J., Daneshtalab, M., & Jenihhin, M. (2023) · IEEE European Test Symposium (ETS) · doi:10.1109/ets56758.2023.10174133 · Cited by 25
“FI outputs different evaluations for DNNs’ reliability by accuracy loss, vulnerability factors, or fault classification [11], [13], [14].”
BackgroundDetecting Errors in Convolutional Neural Networks Using Inter Frame Spatio-Temporal Correlation
Draghetti, L. K., dos Santos, F. F., Carro, L., & Rech, P. (2019) · IEEE International On-Line Testing Symposium (IOLTS) · doi:10.1109/iolts.2019.8854431 · Cited by 17
“The authors granted the access to the data that was previously published on [13].”
BackgroundSelf-citationAssessment of Machine Learning Algorithms for Near-Sensor Computing under Radiation Soft Errors
Trindade, M. G., Bastos, R. P., Garibotti, R., Ost, L. C., Létiche, M., & Beaucour, J. (2020) · IEEE International Conference on Electronics, Circuits and Systems (ICECS) · doi:10.1109/icecs49266.2020.9294938 · Cited by 17
“[9] investigate how the presence of soft errors in GPUs can reduce the reliability of a CNN.”
BackgroundReduced Precision DWC: An Efficient Hardening Strategy for Mixed-Precision Architectures
dos Santos, F. F., Brandalero, M., Sullivan, M. B., Basso, P. M., Hübner, M., Carro, L., & Rech, P. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3058872 · Cited by 21
“Even for some safety-critical applications, such as Convolutional Neural Networks (CNNs) for objects detection, small variations in the output values are unlikely to induce critical faults [38], [39].”
BackgroundMethodSelf-citationBadía, J. M., León, G., Belloch, J. A., Lindoso, A., García-Valderas, M., Morilla, Y., & Entrena, L. (2022) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2022.3155820 · Cited by 21
harDNNing: a machine-learning-based framework for fault tolerance assessment and protection of DNNs
Traiola, M., Kritikakou, A. S., & Sentieys, O. (2023) · IEEE European Test Symposium (ETS) · doi:10.1109/ets56758.2023.10174178 · Cited by 23
“fabrication testing that manifest themselves later in the field of application, silicon ageing, and environmental stress, such as heat, humidity, vibration, and radiation [3].”
BackgroundAssessment of Tiny Machine-Learning Computing Systems Under Neutron-Induced Radiation Effects
Bastos, R. P., Trindade, M. G., Garibotti, R., Gava, J., Reis, R. A. L., & Ost, L. C. (2022) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2022.3176485 · Cited by 17
Understanding fault-tolerance vulnerabilities in advanced SoC FPGAs for critical applications
Cherezova, N., Shibin, K., Jenihhin, M., & Jutman, A. (2023) · Microelectronics Reliability · doi:10.1016/j.microrel.2023.115010 · Cited by 15
Bleier, N., Mubarik, M. H., Swenson, G. R., & Kumar, R. (2023) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3613424.3614271 · Cited by 29
“Due to inherent algorithmic resilience to radiation-induced bit-flip errors for such workloads [125], the overhead of software-based soft-error hardening will be low: 18% [3], 5% for convolution layers [132].”
Impact of High-Level Synthesis on Reliability of Artificial Neural Network Hardware Accelerators
Traiola, M., dos Santos, F. F., Rech, P., Cazzaniga, C., Sentieys, O., & Kritikakou, A. S. (2024) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2024.3377596 · Cited by 15
“For instance, reliability analysis is performed for CNNs on NVIDIA GPUs [10] and Arm micro-processors [11].”
Self-citationSelective Hardening of CNNs based on Layer Vulnerability Estimation
Bolchini, C., Cassano, L., Miele, A., & Nazzari, A. (2022) · Proceedings · doi:10.1109/dft56152.2022.9962339 · Cited by 21
“However, applications exploiting deep learning generally expose an intrinsic degree of fault resiliency [9].”
BackgroundMethodA Systematic Methodology to Compute the Quantum Vulnerability Factors for Quantum Circuits
Oliveira, D., Giusto, E., Baheri, B., Guan, Q., Montrucchio, B., & Rech, P. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3313934 · Cited by 24
“The fact that a fault improves the output quality should not surprise, as it has been observed in probabilisticclassical-suchasartificialneuralnetworks(despite withmuchlowerprobability)[49]-andquantumcomputation applications[50].”
BackgroundSelf-citationQuantifying the Impact of Memory Errors in Deep Learning
Zhang, Z., Huang, L., Huang, R., Xu, W., & Katz, D. S. (2019) · IEEE International Conference on Cluster Computing (CLUSTER) · doi:10.1109/cluster.2019.8890989 · Cited by 17
“Dos Santos et al. examined the failure in time (FIT, failures in 109operatinghours) of several GPUs for inference, where application crash and hang results in severe consequences.”
BackgroundMethodInfluentialAn Emulation Platform for Evaluating the Reliability of Deep Neural Networks
Sio, C. D., Azimi, S., & Sterpone, L. (2020) · Proceedings · doi:10.1109/dft50435.2020.9250872 · Cited by 14
“In [10], the authors perform beam experiments and fault injection on different GPU architectures.”
BackgroundAnalyzing and increasing soft error resilience of Deep Neural Networks on ARM processors
Liu, Z., Liu, Y., Chen, Z., Guo, G., & Wang, H. (2021) · Microelectronics Reliability · doi:10.1016/j.microrel.2021.114331 · Cited by 14
Assessing the Impact of Compiler Optimizations on GPUs Reliability
dos Santos, F. F., Carro, L., Vella, F., & Rech, P. (2024) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3638249 · Cited by 15
“The GPU reliability has already been evaluated through beam experiments in previous works [3, 20, 24, 29, 44, 49, 51].”
Self-citationBadía, J. M., León, G., Belloch, J. A., García-Valderas, M., Lindoso, A., & Entrena, L. (2021) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2021.3128722 · Cited by 17
Experimental Findings on the Sources of Detected Unrecoverable Errors in GPUs
dos Santos, F. F., Malde, S., Cazzaniga, C., Frost, C. D., Carro, L., & Rech, P. (2022) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2022.3141341 · Cited by 14
Self-citationA prediction method of regional water resources carrying capacity based on artificial neural network
Shi, C., & Zhang, Z. (2021) · Earth Sciences Research Journal · doi:10.15446/esrj.v25n2.81615 · Cited by 13
“, that is, the accuracy is 90% (Dos Santos et al., 2019); when the number of iterations is 34735, the network reaches the required accuracy, E0 0999924.”
BackgroundReliability of Vision Transformers and CNNs on Edge AI Systems Under Neutron Radiation
Badia, J. M., Martin-Salinas, I., Leon, G., Amor-Martin, A., Frias-Dominguez, L., Belloch, J. A., Garcia-Valderas, M., Lindoso, A., Cazzaniga, C., & Entrena, L. (2025) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2025.3536519 · Cited by 11
“They can detect and correct between 50% and 60% of the radiation-induced errors. dos Santos et al. [37] combined the ABFT strategy with a redesign of the maxpool layers and can detect up to 98% of the critical SDCs and correct more Authorized licensed use limited to the terms of the applicable…”
InfluentialEvaluating the soft error sensitivity of a GPU-based SoC for matrix multiplication
León, G., Badía, J. M., Belloch, J. A., Lindoso, A., & Entrena, L. (2020) · Microelectronics Reliability · doi:10.1016/j.microrel.2020.113856 · Cited by 13
Evaluation and Mitigation of Faults Affecting Swin Transformers
Gavarini, G., Ruospo, A., & Sánchez, E. (2023) · IEEE International On-Line Testing Symposium (IOLTS) · doi:10.1109/iolts59296.2023.10224882 · Cited by 15
“In particular, they propose to use ABFT to strengthen linear operations and range-restriction for non-linear computations.”
BackgroundMethodInfluentialCross-Layer Reliability Analysis of NVDLA Accelerators: Exploring the Configuration Space
Veronesi, A., Nazzari, A., Passarello, D., Krstic, M. D., Favalli, M., Cassano, L., Miele, A., Bertozzi, D., & Bolchini, C. (2024) · IEEE European Test Symposium (ETS) · doi:10.1109/ets61313.2024.10568018 · Cited by 12
Deligiannis, N. I., Cantoro, R., Reorda, M. S., Traiola, M., & Valea, E. (2021) · IEEE Access · doi:10.1109/access.2021.3129149 · Cited by 11
“In [31], the authors perform an analysis on the reliability of CNNs and employ fault tolerance techniques to enhance their”
MethodOpen-Set Recognition: an Inexpensive Strategy to Increase DNN Reliability
Gavarini, G., Stucchi, D., Ruospo, A., Boracchi, G., & Sánchez, E. (2022) · IEEE International On-Line Testing Symposium (IOLTS) · doi:10.1109/iolts56730.2022.9897805 · Cited by 14
“A high-level solution is provided by [19], where the authors propose a fault detector called Reliable Maxpooling , where the output of all the network’s Maxpool layers is compared against a threshold.”
BackgroundSpecial Session: Reliability Assessment Recipes for DNN Accelerators
Ahmadilivani, M. H., Bosio, A., Deveautour, B., dos Santos, F. F., Guerrero-Balaguera, J.-D., Jenihhin, M., Kritikakou, A. S., Sierra, R. L., Pappalardo, S., Raik, J., et al. (2024) · IEEE VLSI Test Symposium (VTS) · doi:10.1109/vts60656.2024.10538707 · Cited by 13
Self-citationThermal neutrons: a possible threat for supercomputer reliability
Oliveira, D., Blanchard, S. P., DeBardeleben, N., dos Santos, F. F., Dávila, G. P., Navaux, P., Favalli, A., Schappert, O., Wender, S. A., Cazzaniga, C., et al. (2020) · The Journal of Supercomputing · doi:10.1007/s11227-020-03324-9 · Cited by 10
“This result follows previous work that shows low SDC sensitivity in CNN based object detection [47].”
ResultSelf-citationAn Effective Method to Identify Microarchitectural Vulnerabilities in GPUs
Condia, J. E. R., Rech, P., dos Santos, F. F., Carro, L., & Reorda, M. S. (2022) · IEEE Transactions on Device and Materials Reliability · doi:10.1109/tdmr.2022.3166260 · Cited by 13
“Other reliability evaluation methodologies are based on exposing real GPUs to beam experiments [12], [23], [24], which are accurate when targeting application characterizations and consist on executing the target workloads on real devices in the presence of external sources able to produce faults…”
Self-citationJiang, W., Yu, H., Chen, F., & Ha, Y. (2023) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2023.3235803 · Cited by 11
Soft Error and Its Countermeasures in Terrestrial Environment
Hashimoto, M., & Liao, W. (2020) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac47756.2020.9045161 · Cited by 10
“Therefore, soft error rate of GPU, especially, application-level error rate is drawing a lot of attention, and several experiments are performed and reported [22], [23].”
BackgroundFault Injectors for TensorFlow: Evaluation of the Impact of Random Hardware Faults on Deep CNNs
Beyer, M., Morozov, A. S., Valiev, E., Schorn, C., Gauerhof, L., Ding, K., & Janschek, K. (2020) · European Safety and Reliability Conference (ESREL) · doi:10.3850/978-981-14-8593-0_5754-cd · Cited by 12
Impact of Thread Parallelism on the Soft Error Reliability of Convolution Neural Networks
Abich, G., Garibotti, R., Gava, J., Reis, R. A. L., & Ost, L. C. (2022) · IEEE Latin American Symposium on Circuits and Systems (LASCAS) · doi:10.1109/lascas53948.2022.9789088 · Cited by 13
Analysis of Kernel Redundancy for Soft Error Mitigation on Embedded GPUs
Serrano-Cases, A., Alcaide, S., Romero-Maestre, A., Morilla, Y., & Cuenca-Asensi, S. (2023) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2023.3291418 · Cited by 8
“Nevertheless, there is no clear relationship between them since other studies, such as those in [21] and [5], reported a different behavior on Tegra X1 SoC.”
BackgroundMethodResultEvaluating the Reliability of Supervised Compression for Split Computing
Guerrero-Balaguera, J.-D., Condia, J. E. R., Levorato, M., & Reorda, M. S. (2024) · IEEE VLSI Test Symposium (VTS) · doi:10.1109/vts60656.2024.10538938 · Cited by 10
“Indeed, previous works on DNN’s reliability [8], [9] have demonstrated that (transient and permanent) faults affecting hardware structures can induce errors at the algorithmic level of the DNN, resulting in wrong predictions that might lead to system malfunctions, reduced efficiency, or…”
BackgroundImpact of Radiation-Induced Effects on Embedded GPUs Executing Large Machine Learning Models
Coelho, B. L., dos Santos, F. F., Saveriano, M., Allen, G. R., Daniel, A. C., Guertin, S. M., Vartanian, S., Wyrwas, E. J., Frost, C. D., & Rech, P. (2025) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2025.3528764 · Cited by 8
“A similar behavior was observed in related work [28], where the authors analyze the shapes of error distributions in CNNs, showing that line and square (rectangular) errors in convolution layers propagate through the max-pooling layers.”
Self-citationWei, X., Zhou, C., Yue, H., & Zhou, J. T. (2023) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2023.103024 · Cited by 10
Metadata
- Authors
- Santos, F. F. D., Pimenta, P. F., Lunardi, C., Draghetti, L., Carro, L., Kaeli, D., & Rech, P.
- Affiliations
- Universidade Federal do Rio Grande do Sul (BR)
- Northeastern University (US)
- ORCID
- Venue
- IEEE Transactions on Reliability
- Published
- 2019-06
- Volume
- 68
- Issue
- 2
- Pages
- 663-677
- ISSN
- 0018-9529 (ISSN-L)
- 1558-1721
- OpenAlex
- W2901848761
- Cited by
- 226
- References
- 67
- Citations by year
- 2026: 18 · 2025: 18 · 2024: 41 · 2023: 33 · 2022: 42 · 2021: 34 · 2020: 27 · 2019: 13
- FWCI
- 7.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Reliability (publishedVersion)
- Institutional Research Information System (Università degli Studi di Trento) (submittedVersion)
- Science and Technology Facilities Council (submittedVersion)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Radiation Effects in Electronics
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- convolutional neural networks, GPU reliability, fault injection, silent data corruption, error-correcting codes, algorithm-based fault tolerance, matrix multiplication, object detection, safety-critical systems, fault propagation, NVIDIA GPUs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.