Analyzing and Increasing the Reliability of Convolutional Neural Networks on GPUs
The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
Graphics processing units (GPUs) are playing a critical role in convolutional neural networks (CNNs) for image detection. As GPU-enabled CNNs move into safety-critical environments, reliability is becoming a growing concern. In this paper, we evaluate and propose strategies to improve the reliability of object detection algorithms, as run on three NVIDIA GPU architectures. We consider three algorithms: 1) you only look once; 2) a faster region-based CNN (Faster R-CNN); and 3) a residual network, exposing live hardware to neutron beams. We complement our beam experiments with fault injection to better characterize fault propagation in CNNs. We show that a single fault occurring in a GPU tends to propagate to multiple active threads, significantly reducing the reliability of a CNN. Moreover, relying on error correcting codes dramatically reduces the number of silent data corruptions (SDCs), but does not reduce the number of critical errors (i.e., errors that could potentially impact safety-critical applications). Based on observations on how faults propagate on GPU architectures, we propose effective strategies to improve CNN reliability. We also consider the benefits of using an algorithm-based fault-tolerance technique for matrix multiplication, which can correct more than 87% of the critical SDCs in a CNN, while redesigning maxpool layers of the CNN to detect up to 98% of critical SDCs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“as commented in [9], SASSIFI is the only tool working with the NVIDIA proprietary libraries generally employed when implementing CNNs.”
From Fast and Accurate Error Simulation for CNNs Against Soft Errors (2023) · cites this paper as [9]
In the citing paper
“radiation test offers the most adherent solution to cause soft errors, however, apart from the highly expensive equipment and the complex setup, there is little to no controllability on the injection process, and the observability is only on the primary outputs. As a result, it is suitable for final validation and blackbox campaigns, not for in-depth analysis and investigations [9], [36].”
From Fast and Accurate Error Simulation for CNNs Against Soft Errors (2023) · cites this paper as [9]
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“Specifically, neural networks for object detection have the peculiarity of using convolution to reduce the information to be processed [15], a class of computations where reliability has not been studied deeply to date.”
BackgroundYou Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 2×
“5 to compare the corrupted output of a CNN with the radiation-free output, following the T J tradeoff discussion presented in Fawcett et al. [47], which is valid, independent of the source of detection imprecision (intrinsic algorithm detection imprecision or radiation-induced corruption).”
MethodThe pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 1×
“We use the PASCAL VOC 2012 [48] and the Caltech Pedes-trian datasets [49] as input for our study.”
MethodCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · cited 2×
“Convolutional neural networks (CNNs) have, in fact, been deployed in self-driving cars to dynamically identify/classify objects [2], [3].”
BackgroundMethodIdentity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665 · cited 2×
“This process is performed repeatedly for all layers in the network, allowing us to use deeper network structures (e.g., 50 to 1000 layers) without impacting precision, and without increasing the algorithmic complexity [45].”
MethodIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · cited 1×
“As GPUs are highly efficient for accelerating matrix multiplication, they are a great target for CNNs. Specific processors can be designed for DNN, achieving good performance and low power consumption [42], [43].”
BackgroundMulti-column deep neural networks for image classification
Ciresan, D., Meier, U., & Schmidhuber, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248110 · Cited by 3,720
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198 · cited 1×
“GPUs can detect obstacles in a scene in near real time, a critical task in next-generation autonomous vehicles, and are already present in many self-driving cars [4].”
BackgroundRadiation-induced soft errors in advanced semiconductor technologies
Baumann, R. C. (2005) · IEEE Transactions on Device and Materials Reliability · doi:10.1109/tdmr.2005.853449 · Cited by 1,549 · cited 3×
“FIT does not dependonthe execution time (nor frames/s) [8].”
BackgroundInfluentialEvaluation of Pooling Operations in Convolutional Architectures for Object Recognition
Scherer, D., Müller, A., & Behnke, S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15825-4_10 · Cited by 1,503
Pedestrian detection: A benchmark
Dollar, P., Wojek, C., Schiele, B., & Perona, P. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206631 · Cited by 1,313 · cited 1×
“We use the PASCAL VOC 2012 [48] and the Caltech Pedes-trian datasets [49] as input for our study.”
Method(2012) · neural information processing systems · Cited by 1,291
Algorithm-Based Fault Tolerance for Matrix Operations
Huang, K.-H., & Abraham, J. A. (1984) · IEEE Transactions on Computers · doi:10.1109/tc.1984.1676475 · Cited by 1,283 · cited 3×
“Some promising results have shown that algorithm-based fault tolerance (ABFT) [12] is a very efficient strategy to protect matrix multiplication [30], [31] and fast Fourier transform (FFT) [32] in GPUs.”
BackgroundMethodcuDNN: Efficient Primitives for Deep Learning
Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J. J., Catanzaro, B., & Shelhamer, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0759 · Cited by 943 · cited 2×
“However, they are less efficient than GEMM-based convolutions [41].”
BackgroundMukherjee, S. S., Weaver, C. T., Emer, J., Reinhardt, S. K., & Austin, T. M. (2003) · Cited by 840
Mukherjee, S. S., Weaver, C., Emer, J., Reinhardt, S. K., & Austin, T. (2004) · 22nd Digital Avionics Systems Conference. Proceedings (Cat. No.03CH37449) · doi:10.1109/micro.2003.1253181 · Cited by 656 · cited 1×
“RF injections measure RF architectural vulnerability factor (AVF) and are used to study how applications propagate errors in memory elements [65].”
MethodTime redundancy based soft-error tolerance to rescue nanometer technologies
Nicolaidis, M. (2003) · Proceedings 17th IEEE VLSI Test Symposium (Cat. No.PR00146) · doi:10.1109/vtest.1999.766651 · Cited by 500 · cited 1×
“There are several sources of transient errors that can undermine GPU reliability in these systems, including environmental perturbations, software errors, and process/temperature/voltage variations [6], [7].”
BackgroundUnderstanding error propagation in deep learning neural network (DNN) accelerators and applications
Li, G., Hari, S. K. S., Sullivan, M., Tsai, T., Pattabiraman, K., Emer, J., & Keckler, S. W. (2017) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/3126908.3126964 · Cited by 492 · cited 2×
“Some solutions require specific hardware mod-ifications [10], [16].”
BackgroundDEPENDABLE COMPUTING AND FAULT TOLERANCE : CONCEPTS AND TERMINOLOGY
Laprie, J.-C. (2005) · Twenty-Fifth International Symposium on Fault-Tolerant Computing, 1995, ' Highlights from Twenty-Five Years'. · doi:10.1109/ftcsh.1995.532603 · Cited by 460 · cited 1×
“and process/temperature/voltage variations [6], [7].”
Jedec (2001) · Medical Entomology and Zoology · Cited by 454
Statistical fault injection: Quantified error and confidence
Leveugle, R., Calvez, A., Maistri, P., & Vanhauwaert, P. (2009) · 2009 Design, Automation & Test in Europe Conference & Exhibition · doi:10.1109/date.2009.5090716 · Cited by 401 · cited 1×
“As shown in [13] and [66], injecting a higher number of faults would not significantly change our statistics.”
BackgroundFixed-point feedforward deep neural network design using weights +1, 0, and −1
Hwang, K., & Sung, W. (2014) · 2014 IEEE Workshop on Signal Processing Systems (SiPS) · doi:10.1109/sips.2014.6986082 · Cited by 244
Comparison of error rates in combinational and sequential logic
Buchner, S., Baze, M., Brown, D., McMorrow, D., & Melinger, J. (1997) · IEEE Transactions on Nuclear Science · doi:10.1109/23.659037 · Cited by 244
Fixed point optimization of deep convolutional neural networks for object recognition
Anwar, S., Hwang, K., & Sung, W. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178146 · Cited by 239 · cited 1×
“Recent research has been focused on the option of using fixed-point or integer operations for object detection [60]–[62].”
BackgroundReal-Time Pedestrian Detection with Deep Network Cascades
Angelova, A., Krizhevsky, A., Vanhoucke, V., Ogale, A., & Ferguson, D. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.32 · Cited by 239 · cited 1×
“Prior work has tried to adopt DNNs for real-time object detection, showing promising results [39], [40].”
BackgroundDefect and error tolerance in the presence of massive numbers of defects
Breuer, M. A., Gupta, S. K., & Mak, T. M. (2004) · IEEE Design and Test of Computers · doi:10.1109/mdt.2004.8 · Cited by 227 · cited 1×
“These reliability weaknesses arise due to the fact that GPUs were originally designed for graphics rendering, a task where high reliability is less of a concern [28].”
BackgroundSASSIFI: An architecture-level fault injection tool for GPU application resilience evaluation
Hari, S. K. S., Tsai, T., Stephenson, M., Keckler, S. W., & Emer, J. (2017) · 2017 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass.2017.7975296 · Cited by 181 · cited 4×
“Finally, we evaluate error propagation in YOLO when running on a Kepler device using the NVIDIA SASSIFI fault injector [13].”
BackgroundMethodInfluentialTiwari, D., Gupta, S., Rogers, J., Maxwell, D., Rech, P., Vazhkudai, S., Oliveira, D., Londo, D., DeBardeleben, N., Navaux, P., et al. (2015) · 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2015.7056044 · Cited by 179 · cited 1×
“This result is in contrast with the trends observed when running GEMM, as discussed in Section V-C, and for HPC applications, as presented in prior work [26], [55], where the crash rate is lower than the SDC rate.”
ResultSelf-citationDaDianNao: A Neural Network Supercomputer
Luo, T., Liu, S., Li, L., Wang, Y., Zhang, S., Chen, T., Xu, Z., Temam, O., & Chen, Y. (2017) · IEEE Transactions on Computers · doi:10.1109/tc.2016.2574353 · Cited by 175 · cited 1×
“As GPUs are highly efficient for accelerating matrix multiplication, they are a great target for CNNs. Specific processors can be designed for DNN, achieving good performance and low power consumption [42], [43].”
BackgroundComplete and partial fault tolerance of feedforward neural nets
Phatak, D. S., & Koren, I. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.363479 · Cited by 161 · cited 1×
“Most of the available solutions rely on partial duplication or full duplication (or even triplication) of operations [17].”
BackgroundDesign and Evaluation of Hybrid Fault-Detection Systems
Reis, G. A., Chang, J., Vachharajani, N., Rangan, R., August, D. I., & Mukherjee, S. S. (2005) · ACM SIGARCH Computer Architecture News · doi:10.1145/1080695.1069983 · Cited by 150
Comparison of Combinational and Sequential Error Rates for a Deep Submicron Process
Mahatme, N. N., Jagannathan, S., Loveless, T. D., Massengill, L. W., Bhuva, B. L., Wen, S.-J., & Wong, R. (2011) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2011.2171993 · Cited by 149 · cited 1×
“A terrestrial neutron strike may perturb the state of a transistor, generating bit-flips in memory or current spikes in logic circuits that, if latched, lead to an error [34], [35].”
BackgroundFault tolerance in artificial neural networks
Séquin, C. H., & Clay, R. (1990) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1990.137651 · Cited by 144 · cited 2×
“Some studies have used fault injection during network training [18], [19].”
BackgroundGPU-Qin: A methodology for evaluating the error resilience of GPGPU applications
Fang, B., Pattabiraman, K., Ripeanu, M., & Gurumurthi, S. (2014) · 2014 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass.2014.6844486 · Cited by 142 · cited 1×
“Prior work used GPUQin [23] to evaluate CUDA benchmarks reliability.”
MethodRecursive restartability: turning the reboot sledgehammer into a scalpel
Candea, G., & Fox, A. (2005) · Proceedings Eighth Workshop on Hot Topics in Operating Systems · doi:10.1109/hotos.2001.990072 · Cited by 138 · cited 1×
“However, the system must be able to recover before causing any harm to the environment [59].”
BackgroundDesign and Evaluation of Hybrid Fault-Detection Systems
Reis, G. A., Chang, J., Vachharajani, N., Rangan, R., August, D. I., & Mukherjee, S. S. (2005) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2005.21 · Cited by 137 · cited 1×
“We can consider performance-reliability tradeoffs by dividing the frames/s by the FIT rate, gathering the number of correctly computed frames between failures [52].”
MethodEvaluation and Mitigation of Radiation-Induced Soft Errors in Graphics Processing Units
de Oliveira, D. A. G., Pilla, L. L., Santini, T., & Rech, P. (2016) · IEEE Transactions on Computers · doi:10.1109/tc.2015.2444855 · Cited by 101 · cited 6×
“Similar to results from previous studies [26], Fig.”
BackgroundResultInfluentialSelf-citationTracking Interacting Objects Optimally Using Integer Programming
Wang, X., Türetken, E., Fleuret, F., & Fua, P. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10590-1_2 · Cited by 95 · cited 1×
“Recent research has been focused on the option of using fixed-point or integer operations for object detection [60]–[62].”
BackgroundUnderstanding Error Propagation in GPGPU Applications
Li, G., Pattabiraman, K., Cher, C.-Y., & Bose, P. (2016) · SC16: International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1109/sc.2016.20 · Cited by 89 · cited 2×
“Additionally, a single corrupted thread could feed thousands of future parallel threads with erroneous data, again leading to multiple output errors [27].”
BackgroundMethodAnalyzing soft-error vulnerability on GPGPU microarchitecture
Tan, J., Goswami, N., Li, T., & Fu, X. (2011) · 2011 IEEE International Symposium on Workload Characterization (IISWC) · doi:10.1109/iiswc.2011.6114182 · Cited by 82 · cited 1×
“Previous studies have proposed an early evaluation of the reliability of GPUs [20]; they have also evaluated the reliability of GPUs running HPC codes through radiation experiments [21] or fault injection [22]–[24].”
BackgroundPerformance and fault-tolerance of neural networks for optimization
Protzel, P. W., Palumbo, D. L., & Arras, M. K. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238315 · Cited by 82 · cited 1×
“have an inherently higher SDC masking capability than most other codes [18], [56], [57].”
BackgroundSensitivity to errors in artificial neural networks: a behavioral approach
Alippi, C., Piuri, V., & Sami, M. (1995) · IEEE Transactions on Circuits and Systems I Fundamental Theory and Applications · doi:10.1109/81.390269 · Cited by 77
Investigating the Fault Tolerance of Neural Networks
Tchernev, E. B., Mulvaney, R. G., & Phatak, D. S. (2005) · Neural Computation · doi:10.1162/0899766053723096 · Cited by 73 · cited 1×
“Neural networks have an inherently higher SDC masking capability than most other codes [18], [56], [57].”
BackgroundLBANN: livermore big artificial neural network HPC toolkit
Van Essen, B., Kim, H., Pearce, R., Boakye, K., & Chen, B. (2015) · Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments · doi:10.1145/2834892.2834897 · Cited by 71 · cited 1×
“Artificial neural networks (ANNs) are becoming a widely adopted computational approach in many fields, from data mining to pattern recognition, and from robotics to data analytics [1].”
BackgroundRech, P., Aguiar, C., Frost, C., & Carro, L. (2013) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2013.2252625 · Cited by 63 · cited 4×
“Some promising results have shown that algorithm-based fault tolerance (ABFT) [12] is a very efficient strategy to protect matrix multiplication [30], [31] and fast Fourier transform (FFT) [32] in GPUs.”
BackgroundMethodInfluentialSelf-citationAnalysis of Fault Tolerance in Artificial Neural Networks
Piuri, V. (2001) · Journal of Parallel and Distributed Computing · doi:10.1006/jpdc.2000.1663 · Cited by 63 · cited 2×
“This was suggested using the study of the mathematical algorithms present in general neural paradigms, and by evaluating specific implementations [9], [14].”
MethodDynamic Derivation of Application-Specific Error Detectors and their Implementation in Hardware
Pattabiraman, K., Saggese, G. P., Chen, D., Kalbarczyk, Z., & Iyer, R. K. (2006) · 2006 Sixth European Dependable Computing Conference · doi:10.1109/edcc.2006.9 · Cited by 63 · cited 1×
“Even if crashes are more frequent than SDCs, they could be considered less critical since crashes could be, at least, detected [58].”
BackgroundReal-world design and evaluation of compiler-managed GPU redundant multithreading
Wadden, J., Lyashevsky, A., Gurumurthi, S., Sridharan, V., & Kevin Skadron, Y. (2014) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2014.6853227 · Cited by 59
Calculating Architectural Vulnerability Factors for Spatial Multi-Bit Transient Faults
Wilkening, M., Sridharan, V., Li, S., Previlon, F. G., Gurumurthi, S., & Kaeli, D. R. (2014) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2014.15 · Cited by 56 · cited 1×
“Previous studies have proposed an early evaluation of the reliability of GPUs [20]; they have also evaluated the reliability of GPUs running HPC codes through radiation experiments [21] or fault injection [22]–[24].”
BackgroundSelf-citationNoh, J., Correas, V., Lee, S. Y., Jeon, J., Nofal, I., Cerba, J., Belhaddad, H., Alexandrescu, D., Lee, Y., & Kwon, S. (2015) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2015.2450997 · Cited by 55 · cited 3×
“transistors have a much lower neutron-induced error rate as compared to CMOS [36].”
BackgroundResultCharacterization of Single Bit and Multiple Cell Soft Error Events in Planar and FinFET SRAMs
Fang, Y.-P., & Oates, A. S. (2016) · IEEE Transactions on Device and Materials Reliability · doi:10.1109/tdmr.2016.2535663 · Cited by 54 · cited 1×
“It is worth noting that with the shrinkage of transistor di-mension, the percentage of neutron-induced faults that generate multiple cell upsets will increase—for the current technology node, the increase is 20%–30% [54].”
BackgroundReal-world design and evaluation of compiler-managed GPU redundant multithreading
Wadden, J., Lyashevsky, A., Gurumurthi, S., Sridharan, V., & Kevin Skadron, Y. (2014) · ACM SIGARCH Computer Architecture News · doi:10.1145/2678373.2665686 · Cited by 51 · cited 1×
“There are several ways to mitigate SDCs in a software, such as using code replication [29].”
MethodUnderstanding error propagation in GPGPU applications
Li, G., Pattabiraman, K., Cher, C.-Y., & Bose, P. K. (2016) · IEEE International Conference on High Performance Computing, Data, and Analytics · doi:10.5555/3014904.3014932 · Cited by 50
A low power and highly reliable 400Mbps mobile DDR SDRAM with on-chip distributed ECC
Kim, S.-H., Lee, W., Kim, J.-H., Lee, S.-S., Hwang, S.-Y., Kim, C.-I., Kwon, T.-W., Han, B.-S., Cho, S.-K., Kim, D.-H., et al. (2007) · IEEE Asian Solid-State Circuits Conference (A-SSCC) · doi:10.1109/asscc.2007.4425789 · Cited by 44 · cited 1×
“DRAM sensitivity has already been extensively studied [50], [51], and is beyond the scope of this paper.”
BackgroundRadiation-Induced Error Criticality in Modern HPC Parallel Accelerators
Oliveira, D., Pilla, L. L., Hanzich, M., Netto, V. F., dos Santos, F. F., Lunardi, C., María, J., Navaux, P. O. A., Carro, L., & Rech, P. (2017) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2017.41 · Cited by 44 · cited 3×
“We further inspect GEMM reliability, showing the spatial distribution of output errors, as proposed in Oliveira et al. [21], which used a distribution that classifies errors based on their co-ordinates.”
BackgroundMethodSelf-citationBraun, C., Halder, S., & Wunderlich, H. J. (2014) · 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks · doi:10.1109/dsn.2014.48 · Cited by 40 · cited 1×
“Some promising results have shown that algorithm-based fault tolerance (ABFT) [12] is a very efficient strategy to protect matrix multiplication [30], [31] and fast Fourier transform (FFT) [32] in GPUs.”
BackgroundSoftware-Based Hardening Strategies for Neutron Sensitive FFT Algorithms on GPUs
Pilla, L. L., Rech, P., Silvestri, F., Frost, C. D., Navaux, P. O. A., Reorda, M. S., & Carro, L. (2014) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2014.2301768 · Cited by 38 · cited 1×
“Some promising results have shown that algorithm-based fault tolerance (ABFT) [12] is a very efficient strategy to protect matrix multiplication [30], [31] and fast Fourier transform (FFT) [32] in GPUs.”
BackgroundSelf-citationGPU Behavior on a Large HPC Cluster
DeBardeleben, N., Blanchard, S. P., Monroe, L., Romero, P., Grunau, D., Idler, C., & Wright, C. (2014) · Lecture notes in computer science · doi:10.1007/978-3-642-54420-0_66 · Cited by 37 · cited 1×
“Preliminary studies have found some inherent reliability weaknesses in the architecture of GPUs [25], [26].”
BackgroundHigh performance fault-tolerant digital neural networks
Bettola, S., & Piuri, V. (1998) · IEEE Transactions on Computers · doi:10.1109/12.660173 · Cited by 30 · cited 3×
“While previous work has focused on understanding and improving the reliability of neural networks [9], [10], to the best of our knowledge, this is the first study that targets CNN reliability on GPUs.”
BackgroundMethodA Real-Time Pedestrian Detector using Deep Learning for Human-Aware Navigation.Preprint
Ribeiro, D. A., Mateus, A., Nascimento, J. C., & Miraldo, P. (2016) · arXiv (Cornell University) · Cited by 16
Fault mitigation strategies for CUDA GPUs
Carlo, S. D., Gambardella, G., Martella, I., Prinetto, P., Rolfo, D., & Trotta, P. (2013) · IEEE International Test Conference (ITC) · doi:10.1109/test.2013.6651908 · Cited by 15 · cited 1×
“Previous studies have proposed an early evaluation of the reliability of GPUs [20]; they have also evaluated the reliability of GPUs running HPC codes through radiation experiments [21] or fault injection [22]–[24].”
BackgroundUsing Low Cost Erasure and Error Correction Schemes to Improve Reliability of Commodity DRAM Systems
Chen, H.-M., Jeloka, S., Arunkumar, A., Blaauw, D. T., Wu, C.-J., Mudge, T., & Chakrabarti, C. (2016) · IEEE Transactions on Computers · doi:10.1109/tc.2016.2550455 · Cited by 13 · cited 1×
“DRAM sensitivity has already been extensively studied [50], [51], and is beyond the scope of this paper.”
BackgroundFault Tolerance in Distributed Neural ComputingPreprint
Кулаков, А. Ю., Zwoliński, M., & Reeve, J. S. (2015) · arXiv (Cornell University) · doi:10.13140/rg.2.1.1387.0800 · Cited by 7
GPUs Neutron Sensitivity Dependence on Data Type
Rech, P., Frost, C. D., & Carro, L. (2014) · Journal of Electronic Testing · doi:10.1007/s10836-014-5456-6 · Cited by 2 · cited 1×
“Unfortunately, it has been shown that faults in inte-ger data are more likely to impact the execution of a GPU than faults in FP32 data [63].”
BackgroundSelf-citation
Research works printed but not linked: 4
- sridharan, 2009, The effect of input data on program vulnerability, Workshop on Silicon Errors in Logic-System Effects, 1
- 10.1109/TNS.2014.2301768 · DOI
- 2018, Log database for IEEE transaction on reliability, Github
- collobert, 0, BigLearn NIPS Workshop
Cited by
A survey on modeling and improving reliability of DNN algorithms and accelerators
Mittal, S. (2020) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2019.101689 · Cited by 175
Reliable Data Transmission Model for Mobile Ad Hoc Network Using Signcryption Technique
Elhoseny, M., & Shankar, K. (2020) · IEEE Transactions on Reliability · doi:10.1109/tr.2019.2915800 · Cited by 159
“Steering overhead is the principle issue with course disclosure, which causes data loss [19].”
BackgroundA Low-cost Fault Corrector for Deep Neural Networks through Range Restriction
Chen, Z., Li, G., & Pattabiraman, K. (2021) · 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) · doi:10.1109/dsn48987.2021.00018 · Cited by 104
“Techniques based on Algorithm-based Fault Tolerance (ABFT) have also been proposed for detecting faults in particular layers in DNNs (e.g., Conv layer [17], [18], [61]; matrix multiplication [62]).”
Soft errors in DNN accelerators: A comprehensive review
Ibrahim, Y., Wang, H., Liu, J., Wei, J., Chen, L., Rech, P., Adam, K., & Guo, G. (2020) · Microelectronics Reliability · doi:10.1016/j.microrel.2020.113969 · Cited by 102
Self-citationA Systematic Literature Review on Hardware Reliability Assessment Methods for Deep Neural Networks
Ahmadilivani, M. H., Taheri, M., Raik, J., Daneshtalab, M., & Jenihhin, M. (2024) · ACM Computing Surveys · doi:10.1145/3638242 · Cited by 134
“The general model for classifying faults in the mentioned works is as follows: — Masked: Fault does not affect the output, — SDC: Output confidence score differs from that of the golden model, — DUE: The program hangs or the system reboots (also called Crash in References [10, 121]).”
BackgroundMethodInfluential
Metadata
- Authors
- Santos, F. F. D., Pimenta, P. F., Lunardi, C., Draghetti, L., Carro, L., Kaeli, D., & Rech, P.
- Affiliations
- Universidade Federal do Rio Grande do Sul (BR)
- Northeastern University (US)
- ORCID
- Venue
- IEEE Transactions on Reliability
- Published
- 2019-06
- Volume
- 68
- Issue
- 2
- Pages
- 663-677
- ISSN
- 0018-9529 (ISSN-L)
- 1558-1721
- OpenAlex
- W2901848761
- Cited by
- 226
- References
- 67
- Citations by year
- 2026: 18 · 2025: 18 · 2024: 41 · 2023: 33 · 2022: 42 · 2021: 34 · 2020: 27 · 2019: 13
- FWCI
- 7.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Reliability (publishedVersion)
- Institutional Research Information System (Università degli Studi di Trento) (submittedVersion)
- Science and Technology Facilities Council (submittedVersion)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Radiation Effects in Electronics
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- convolutional neural networks, GPU reliability, fault injection, silent data corruption, error-correcting codes, algorithm-based fault tolerance, matrix multiplication, object detection, safety-critical systems, fault propagation, NVIDIA GPUs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.