The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Understanding error propagation in deep learning neural network (DNN) accelerators and applications
Abstract
Deep learning neural networks (DNNs) have been successful in solving a wide range of machine learning problems. Specialized hardware accelerators have been proposed to accelerate the execution of DNN algorithms for high-performance and energy efficiency. Recently, they have been deployed in datacenters (potentially for business-critical or industrial applications) and safety-critical systems such as self-driving cars. Soft errors caused by high-energy particles have been increasing in hardware systems, and these can lead to catastrophic failures in DNN systems. Traditional methods for building resilient systems, e.g., Triple Modular Redundancy (TMR), are agnostic of the DNN algorithm and the DNN accelerator's architecture. Hence, these traditional resilience approaches incur high overheads, which makes them challenging to deploy. In this paper, we experimentally evaluate the resilience characteristics of DNN systems (i.e., DNN software running on specialized accelerators). We find that the error resilience of a DNN system depends on the data types, values, data reuses, and types of layers in the design. Based on our observations, we propose two efficient protection techniques for DNN systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · cited 1×
“Deep learning neural network (DNN) applications are widely used in high-performance computing systems and datacenters [4, 18, 47].”
BackgroundConvolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213 · cited 1×
“A deep learning neural network (DNN) is a directed acyclic graph consisting of multiple computation layers [34].”
BackgroundOptimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158 · cited 1×
“[61], Diannao [12], Dadiannao [14] x x x”
BackgroundDaDianNao: A Machine-Learning Supercomputer
Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., et al. (2014) · 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1109/micro.2014.58 · Cited by 1,492
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364
Borkar, S. (2005) · IEEE Micro · doi:10.1109/mm.2005.110 · Cited by 1,320
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks
Chen, Y.-H., Emer, J., & Sze, V. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001177 · Cited by 1,182
Self-citationEIE: Efficient Inference Engine on Compressed Deep Neural Network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.30 · Cited by 1,077
ShiDianNao: shifting vision processing closer to the sensor
Du, Z., Fasthuber, R., Chen, T., Ienne, P., Li, L., Luo, T., Feng, X., Chen, Y., & Temam, O. (2015) · Proceedings of the 42nd Annual International Symposium on Computer Architecture · doi:10.1145/2749469.2750389 · Cited by 933
Error detection by duplicated instructions in super-scalar processors
Oh, N., Shirvani, P. P., & McCluskey, E. J. (2002) · IEEE Transactions on Reliability · doi:10.1109/24.994913 · Cited by 584
Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks
Chen, Y.-H., Emer, J., & Sze, V. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.40 · Cited by 534
Self-citationMinerva: enabling low-power, highly-accurate deep neural network accelerators
Reagen, B., Whatmough, P., Adolf, R., Rama, S., Lee, H., Lee, S. K., Hernández-Lobato, J. M., Wei, G.-Y., & Brooks, D. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001165 · Cited by 446
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343
Memory-centric accelerator design for Convolutional Neural Networks
Peemen, M., Setio, A. A. A., Mesman, B., & Corporaal, H. (2013) · 2013 IEEE 31st International Conference on Computer Design (ICCD) · doi:10.1109/iccd.2013.6657019 · Cited by 332
A 240 G-ops/s Mobile Coprocessor for Deep Neural Networks
Gokhale, V., Jin, J., Dundar, A., Martini, B., & Culurciello, E. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.106 · Cited by 300
Can Deep Learning Revolutionize Mobile Sensing?
Lane, N. D., & Georgiev, P. (2015) · Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications · doi:10.1145/2699343.2699349 · Cited by 282 · cited 1×
“This makes them very challenging to deploy in self-driving cars, which need to detect objects such as pedestrians in real time [33] and have strict cost constraints.”
BackgroundDianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · ACM SIGPLAN Notices · doi:10.1145/2644865.2541967 · Cited by 281
Minerva: Enabling Low-Power, Highly-Accurate Deep Neural Network Accelerators
Reagen, B., Whatmough, P., Adolf, R., Rama, S., Lee, H., Lee, S. K., Hernandez-Lobato, J. M., Wei, G.-Y., & Brooks, D. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.32 · Cited by 246
A Massively Parallel Coprocessor for Convolutional Neural Networks
Sankaradas, M., Jakkula, V., Cadambi, S., Chakradhar, S., Durdanovic, I., Cosatto, E., & Graf, H. P. (2009) · 2009 20th IEEE International Conference on Application-specific Systems, Architectures and Processors · doi:10.1109/asap.2009.25 · Cited by 224
Soft Error Susceptibilities of 22 nm Tri-Gate Devices
Seifert, N., Gill, B., Jahinuzzaman, S., Basile, J., Ambrose, V., Shi, Q., Allmon, R., & Bramnik, A. (2012) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2012.2218128 · Cited by 192
Quantifying the Accuracy of High-Level Fault Injection Techniques for Hardware Faults
Wei, J., Thomas, A., Li, G., & Pattabiraman, K. (2014) · 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks · doi:10.1109/dsn.2014.2 · Cited by 182 · cited 2×
“This is in line with the definition used in other work [20, 26, 36, 59].”
MethodResultSelf-citationOrigami: A Convolutional Network Accelerator
Cavigelli, L., Gschwend, D., Mayer, C., Willi, S., Muheim, B., & Benini, L. (2015) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/2742060.2743766 · Cited by 180
Low-cost program-level detectors for reducing silent data corruptions
Hari, S. K. S., Adve, S. V., & Naeimi, H. (2012) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2012.6263960 · Cited by 161
Self-citationA defect-tolerant accelerator for emerging high-performance applications
Temam, O. (2012) · ACM SIGARCH Computer Architecture News · doi:10.1145/2366231.2337200 · Cited by 143
Park, S., Bong, K., Shin, D., Lee, J., Choi, S., & Yoo, H. (2015) · Digest of technical papers/Digest of technical papers - IEEE International Solid-State Circuits Conference · doi:10.1109/isscc.2015.7062935 · Cited by 127
Proteus: Exploiting Numerical Precision Variability in Deep Neural Networks
Judd, P., Albericio, J., Hetherington, T., Aamodt, T. M., Jerger, N. E., & Moshovos, A. (2016) · International Conference on Supercomputing · doi:10.1145/2925426.2926294 · Cited by 116
Gill, B., Seifert, N., & Zia, V. (2009) · IEEE International Reliability Physics Symposium proceedings · doi:10.1109/irps.2009.5173251 · Cited by 108
Intermittent faults and effects on reliability of integrated circuits
Constantinescu, C. S. (2008) · Annual Reliability and Maintainability Symposium (RAMS) · doi:10.1109/rams.2008.4925824 · Cited by 99
A defect-tolerant accelerator for emerging high-performance applications
Temam, O. (2012) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2012.6237031 · Cited by 94
IPAS: intelligent protection against silent output corruption in scientific applications
Laguna, I., Schulz, M., Richards, D. F., Calhoun, J. C., & Olson, L. N. (2016) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1145/2854038.2854059 · Cited by 79
Shoestring: probabilistic soft error reliability on the cheap
Feng, S., Gupta, S., Ansari, A., & Mahlke, S. A. (2010) · ACM SIGARCH Computer Architecture News · doi:10.1145/1735970.1736063 · Cited by 79
Sensitivity to errors in artificial neural networks: a behavioral approach
Alippi, C., Piuri, V., & Sami, M. (1995) · IEEE Transactions on Circuits and Systems I Fundamental Theory and Applications · doi:10.1109/81.390269 · Cited by 77
1968 International Solid-State Circuits Conference
(1968) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.1968.1049838 · Cited by 74
Analysis of Fault Tolerance in Artificial Neural Networks
Piuri, V. (2001) · Journal of Parallel and Distributed Computing · doi:10.1006/jpdc.2000.1663 · Cited by 63
Understanding error propagation in GPGPU applications
Li, G., Pattabiraman, K., Cher, C.-Y., & Bose, P. K. (2016) · IEEE International Conference on High Performance Computing, Data, and Analytics · doi:10.5555/3014904.3014932 · Cited by 53
Self-citationDeep compression and EIE: Efficient inference engine on compressed deep neural network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M., & Dally, B. (2016) · IEEE Hot Chips Symposium · doi:10.1109/hotchips.2016.7936226 · Cited by 46
Towards an embedded biologically-inspired machine vision processor
Sriram, V., Cox, D., Tsoi, K. H., & Luk, W. W. (2010) · International Conference on Field-Programmable Technology (FPT) · doi:10.1109/fpt.2010.5681487 · Cited by 36
Automated Derivation of Application-Specific Error Detectors Using Dynamic Analysis
Pattabiraman, K., Saggese, G. P., Chen, D., Kalbarczyk, Z., & Iyer, R. K. (2010) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2010.19 · Cited by 34
Self-citationConfigurable Detection of SDC-causing Errors in Programs
Lu, Q., Li, G., Pattabiraman, K., Gupta, M. S., & Rivers, J. A. (2017) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3014586 · Cited by 32
Self-citationHigh performance fault-tolerant digital neural networks
Bettola, S., & Piuri, V. (1998) · IEEE Transactions on Computers · doi:10.1109/12.660173 · Cited by 30
Fine-Grained Characterization of Faults Causing Long Latency Crashes in Programs
Li, G., Lu, Q., & Pattabiraman, K. (2015) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2015.36 · Cited by 28
Self-citationExploiting spatial architectures for edit distance algorithms
Tithi, J. J., Crago, N. C., & Emer, J. (2014) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass.2014.6844458 · Cited by 17
Self-citationEvaluation of Histogram of Oriented Gradients Soft Errors Criticality for Automotive Applications
Fernandes, F., Weigel, L., Jung, C., Navaux, P., Carro, L., & Rech, P. (2016) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/2998573 · Cited by 14
Energy-reliability limits in nanoscale neural networks
Chatterjee, A., & Varshney, L. R. (2017) · Conference on Information Sciences and Systems · doi:10.1109/ciss.2017.7926139 · Cited by 7
Research works printed but not linked: 11
- Autonomous car facts 2016. Keynote: Autonomous Car A New Driver for Resilient Computing and Design-for-Test. (2016). Retrieved Oct. 2016 from https://nepp.nasa.gov/workshops/etw2016/talks/15WED/20160615-0930-Autonomous_Saxena-Nirmal-Saxena-Rec2016Jun16-nasaNEPP.pdf Autonomous car facts 2016. Keynote: Autonomous Car A New Driver for Resilient Computing and Design-for-Test. (2016). Retrieved Oct. 2016 from https://nepp.nasa.gov/workshops/etw2016/talks/15WED/20160615-0930-Autonomous_Saxena-Nirmal-Saxena-Rec2016Jun16-nasaNEPP.pdf
- Caffe Model 2014. Caffe Model Zoo. (2014). Retrieved Oct. 2016 from http://caffe.berkeleyvision.org/model_zoo.html Caffe Model 2014. Caffe Model Zoo. (2014). Retrieved Oct. 2016 from http://caffe.berkeleyvision.org/model_zoo.html
- CaffeNet 2014. CaffeNet . ( 2014 ). Retrieved Oct. 2016 from http://caffe.berkeleyvision.org/model_zoo.html CaffeNet 2014. CaffeNet. (2014). Retrieved Oct. 2016 from http://caffe.berkeleyvision.org/model_zoo.html
- ConvNet 2014. High-performance C++/CUDA implementation of convolutional neural networks. ( 2014 ). Retrieved Oct. 2016 from https://code.google.com/p/cuda-convnet ConvNet 2014. High-performance C++/CUDA implementation of convolutional neural networks. (2014). Retrieved Oct. 2016 from https://code.google.com/p/cuda-convnet
- 10.5555/2354410.2355132 · doi:10.5555/2354410.2355132
- IEC 61508 2016. Functional Safety and IEC 61508. (2016). Retrieved Oct. 2016 from http://www.iec.ch/functionalsafety/ IEC 61508 2016. Functional Safety and IEC 61508. (2016). Retrieved Oct. 2016 from http://www.iec.ch/functionalsafety/
- Lin , Min and Chen , Qiang and Yan , Shuicheng . 2013. Network in network. arXiv preprint arXiv:1312.4400 ( 2013 ). Lin, Min and Chen, Qiang and Yan, Shuicheng. 2013. Network in network. arXiv preprint arXiv:1312.4400 (2013). · arXiv
- Neale Adam and Manoj Sachdev. 2016. Neutron Radiation Induced Soft Error Rates for an Adjacent-ECC Protected SRAM in 28 nm CMOS. (2016). Neale Adam and Manoj Sachdev. 2016. Neutron Radiation Induced Soft Error Rates for an Adjacent-ECC Protected SRAM in 28 nm CMOS. (2016). · doi:10.1109/tns.2016.2547963
- 10.1109/CGO.2005.34 · doi:10.1109/cgo.2005.34
- TPU {n. d.}. Google supercharges machine learning tasks with TPU custom chip. ({n. d.}). Retrieved Oct. 2016 from https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html TPU {n. d.}. Google supercharges machine learning tasks with TPU custom chip. ({n. d.}). Retrieved Oct. 2016 from https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html
- Yann LeCun . 2000. Deep Learning and the Future of AI. (2000). Retrieved Apr. 2017 from https://indico.cern.ch/event/510372/ Yann LeCun. 2000. Deep Learning and the Future of AI. (2000). Retrieved Apr. 2017 from https://indico.cern.ch/event/510372/
Cited by
A Survey on Distributed Machine Learning
Verbraeken, J., Wolting, M., Katzy, J., Kloppenburg, J., Verbelen, T., & Rellermeyer, J. S. (2021) · ACM Computing Surveys · doi:10.1145/3377454 · Cited by 872
The Hidden Vulnerability of Distributed Learning in Byzantium
Mhamdi, E. M. E., Guerraoui, R., & Rouault, S. (2018) · arXiv · doi:10.48550/arxiv.1802.07927 · Cited by 648
Analyzing and Increasing the Reliability of Convolutional Neural Networks on GPUs
Santos, F. F. D., Pimenta, P. F., Lunardi, C., Draghetti, L., Carro, L., Kaeli, D., & Rech, P. (2019) · IEEE Transactions on Reliability · doi:10.1109/tr.2018.2878387 · Cited by 226
Ares: A framework for quantifying the resilience of deep neural networks
Reagen, B., Gupta, U., Pentecost, L., Whatmough, P., Lee, S. K., Mulholland, N., Brooks, D., & Wei, G.-Y. (2018) · 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC) · doi:10.1109/dac.2018.8465834 · Cited by 189
Perez-Cerrolaza, J., Abella, J., Borg, M., Donzella, C., Cerquides, J., Cazorla, F. J., Englund, C., Tauber, M., Nikolakopoulos, G., & Flores, J. L. (2024) · ACM Computing Surveys · doi:10.1145/3626314 · Cited by 185
Metadata
- Authors
- Li, G., Hari, S. K. S., Sullivan, M., Tsai, T., Pattabiraman, K., Emer, J., & Keckler, S. W.
- Venue
- Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
- Publisher
- ACM
- Published
- 2017-11-12
- Pages
- 1-12
- ISSN
- 2167-4337 (ISSN-L)
- OpenAlex
- W2767260595
- Cited by
- 492
- References
- 52
- FWCI
- 50.59
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.