Winograd convolution: a perspective from fault tolerance
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Winograd convolution is originally proposed to reduce the computing overhead by converting multiplication in neural network (NN) with addition via linear transformation. Other than the computing efficiency, we observe its great potential in improving NN fault tolerance and evaluate its fault tolerance comprehensively for the first time. Then, we explore the use of fault tolerance of winograd convolution for either fault-tolerant or energy-efficient NN processing. According to our experiments, winograd convolution can be utilized to reduce fault-tolerant design overhead by 27.49% or energy consumption by 7.19% without any accuracy loss compared to that without being aware of the fault tolerance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A survey of deep neural network architectures and their applications
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.038 · Cited by 3,239 · cited 2×
“More fault-tolerant neural network design approaches can be found in recent surveys [39] [25] [37].”
BackgroundProceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1994) · 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1994.323798 · Cited by 1,477
Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
Deng, B. L., Li, G., Han, S., Shi, L., & Xie, Y. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2976475 · Cited by 996 · cited 2×
“It attracts worldwide attentions and substantial research efforts have been devoted to improving DNNs from the perspective of precision, performance, energy efficiency, and so on [6].”
BackgroundMethodMinerva: enabling low-power, highly-accurate deep neural network accelerators
Reagen, B., Whatmough, P., Adolf, R., Rama, S., Lee, H., Lee, S. K., Hernández-Lobato, J. M., Wei, G.-Y., & Brooks, D. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001165 · Cited by 446 · cited 4×
“…the soft errors, the fault tolerance of DNNs can also be explored to relax the requirements of absolute correctness of the DNN execution, which can improve the energy efficiency substantially with techniques such as aggressive voltage scaling[32] [48], overclocking [5] [22], and model pruning [6].”
BackgroundMethodInfluentialCMOS Scaling Trends and Beyond
Bohr, M. T., & Young, I. A. (2017) · IEEE Micro · doi:10.1109/mm.2017.4241347 · Cited by 336
Fault and Error Tolerance in Neural Networks: A Review
Torres-Huitzil, C., & Girau, B. (2017) · IEEE Access · doi:10.1109/access.2017.2742698 · Cited by 228
Ares: A framework for quantifying the resilience of deep neural networks
Reagen, B., Gupta, U., Pentecost, L., Whatmough, P., Lee, S. K., Mulholland, N., Brooks, D., & Wei, G.-Y. (2018) · 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC) · doi:10.1109/dac.2018.8465834 · Cited by 189
SCALE-Sim: Systolic CNN Accelerator Simulator
Samajdar, A., Zhu, Y., Whatmough, P. N., Mattina, M., & Krishna, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.02883 · Cited by 188
A survey on modeling and improving reliability of DNN algorithms and accelerators
Mittal, S. (2020) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2019.101689 · Cited by 175 · cited 1×
“More fault-tolerant neural network design approaches can be found in recent surveys [39] [25] [37].”
BackgroundRobust Machine Learning Systems: Challenges,Current Trends, Perspectives, and the Road Ahead
Shafique, M., Naseer, M., Theocharides, T., Kyrkou, C., Mutlu, O., Orosa, L., & Choi, J. (2020) · IEEE Design and Test · doi:10.1109/mdat.2020.2971217 · Cited by 156
Efficient Sparse-Winograd Convolutional Neural NetworksPreprint
Liu, X., Pool, J., Han, S., & Dally, W. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06367 · Cited by 147
Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
Ng, H.-C., Liu, S., & Luk, W. W. (2018) · doi:10.1145/3174243 · Cited by 103
DNN Engine: A 28-nm Timing-Error Tolerant Sparse Deep Neural Network Processor for IoT Applications
Whatmough, P. N., Lee, S. K., Brooks, D., & Wei, G.-Y. (2018) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2018.2841824 · Cited by 96
TensorFI: A Flexible Fault Injection Framework for TensorFlow Applications
Chen, Z., Narayanan, N., Fang, B., Li, G., Pattabiraman, K., & DeBardeleben, N. (2020) · 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre5003.2020.00047 · Cited by 93 · cited 2×
“…winograd-based DNNs, we develop an operation-level fault injection approach rather than reusing existing neuron-level fault injection methods [27] [4] [33] which can hardly differentiate the standard convolution and wino-grad convolution due to the lack of fault simulation of details in each…”
BackgroundMethodA defect-tolerant accelerator for emerging high-performance applications
Temam, O. (2012) · 2012 39th Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2012.6237031 · Cited by 92
RANA: Towards Efficient Neural Acceleration with Refresh-Optimized Embedded DRAM
Tu, F., Wu, W., Yin, S., Liu, L., & Wei, S. (2018) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2018.00037 · Cited by 91
Sanity-Check: Boosting the Reliability of Safety-Critical Deep Neural Network Applications
Ozen, E., & Orailoglu, A. (2019) · 2019 IEEE 28th Asian Test Symposium (ATS) · doi:10.1109/ats47505.2019.000-8 · Cited by 74
Proceedings of the 40th annual Design Automation Conference
(2003) · doi:10.1145/775832 · Cited by 70
Arithmetic-intensity-guided fault tolerance for neural network inference on GPUs
Kosaian, J., & Rashmi, K. V. (2021) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/3458817.3476184 · Cited by 53
Shen, J., Huang, Y. R., Wen, M., & Zhang, C. (2019) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2019.2912894 · Cited by 47
Searching for Winograd-aware Quantized NetworksPreprint
Fernández-Marqués, J., Whatmough, P. N., Mundy, A., & Mattina, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10711 · Cited by 45
Regularizing Multilayer Perceptron for Robustness
Dey, P., Nag, K., Pal, T., & Pal, N. R. (2017) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2017.2664143 · Cited by 43
AxTrain: Hardware-Oriented Neural Network Training for Approximate Inference
He, X. J., Ke, L., Lü, W., Yan, G., & Zhang, X. (2018) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/3218603.3218643 · Cited by 42
Training Modern Deep Neural Networks for Memory-Fault Robustness
Hacene, G. B., Leduc-Primeau, F., Soussia, A. B., Gripon, V., & Gagnon, F. (2019) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2019.8702382 · Cited by 39
DWM: A Decomposable Winograd Method for Convolution Acceleration
Huang, D., Zhang, X., Zhang, R., Zhi, T., He, D., Guo, J., Liu, C., Guo, Q., Du, Z., Liu, S., et al. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5838 · Cited by 39
Self-citationSafe Overclocking for CNN Accelerators Through Algorithm-Level Error Detection
Marty, T., Yuki, T., & Derrien, S. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.2981056 · Cited by 39
SNR: S queezing N umerical R ange Defuses Bit Error Vulnerability Surface in Deep Neural Networks
Ozen, E., & Orailoğlu, A. (2021) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3477007 · Cited by 39
GreenTPU: Improving Timing Error Resilience of a Near-Threshold Tensor Processing Unit
Pandey, P., Basu, P., Chakraborty, K., & Roy, S. (2019) · Proceedings of the 56th Annual Design Automation Conference 2019 · doi:10.1145/3316781.3317835 · Cited by 36
FTT-NAS: Discovering Fault-Tolerant Neural Architecture
Li, W., Ning, X., Ge, G., Chen, X., Wang, Y., & Yang, H. (2020) · 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac47756.2020.9045324 · Cited by 34
Self-citationJia, K., Liu, Z., Wei, Q., Qiao, F., Liu, X.-J., Yang, Y., Fan, H., & Yang, H. (2018) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3195970.3196004 · Cited by 28
FTT-NAS: Discovering Fault-tolerant Convolutional Neural Architecture
Ning, X., Ge, G., Li, W., Zhu, Z., Zheng, Y., Chen, X., Gao, Z., Wang, Y., & Yang, H. (2021) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3460288 · Cited by 20
Self-citationCodeNet: Training Large Scale Neural Networks in Presence of Soft-ErrorsPreprint
Dutta, S., Bai, Z., Low, T. M., & Grover, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.01042 · Cited by 15
R2F: A Remote Retraining Framework for AIoT Processors With Computing Errors
Xu, D., He, M., Liu, C., Wang, Y., Cheng, L., Li, H., Li, X., & Cheng, K.-T. T. (2021) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2021.3089224 · Cited by 12
Self-citationSqueezing the Last MHz for CNN Acceleration on FPGAs
Li, L., Xu, D., Xing, K., Liu, C., Wang, Y., Li, H., & Li, X. (2019) · IEEE International Test Conference in Asia (ITC-Asia) · doi:10.1109/itc-asia.2019.00039 · Cited by 11
Self-citationError Probability Models for Voltage-Scaled Multiply-Accumulate Units
Rathore, M., Milder, P. A., & Salman, E. (2020) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2020.2988204 · Cited by 9
Exploiting Asymmetry in eDRAM Errors for Redundancy-Free Error-Tolerant Design
Liu, S., Reviriego, P., Guo, J., Han, J., & Lombardi, F. (2019) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2019.2960491 · Cited by 6
AEP: An error-bearing neural network accelerator for energy efficiency and model protection
Zhao, L., Zhang, Y., & Yang, J. (2017) · International Conference on Computer Aided Design · doi:10.1109/iccad.2017.8203854 · Cited by 1
Research works printed but not linked: 8
- Deng Jiacnao et al. 2015. Retraining-based timing error mitigation for hardware neural networks. In 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 593--596.
- Du Yuxuan et al. 2020. An Energy-Efficient Time-Domain Binary Neural Network Accelerator with Error-Detection in 28nm CMOS. In 2020 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS). IEEE, 70--73.
- He Yi et al. 2020. Fidelity: Efficient resilience analysis framework for deep learning accelerators. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 270--281.
- Li Guangli et al. 2020. Lance: efficient low-precision quantized winograd convolution for neural networks based on graphics processing units. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3842--3846.
- Lu Liqiang et al. 2017. Evaluating fast algorithms for convolutional neural networks on FPGAs. In 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, 101--108.
- Mahmoud Abdulrahman et al. 2020. Pytorchfi: A runtime perturbation tool for dnns. In 2020 50th Annual LEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W). IEEE, 25--31.
- Xygkis Athanasios et al. 2018. Efficient winograd-based convolution kernel implementation on edge devices. In 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC). IEEE, 1--6.
- Zhang Jeff et al. 2018. Thundervolt: enabling aggressive voltage underscaling and timing error resilience for energy efficient deep learning accelerators. In Proceedings of the 55th Annual Design Automation Conference. 1--6.
Cited by
Soft Error Reliability Analysis of Vision Transformers
Xue, X., Liu, C., Wang, Y., Yang, B., Luo, T., Zhang, L., Li, H., & Li, X. (2023) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2023.3317138 · Cited by 35
“Particularly, following prior reliability analysis works [18] [20] [24] , we also use bit error rate (BER), which represents the ratio of bit flip errors over the total number of bits of operations in the model as the soft error intensity metric.”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Xue, X., Huang, H., Liu, C., Luo, T., Zhang, L., & Wang, Y.
- Affiliations
- University of Chinese Academy of Sciences (CN)
- Agency for Science, Technology and Research (SG)
- Venue
- Proceedings of the 59th ACM/IEEE Design Automation Conference
- Publisher
- ACM
- Published
- 2022-07-10
- Pages
- 853-858
- ISSN
- 0738-100X (ISSN-L)
- OpenAlex
- W4293023174
- Cited by
- 10
- References
- 38
- Citations by year
- 2026: 1 · 2024: 1 · 2023: 6 · 2022: 2
- FWCI
- 1.81
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM/IEEE Design Automation Conference (DAC) (publishedVersion)
- Funders
- Singapore Government (A1687b0033)
- National Natural Science Foundation of China (62174162, 61902375)
- Chinese Academy of Sciences (XDC05030201)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Keywords
- Winograd convolution, hardware fault tolerance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.