ShiftAddNet: A Hardware-Inspired Deep Network
The DOI matches its DataCite record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Multiplication (e.g., convolution) is arguably a cornerstone of modern deep neural networks (DNNs). However, intensive multiplications cause expensive resource costs that challenge DNNs' deployment on resource-constrained edge devices, driving several attempts for multiplication-less deep networks. This paper presented ShiftAddNet, whose main inspiration is drawn from a common practice in energy-efficient hardware implementation, that is, multiplication can be instead performed with additions and logical bit-shifts. We leverage this idea to explicitly parameterize deep networks in this way, yielding a new type of deep network that involves only bit-shift and additive weight layers. This hardware-inspired ShiftAddNet immediately leads to both energy-efficient inference and training, without compromising the expressive capacity compared to standard DNNs. The two complementary operation types (bit-shift and add) additionally enable finer-grained control of the model's learning capacity, leading to more flexible trade-off between accuracy and (training) efficiency, as well as improved robustness to quantization and pruning. We conduct extensive experiments and ablation studies, all backed up by our FPGA-based ShiftAddNet implementation and energy measurements. Compared to existing DNNs or other multiplication-less models, ShiftAddNet aggressively reduces over 80% hardware-quantified energy cost of DNNs training and inference, while offering comparable or better accuracies. Codes and pre-trained models are available at https://github.com/RICE-EIC/ShiftAddNet.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Alibaba Damo Academy and other institutions jointly proposed ShiftAddNet [143], which uses the combination of addition operation and shift operation instead of multiplication.”
From A survey of model compression strategies for object detection (2023) · cites this paper as [143]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
GhostNet: More Features From Cheap Operations
Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00165 · Cited by 5,222
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · arXiv · doi:10.48550/arxiv.1906.02243 · Cited by 3,735
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684
VGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032
Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1602.02830 · Cited by 2,106
Rethinking the Value of Network Pruning
Liu, Z., Sun, M., Zhou, T., Huang, G., & Darrell, T. (2018) · arXiv · doi:10.48550/arxiv.1810.05270 · Cited by 1,625
Self-citationHuman Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
Jiang, W., & Yin, Z. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806333 · Cited by 676 · cited 1×
“For the trainable shift layers, we follow [4] to adopt 50% sparsity by default; and for the MHEALTH and USCHAD datasets, we follow [44] to use a DCNN model and train it for 40 epochs.”
MethodmHealthDroid: A Novel Framework for Agile Development of Mobile Health Applications
Banos, O., Garcia, R., Holgado-Terriza, J. A., Damas, M., Pomares, H., Rojas, I., Saez, A., & Villalonga, C. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-13105-4_14 · Cited by 583
USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors
Zhang, M., & Sawchuk, A. A. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370438 · Cited by 554 · cited 3×
“We also observe similar improved efficiency-accuracy trade-offs on the MHEALTH [37] and USCHAD [39] datasets and report the performance in the supplement.”
Cited by
Lightweight Deep Learning for Resource-Constrained Environments: A Survey
Liu, H.-I., Galindo, M., Xie, H., Wong, L.-K., Shuai, H.-H., Li, Y.-H., & Cheng, W.-H. (2024) · ACM Computing Surveys · doi:10.1145/3657282 · Cited by 259
A Survey of Techniques for Optimizing Transformer Inference
Chitty-Venkata, K. T., Mittal, S., Emani, M., Vishwanath, V., & Somani, A. K. (2023) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2023.102990 · Cited by 154
Neural Architecture Search for Transformers: A Survey
Chitty-Venkata, K. T., Emani, M., Vishwanath, V., & Somani, A. K. (2022) · IEEE Access · doi:10.1109/access.2022.3212767 · Cited by 118
DeepShift: Towards Multiplication-Less Neural Networks
Elhoushi, M., Chen, Z., Shafiq, F., Tian, Y. H., & Li, J. Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00268 · Cited by 117
HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark
Li, C., Yu, Z., Fu, Y., Zhang, Y., Zhao, Y., You, H., Yu, Q., Wang, Y., & Lin, Y. C. (2021) · arXiv · doi:10.48550/arxiv.2103.10584 · Cited by 88
“Note that we consider the Kendall Rank Correlation Coefficients (Abdi, 2007), which is a commonly used correlation coefficient in both recent NAS frameworks and benchmarks (You et al., 2020b; Siems et al., 2020; Yang et al., 2020).”
BackgroundMethodSelf-citation
Metadata
- Authors
- You, H., Chen, X., Zhang, Y., Li, C., Li, S., Liu, Z., Wang, Z., & Lin, Y. C.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- arXiv
- 2010.12785
- References
- 15
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ShiftAddNet: A Hardware-Inspired Deep Network. https://app.cochat.ai/papers/10.48550/arxiv.2010.12785
- The paper
- You, H., Chen, X., Zhang, Y., Li, C., Li, S., Liu, Z., Wang, Z., & Lin, Y. C. (2020). ShiftAddNet: A Hardware-Inspired Deep Network. arXiv. https://doi.org/10.48550/arxiv.2010.12785
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.