SqueezeNext: Hardware-Aware Neural Network Design
Open accessAbstract
One of the main barriers for deploying neural networks on embedded systems has been large memory and power consumption of existing neural networks. In this work, we introduce SqueezeNext, a new family of neural network architectures whose design was guided by considering previous architectures such as SqueezeNet, as well as by simulation results on a neural network accelerator. This new network is able to match AlexNet's accuracy on the ImageNet benchmark with 112× fewer parameters, and one of its deeper variants is able to achieve VGG-19 accuracy with only 4.4 Million parameters, (31× smaller than VGG-19). SqueezeNext also achieves better top-5 classification accuracy with 1.3× fewer parameters as compared to MobileNet, but avoids using depthwise-separable convolutions that are inefficient on some mobile processor platforms. This wide range of accuracy gives the user the ability to make speed-accuracy tradeoffs, depending on the available resources on the target hardware. Using hardware simulation results for power and inference speed on an embedded system has guided us to design variations of the baseline model that are 2.59×/8.26× faster and 2.25×/7.5× more energy efficient as compared to SqueezeNet/AlexNet without any accuracy degradation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Achieving efficient real-time neural networks with optimal accuracy requires rethinking how models are designed, trained, and deployed.
“Achieving efficient, real-time NNs with optimal accuracy requires rethinking the design, training, and deployment of NN models [71].”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [71]
The Introduction and Conclusions support rethinking network design around hardware constraints to improve inference speed and energy efficiency without sacrificing accuracy. The paper discusses training procedures and barriers to deployment, but does not say that rethinking training and deployment is required; those parts of the attributed claim go beyond its findings. (LLM, conf=0.84)
Experimental — accuracy not yet measuredSqueezeNext improved parameter compression by directly incorporating separation convolution.
“On the basis of SqueezeNet, they proposed SqueezeNext [126] by directly incorporating separation convolution to achieve improved parameter compression.”
From A survey of model compression strategies for object detection (2023) · cites this paper as [126]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 33 · Cited by 350
Research works printed but not linked: 22
- L. Cavigelli and L. Benini. Origami: A 803-gop/s/w convolutional network accelerator. IEEE Transac- tions on Circuits and Systems for Video Technology, 27(11):2461-2475, 2017.
- C. Farabet, B. Martini, B. Corda, P. Akselrod, E. Cu- lurciello, and Y. LeCun. Neuflow: A runtime recon- figurable dataflow processor for vision. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 IEEE Computer Society Conference on, pages 109-116, 2011.
- C. Farabet, C. Poulet, J. Y. Han, and Y. LeCun. Cnp: An fpga-based processor for convolutional networks. In Field Programmable Logic and Applications, 2009. FPL 2009. International Conference on, pages 32-37, 2009.
- A. Gholami, A. Azad, P. Jin, K. Keutzer, and A. Bu- luc. Integrated model, batch and domain parallelism in training neural networks. ACM Symposium on Paral- lelism in Algorithms and Architectures (SPAA), 2018.
- V. Gokhale, J. Jin, A. Dundar, B. Martini, and E. Cu- lurciello. A 240 g-ops/s mobile coprocessor for deep neural networks. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition Workshops, pages 682-687, 2014.
- S. Han, H. Mao, and W. J. Dally. Deep compres- sion: Compressing deep neural networks with prun- ing, trained quantization and huffman coding. In- ternational Conference on Learning Representations (ICLR), 2016.
- S. Han, J. Pool, J. Tran, and W. Dally. Learning both weights and connections for efficient neural network. In Advances in neural information processing systems, pages 1135-1143, 2015.
- M. G. Hluchyj and M. J. Karol. Shuffle net: An ap- plication of generalized perfect shuffles to multihop lightwave networks. Journal of Lightwave Technol- ogy, 9(10):1386-1397, 1991.
- A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017. · arXiv
- G. Huang, S. Liu, L. van der Maaten, and K. Q. Weinberger. Condensenet: An efficient densenet using learned group convolutions. arXiv preprint arXiv:1711.09224, 2017. · arXiv
- G. Huang, Z. Liu, K. Q. Weinberger, and L. van der Maaten. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, volume 1, page 3, 2017.
- I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio. Quantized neural networks: Training neural networks with low precision weights and acti- vations. arXiv preprint arXiv:1609.07061, 2016. · arXiv
- F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer. SqueezeNet: AlexNet- level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360, 2016. · arXiv
- M. Jaderberg, A. Vedaldi, and A. Zisserman. Speed- ing up convolutional neural networks with low rank expansions. arXiv preprint arXiv:1405.3866, 2014. · arXiv
- N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers, et al. In-datacenter performance analysis of a tensor processing unit. In Proceedings of the 44th Annual International Symposium on Computer Archi- tecture, pages 1-12. ACM, 2017.
- E. Park, J. Ahn, and S. Yoo. Weighted-entropy-based quantization for deep neural networks. In IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), 2017.
- M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi. Xnor-net: Imagenet classification using binary convo- lutional neural networks. In European Conference on Computer Vision, pages 525-542, 2016.
- K. Simonyan and A. Zisserman. Very deep convo- lutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. · arXiv
- B. Wu, F. Iandola, P. H. Jin, and K. Keutzer. Squeezedet: Unified, small, low power fully con- volutional neural networks for real-time object de- tection for autonomous driving. arXiv preprint arXiv:1612.01051, 2016. · arXiv
- B. Wu, A. Wan, X. Yue, and K. Keutzer. Squeeze- seg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud. arXiv preprint arXiv:1710.07368, 2017. · arXiv
- F. Yu, D. Wang, and T. Darrell. Deep layer aggrega- tion. arXiv preprint arXiv:1707.06484, 2017. · arXiv
- C. Zhu, S. Han, H. Mao, and W. J. Dally. Trained ternary quantization. arXiv preprint arXiv:1612.01064, 2016. · arXiv
- Authors
- Gholami, A., Kwon, K., Wu, B., Tai, Z., Yue, X., Jin, P., Zhao, S., & Keutzer, K.
- Affiliations
- University of California, Berkeley (US)
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 1719-171909
- OpenAlex
- W2795077673
- Cited by
- 350
- References
- 33
- Citations by year
- 2026: 8 · 2025: 42 · 2024: 46 · 2023: 66 · 2022: 50 · 2021: 47 · 2020: 48 · 2019: 39 · 2018: 4
- FWCI
- 10.40
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- neural network design, model compression, parameter reduction, hardware-aware design, neural network accelerators, embedded systems, energy efficiency, inference speed, speed-accuracy trade-off, SqueezeNet, MobileNet, ImageNet classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
