CoChat

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Sze, Vivienne, Chen, Yu-Hsin, Yang, Tien-Ju, Emer, Joel S. · 2017 · Proceedings of the IEEE

The DOI matches its Crossref record; 144 of the 157 references it cites were found in a registry or an index.

Abstract

Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances toward the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic codesigns, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the tradeoffs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

Overview

Integrity signals

Registry

Crossref · 10.1109/jproc.2017.2761740

Cites

144 of 144 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 34

  1. 10.13140/RG.2.2.18893.74727 · DOI · Paper page
  2. Li, Stanford CS Class CS231n: Convolutional Neural Networks for Visual Recognition
  3. 2016, Complete Visual Networking Index (VNI) Forecast
  4. Woodhouse, 2016, Big, Big, Big Data: Higher and Higher Resolution Video Surveillance
  5. Maas, Rectifier nonlinearities improve neural network acoustic models, 1
  6. Caffe LeNet MNIST
  7. Caffe Model Zoo
  8. Matconvnet Pretrained Models
  9. TensorFlow-Slim Image Classification Library
  10. Deep Learning Frameworks
  11. Yann, The MNIST DATABASE of Handwritten Digits
  12. Krizhevsky, The CIFAR-10 Dataset
  13. Krizhevsky, 2010, Convolutional deep belief networks on cifar-10, Unpublished Manuscript, 40, 1
  14. Pascal VOC Data Sets
  15. Microsoft Common Objects in Context (COCO) Dataset
  16. Google Open Images
  17. YouTube-8M
  18. AudioSet
  19. 2017, Intel Architecture Instruction Set Extensions Programming Reference
  20. Condon, Facebook unveils Big Basin, new server geared for deep learning
  21. Intel Math Kernel Library
  22. 10.1145/1816038.1815993 · DOI
  23. 10.1145/2872887.2750389 · DOI
  24. 10.1145/2644865.2541967 · DOI · Paper page
  25. Standard, High bandwidth memory (HBM) DRAM
  26. 10.1145/3093336.3037702 · DOI
  27. Wilson, International technology roadmap for semiconductors (ITRS)
  28. 10.1109/icvd.2004.1260946 · doi:10.1109/icvd.2004.1260946
  29. Lee, A 2.5 GHz 7.7 TOPS/W switched-capacitor matrix multiplier with co-designed local memory in 40 nm, 418
  30. Lee, LogNet: Energy-efficient neural networks using logrithmic computations, 5900
  31. DNN Energy Estimation
  32. 10.1145/3140659.3080254 · DOI · Paper page
  33. Iandola, SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1 MB model size
  34. Benchmarking DNN Processors

Cited by

See all 4,215

Metadata

Authors
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S.
Affiliations
  • Massachusetts Institute of Technology (US)
  • Nvidia (United States) (US)
Venue
Proceedings of the IEEE
Published
2017-12
Volume
105
Issue
12
Pages
2295-2329
ISSN
  • 0018-9219 (ISSN-L)
  • 1558-2256
OpenAlex
W2604319603
Cited by
4,215
References
157
Citations by year
2026: 504 · 2025: 523 · 2024: 474 · 2023: 466 · 2022: 508 · 2021: 568 · 2020: 551 · 2019: 392 · 2018: 199 · 2017: 28
FWCI
132.75
Type
journal article
Language
en
Open access
closed
Locations
Proceedings of the IEEE (publishedVersion)
Funders
  • DARPA YFA
  • MIT CICS
  • gift from Nvidia
  • gift from Intel
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
Keywords
deep neural network, energy efficiency, throughput, hardware accelerators, computation cost reduction, benchmarking metrics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.