Efficient Processing of Deep Neural Networks: A Tutorial and Survey
The DOI matches its Crossref record; 144 of the 157 references it cites were found in a registry or an index.
Abstract
Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances toward the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic codesigns, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the tradeoffs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Classical arithmetic operations, such as MAC, constitute the bulk of the computational requirements for models used in artificial intelligence [13].”
From Brain-inspired computing systems: a systematic literature review (2024) · cites this paper as [13]
In the citing paper
“To clarify, consider the multiply-and-accumulate (MAC) operation in digital processors, where the operation energy consumption is significantly dependent on the bit width of the input operands [13, 235] or equivalently to the signal-to-noise ratio (SNR) in analog computing [236].”
From Brain-inspired computing systems: a systematic literature review (2024) · cites this paper as [13]
In the citing paper
“Altogether, processing one 224 × 224 input image needs a VGG-16 model of 15.5 G multiply-and-accumulates and 138 M weights [40].”
From Face Recognition Based on Deep Learning and FPGA for Ethnicity Identification (2022) · cites this paper as [40]
In the citing paper
“The energy consumption of executing a DNN model is highly dictated by the computation architecture and methods (e.g., ultra-low power compute-in-memory AI accelerator) at the edge computation nodes [249], the architecture of DNN models [250], and the wireless transmission for data exchange during the model inference procedure.”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [250]
In the citing paper
“but there is no review discussing all three categories which in future will be integrated seamlessly to design effective and efficient DNNs in the new edge intelligence era.”
From Bringing ai to edge: from deep learning’s perspective (2022) · cites this paper as [25]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 15×
“In 2015, the ImageNet winning entry, ResNet [15], exceeded human-level accuracy with a top-5 error rate4 below 5%.”
InfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 10×
“Used in VGG-16. decompose 5x5 filter 5x1 filter 1x5 filter Apply sequentially (b) Constructing a 5×5 support from 1×5 and 5×1 filter.”
InfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 10×
“this, local response normalization (LRN) [3] was used, which”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“In contrast, recurrent neural networks (RNNs), of which Long Short-Term Memory networks (LSTMs) [38] are a popular variant, have internal memory to allow long-term dependencies to affect the output.”
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 4×
“Table 2 Summary of Popular DNNs [3], [15], [48], [50], [51]”
InfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 8×
“†ACCURACY IS MEASURED BASED ON TOP-5 ERROR ON IMAGENET [14].”
InfluentialFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Research works printed but not linked: 34
- 10.13140/RG.2.2.18893.74727 · DOI · Paper page
- Li, Stanford CS Class CS231n: Convolutional Neural Networks for Visual Recognition
- 2016, Complete Visual Networking Index (VNI) Forecast
- Woodhouse, 2016, Big, Big, Big Data: Higher and Higher Resolution Video Surveillance
- Maas, Rectifier nonlinearities improve neural network acoustic models, 1
- Caffe LeNet MNIST
- Caffe Model Zoo
- Matconvnet Pretrained Models
- TensorFlow-Slim Image Classification Library
- Deep Learning Frameworks
- Yann, The MNIST DATABASE of Handwritten Digits
- Krizhevsky, The CIFAR-10 Dataset
- Krizhevsky, 2010, Convolutional deep belief networks on cifar-10, Unpublished Manuscript, 40, 1
- Pascal VOC Data Sets
- Microsoft Common Objects in Context (COCO) Dataset
- Google Open Images
- YouTube-8M
- AudioSet
- 2017, Intel Architecture Instruction Set Extensions Programming Reference
- Condon, Facebook unveils Big Basin, new server geared for deep learning
- Intel Math Kernel Library
- 10.1145/1816038.1815993 · DOI
- 10.1145/2872887.2750389 · DOI
- 10.1145/2644865.2541967 · DOI · Paper page
- Standard, High bandwidth memory (HBM) DRAM
- 10.1145/3093336.3037702 · DOI
- Wilson, International technology roadmap for semiconductors (ITRS)
- 10.1109/icvd.2004.1260946 · doi:10.1109/icvd.2004.1260946
- Lee, A 2.5 GHz 7.7 TOPS/W switched-capacitor matrix multiplier with co-designed local memory in 40 nm, 418
- Lee, LogNet: Energy-efficient neural networks using logrithmic computations, 5900
- DNN Energy Estimation
- 10.1145/3140659.3080254 · DOI · Paper page
- Iandola, SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1 MB model size
- Benchmarking DNN Processors
Cited by
A survey of the recent architectures of deep convolutional neural networks
Khan, A., Sohail, A., Zahoora, U., & Qureshi, A. S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09825-6 · Cited by 3,137
“However, the training of deep and high capacity architectures is still a significant overhead on memory usage and computational resources (Lacey et al. 2016; Sze et al. 2017; Justus et al. 2019).”
BackgroundEdge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455
“Interested readers are encouraged to refer to the recent monograph [114] for more comprehensive discussions about recent advancements on hardware acceleration for DNN processing.”
BackgroundMemory devices and applications for in-memory computing
Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R., & Eleftheriou, E. (2020) · Nature Nanotechnology · doi:10.1038/s41565-020-0655-z · Cited by 2,289
“NaTuRe NaNoTecHNology This non-volatile accumulative behaviour, in spite of its nonlinear and stochastic nature, can be exploited in several applications, such as training deep neural networks, where the conductance values need to be incrementally modified.”
BackgroundDeep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
Photonics for artificial intelligence and neuromorphic computing
Shastri, B. J., Tait, A. N., Ferreira de Lima, T., Pernice, W. H. P., Bhaskaran, H., Wright, C. D., & Prucnal, P. R. (2021) · Nature Photonics · doi:10.1038/s41566-020-00754-y · Cited by 1,756
A State-of-the-Art Survey on Deep Learning Theory and Architectures
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., Van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019) · Electronics · doi:10.3390/electronics8030292 · Cited by 1,620
Deep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540
SLIM: Simultaneous Logic-in-Memory Computing Exploiting Bilayer Analog OxRAM Devices
Kingra, S. K., Parmar, V., Chang, C.-C., Hudec, B., Hou, T.-H., & Suri, M. (2020) · Scientific Reports · doi:10.1038/s41598-020-59121-0 · Cited by 1,439
Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
Deng, S., Zhao, H., Fang, W., Yin, J., Dustdar, S., & Zomaya, A. Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2984887 · Cited by 1,147
Convolutional neural network: a review of models, methodologies and applications to object detection
Dhillon, A., & Verma, G. K. (2020) · Progress in Artificial Intelligence · doi:10.1007/s13748-019-00203-0 · Cited by 1,119
“This segment presents some of those instances where DNNs are presently having a considerable effect: • Human activity recognition [27, 28].”
BackgroundInfluentialEyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices
Chen, Y.-H., Yang, T.-J., Emer, J. S., & Sze, V. (2019) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2019.2910232 · Cited by 1,106
“To read the non-zero weights from a specific column, e.g., column 1 (assuming indexing starts from 0), the PE first reads address[1] and address[2] from the address vector in the CSC compressed weights, which gives the inclusive lower bound and non-inclusive upper bound of the addresses, i.e., 2 and…”
BackgroundSelf-citationA survey on deep learning for big data
Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.006 · Cited by 1,226
Application and theory gaps during the rise of artificial intelligence in education
Chen, X., Xie, H., Zou, D., & Hwang, G.-J. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100002 · Cited by 1,003
Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
Deng, B. L., Li, G., Han, S., Shi, L., & Xie, Y. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2976475 · Cited by 996
“The weights and psums propagate across Adapted from [14], [257], and [263].”
BackgroundMethodPruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 943
CNN Variants for Computer Vision: History, Architecture, Application, Challenges and Future Scope
Bhatt, D., Patel, C., Talsania, H., Patel, J., Vaghela, R., Pandya, S., Modi, K., & Ghayvat, H. (2021) · Electronics · doi:10.3390/electronics10202470 · Cited by 890
“Nonetheless, training more deep and high-capacity CNN architectures consumes a substantial amount of memory and computing resources [68,69]; • The fundamental disadvantage of CNNs is their inability to be applied in real-time.”
BackgroundMachine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey
Nguyen, G., Dlugolinsky, S., Bobák, M., Tran, V., López García, Á., Heredia, I., Malík, P., & Hluchý, L. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-018-09679-z · Cited by 864
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S. K. E., Sidike, P., Nasrin, M. S., Essen, B. C. V., Awwal, A. A. S., & Asari, V. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.01164 · Cited by 426
“AI: Artificial Intelligence, ML, NN, DL, and Spiking Neural Networks (SNN) according to [294].”
BackgroundMethodTang, J., Yuan, F., Shen, X., Wang, Z., Rao, M., He, Y., Sun, Y., Li, X., Zhang, W., Li, Y., et al. (2019) · Advanced Materials · doi:10.1002/adma.201902761 · Cited by 834
A survey on deep learning for multimodal data fusion
Gao, J., Li, P., Chen, Z., & Zhang, J. (2020) · Neural Computation · doi:10.1162/neco_a_01273 · Cited by 821
“A convolutional neural network is a special kind of deep network that considers the local topology of data (Li, Xia, Du, Lin, & Samat, 2017; Sze, Chen, Yang, & Emer, 2017).”
BackgroundEdge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“, ultra-low power compute-in-memory AI accelerator) at the edge computation nodes [249], the architecture of DNN models [250], and the wireless transmission for data exchange during the model inference procedure.”
MethodA survey of deep learning-based network anomaly detection
Kwon, D., Kim, H., Kim, J., Suh, S. C., Kim, I., & Kim, K. J. (2019) · Cluster Computing · doi:10.1007/s10586-017-1117-8 · Cited by 778
Deep Learning With Spiking Neurons: Opportunities and Challenges
Pfeiffer, M., & Pfeil, T. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00774 · Cited by 766
COVID-CAPS: A capsule network-based framework for identification of COVID-19 cases from X-ray images
Afshar, P., Heidarian, S., Naderkhani, F., Oikonomou, A., Plataniotis, K. N., & Mohammadi, A. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.09.010 · Cited by 705
An optical neural chip for implementing complex-valued neural network
Zhang, H., Gu, M., Jiang, X. D., Thompson, J., Cai, H., Paesani, S., Santagati, R., Laing, A., Zhang, Y., Yung, M. H., et al. (2021) · Nature Communications · doi:10.1038/s41467-020-20719-7 · Cited by 666
Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review
Li, Y., Ma, L., Zhong, Z., Liu, F., Chapman, M. A., Cao, D., & Li, J. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3015992 · Cited by 611
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
Menghani, G. (2023) · ACM Computing Surveys · doi:10.1145/3578938 · Cited by 605
“There are other survey papers that cover a limited part of the field of efficient deep learning such as specific compression techniques like quantization and distillation [26, 94], architecture search [3], or hardware design [129].”
BackgroundMetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
Liu, Z., Mu, H., Zhang, X., Guo, Z., Yang, X., Cheng, K.-T., & Sun, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00339 · Cited by 563
“A comprehensive survey is provided in [47].”
Demystifying Parallel and Distributed Deep Learning: An In-depth Concurrency Analysis
Ben-Nun, T., & Hoefler, T. (2020) · ACM Computing Surveys · doi:10.1145/3320060 · Cited by 577
“Since mobile hardware is limited in memory capacity and has to be energy efficient, specialized DNN computational hardware is frequently proposed [233].”
BackgroundWhat is the State of Neural Network Pruning?
Blalock, D., Ortiz, J. J. G., Frankle, J., & Guttag, J. (2020) · arXiv · doi:10.48550/arxiv.2003.03033 · Cited by 560
“These requirements not only increase infrastructure costs, but also make deployment of networks to resource-constrained environments such as mobile phones or smart devices challenging (Han et al., 2015; Sze et al., 2017; Yang et al., 2017).”
BackgroundProtecting Intellectual Property of Deep Neural Networks with Watermarking
Zhang, J., Gu, Z., Jang, J., Wu, H., Stoecklin, M. P., Huang, H., & Molloy, I. (2018) · Proceedings of the 2018 on Asia Conference on Computer and Communications Security · doi:10.1145/3196494.3196550 · Cited by 540
“The goal of model pruning is to reduce redundant parameters, but still keep the performance of original deep neural networks [8, 21, 41, 49, 51].”
BackgroundCowls, J., Tsamados, A., Taddeo, M., & Floridi, L. (2023) · AI & SOCIETY · doi:10.1007/s00146-021-01294-x · Cited by 552
“In the medium term, the energy consumption of the inference phase scales with usage, as inference can usually occur millions of times per day for an indefinite amount of time (Sze et al. 2017).”
BackgroundMethodCowls, J., Tsamados, A., Taddeo, M., & Floridi, L. (2021) · doi:10.2139/ssrn.3804983 · Cited by 552
A Configurable Cloud-Scale DNN Processor for Real-Time AI
Fowers, J., Ovtcharov, K., Papamichael, M., Massengill, T., Liu, M., Lo, D., Alkalay, S., Haselman, M., Adams, L., Ghandi, M., et al. (2018) · 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2018.00012 · Cited by 520
“For a more detailed survey on efficient DNN accelerators, refer to [48] and [100].”
BackgroundMetrology for the next generation of semiconductor devices
Orji, N. G., Badaroglu, M., Barnes, B. M., Beitia, C., Bunday, B. D., Celano, U., Kline, R. J., Neisser, M., Obeng, Y., & Vladar, A. E. (2018) · Nature Electronics · doi:10.1038/s41928-018-0150-9 · Cited by 501
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review
Shawahna, A., Sait, S. M., & El-Maleh, A. (2019) · IEEE Access · doi:10.1109/access.2018.2890150 · Cited by 496
“Similar approaches to the DianNao family of techniques are presented in [115] with similar limitations.”
MethodGazelle: A Low Latency Framework for Secure Neural Network Inference
Juvekar, C., Vaikuntanathan, V., & Chandrakasan, A. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.05507 · Cited by 164
“For more details on CNNs, we refer the reader to [37].”
BackgroundCommunication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
“Such extremely high network bandwidth and low latency requirements would place unprecedented pressures on traditional cloud-based AI, where massive sensors/embedded devices transfer collected data to the cloud [8], often under varying network qualities (e.”
BackgroundIn-Memory Computing: Advances and Prospects
Verma, N., Jia, H., Valavi, H., Tang, Y., Ozatay, M., Chen, L.-Y., Zhang, B., & Deaville, P. (2019) · IEEE Solid-State Circuits Magazine · doi:10.1109/mssc.2019.2922889 · Cited by 473
“Here, processing engines (PEs) are arranged in a 2D array, matching the dataflow of MVM [5], [6].”
MethodScaling for edge inference of deep neural networks
Xu, X., Ding, Y., Hu, S. X., Niemier, M., Cong, J., Hu, Y., & Shi, Y. (2018) · Nature Electronics · doi:10.1038/s41928-018-0059-3 · Cited by 462
Software Vulnerability Detection Using Deep Neural Networks: A Survey
Lin, G., Wen, S., Han, Q.-L., Zhang, J., & Xiang, Y. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2993293 · Cited by 457
“1) enable the learning of high-level features or representations with more complexity and abstraction [104], compared with conventional ML algorithms; 2) are capable of automatically learning latent features or representations which are more generalizable, thus relieving practitioners from labor-intensive, subjective,…”
BackgroundPareek, P., & Thakkar, A. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09904-8 · Cited by 455
XNOR-SRAM: In-Memory Computing SRAM Macro for Binary/Ternary Deep Neural Networks
Yin, S., Jiang, Z., Seo, J.-S., & Seok, M. (2020) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2019.2963616 · Cited by 429
Emerging Memristive Artificial Synapses and Neurons for Energy‐Efficient Neuromorphic Computing
Choi, S., Yang, J., & Wang, G. (2020) · Advanced Materials · doi:10.1002/adma.202004659 · Cited by 424
PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference
Ankit, A., Hajj, I. E., Chalamalasetti, S. R., Ndu, G., Foltin, M., Williams, R. S., Faraboschi, P., Hwu, W.-M. W., Strachan, J. P., Roy, K., et al. (2019) · Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3297858.3304049 · Cited by 424
“[103] provide a thorough survey of deep learning accelerators.”
BackgroundCryptocurrency trading: a comprehensive survey
Fang, F., Ventre, C., Basios, M., Kanthan, L., Martinez-Rego, D., Wu, F., & Li, L. (2022) · Financial Innovation · doi:10.1186/s40854-021-00321-6 · Cited by 413
“Deep learning algorithms are currently the basis for many modern artificial intelligence applications [220].”
BackgroundEfficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3226481 · Cited by 436
“Block diagram of the generic DNN inference accelerator [2].”
BackgroundMethodInfluentialBiswas, A., & Chandrakasan, A. P. (2019) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2018.2880918 · Cited by 414
“In the conventional all-digital implementation of CNNs [4], [5], [9], with the memory and the processing elements being physically separate, reading wi s and the partial sums from the on-chip SRAMs lead to a lot of data movement per computation [10] and, hence, make them energy-hungry.”
BackgroundWan, J., Li, X., Dai, H.-N., Kusiak, A., Martinez-Garcia, M., & Li, D. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3034808 · Cited by 382
“For example, high-performance computing servers equipped with GPUs are often required to fasten the training process on massive data [49] while exiting manufacturing facilities may not fulfill the stringent requirement on computing capability.”
BackgroundDistributed Artificial Intelligence Empowered by End-Edge-Cloud Computing: A Survey
Duan, S., Wang, D., Ren, J., Lyu, F., Zhang, Y., Wu, H., & Shen, X. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3218527 · Cited by 408
“or more hidden layers are ultimately propagated to the output layer, which presents the final outputs of the network to the user [25].”
BackgroundLarge-Scale Optical Neural Networks Based on Photoelectric Multiplication
Hamerly, R., Bernstein, L., Sludds, A., Soljačić, M., & Englund, D. (2019) · Physical Review X · doi:10.1103/physrevx.9.021032 · Cited by 388
“3(a) shows the standard approach: a scatterplot comparing error rate with number of MACs, a rough proxy for time or energy consumption [12, 13].”
BackgroundMethodInfluentialFastDepth: Fast Monocular Depth Estimation on Embedded Systems
Wofk, D., Ma, F., Yang, T.-J., Karaman, S., & Sze, V. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794182 · Cited by 384
“Prior research on designing fast and efficient networks has primarily focused on encoder networks for tasks such as image classification and object detection [1].”
BackgroundSelf-citationCompute-in-Memory Chips for Deep Learning: Recent Trends and Prospects
Yu, S., Jiang, H., Huang, S., Peng, X., & Lu, A. (2021) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2021.3092533 · Cited by 391
JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services
Eshratifar, A. E., Abrishami, M. S., & Pedram, M. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2947893 · Cited by 381
“Recent studies show representing the parameters of DNNs with only 4-bits affect the accuracy not more than 1% [38].”
BackgroundReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions
Liu, Z., Shen, Z., Savvides, M., & Cheng, K.-T. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58568-6_9 · Cited by 362
Structured Pruning for Deep Convolutional Neural Networks: A Survey
He, Y., & Xiao, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3334614 · Cited by 363
Thakkar, A., & Lohiya, R. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-020-09496-0 · Cited by 366
NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
Yang, T.-J., Howard, A., Chen, B., Zhang, X., Go, A., Sandler, M., Sze, V., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_18 · Cited by 572
Self-citationA Review of Face Recognition Technology
Li, L., Mu, X., Li, S., & Peng, H. (2020) · IEEE Access · doi:10.1109/access.2020.3011028 · Cited by 360
“It is the most widely used deep learning framework [28], [29].”
SAR Automatic Target Recognition Based on Multiview Deep Learning Framework
Pei, J., Huang, Y., Huo, W., Zhang, Y., Yang, J., & Yeo, T.-S. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2776357 · Cited by 358
“Furthermore, there are many hardware accelerations and computational methods [38], [39]”
MethodA critical review of RNN and LSTM variants in hydrological time series predictions
Waqas, M., & Humphries, U. W. (2024) · MethodsX · doi:10.1016/j.mex.2024.102946 · Cited by 354
Bayesian Compression for Deep Learning
Louizos, C., Ullrich, K., & Welling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08665 · Cited by 312
“As an illustration, consider the convolutional layers of Alexnet, which account for only 4% of the parameters but 91% of the computation [68].”
MethodPruning neural networks without any data by iteratively conserving synaptic flowPreprint
Tanaka, H., Kunin, D., Yamins, D. L. K., & Ganguli, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05467 · Cited by 829
“Network pruning, or the compression of neural networks by removing parameters, has been an important subject both for reasons of practical deployment [1, 2, 3, 4, 5, 6, 7] and for theoretical understanding of artificial [8] and biological [9] neural networks.”
BackgroundState-of-Charge Estimation of Li-Ion Battery in Electric Vehicles: A Deep Neural Network Approach
How, D. N. T., Hannan, M. A., Lipu, M. S. H., Sahari, K. S. M., Ker, P. J., & Muttaqi, K. M. (2020) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2020.3004294 · Cited by 337
Training and Inference with Integers in Deep Neural Networks
Wu, S., Li, G., Chen, F., & Shi, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04680 · Cited by 306
Mehonic, A., Sebastian, A., Rajendran, B., Simeone, O., Vasilaki, E., & Kenyon, A. J. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000085 · Cited by 331
Cust, E. E., Sweeting, A. J., Ball, K., & Robertson, S. (2019) · Journal of Sports Sciences · doi:10.1080/02640414.2018.1521769 · Cited by 324
“An unsupervised approach could suit specific problems to explain key data features, via clustering (Mohammed et al., 2016; Sze et al., 2017).”
BackgroundMethodInfluentialAuto-Encoders in Deep Learning—A Review with New Perspectives
Chen, S., & Guo, W. (2023) · Mathematics · doi:10.3390/math11081777 · Cited by 318
A Survey of Android Malware Detection with Deep Neural Models
Qiu, J., Zhang, J., Luo, W., Pan, L., Nepal, S., & Xiang, Y. (2021) · ACM Computing Surveys · doi:10.1145/3417978 · Cited by 321
“Compared 126:7 with the traditional ML, in the application of Android malware analysis, DL techniques have the following advantages: • Comparing with traditional ML algorithms, DL is good at learning the high-level abstract representations with more complex patterns within Android apps [69].”
BackgroundMethodIelmini, D., & Ambrogio, S. (2020) · Nanotechnology · doi:10.1088/1361-6528/ab554b · Cited by 321
Automated identification and grading system of diabetic retinopathy using deep neural networks
Zhang, W., Zhong, J., Yang, S., Gao, Z., Hu, J., Chen, Y., & Yi, Z. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.03.016 · Cited by 311
Self-citationZeng, L., Chen, X., Zhou, Z., Yang, L., & Zhang, J. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2020.3042320 · Cited by 313
“Meanwhile, advances in Deep Neural Networks (DNNs) have shown unprecedented ability in learning abstract representation and extracting high-level features, promoting significant improvement in processing human-centric contents [7].”
BackgroundFP-BNN: Binarized neural network on FPGA
Liang, S., Yin, S., Liu, L., Luk, W., & Wei, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.09.046 · Cited by 298
“Table 1 Summary of weight and MAC number of popular CNNs [21] .”
BackgroundInfluentialShrestha, Y. R., Krishna, V., & von Krogh, G. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2020.09.068 · Cited by 331
Liu, L., Wang, R., Xie, C., Yang, P., Wang, F., Sudirman, S., & Liu, W. (2019) · IEEE Access · doi:10.1109/access.2019.2909522 · Cited by 291
“In deep networks that contain residual blocks, we use bottleneck [43] to reduce computational cost by adopting extra convolution operation.”
MethodHemanth, D. J., Deperlioglu, O., & Kose, U. (2020) · Neural Computing and Applications · doi:10.1007/s00521-018-03974-0 · Cited by 285
RetractedComputing Graph Neural Networks: A Survey from Algorithms to Accelerators
Abadal, S., Jain, A., Guirado, R., López-Alonso, J., & Alarcón, E. (2022) · ACM Computing Surveys · doi:10.1145/3477141 · Cited by 285
The Road for 2D Semiconductors in the Silicon Age
Wang, S., Liu, X., & Zhou, P. (2022) · Advanced Materials · doi:10.1002/adma.202106886 · Cited by 277
A Survey on Deep Learning for Human Activity Recognition
Gu, F., Chung, M.-H., Chignell, M., Valaee, S., Zhou, B., & Liu, X. (2022) · ACM Computing Surveys · doi:10.1145/3472290 · Cited by 272
“To relieve the burden of hand-engineering features, deep learning methods have been proposed in recent years [112, 207].”
MethodCMOS-integrated memristive non-volatile computing-in-memory for AI edge processors
Chen, W.-H., Dou, C., Li, K.-X., Lin, W.-Y., Li, P.-Y., Huang, J.-H., Wang, J.-H., Wei, W.-C., Xue, C.-X., Chiu, Y.-C., et al. (2019) · Nature Electronics · doi:10.1038/s41928-019-0288-0 · Cited by 276
An optical neural network using less than 1 photon per multiplication
Wang, T., Ma, S.-Y., Wright, L. G., Onodera, T., Richard, B. C., & McMahon, P. L. (2022) · Nature Communications · doi:10.1038/s41467-021-27774-8 · Cited by 268
A comprehensive review of model compression techniques in machine learning
Dantas, P. V., Sabino da Silva, W., Cordeiro, L. C., & Carvalho, C. B. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05747-w · Cited by 261
“Neural Architecture [74, 119, 151, 153–156, 172, 173, 205, 206, 233, 236, 263, 268, 289, 290] Optimization [15, 124, 184, 222, 228, 230, 261, 277] Recurrent neural networks [38, 39, 159, 267, 291] Miscellaneous [3, 5, 6, 16, 18, 28, 31, 32, 34, 41, 46–48,50,51,53,54,61,68,73,76,…”
Pruning by explaining: A novel criterion for deep neural network pruning
Yeom, S.-K., Seegerer, P., Lapuschkin, S., Binder, A., Wiedemann, S., Müller, K.-R., & Samek, W. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.107899 · Cited by 263
Lu, Y., Saeys, W., Kim, M., Peng, Y., & Lu, R. (2020) · Postharvest Biology and Technology · doi:10.1016/j.postharvbio.2020.111318 · Cited by 263
NestDNN: Resource-Aware Multi-Tenant On-Device Deep Learning for Continuous Mobile Vision
Fang, B., Zeng, X., & Zhang, M. (2018) · Proceedings of the 24th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3241539.3241559 · Cited by 291
“Unfortunately, deep learning models are known to be resource-demanding [35].”
BackgroundAutomated Categorization of Brain Tumor from MRI Using CNN features and SVM
Deepak, S., & Ameer, P. M. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-02568-w · Cited by 260
Ju, Y., Chen, Y., Cao, Z., Liu, L., Pei, Q., Xiao, M., Ota, K., Dong, M., & Leung, V. C. M. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3242997 · Cited by 245
Self-citationNamed Entity Recognition and Relation Extraction: State-of-the-Art
Nasar, Z., Jaffry, S. W., & Malik, M. K. (2022) · ACM Computing Surveys · doi:10.1145/3445965 · Cited by 245
“These types of solutions require understanding of the datasets at hand along with the idea about current and future computational requirements, as discussed in Reference [157].”
BackgroundThe Next Generation of Deep Learning Hardware: Analog Computing
Haensch, W., Gokmen, T., & Puri, R. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2871057 · Cited by 235
“To drive further improvement of compute efficiency, features of deep learning algorithms can be exploited that are unique to that space [6], [7].”
BackgroundMethodArtificial intelligence focus and firm performance
Mishra, S., Ewing, M. T., & Cooper, H. B. (2022) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-022-00876-5 · Cited by 231
“AI refers to the ability of `machines to mimic intelligent human behaviour´ (Syam & Sharma, 2018, p. 136), encompassing the subdomains of machine learning, neural networks and deep learning (Shrestha et al., 2021; Sze et al., 2017).”
BackgroundA Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification
Rokh, B., Azarpeyvand, A., & Khanteymoori, A. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3623402 · Cited by 232
“Nevertheless, some works have performed the quantization of these parameters, such as paper [106] which represents biases with 1-bit precision. after training, activations change during the inference phase according to the input data.”
BackgroundMachine learning for structural health monitoring: challenges and opportunities
Yuan, F.-G., Zargar, S. A., Chen, Q., & Wang, S. (2020) · Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2020 · doi:10.1117/12.2561610 · Cited by 207
“Deep learning automatically discovers intricate features in high-dimensional data which has led to its widespread adoption across many application domains [38-41].”
BackgroundLiao, L., Li, H., Shang, W., & Ma, L. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3506695 · Cited by 222
“Energy and power are important when executing DNN models in mobile devices with the limited battery capacity [79].”
BackgroundRecognition of peripheral blood cell images using convolutional neural networks
Acevedo, A., Alférez, S., Merino, A., Puigví, L., & Rodellar, J. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.105020 · Cited by 224
Recent progress in analog memory-based accelerators for deep learning
Tsai, H., Ambrogio, S., Narayanan, P., Shelby, R. M., & Burr, G. W. (2018) · Journal of Physics D: Applied Physics · doi:10.1088/1361-6463/aac8a5 · Cited by 225
A Programmable Heterogeneous Microprocessor Based on Bit-Scalable In-Memory Computing
Jia, H., Valavi, H., Tang, Y., Zhang, J., & Verma, N. (2020) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2020.2987714 · Cited by 224
“the trend toward modular CNN design, where other kernel sizes, such as 5 × 5 and 11 × 11, can be composed of 3 × 3 kernels [4], [11].”
BackgroundSurvey and Benchmarking of Machine Learning Accelerators
Reuther, A., Michaleas, P., Jones, M., Gadepally, V., Samsi, S., & Kepner, J. (2019) · 2019 IEEE High Performance Extreme Computing Conference (HPEC) · doi:10.1109/hpec.2019.8916327 · Cited by 175
“Conversely, pooling, dropout, softmax, and recurrent/skip connection layers are not computationally intensive since these types of layers stipulate datapaths for weight and data operands. including integer representations, have been shown to be reasonably effective for inference [7], [8].”
BackgroundCapra, M., Bussolino, B., Marchisio, A., Masera, G., Martina, M., & Shafique, M. (2020) · IEEE Access · doi:10.1109/access.2020.3039858 · Cited by 222
“[336] 2016 2 ≈ 5 5 5 5 5 5 ≈ ≈ 5 5 ≈ 5 5 [333] 2017 - 3 ≈ 5 5 5 5 ≈ 3 3 5 3 5 5 3 [92] 2017 247 3 3 5 3 ≈ 3 3 5 5 5 3 3 3 3”
BackgroundA Survey of Machine Learning for Computer Architecture and Systems
Wu, N., & Xie, Y. (2023) · ACM Computing Surveys · doi:10.1145/3494523 · Cited by 213
“To enable practical implementations of ML-based techniques, improvement can be made from either the model side or software/hardware co-design [215].”
MethodMa, Z., Ge, H., Liu, Y., Zhao, M., & Ma, J. (2019) · IEEE Access · doi:10.1109/access.2019.2896003 · Cited by 215
“DNN is part of the broad field of AI, which is the science and engineering of creating intelligent machines that have the ability to achieve goals like humans do [21].”
Background
Metadata
- Authors
- Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S.
- Affiliations
- Massachusetts Institute of Technology (US)
- Nvidia (United States) (US)
- Venue
- Proceedings of the IEEE
- Published
- 2017-12
- Volume
- 105
- Issue
- 12
- Pages
- 2295-2329
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W2604319603
- Cited by
- 4,215
- References
- 157
- Citations by year
- 2026: 504 · 2025: 523 · 2024: 474 · 2023: 466 · 2022: 508 · 2021: 568 · 2020: 551 · 2019: 392 · 2018: 199 · 2017: 28
- FWCI
- 132.75
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the IEEE (publishedVersion)
- Funders
- DARPA YFA
- MIT CICS
- gift from Nvidia
- gift from Intel
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- deep neural network, energy efficiency, throughput, hardware accelerators, computation cost reduction, benchmarking metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.