Efficient Processing of Deep Neural Networks: A Tutorial and Survey
The DOI matches its Crossref record; 144 of the 157 references it cites were found in a registry or an index.
Abstract
Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances toward the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic codesigns, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the tradeoffs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Classical arithmetic operations, such as MAC, constitute the bulk of the computational requirements for models used in artificial intelligence [13].”
From Brain-inspired computing systems: a systematic literature review (2024) · cites this paper as [13]
In the citing paper
“To clarify, consider the multiply-and-accumulate (MAC) operation in digital processors, where the operation energy consumption is significantly dependent on the bit width of the input operands [13, 235] or equivalently to the signal-to-noise ratio (SNR) in analog computing [236].”
From Brain-inspired computing systems: a systematic literature review (2024) · cites this paper as [13]
In the citing paper
“Altogether, processing one 224 × 224 input image needs a VGG-16 model of 15.5 G multiply-and-accumulates and 138 M weights [40].”
From Face Recognition Based on Deep Learning and FPGA for Ethnicity Identification (2022) · cites this paper as [40]
In the citing paper
“The energy consumption of executing a DNN model is highly dictated by the computation architecture and methods (e.g., ultra-low power compute-in-memory AI accelerator) at the edge computation nodes [249], the architecture of DNN models [250], and the wireless transmission for data exchange during the model inference procedure.”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [250]
In the citing paper
“but there is no review discussing all three categories which in future will be integrated seamlessly to design effective and efficient DNNs in the new edge intelligence era.”
From Bringing ai to edge: from deep learning’s perspective (2022) · cites this paper as [25]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 15×
“In 2015, the ImageNet winning entry, ResNet [15], exceeded human-level accuracy with a top-5 error rate4 below 5%.”
InfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 10×
“Used in VGG-16. decompose 5x5 filter 5x1 filter 1x5 filter Apply sequentially (b) Constructing a 5×5 support from 1×5 and 5×1 filter.”
InfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 10×
“this, local response normalization (LRN) [3] was used, which”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“In contrast, recurrent neural networks (RNNs), of which Long Short-Term Memory networks (LSTMs) [38] are a popular variant, have internal memory to allow long-term dependencies to affect the output.”
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 4×
“Table 2 Summary of Popular DNNs [3], [15], [48], [50], [51]”
InfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 8×
“†ACCURACY IS MEASURED BASED ON TOP-5 ERROR ON IMAGENET [14].”
InfluentialFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Research works printed but not linked: 34
- 10.13140/RG.2.2.18893.74727 · DOI · Paper page
- Li, Stanford CS Class CS231n: Convolutional Neural Networks for Visual Recognition
- 2016, Complete Visual Networking Index (VNI) Forecast
- Woodhouse, 2016, Big, Big, Big Data: Higher and Higher Resolution Video Surveillance
- Maas, Rectifier nonlinearities improve neural network acoustic models, 1
- Caffe LeNet MNIST
- Caffe Model Zoo
- Matconvnet Pretrained Models
- TensorFlow-Slim Image Classification Library
- Deep Learning Frameworks
- Yann, The MNIST DATABASE of Handwritten Digits
- Krizhevsky, The CIFAR-10 Dataset
- Krizhevsky, 2010, Convolutional deep belief networks on cifar-10, Unpublished Manuscript, 40, 1
- Pascal VOC Data Sets
- Microsoft Common Objects in Context (COCO) Dataset
- Google Open Images
- YouTube-8M
- AudioSet
- 2017, Intel Architecture Instruction Set Extensions Programming Reference
- Condon, Facebook unveils Big Basin, new server geared for deep learning
- Intel Math Kernel Library
- 10.1145/1816038.1815993 · DOI
- 10.1145/2872887.2750389 · DOI
- 10.1145/2644865.2541967 · DOI · Paper page
- Standard, High bandwidth memory (HBM) DRAM
- 10.1145/3093336.3037702 · DOI
- Wilson, International technology roadmap for semiconductors (ITRS)
- 10.1109/icvd.2004.1260946 · doi:10.1109/icvd.2004.1260946
- Lee, A 2.5 GHz 7.7 TOPS/W switched-capacitor matrix multiplier with co-designed local memory in 40 nm, 418
- Lee, LogNet: Energy-efficient neural networks using logrithmic computations, 5900
- DNN Energy Estimation
- 10.1145/3140659.3080254 · DOI · Paper page
- Iandola, SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1 MB model size
- Benchmarking DNN Processors
Cited by
Items 101–200 of 1,011
Ma, Z., Ge, H., Liu, Y., Zhao, M., & Ma, J. (2019) · IEEE Access · doi:10.1109/access.2019.2896003 · Cited by 215
“DNN is part of the broad field of AI, which is the science and engineering of creating intelligent machines that have the ability to achieve goals like humans do [21].”
BackgroundOnline prediction of mechanical properties of hot rolled steel plate using machine learning
Xie, Q., Suvarna, M., Li, J., Zhu, X., Cai, J., & Wang, X. (2021) · Materials & Design · doi:10.1016/j.matdes.2020.109201 · Cited by 215
Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications
Azghadi, M. R., Lammie, C., Eshraghian, J. K., Payvand, M., Donati, E., Linares-Barranco, B., & Indiveri, G. (2020) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2020.3036081 · Cited by 212
“tribute to increasing throughput of a larger processor [81], [82].”
BackgroundQuantum optical neural networks
Steinbrecher, G. R., Olson, J. P., Englund, D., & Carolan, J. (2019) · npj Quantum Information · doi:10.1038/s41534-019-0174-7 · Cited by 209
Kumar, P., Chauhan, S., & Awasthi, L. K. (2024) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09986-x · Cited by 206
“In recent years, deep learning approaches have been proposed [12, 13] to alleviate the need for human intervention in the feature engineering process.”
Bayesian State Estimation for Unobservable Distribution Systems Via Deep Learning
Mestav, K. R., Luengo-Rozas, J., & Tong, L. (2019) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2019.2919157 · Cited by 216
“Special hardware that exploits massive parallelism can greatly speed up the computation [49].”
BackgroundDemystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
Ben‐Nun, T., & Hoefler, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09941 · Cited by 203
“Since mobile hardware is limited in memory capacity and has to be energy efficient, specialized DNN computational hardware is frequently proposed [233].”
BackgroundUrban form and composition of street canyons: A human-centric big data and deep learning approach
Middel, A., Lukasczyk, J., Zakrzewski, S., Arnold, M., & Maciejewski, R. (2019) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2018.12.001 · Cited by 204
A Deep Learning Method for Short-Term Residential Load Forecasting in Smart Grid
Hong, Y., Zhou, Y., Li, Q., Xu, W., & Zheng, X. (2020) · IEEE Access · doi:10.1109/access.2020.2981817 · Cited by 201
“ral Network (DNN) [43], Extreme Learning Machine [36].”
Robles-Velasco, A., Cortés, P., Muñuzuri, J., & Onieva, L. (2020) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2019.106754 · Cited by 195
“The learning of networks is achieved by adjusting those weights, while its structure does not usually vary [33].”
BackgroundToward Edge-Based Deep Learning in Industrial Internet of Things
Liang, F., Yu, W., Liu, X., Griffith, D., & Golmie, N. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2963635 · Cited by 199
“The number of convolutional layers is an important parameter in a CNN model, affecting performance directly [29].”
BackgroundBayesian neural networks for flight trajectory prediction and safety assessment
Zhang, X., & Mahadevan, S. (2020) · Decision Support Systems · doi:10.1016/j.dss.2020.113246 · Cited by 195
Capra, M., Bussolino, B., Marchisio, A., Shafique, M., Masera, G., & Martina, M. (2020) · Future Internet · doi:10.3390/fi12070113 · Cited by 195
Liu, J., Xiang, J., Jin, Y., Liu, R., Yan, J., & Wang, L. (2021) · Remote Sensing · doi:10.3390/rs13214387 · Cited by 193
Deep learning with coherent VCSEL neural networks
Chen, Z., Sludds, A., Davis, R., Christen, I., Bernstein, L., Ateshian, L., Heuser, T., Heermeier, N., Lott, J. A., Reitzenstein, S., et al. (2023) · Nature Photonics · doi:10.1038/s41566-023-01233-w · Cited by 191
Analog architectures for neural network acceleration based on non-volatile memory
Xiao, T. P., Bennett, C. H., Feinberg, B., Agarwal, S., & Marinella, M. J. (2020) · Applied Physics Reviews · doi:10.1063/1.5143815 · Cited by 193
Delocalized photonic deep learning on the internet’s edge
Sludds, A., Bandyopadhyay, S., Chen, Z., Zhong, Z., Cochrane, J., Bernstein, L., Bunandar, D., Dixon, P. B., Hamilton, S. A., Streshinsky, M., et al. (2022) · Science · doi:10.1126/science.abq8271 · Cited by 189
Asymptotic Soft Filter Pruning for Deep Convolutional Neural Networks
He, Y., Dong, X., Kang, G., Fu, Y., Yan, C., & Yang, Y. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2933477 · Cited by 191
“Besides, Chen et al. [18] and Sze et al. [19] claimed that VGGNet [13] requires 321.1-MB memory and 236-mW power for a batch of three frames and processes 0.7 frames/s even when it is deployed on the optimized energy-efficient chip.”
MethodInfluentialOmbabi, A. H., Ouarda, W., & Alimi, A. M. (2020) · Social Network Analysis and Mining · doi:10.1007/s13278-020-00668-1 · Cited by 192
A systematic literature review on hardware implementation of artificial intelligence algorithms
Talib, M. A., Majzoub, S., Nasir, Q., & Jamal, D. (2021) · The Journal of Supercomputing · doi:10.1007/s11227-020-03325-8 · Cited by 188
A Survey of ReRAM-Based Architectures for Processing-In-Memory and Neural Networks
Mittal, S. (2018) · Machine Learning and Knowledge Extraction · doi:10.3390/make1010005 · Cited by 185
Reinforcement learning for neural architecture search: A review
Jaafra, Y., Luc Laurent, J., Deruyver, A., & Saber Naceur, M. (2019) · Image and Vision Computing · doi:10.1016/j.imavis.2019.06.005 · Cited by 183
A Survey on Kolmogorov-Arnold Network
Somvanshi, S., Javed, S. A., Islam, M. M., Pandit, D., & Das, S. (2026) · ACM Computing Surveys · doi:10.1145/3743128 · Cited by 175
“This architecture, grounded in the Kolmogorov-Arnold representation theorem, enables KANs to approximate complex, multivariate functions while maintaining a degree of transparency often absent in standard models like MLPs [21, 33, 92].”
On-Demand Deep Model Compression for Mobile Devices: A Usage-Driven Model Selection Framework
Liu, S., Lin, Y., Zhou, Z., Nan, K., Liu, H., & Du, J. (2018) · Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3210240.3210337 · Cited by 217
“Compression is a commonly employed technique to trim down the complexity of DNNs, which can be performed by reducing the weight precision, or the number of operations, or both [41].”
BackgroundMethodA phase-change memory model for neuromorphic computing
Nandakumar, S. R., Le Gallo, M., Boybat, I., Rajendran, B., Sebastian, A., & Eleftheriou, E. (2018) · Journal of Applied Physics · doi:10.1063/1.5042408 · Cited by 178
Prospects and applications of photonic neural networks
Huang, C., Sorger, V. J., Miscuglio, M., Al-Qadasi, M., Mukherjee, A., Lampe, L., Nichols, M., Tait, A. N., Ferreira de Lima, T., Marquez, B. A., et al. (2022) · Advances in Physics: X · doi:10.1080/23746149.2021.1981155 · Cited by 180
Rasch, M. J., Mackin, C., Le Gallo, M., Chen, A., Fasoli, A., Odermatt, F., Li, N., Nandakumar, S. R., Narayanan, P., Tsai, H., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-40770-4 · Cited by 213
“Numerous hardware concepts have been introduced to accelerate DNN training and/or inference 2 – 4 , by approximating matrix-vector multiplications (MVMs) and other arithmetic with custom fl oating-point representations such as b fl oat16 5 or DLFloat 6 , or with reduced-precision fi xed-point…”
BackgroundAnomaly detection using edge computing in video surveillance system: review
Patrikar, D. R., & Parate, M. R. (2022) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-022-00227-8 · Cited by 177
“Despite the rapid development of learning methods, CNNs, and edge devices for computer vision purposes particularly, the gap between software and hardware implementations is already considerable [101].”
BackgroundI Know What You See: Power Side-Channel Attack on Convolutional Neural Network Accelerators
Wei, L., Luo, B., Li, Y., Liu, Y., & Xu, Q. (2018) · Proceedings of the 34th Annual Computer Security Applications Conference · doi:10.1145/3274694.3274696 · Cited by 174
Deep and Embedded Learning Approach for Traffic Flow Prediction in Urban Informatics
Zheng, Z., Yang, Y., Liu, J., Dai, H.-N., & Zhang, Y. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2909904 · Cited by 175
“The standard convolution computational cost per time step in CNN is O ( N CNN ) , where N CNN is the total number of parameters in a CNN [46], [47].”
BackgroundXiao, B., Xu, Y., Bi, X., Zhang, J., & Ma, X. (2020) · Neurocomputing · doi:10.1016/j.neucom.2018.09.101 · Cited by 174
Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications
Azghadi, M. R., Lammie, C., Eshraghian, J. K., Payvand, M., Donati, E., Linares-Barranco, B., & Indiveri, G. (2020) · ResearchOnline - JCU (James Cook University) · doi:10.3929/ethz-b-000465927 · Cited by 176
Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference
Verelst, T., & Tuytelaars, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00239 · Cited by 170
“Parallel execution on GPU or FPGA relies on the regularity of standard convolutions to pipeline operations [26, 42] and adding element-wise conditional statements might strongly slow down inference.”
Survey of Machine Learning Accelerators
Reuther, A., Michaleas, P., Jones, M., Gadepally, V., Samsi, S., & Kepner, J. (2020) · 2020 IEEE High Performance Extreme Computing Conference (HPEC) · doi:10.1109/hpec43674.2020.9286149 · Cited by 163
“There are many surveys [12], [21]–[29] and other papers that cover various aspects of AI accelerators; this paper focuses on gathering a comprehensive list of AI accelerators with their computational capability, power efficiency, and ultimately the computational effectiveness of utilizing…”
BackgroundElectricity Theft Detection in Smart Grids Based on Deep Neural Network
Lepolesa, L. J., Achari, S., & Cheng, L. (2022) · IEEE Access · doi:10.1109/access.2022.3166146 · Cited by 165
“With present technology permission, DNNs can have a count of layers that is beyond a thousand [33].”
BackgroundInfluentialFine-grained leukocyte classification with deep residual learning for microscopic images
Qin, F., Gao, N., Peng, Y., Wu, Z., Shen, S., & Grudtsin, A. (2018) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.05.024 · Cited by 167
Bringing ai to edge: from deep learning’s perspective
Liu, D., Kong, H., Luo, X., Liu, W., & Subramaniam, R. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.04.141 · Cited by 171
“Some prior DNN reviews cover one of the three categories, like [25] [26] for model compression, [27] for early works on efficient DNN processing and [28] for NAS1 but there is no review discussing all three categories which in future will be integrated seamlessly to design effective and efficient DNNs in the new edge…”
BackgroundEnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networks
Liang, S., Wang, Y., Liu, C., He, L., Li, H., Xu, D., & Li, X. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2020.3014632 · Cited by 166
Deep Neural Network Approximation for Custom Hardware: Where We've Been, Where We're Going
Wang, E., Davis, J. J., Zhao, R., Ng, H.-C., Niu, X., Luk, W., Cheung, P. Y. K., & Constantinides, G. A. (2020) · ACM Computing Surveys · doi:10.1145/3309551 · Cited by 166
“For example, a small LeNet-5 model targeing the simple MNIST handwrien digitclassication task requires 680 kop/cl (thousand arithmetic operations per classication, where an arithmetic operation is either an addition or multiplication), while a VGG16 implementation executing the 1000-class ImageNet task requires 31…”
MethodReview of Lightweight Deep Convolutional Neural Networks
Chen, F., Li, S., Han, J., Ren, F., & Yang, Z. (2024) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-10032-z · Cited by 164
“More related works refer to FPGAs and ASICs are detailed in [16, 168, 169, 175].”
Learning Object Grasping for Soft Robot Hands
Choi, C., Schwarting, W., DelPreto, J., & Rus, D. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2810544 · Cited by 161
“However, it was recently shown that most of the energy, about 80% of the entire computational effort, is consumed by the convolution layers [29].”
An End-to-End Deep Neural Network for Autonomous Driving Designed for Embedded Automotive Platforms
Kocić, J., Jovičić, N., & Drndarević, V. (2019) · Sensors · doi:10.3390/s19092064 · Cited by 161
Moving Convolutional Neural Networks to Embedded Systems: The AlexNet and VGG-16 Case
Alippi, C., Disabato, S., & Roveri, M. (2018) · 2018 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2018.00049 · Cited by 161
“A relatively large literature about precision scaling in CNNs exists [42].”
BackgroundZeng, L., Li, E., Zhou, Z., & Chen, X. (2019) · IEEE Network · doi:10.1109/mnet.001.1800506 · Cited by 159
“[3] V.”
BackgroundInfluentialDeep Learning on Mobile and Embedded Devices: State-of-the-art, Challenges, and Future Directions
Chen, Y., Zheng, B., Zhang, Z., Wang, Q., Shen, C., & Zhang, Q. (2021) · ACM Computing Surveys · doi:10.1145/3398209 · Cited by 159
“Reference [168] mainly focused on hardware solutions, such as hardware platforms, circuits design, and the use of new memory technologies.”
BackgroundSelf-citationMixed-Signal Computing for Deep Neural Network Inference
Murmann, B. (2021) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2020.3020286 · Cited by 160
“In particular, we focus on convolution layers, which tend to dominate the size and workload of modern DNNs with millions of parameters [6].”
BackgroundEdge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI
Yao, J., Zhang, S., Yao, Y., Wang, F., Ma, J., Zhang, J., Chu, Y., Ji, L., Jia, K., Shen, T., et al. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3178211 · Cited by 229
“Except the storage, querying the model parameters for processing and inference is both time-consuming and energy-intensive [14], yielding power consumption and thermal performance both are bottlenecks.”
BackgroundWang, H., Liu, R., Dorrance, R., Dasalukunte, D., Lake, D., & Carlton, B. R. (2023) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2022.3232601 · Cited by 171
“Traditional digital schemes, as represented by the triangle markers, are still limited to the sub-10-TOPS/W region, making them difficult to deploy in energy-constrained edge devices [11], [12], [13], [14], [15].”
BackgroundFu, F., Wei, J., Zhang, M., Yu, F., Xiao, Y., Rong, D., Shan, Y., Li, Y., Zhao, C., Liao, F., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-18606-2 · Cited by 164
Self-citationDeep Reinforcement Learning Based Resource Management for DNN Inference in Industrial IoT
Zhang, W., Yang, D., Peng, H., Wu, W. R., Quan, W., Zhang, H., & Shen, X. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3068255 · Cited by 172
“1, implementing DNN inference requires two stages, namely, offline training and online inference [8].”
MethodReinforcement learning applied to production planning and control
Esteso, A., Peidro, D., Mula, J., & Díaz‐Madroñero, M. (2022) · International Journal of Production Research · doi:10.1080/00207543.2022.2104180 · Cited by 180
A computing-in-memory macro based on three-dimensional resistive random-access memory
Huo, Q., Yang, Y., Wang, Y., Lei, D., Fu, X., Ren, Q., Xu, X., Luo, Q., Xing, G., Chen, C., et al. (2022) · Nature Electronics · doi:10.1038/s41928-022-00795-x · Cited by 159
“T he use of convolutional neural networks (CNNs) has led to improvements in computer vision, natural language processing and speech recognition. tremendous These methods do, however, have computational and storage demands 1–3 .”
BackgroundXie, T., Zhang, G., Hou, J., Xie, J., Lv, M., & Liu, F. (2019) · Journal of Hydrology · doi:10.1016/j.jhydrol.2019.123915 · Cited by 156
Photonic Neural Networks: A Survey
Marinis, L. D., Cococcioni, M., Castoldi, P., & Andriolli, N. (2019) · IEEE Access · doi:10.1109/access.2019.2957245 · Cited by 164
Boveiri, H. R., Khayami, R., Elhoseny, M., & Gunasekaran, M. (2018) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-018-1071-1 · Cited by 149
A Survey of Techniques for Mobile Service Encrypted Traffic Classification Using Deep Learning
Wang, P., Chen, X., Ye, F., & Sun, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2912896 · Cited by 148
“The pooling provides a form of translation invariance and reduces computation for upper layers [18].”
BackgroundDNNOff: Offloading DNN-Based Intelligent IoT Applications in Mobile Edge Computing
Chen, X., Li, M., Zhong, H., Ma, Y., & Hsu, C. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3075464 · Cited by 155
“3075464 machine learning technique [1], the DNN has been applied in 30 industrial Internet of things (IoT) scenarios such as computer 31 vision [2] and self-driving cars [3].”
BackgroundMethodCompeting memristors for brain-inspired computing
Kim, S. J., Kim, S., & Jang, H. W. (2020) · iScience · doi:10.1016/j.isci.2020.101889 · Cited by 159
Cavalcante, M., Schuiki, F., Zaruba, F., Schaffner, M., & Benini, L. (2019) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2019.2950087 · Cited by 202
“The strong emergence of massively data-parallel workloads, such as data analytics and machine learning [6], created a major window of opportunity for architectures that effectively exploit data parallelism to achieve energy efficiency.”
BackgroundA Survey of On-Device Machine Learning: An Algorithms and Learning Theory Perspective
Dhar, S., Guo, J., Liu, J., Tripathi, S., Kurup, U., & Shah, M. (2021) · ACM Transactions on Internet of Things · doi:10.1145/3450494 · Cited by 276
“Besides the model size, querying the model parameters for processing is both time-consuming and energy intensive [167].”
BackgroundMethodThe Impact of Artificial Intelligence on Branding: A Bibliometric Analysis (1982-2019)
Varsha, P. S., Akter, S., Kumar, A., Gochhait, S., & Patagundi, B. (2021) · Journal of Global Information Management · doi:10.4018/jgim.20210701.oa10 · Cited by 192
“This is a broad subset of AI science, integrating neural networks (Sze et al., 2017).”
BackgroundEnable Deep Learning on Mobile Devices: Methods, Systems, and Applications
Cai, H., Lin, J., Lin, Y., Liu, Z., Tang, H., Wang, H., Zhu, L., & Han, S. (2022) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3486618 · Cited by 195
“General deep learning accelerators are mostly spatial architectures using dataflow processing [272] where ALUs form a processing chain so that they can pass data from one to another directly.”
BackgroundHoang, L.-H., Hanif, M. A., & Shafique, M. (2020) · Design, Automation, and Test in Europe · doi:10.23919/date48585.2020.9116571 · Cited by 161
“A more comprehensive overview of the neural networks can be found in [19].”
del Hougne, P., Imani, M. F., Diebold, A. V., Horstmeyer, R., & Smith, D. R. (2020) · Advanced Science · doi:10.1002/advs.201901913 · Cited by 143
“In future implementations, it may be worthwhile to include additional measurement and processing constraints in the LISP, such as phaseless measurements and few-bit processing [76], to lower the hardware requirements further.”
MethodMAGNet: A Modular Accelerator Generator for Neural Networks
Venkatesan, R., Shao, Y. S., Wang, M., Clemons, J., Dai, S., Fojtik, M., Keller, B., Klinefelter, A., Pinckney, N., Raina, P., et al. (2019) · 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) · doi:10.1109/iccad45719.2019.8942127 · Cited by 138
“In addition to previously published dataflows, i.e, weight stationary (WS), output stationary (OS), and input stationary (IS) [18], MAGNet features new multi-level weight-output stationary dataflows: Weight Stationary - Local Output Station-ary (WS-LOS) and Output Stationary - Local Weight Stationary…”
BackgroundSelf-citationHardware Approximate Techniques for Deep Neural Network Accelerators: A Survey
Armeniakos, G., Zervakis, G., Soudris, D., & Henkel, J. (2022) · ACM Computing Surveys · doi:10.1145/3527156 · Cited by 204
“Though, the latter assumes that the accumulators of the DNN accelerator have enough precision to avoid any overlow and accurately accumulate the partial sums [18, 40, 58, 116].”
BackgroundInfluentialMemristive and CMOS Devices for Neuromorphic Computing
Milo, V., Malavena, G., Compagnoni, C. M., & Ielmini, D. (2020) · Materials · doi:10.3390/ma13010166 · Cited by 158
“The field of neuromorphic networks includes both the DNN [36], and SNN, the latter more directly inspired by the human brain [37].”
BackgroundMethodInfluentialA Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platforms
Silvano, C., Ielmini, D., Ferrandi, F., Fiorin, L., Curzel, S., Benini, L., Conti, F., Garofalo, A., Zambelli, C., Calore, E., et al. (2025) · ACM Computing Surveys · doi:10.1145/3729215 · Cited by 262
“These platforms can be developed using CPUs, GPUs, FPGAs, and ASICs [47, 51, 73, 127, 131, 193, 213].”
Yang, Z., Zhang, W., & Feng, J. (2021) · Safety Science · doi:10.1016/j.ssci.2021.105522 · Cited by 163
Efficient Hardware Architectures for Accelerating Deep Neural Networks: Survey
Pudi, D., Boppu, S., Manikandan, M. S., & Cenkeramaddi, L. R. (2022) · IEEE Access · doi:10.1109/access.2022.3229767 · Cited by 180
Artificial intelligence explainability: the technical and ethical dimensions
McDermid, J. A., Jia, Y., Porter, Z., & Habli, I. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0363 · Cited by 136
“NNs have sub-classes, e.g. convolutional neural networks (CNNs) [7] and deep neural networks (DNN) [8].”
BackgroundInfluentialData-Free Network Quantization With Adversarial Knowledge Distillation
Choi, Y., Choi, J., El-Khamy, M., & Lee, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00363 · Cited by 135
“The benefit of over-parameterization is empirically shown to be the key factor of the great success of deep learning, but once we find a well-trained high-accuracy model, its deployment on various inference platforms faces different requirements and challenges [3, 4].”
Recurrent Neural Networks: An Embedded Computing Perspective
Rezk, N. M., Purnaprajna, M., Nordström, T., & Ul-Abdin, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2982416 · Cited by 168
“For instance, a survey on neural networks efficient processing [102] studied CNNs, CNN optimizations, and CNN implementations and another CNN survey [106] studied CNN mappings on FPGAs.”
MethodMa, X., Chen, Z., Chen, W., Li, Z., Chi, Y., Han, C., & Li, S. (2020) · IEEE Access · doi:10.1109/access.2020.2994100 · Cited by 134
Toward Intelligent Cooperation of UAV Swarms: When Machine Learning Meets Digital Twin
Lei, L., Shen, G., Zhang, L., & Li, Z. (2020) · IEEE Network · doi:10.1109/mnet.011.2000388 · Cited by 148
Low-Power, Adaptive Neuromorphic Systems: Recent Progress and Future Directions
Basu, A., Acharya, J., Karnik, T., Liu, H., Li, H. H., Seo, J.-S., & Song, C. (2018) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2018.2816339 · Cited by 135
“Recent surveys have either focused solely on deep learning hardware for inference [14], CMOS adaptive synapses [15] or on some specific aspects of neuromorphic systems [16], [17].”
BackgroundAhmed, F., Xiang, X., Jiang, C., Xiang, G., & Yang, S. (2022) · Ocean Engineering · doi:10.1016/j.oceaneng.2022.113300 · Cited by 137
A survey on optimization techniques for edge artificial intelligence (ai)
Surianarayanan, C., Lawrence, J. J., Chelliah, P. R., Prakash, E., & Hewage, C. (2023) · Sensors · doi:10.3390/s23031279 · Cited by 131
“So, the system has to access off-chip DRAM access which has latency and energy requirements higher than other computer operations [78].”
BackgroundRobust Machine Learning Systems: Challenges,Current Trends, Perspectives, and the Road Ahead
Shafique, M., Naseer, M., Theocharides, T., Kyrkou, C., Mutlu, O., Orosa, L., & Choi, J. (2020) · IEEE Design and Test · doi:10.1109/mdat.2020.2971217 · Cited by 156
“A detailed overview of neural networks can be found in [16][17].”
BackgroundBig Data Analytics for Large-scale Wireless Networks: Challenges and Opportunities
Dai, H.-N., Wong, R. C.-W., Wang, H., Zheng, Z., & Vasilakos, A. V. (2020) · ACM Computing Surveys · doi:10.1145/3337065 · Cited by 124
“Recently, deep learning architectures have received extensive attention [190].”
BackgroundApplying machine learning approach in recycling
Özdemir, M. E., Ali, Z., Subeshan, B., & Asmatulu, E. (2021) · Journal of Material Cycles and Waste Management · doi:10.1007/s10163-021-01182-y · Cited by 148
Li, K., Duan, H., Liu, L., Qiu, R., van den Akker, B., Ni, B., Chen, T., Yin, H., Yuan, Z., & Ye, L. (2022) · Environmental Science & Technology · doi:10.1021/acs.est.1c05020 · Cited by 138
Optical Computing: Status and Perspectives
Kazanskiy, N. L., Butt, M. A., & Khonina, S. N. (2022) · Nanomaterials · doi:10.3390/nano12132171 · Cited by 165
Magnetic Domain Wall Based Synaptic and Activation Function Generator for Neuromorphic Accelerators
Siddiqui, S. A., Dutta, S., Tang, A. S., Liu, L., Ross, C. A., & Baldo, M. A. (2019) · Nano Letters · doi:10.1021/acs.nanolett.9b04200 · Cited by 144
A bird's-eye view of deep learning in bioimage analysis
Meijering, E. (2020) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2020.08.003 · Cited by 126
“Codesigning architectural, algorithmic, software, and hardware solutions to allow multi-machine parallelism and scalable distributed deep learning for this purpose is an ongoing engineering challenge [261,262,304].”
BackgroundDeep Learning-Constructed Joint Transmission-Recognition for Internet of Things
Lee, C.-H., Lin, J.-W., Chen, P.-H., & Chang, Y.-C. (2019) · IEEE Access · doi:10.1109/access.2019.2920929 · Cited by 126
“The DNN-constructed approach also offers other advantages such as low computational complexity [53], meeting the computing requirement of IoT devices.”
BackgroundLogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications
Umuroglu, Y., Akhauri, Y., Fraser, N. J., & Blott, M. (2020) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl50879.2020.00055 · Cited by 189
“A variety of hardware platform options exist for accelerating DNN inference, including GPGPUs, FPGAs and specialized tensor processors [4], [5].”
BackgroundAdvancements in TinyML: Applications, Limitations, and Impact on IoT Devices
Elhanashi, A., Dini, P., Saponara, S., & Zheng, Q. (2024) · Electronics · doi:10.3390/electronics13173562 · Cited by 190
Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing
Sarwar, S. S., Ankit, A., & Roy, K. (2020) · IEEE Access · doi:10.1109/access.2019.2963056 · Cited by 120
“Note that we do not consider the energy expended in off-chip data movement (movement of weight from DRAM to local memory in cores) and inter-core data movement (over network) as these can vary based on the layer configurations, chip size, network design and several optimizations obtained from the software layers [33].”
BackgroundHuynh, A. T., Nguyen, Q. D., Lieu, Q. X., Magee, B. J., Chung, T., Tran, K. T., & Nguyen, K. T. (2020) · Applied Sciences · doi:10.3390/app10217726 · Cited by 131
“With powerful representational ability, DNN is able to achieve high accuracy in various tasks [31].”
BackgroundMethodJeong, K., Abbas, A., Shin, J., Son, M., Kim, Y. M., & Cho, K. H. (2021) · Water Research · doi:10.1016/j.watres.2021.117697 · Cited by 139
Deep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04311 · Cited by 122
“Hardware: There exist several efforts that attempt to shift deep learning computing from the cloud side to mobile devices [154].”
BackgroundOsarogiagbon, A. U., Khan, F. I., Venkatesan, R., & Gillard, P. (2020) · Process Safety and Environmental Protection · doi:10.1016/j.psep.2020.09.038 · Cited by 131
Ahmed, A. N., Lam, T. V., Hùng, N. D., Thieu, N. V., Kişi, Ö., & El‐Shafie, A. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107282 · Cited by 136
Okewu, E., Adewole, P., Misra, S., Maskeliūnas, R., & Damaševičius, R. (2021) · Applied Artificial Intelligence · doi:10.1080/08839514.2021.1922847 · Cited by 159
“Total = 6 (26.1%); Journals = 3; Book Chapters = 1; Conference = 2 (Lu, Plataniotis, and Venetsanopoulos), (Yu and Deng 2010), (Nguyen, Yosinski, and Clune 2014), (Hinton et al. 2012), (Bengio 2012), (Sze et al. 2017), (Hinton 2010), (Fischer and Igel 2014)”
InfluentialTemporally-focused analytics of self-regulated learning: A systematic review of literature
Saint, J., Fan, Y., Gašević, D., & Pardo, A. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100060 · Cited by 82
Artificial Intelligence and Advanced Materials
López, C. (2023) · Advanced Materials · doi:10.1002/adma.202208683 · Cited by 144
DeepShift: Towards Multiplication-Less Neural Networks
Elhoushi, M., Chen, Z., Shafiq, F., Tian, Y. H., & Li, J. Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00268 · Cited by 117
“This reduces the memory footprint of the model and increases energy savings for transferring weights between different layers of memory of a GPU or CPU ([58]).”
BackgroundScalable and Programmable Neural Network Inference Accelerator Based on In-Memory Computing
Jia, H., Ozatay, M., Tang, Y., Valavi, H., Pathak, R., Lee, J., & Verma, N. (2021) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2021.3119018 · Cited by 135
“In terms of accelerators, a key focus has been on matrix-vector multiplies (MVMs), which dominate NN inference computations, with data movement/accessing being a critical concern due to the high-dimensionality MVMs typically involved [22].”
BackgroundMachine Learning for Human Learners: Opportunities, Issues, Tensions and Threats
Webb, M. E., Fluck, A., Magenheim, J., Malyn-Smith, J., Waters, J., Deschênes, M., & Zagami, J. (2021) · Educational Technology Research and Development · doi:10.1007/s11423-020-09858-2 · Cited by 116
“Deep learning is now widely used in speech recognition and computer vision and is expected to make a large contribution in many other fields in the near future (Sze et al. 2017).”
Background
Metadata
- Authors
- Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S.
- Affiliations
- Massachusetts Institute of Technology (US)
- Nvidia (United States) (US)
- Venue
- Proceedings of the IEEE
- Published
- 2017-12
- Volume
- 105
- Issue
- 12
- Pages
- 2295-2329
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W2604319603
- Cited by
- 4,215
- References
- 157
- Citations by year
- 2026: 504 · 2025: 523 · 2024: 474 · 2023: 466 · 2022: 508 · 2021: 568 · 2020: 551 · 2019: 392 · 2018: 199 · 2017: 28
- FWCI
- 132.75
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the IEEE (publishedVersion)
- Funders
- DARPA YFA
- MIT CICS
- gift from Nvidia
- gift from Intel
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- deep neural network, energy efficiency, throughput, hardware accelerators, computation cost reduction, benchmarking metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.