Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
The DOI matches its Crossref record; 317 of the 334 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Domain-specific hardware is becoming a promising topic in the backdrop of improvement slow down for general-purpose processors due to the foreseeable end of Moore's Law. Machine learning, especially deep neural networks (DNNs), has become the most dazzling domain witnessing successful applications in a wide spectrum of artificial intelligence (AI) tasks. The incomparable accuracy of DNNs is achieved by paying the cost of hungry memory consumption and high computational complexity, which greatly impedes their deployment in embedded systems. Therefore, the DNN compression concept was naturally proposed and widely used for memory saving and compute acceleration. In the past few years, a tremendous number of compression techniques have sprung up to pursue a satisfactory tradeoff between processing efficiency and application accuracy. Recently, this wave has spread to the design of neural network accelerators for gaining extremely high performance. However, the amount of related works is incredibly huge and the reported approaches are quite divergent. This research chaos motivates us to provide a comprehensive survey on the recent advances toward the goal of efficient compression and execution of DNNs without significantly compromising accuracy, involving both the high-level algorithms and their applications in hardware design. In this article, we review the mainstream compression approaches such as compact model, tensor decomposition, data quantization, and network sparsification…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“but there is no review discussing all three categories which in future will be integrated seamlessly to design effective and efficient DNNs in the new edge intelligence era.”
From Bringing ai to edge: from deep learning’s perspective (2022) · cites this paper as [28]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 3×
“Specifically, an example of the identity connection used in ResNet [30] [see Fig.”
BackgroundMethodVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“In VGG-Nets [27], the interlayer topology is rather simple, where multiple Conv layers are stacked head-to-tail in sequence.”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 3×
“1078 [20] A.”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“…we discuss recent advances in RNNs that aim to improve the execution efficiency by designing compact RNNs in the level of unit or network. a) Unit level: LSTM is designed to improve the long-term dependence over vanilla RNNs by alleviating the issue of gradient explosion or vanishing [18], [46].”
BackgroundMethodLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
(2014) · Cited by 33,419
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · cited 35×
“The feature propagation gradually improves the abstraction level from local to global, which finally enables the network decision such as image recognition. a) Spatial correlation to enlarge RF: Compared to MLP, the key feature of CNNs is the local connections with shared weights, which results in high spatial correlat…”
BackgroundMethodInfluential
Research works printed but not linked: 2
- lin, 2013, Network in network, arXiv 1312 4400
- han, 2015, Learning both weights and connections for efficient neural network, Proc Adv Neural Inf Process Syst, 1135
Cited by
Items 201–300 of 1,002
Unsupervised Deep-Learning for Distributed Clock Synchronization in Wireless Networks
Abakasanga, E., Shlezinger, N., & Dabora, R. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3269381 · Cited by 26
“5 · 10 3 products on inference – a computational burden which is feasible on real-time modern micro-controllers, and can support parallelization implemented by dedicated DNN hardware accelerators [40].”
BackgroundSystematic Literature Review on Cost-Efficient Deep Learning
Klemetti, A., Raatikainen, M., Myllyaho, L., Mikkonen, T., & Nurminen, J. K. (2023) · IEEE Access · doi:10.1109/access.2023.3275431 · Cited by 27
“and software co-design [22], [23], [24], [25], [26], [27], [28].”
A Comprehensive Survey of Compression Algorithms for Language Models
Park, S., Choi, J., Lee, S., & Kang, U. (2026) · ACM Computing Surveys · doi:10.1145/3819816 · Cited by 32
“…) [41, 84, 107, 126], low-rank approximation ( LRA ) [50, 101, 135, 164], and dynamic inference [94, 100, 119] have been proposed, and they significantly improve the efficiency of language models; however, an excessive number of research papers make it hard to figure out the overall trend and…”
Bhalgaonkar, S., Munot, M. V., & Anuse, A. D. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109180 · Cited by 29
ChatIoT: Zero-code Generation of Trigger-action Based IoT Programs
Gao, Y. C., Xiao, K., Li, F., Xu, W., Huang, J., & Dong, W. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3678585 · Cited by 32
“Traditionally, techniques such as model quantization, model pruning, and knowledge distillation have been extensively employed to compress models, thereby reducing their computational demands and memory foot-print [16, 18, 21].”
EdgeCI: Distributed Workload Assignment and Model Partitioning for CNN Inference on Edge Clusters
Chen, Y., Luo, T., Fang, W. W., & Xiong, N. N. (2024) · ACM Transactions on Internet Technology · doi:10.1145/3656041 · Cited by 27
“To alleviate discrepancy between the intensive resource requirements of CNNs and the limited computing powers of IoT devices, many researchers have explored a lot of optimization technologies, including model compression [12, 13], dynamic inference [14, 15], and edge-cloud synergy [16–18].”
BackgroundLLMs and IoT: A Comprehensive Survey on Large Language Models and the Internet of Things
Sarhaddi, F., Nguyen, N. T., Zuniga, A., Khan, M., Hui, P., Tarkoma, S., Flores, H., & Nurmi, P. (2026) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2026.3680064 · Cited by 33
“Key methods include pruning, quantization, and knowledge distillation [82], [83].”
Lastilla, L., Ammirati, S., Firmani, D., Komodakis, N., Merialdo, P., & Scardapane, S. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.102875 · Cited by 29
AdaSTE: An Adaptive Straight-Through Estimator to Train Binary Neural Networks
Le, H., Høier, R. K., Lin, C.-T., & Zach, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00055 · Cited by 30
“For additional surveys of weight quantization we refer to the review papers [16, 32] as well as section III of [11].”
MethodOptical processor for a binarized neural network
Huang, L., & Yao, J. (2022) · Optics Letters · doi:10.1364/ol.464214 · Cited by 24
DiSparse: Disentangled Sparsification for Multitask Model Compression
Sun, X., Hassani, A., Wang, Z., Huang, G. M., & Shi, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01206 · Cited by 34
“Therefore, effectively compressing deep convolutional networks for efficient storage and computation has been a very active research area, and var-30 ious approaches have been proposed and developed [6, 9] over the years.”
BackgroundEo, M., Kang, S., & Rhee, W. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.01.024 · Cited by 28
Liang, J., Zhang, L., Han, C., Bu, C., Wu, H., & Song, A. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3271306 · Cited by 26
“Different from previous compression methods [26], [32], instead of sequentially performing one compression operation at each time in an isolated manner, we adopt a new collaborative compression scheme [34] to accelerate online activity inference.”
BackgroundMethodMitigating carbon footprint for knowledge distillation based deep learning model compression
Rafat, K., Islam, S., Mahfug, A. A., Hossain, M. I., Rahman, F., Momen, S., Rahman, S., & Mohammed, N. (2023) · PLoS ONE · doi:10.1371/journal.pone.0285668 · Cited by 31
“From the literature, larger and lighter models are called the teacher and student models, respectively [25].”
BackgroundLoss-aware automatic selection of structured pruning criteria for deep neural network acceleration
Ghimire, D., Lee, K., & Kim, S. (2023) · Image and Vision Computing · doi:10.1016/j.imavis.2023.104745 · Cited by 24
“Other commonly used solutions include, neural architecture search [16], hardware optimization [17], and hardware-software co-design [18, 19].”
Jeon, Y.-J., Hong, M. J., Ko, C. S., Park, S. J., Lee, H., Lee, W. S., & Jung, D.-H. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.109882 · Cited by 24
XOR-CIM: compute-in-memory SRAM architecture with embedded XOR encryption
Huang, S., Jiang, H., Peng, X., Li, W., & Yu, S. (2020) · doi:10.1145/3400302.3415678 · Cited by 42
“The inference engines are embracing the application-specific-integrated-circuit (ASIC) for hardware acceleration [1].”
BackgroundEnvironmental sensing in autonomous construction robots: Applicable technologies and systems
Okonkwo, C., & Awolusi, I. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106075 · Cited by 29
Tian, X., Zheng, Q., Li, B., Qiao, D., Yu, K., Wei, Z., Li, B., Jiang, H., Li, X., Lin, Y., et al. (2025) · Signal Processing · doi:10.1016/j.sigpro.2025.110444 · Cited by 28
NeuroSim Validation with 40nm RRAM Compute-in-Memory Macro
Lu, A., Peng, X., Li, W., Jiang, H., & Yu, S. (2021) · IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS) · doi:10.1109/aicas51828.2021.9458501 · Cited by 23
“State-of-the-art deep neural network (DNN) based machine learning algorithms have demonstrated remarkable effectiveness for various artificial intelligence applications such as image processing, speech recognition and language translation, etc. [1].”
BackgroundHeSA: Heterogeneous Systolic Array Architecture for Compact CNNs Hardware Accelerators
Xu, R., Ma, S., Wang, Y., & Guo, Y. (2021) · Design, Automation, and Test in Europe · doi:10.23919/date51398.2021.9474145 · Cited by 23
Weak self-supervised learning for seizure forecasting: a feasibility study
Yang, Y., Truong, N. D., Eshraghian, J. K., Nikpour, A., & Kavehei, O. (2022) · Royal Society Open Science · doi:10.1098/rsos.220374 · Cited by 26
Multi-teacher knowledge distillation based on joint Guidance of Probe and Adaptive Corrector
Shang, R., Li, W., Zhu, S., Jiao, L., & Li, Y. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.04.015 · Cited by 25
Zhang, Y., Yang, G., Gao, A., Lv, W., Xie, R., Huang, M., & Liu, S. (2023) · International Journal of Remote Sensing · doi:10.1080/01431161.2023.2214274 · Cited by 24
“These two methods have become the general method to adjust the lightweight deep network (Deng et al. 2020).”
MethodGul, W., Shams, M., & Al-Khalili, D. (2023) · Micromachines · doi:10.3390/mi14081535 · Cited by 24
“In the second case [56], BLs are set to the input values.”
BackgroundEdge AI: Reshaping the Future of Edge Computing with Artificial Intelligence
Banjanović-Mehmedović, L., & Husaković, A. (2023) · doi:10.5644/pi2023.209.07 · Cited by 23
Sparse optimization guided pruning for neural networks
Shi, Y., Tang, A., Niu, L., & Zhou, R. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127280 · Cited by 26
Structured Model Pruning of Convolutional Networks on Tensor Processing Units
Chen, K., Franko, K., & Sang, R. (2021) · arXiv · doi:10.48550/arxiv.2107.04191 · Cited by 22
Camel: Managing Data for Efficient Stream Learning
Li, Y., Shen, Y., & Chen, L. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517836 · Cited by 38
“Quantization techniques are widely used for model and gradient compression [13, 17, 25, 34], but seldom explored in the context of training data compression.”
MethodLightweight Deep Learning Model for Radar-Based Fall Detection With Metric Learning
Ou, Z., & Ye, W. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3229462 · Cited by 26
“Oneofthe main ways for optimizing CNNs architectures is to design computation-efficient models with separable convo-lutions [16].”
Horizontally Distributed Inference of Deep Neural Networks for AI-Enabled IoT
Rodríguez, I., Campos, C., & Fdez‐Riverola, F. (2023) · Sensors · doi:10.3390/s23041911 · Cited by 28
“…but also how to conceive more compact and lightweight DNNs, which is in turn pursued through the compression and simplification of existing networks, e.g., quantization [44], sparsification [45,46], and pruning [44], as well as the creation from the ground up of more efficient architectures [47,48].”
Chen, C., Jiang, B., Liu, S., Li, C., Wu, C., & Yin, R. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3318080 · Cited by 26
“…reached an upper bound, emerging compression algorithms, such as compact model, tensor decomposition, data quantization, and network sparsification, have demonstrated great potential in reducing the size and complexity of deep neural networks without significantly compromising their performance [17].”
MethodHFNet: A CNN Architecture Co-designed for Neuromorphic Hardware With a Crossbar Array of Synapses
Gopalakrishnan, R., Chua, Y., Sun, P., Kumar, A. J. S., & Basu, A. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00907 · Cited by 22
“Considering such a neuromorphic chip, there are several hardware constraints: firstly, at the single device, we may have low bit precision of synaptic weights and output activations (Ji et al., 2018; Deng et al., 2020), synaptic noise and variability (Ambrogio et al.”
BackgroundAnchor pruning for object detection
Bonnaerens, M., Freiberger, M., & Dambre, J. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103445 · Cited by 26
Efficient Visual Recognition: A Survey on Recent Advances and Brain-inspired Methodologies
Wu, Y., Wang, D., Lu, X., Yang, F., Yao, M., Dong, W., Shi, J., & Li, G. (2022) · Machine Intelligence Research · doi:10.1007/s11633-022-1340-5 · Cited by 25
“Deng et al. (2020) [11], one of the latest surveys, is comprehensive and professional in both DNN compression and hardware design.”
BackgroundSelf-citationDual-attention deep learning model for real-time advance rate prediction in TBM operations
Ji, A., Xie, Y., Fan, H., Xue, X., & Zhang, M. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106210 · Cited by 23
Self-citationMultimodal fusion based few-shot network intrusion detection system
Xu, C., Zhan, Y., Wang, Z., & Yang, J. (2025) · Scientific Reports · doi:10.1038/s41598-025-05217-4 · Cited by 27
“Simultaneously, we plan to leverage the latest hardware acceleration 54 technologies and optimized distributed computing strategies to enhance our capability to process large-scale data.”
Sotoudeh, M., & Thakur, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-65474-0_4 · Cited by 21
BMPQ: Bit-Gradient Sensitivity-Driven Mixed-Precision Quantization of DNNs from Scratch
Kundu, S., Wang, S., Sun, Q., Beerel, P. A., & Pedram, M. (2022) · Design, Automation, and Test in Europe · doi:10.23919/date54114.2022.9774740 · Cited by 28
Multi‐objective evolutionary optimization for hardware‐aware neural network pruning
Hong, W., Li, G., Liu, S., Yang, P., & Tang, K. (2022) · Fundamental Research · doi:10.1016/j.fmre.2022.07.013 · Cited by 24
“…pruning methods have been proposed based on various proxy metrics of network complexity, such as the number of Multiply-Accumulate operations (MACs) [4] , the number of FLoating-point Operations Per Second (FLOPS) [6–10] , the number of connection weights [11–13] , the number of blocks [14] , etc.”
MethodSelf-citationLi, T., Ma, Y., & Endoh, T. (2020) · IEEE Access · doi:10.1109/access.2020.3013934 · Cited by 23
Lu, Q., & Murmann, B. (2023) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3591466 · Cited by 25
“There exists a considerable body of literature on reducing the footprint of CNNs using compression techniques like weight and channel pruning, knowledge distillation as well as quantization [10, 13].”
BackgroundMethodEmerging Paradigms of Neural Network PruningPreprint
Wang, H., Qin, C., Zhang, Y., & Fu, Y. (2021) · arXiv (Cornell University) · Cited by 23
Physics informed neural network based multi-zone electric water heater modeling for demand response
Pandiyan, S. V., Gros, S., & Rajasekharan, J. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.125037 · Cited by 20
Xu, X., & Feng-hu, L. (2021) · Computational Intelligence and Neuroscience · doi:10.1155/2021/8785127 · Cited by 20
RetractedMultimodal Neural Network Acceleration on a Hybrid CPU-FPGA Architecture: A Case Study
Ajili, M. T., & Hara–Azumi, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3144977 · Cited by 24
“The achieved accuracy in our implementation had some accuracy loss but is still acceptable, considering other methods in [1], [28].”
BackgroundMethodMende, H., Peters, A., Ibrahim, F., & Schmitt, R. H. (2022) · Procedia CIRP · doi:10.1016/j.procir.2022.05.254 · Cited by 20
An Efficient Lightweight Event Detection Algorithm for On-Site Non-Intrusive Load Monitoring
Kotsilitis, S., Kalligeros, E., Marcoulaki, E., & Karybali, I. G. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3232169 · Cited by 20
“Furthermore, the implementation of deep learning approaches on IoT edge devices with low resources is a complex task [18].”
BackgroundAn Intelligent Edge Diagnosis System Based on Multiplication–Convolution Sparse Network
Wu, Q., Ding, X., Zhang, Q., Liu, R., Siegfried Zhiqiang, W. U., & He, Q. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3304301 · Cited by 20
“…processing units (GPU) with high processing parallelism and memory bandwidth have been the main platform for training and reasoning DNN [32], [33]; however, some DNNs are capable of reaching 10 000 layers and beyond with millions to billions of parameters [34], which makes them difficult to…”
BackgroundOnline Optimization of DNN Inference Network Utility in Collaborative Edge Computing
Li, R., Ouyang, T., Zeng, L., Liao, G., Zhou, Z., & Chen, X. (2024) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2024.3421356 · Cited by 23
“Specially, this type of computing task is allowed to be served by a set of heterogeneous DNN models featuring diverse performance and resource requirements (different model architectures: CNN [15] or Transformer [16], multiple downsized version of the same pre-trained model [17]).”
BackgroundDesigning on-board explainable passenger flow prediction
Barbareschi, M., Emmanuele, A., Mazzocca, N., & di Torrepadula, F. R. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109648 · Cited by 21
Learning on Hardware: A Tutorial on Neural Network Accelerators and Co-ProcessorsPreprint
Baischer, L., Wess, M., & TaheriNejad, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09252 · Cited by 22
“For this reason, tensor decomposition methods are well suited to compress and accelerate neural network models [13].”
BackgroundMethodInfluentialModeling and Optimization of SRAM-based In-Memory Computing Hardware Design
Saikia, J., Yin, S., Cherupally, S. K., Zhang, B., Meng, J., Seok, M., & Seo, J.-S. (2021) · Design, Automation, and Test in Europe · doi:10.23919/date51398.2021.9473973 · Cited by 21
“…DNN algorithms continue to achieve higher accuracy with a smaller number of operations and parameters, the most compact DNN model for ImageNet dataset still requires > 0.5 billion multiply-and-accumulate (MAC) operations and > 2 million weights [1] for the inference of a single image.”
InfluentialJeniffer, J. T., & Chandrasekar, A. (2022) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-021-01263-7 · Cited by 19
“Furthermore, since an adaptive kernel's total fan-in is greater than its individual fan-in, it is unclear what value should be used to calculate weight variance, as suggested by prominent initialization schemes [36, 37].”
BackgroundMachine Learning for Multimedia Communications
Thomos, N., Maugey, T., & Toni, L. (2022) · Sensors · doi:10.3390/s22030819 · Cited by 20
“Solutions have been investigated to compress the weights of neural networks architecture, as reviewed in [202].”
MethodA Survey of Intelligent Chip Design Research Based on Spiking Neural Networks
Chen, L., Xiong, X., & Liu, J. (2022) · IEEE Access · doi:10.1109/access.2022.3200454 · Cited by 21
“[118].”
Neural Adaptive Loop Filtering for Video Coding: Exploring Multi-Hypothesis Sample Refinement
Ding, D., Wang, J., Zhen, G., Mukherjee, D., Joshi, U. B., & Ma, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3260266 · Cited by 19
Synthetic data generation method for data-free knowledge distillation in regression neural networks
Zhou, T., & Chiam, K. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120327 · Cited by 28
Zheng, X., Qian, S., Wei, S., Zhou, S., & Hou, Y. M. (2023) · Applied Sciences · doi:10.3390/app13169211 · Cited by 21
“In deep convolution, a single filter is applied to the apiece input channel, which means that spatial convolution is performed independently on each channel of the input feature map (ifmap) [42].”
MethodA Masked Hardware Accelerator for Feed-Forward Neural Networks With Fixed-Point Arithmetic
Brosch, M., Probst, M., Glaser, M., & Sigl, G. (2023) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2023.3340553 · Cited by 22
“To execute NNs, optimization methods are used to reduce the complexity and number of operations on the edge [1].”
MethodBackpropagation through space, time and the brain
Ellenberger, B., Haider, P., Benitez, F., Jordan, J., Max, K., Jaras, I., Kriener, L., & Petrovici, M. A. (2025) · Nature Communications · doi:10.1038/s41467-025-66666-z · Cited by 26
“The instantaneous target error for a mean-squared error (MSE) or cross-entropy (CE) loss in the last layer is given by”
IVQ: In-Memory Acceleration of DNN Inference Exploiting Varied Quantization
Liu, F., Zhao, W., Wang, Z., Zhao, Y., Yang, T., Chen, Y., & Jiang, L. (2022) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2022.3156017 · Cited by 21
“Since the weights are quantized into different bit widths, most of them actually have small values [12], [24], [54], making shorter bit widths can represent them.”
BackgroundInfluentialEdge ai: leveraging the full potential of deep learning
Shuvo, M. M. H. (2022) · Studies in Computational Intelligence · doi:10.1007/978-3-031-14748-7_2 · Cited by 18
Khan, Z., Yoon, S.-C., & Bhandarkar, S. M. (2025) · Sensors · doi:10.3390/s25030970 · Cited by 20
“Two primary model compression methods are pruning and quantization [22,23].”
Lossy Neural Compression for Geospatial Analytics: A review
Gomes, C. A. F., Wittmann, I., Robert, D., Jakubik, J., Reichelt, T., Maurogiovanni, S., Vinge, R., Hurst, J., Scheurer, E., Sedona, R., et al. (2025) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2025.3546527 · Cited by 23
“There exists an entire field of model compression [36] that revolves around pruning the size of artificial neural networks.”
Enabling and Leveraging AI in the Intelligent Edge: A Review of Current Trends and Future Directions
Goethals, T., Volckaert, B., & Turck, F. D. (2021) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2021.3116437 · Cited by 23
The Case for Hierarchical Deep Learning Inference at the Network Edge
Al-Atat, G., Fresa, A., Behera, A. P., Moothedath, V. N., Gross, J., & Champati, J. P. (2023) · doi:10.1145/3597062.3597278 · Cited by 23
Efficient Processing of MLPerf Mobile Workloads Using Digital Compute-In-Memory Macros
Sun, X., Cao, W.-D., Crafton, B., Akarvardar, K., Mori, H., Fujiwara, H., Noguchi, H., Chih, Y., Chang, M.-F., Wang, Y., et al. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2023.3333290 · Cited by 15
“While 8-bit weights and activations have been assumed throughout the evaluations, it is worth noting that the weight precision and activation precision may not be always equal in practice [42], which would lead to differences in mapping between the dataflows.”
BackgroundTowards Better Structured Pruning Saliency by Reorganizing Convolution
Sun, X., & Shi, H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00220 · Cited by 31
“In general, neural network compression techniques can be categorized [7] into pruning [2,16,17,32,36,42], quantization [5,44,61], low-rank factorization [8,38,64] or knowledge distillation [23, 28, 41].”
BackgroundResource-aware Deployment of Dynamic DNNs over Multi-tiered Interconnected Systems
Singhal, C., Wu, Y., Malandrino, F., Levorato, M., & Chiasserini, C. F. (2024) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom52122.2024.10621218 · Cited by 20
“Two main technical strategies attempt to address this issue: model compression [6] and edge computing [7].”
BackgroundTransformer fault diagnosis method based on the three-stage lightweight residual neural network
Liu, H., Niu, B., Liu, Z., Li, M., & Shi, Z. (2024) · Electric Power Systems Research · doi:10.1016/j.epsr.2024.111142 · Cited by 21
Novel Hybrid UNet++ and LSTM Model for Enhanced Attack Detection and Classification in IoMT Traffic
Mezina, A., Nurmi, J., & Ometov, A. (2025) · IEEE Access · doi:10.1109/access.2025.3553966 · Cited by 18
Palarimath, S., Blessing, N. R. W., Sujatha, T., Pyingkodi, M., Ugalde, B. H., & Palarimath, R. D. (2022) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-16-7610-9_77 · Cited by 17
Lee, Y. J., Jung, H. G., & Suhr, J. K. (2023) · Electronics · doi:10.3390/electronics12234795 · Cited by 17
Multi-Modal Data and Model Reduction for Enabling Edge Fusion in Connected Vehicle Environments
Thornton, S., & Dey, S. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3378182 · Cited by 20
“Many of these improvements fall under the umbrella of model compression [58] [59].”
BackgroundCARLA: A Convolution Accelerator With a Reconfigurable and Low-Energy Architecture
Ahmadi, M., Vakili, S., & Langlois, J. M. P. (2021) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2021.3066967 · Cited by 20
Griffin: Rethinking Sparse Optimization for Deep Learning Architectures
Shin, J. H., Shafiee, A. R., Pedram, A., Abdel-Aziz, H. A., Li, L., & Hassoun, J. (2022) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca53966.2022.00068 · Cited by 24
“Recently, several optimization techniques are proposed to improve compute and memory efficiency for DNNs [14], [39], [63].”
ECQ$$^{\text {x}}$$: Explainability-Driven Quantization for Low-Bit and Sparse DNNs
Becking, D., Dreyer, M., Samek, W., Müller, K., & Lapuschkin, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-04083-2_14 · Cited by 27
“In order to tailor deep learning to resource-constrained hardware, a large research community has emerged in recent years [11, 43].”
BackgroundEr, M. J., Jie, C., Zhang, Y., & Gao, W. (2023) · doi:10.36227/techrxiv.19350389 · Cited by 17
Er, M. J., Chen, J., Zhang, Y., & Gao, W. (2023) · doi:10.36227/techrxiv.19350389.v4 · Cited by 19
A review of state-of-the-art techniques for large language model compression
Dantas, P. V., Cordeiro, L. C., & Waldir, S. S. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-02019-z · Cited by 32
“2 × ( U ) [ 33 ] 0 .”
InfluentialProspects and challenges of electrochemical random-access memory for deep-learning accelerators
Cui, J., Liu, H., & Cao, Q. (2024) · Current Opinion in Solid State and Materials Science · doi:10.1016/j.cossms.2024.101187 · Cited by 17
“1) [7].”
A Refined and Efficient CNN Algorithm for Remote Sensing Object Detection
Liu, B., Mo, P., Wang, S., Cui, Y., & Wu, Z. (2024) · Sensors · doi:10.3390/s24227166 · Cited by 17
Zhang, Q., Xin, C., & Wu, H. (2020) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2020.3040704 · Cited by 16
AILC: Accelerate On-Chip Incremental Learning With Compute-in-Memory Technology
Luo, Y., & Yu, S. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3053199 · Cited by 16
Compressing Deep Neural Networks for Efficient Speech Enhancement
Tan, K., & Wang, D. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413536 · Cited by 20
Towards Inference Delivery Networks: Distributing Machine Learning with Optimality Guarantees
Salem, T. S., Castellano, G., Neglia, G., Pianese, F., & Araldo, A. (2021) · Mediterranean Communication and Computer Networking Conference (MedComNet) · doi:10.1109/medcomnet52149.2021.9501272 · Cited by 20
Learning broad learning system with controllable sparsity through L0 regularization
Chu, F., Wang, G., Wang, J., Chen, C. L. P., & Wang, X. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110068 · Cited by 16
Tuning-Free One-Bit Covariance Estimation Using Data-Driven Dithering
Dirksen, S., & Maly, J. (2024) · IEEE Transactions on Information Theory · doi:10.1109/tit.2024.3358994 · Cited by 16
“In this paper we focus on memoryless one-bit quantizers of the form Q : R p → {− 1 , 1 } p , Q ( X ) = sign( X + τ ) , where sign is the sign function acting element-wise on its input and τ ∈ R p represents a vector of quantization thresholds.”
BackgroundGetting the Best Out of Both Worlds: Algorithms for Hierarchical Inference at the Edge
Moothedath, V. N., Champati, J. P., & Gross, J. (2024) · IEEE Transactions on Machine Learning in Communications and Networking · doi:10.1109/tmlcn.2024.3366501 · Cited by 19
“Therefore, performing local inference using embedded DL models, which we refer to as S-ML (Small-ML) models, on EDs has received significant research interest in the recent past [4]–[6].”
MethodLearning Sparse Neural Networks Using Non-Convex Regularization
Pandit, M. K., Naaz, R., & Chishti, M. A. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3058672 · Cited by 16
Photon-Driven Neural Reconstruction for Path Guiding
Zhu, S., Xu, Z., Sun, T., Kuznetsov, A. A., Meyer, M., Jensen, H. W., Su, H., & Ramamoorthi, R. (2021) · ACM Transactions on Graphics · doi:10.1145/3476828 · Cited by 20
RL-Pruner: Structured Pruning Using Reinforcement Learning for CNN Compression and AccelerationPreprint
Wang, B., & Kindratenko, V. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.06463 · Cited by 24
Performance Modeling of Computer Vision-based CNN on Edge GPUs
Bouzidi, H., Ouarnoughi, H., Niar, S., & Cadi, A. A. E. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3527169 · Cited by 19
“Other model-level optimization techniques have been widely investigated such as quantization [18, 51] and pruning [38, 39, 73].”
MethodCompressing neural networks via formal methods
Ressi, D., Romanello, R., Rossi, S., & Piazza, C. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106411 · Cited by 16
Cooperative Knowledge-Distillation-Based Tiny DNN for UAV-Assisted Mobile-Edge Network
Sun, L., Liu, Z., Wan, L., Lin, Y., Lin, L., Wang, J., & Gen, M. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3409813 · Cited by 17
“This approach encompasses four main categories: 1) numerical quantization [7]; 2) model pruning [8]; 3) tensor decomposition [9]; and 4) knowledge distillation [10].”
BackgroundFully Tensorized Lightweight ConvLSTM Neural Networks for Hyperspectral Image Classification
Ma, T.-Y., Li, H.-C., Zheng, Y., Du, Q., & Plaza, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3511575 · Cited by 15
“Clearly, the substantial size of the input-to-state and state-to-state transition weight tensors in RNNs is the root cause of their elevated complexity, as indicated in [36].”
Characterization and Mitigation of Relaxation Effects on Multi-level RRAM based In-Memory Computing
He, W., Shim, W., Yin, S., Sun, X., Fan, D., Yu, S., & Seo, J.-S. (2021) · IEEE International Reliability Physics Symposium proceedings · doi:10.1109/irps46558.2021.9405228 · Cited by 15
“While state-of-the-art DNN algorithms continue to achieve higher accuracy with less number of parameters, the most compact models still require >3 million weights to achieve >70% top-1 ImageNet accuracy [1].”
BackgroundA Novel Ultra-Low Power 8T SRAM-Based Compute-in-Memory Design for Binary Neural Networks
Kim, Y., Li, S., Yadav, N., & Choi, K. K. (2021) · Electronics · doi:10.3390/electronics10172181 · Cited by 15
“After 33 cycles, 1 bit results of each cycle TH[0], TH[1], .”
BackgroundYuan, Y., Hong-yu, L., & Li-li, J. (2021) · Computational Intelligence and Neuroscience · doi:10.1155/2021/5672227 · Cited by 15
“For example, an automatic vehicle logistics system (AVLS) is recently proposed, which uses automatic vehicles (AVS) as the logistics carrier [15–17].”
MethodRetracted
Metadata
- Authors
- Deng, B. L., Li, G., Han, S., Shi, L., & Xie, Y.
- Affiliations
- Tsinghua University (CN)
- Massachusetts Institute of Technology (US)
- University of California, Santa Barbara (US)
- Venue
- Proceedings of the IEEE
- Published
- 2020-04
- Volume
- 108
- Issue
- 4
- Pages
- 485-532
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W3012561096
- Cited by
- 996
- References
- 334
- Citations by year
- 2026: 131 · 2025: 171 · 2024: 193 · 2023: 174 · 2022: 152 · 2021: 142 · 2020: 31
- FWCI
- 46.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Proceedings of the IEEE (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- National Science Foundation (1725447)
- Beijing Academy of Artificial Intelligence (BAAI), Tsinghua University Initiative Scientific Research Program
- Institute for Guo Qiang, Tsinghua University
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- neural network compression, neural network accelerators, deep neural network, data quantization, network sparsification, tensor decomposition, compact modeling, memory efficiency, embedded systems, hardware architecture
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.