Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
The DOI matches its Crossref record; 317 of the 334 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Domain-specific hardware is becoming a promising topic in the backdrop of improvement slow down for general-purpose processors due to the foreseeable end of Moore's Law. Machine learning, especially deep neural networks (DNNs), has become the most dazzling domain witnessing successful applications in a wide spectrum of artificial intelligence (AI) tasks. The incomparable accuracy of DNNs is achieved by paying the cost of hungry memory consumption and high computational complexity, which greatly impedes their deployment in embedded systems. Therefore, the DNN compression concept was naturally proposed and widely used for memory saving and compute acceleration. In the past few years, a tremendous number of compression techniques have sprung up to pursue a satisfactory tradeoff between processing efficiency and application accuracy. Recently, this wave has spread to the design of neural network accelerators for gaining extremely high performance. However, the amount of related works is incredibly huge and the reported approaches are quite divergent. This research chaos motivates us to provide a comprehensive survey on the recent advances toward the goal of efficient compression and execution of DNNs without significantly compromising accuracy, involving both the high-level algorithms and their applications in hardware design. In this article, we review the mainstream compression approaches such as compact model, tensor decomposition, data quantization, and network sparsification…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“but there is no review discussing all three categories which in future will be integrated seamlessly to design effective and efficient DNNs in the new edge intelligence era.”
From Bringing ai to edge: from deep learning’s perspective (2022) · cites this paper as [28]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 3×
“Specifically, an example of the identity connection used in ResNet [30] [see Fig.”
BackgroundMethodVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“In VGG-Nets [27], the interlayer topology is rather simple, where multiple Conv layers are stacked head-to-tail in sequence.”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 3×
“1078 [20] A.”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“…we discuss recent advances in RNNs that aim to improve the execution efficiency by designing compact RNNs in the level of unit or network. a) Unit level: LSTM is designed to improve the long-term dependence over vanilla RNNs by alleviating the issue of gradient explosion or vanishing [18], [46].”
BackgroundMethodLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
(2014) · Cited by 33,419
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · cited 35×
“The feature propagation gradually improves the abstraction level from local to global, which finally enables the network decision such as image recognition. a) Spatial correlation to enlarge RF: Compared to MLP, the key feature of CNNs is the local connections with shared weights, which results in high spatial correlat…”
BackgroundMethodInfluential
Research works printed but not linked: 2
- lin, 2013, Network in network, arXiv 1312 4400
- han, 2015, Learning both weights and connections for efficient neural network, Proc Adv Neural Inf Process Syst, 1135
Cited by
Items 801–900 of 1,002
Optimization of the Pedestrian and Vehicle Detection Model based on Cloud-Edge Collaboration
Wang, H., Chen, Y., Liu, B., Mo, R., & Lin, W. (2022) · IEEE Smart World Congress (SWC) · doi:10.1109/smartworld-uic-atc-scalcom-digitaltwin-pricomp-metaverse56740.2022.00300 · Cited by 1
“On the other hand, model compression [12] [13] has also aroused a large number of scholars’ research upsurge.”
BackgroundHardware Acceleration of PPG Waveform and Heart Rate Detection System
Mo, H., & Zhou, C. (2023) · International Conference on Electronics Technology (ICET) · doi:10.1109/icet58434.2023.10212025 · Cited by 1
“For a specific function, especially for the neural network[9, 10] or other fields with high computing demand, hardware acceleration research is often carried out to improve execution rate and reduce power consumption and area[11].”
BackgroundPruning Convolutional Filters via Reinforcement Learning with Entropy Minimization
Muşat, B., & Andonie, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42505-9_15 · Cited by 1
“Techniques like pruning and quantization [6–8, 15, 46] have recently become very popular, since they can generate power-efficient sub-versions of these overparametrized networks.”
BackgroundA Pedestrian Detection Case Study for a Traffic Light Controller
Wendt, A., Possegger, H., Bittner, M., Schnöll, D., Wess, M., Malić, D., Bischof, H., & Jantsch, A. (2023) · Springer eBooks · doi:10.1007/978-3-031-39932-9_4 · Cited by 1
Federated Boolean Neural Networks Learning
Leconte, L., Nguyen, V. M., & Moulines, É. (2023) · International Conference on Fog and Mobile Edge Computing (FMEC) · doi:10.1109/fmec59375.2023.10306211 · Cited by 1
“Quantized neural networks (QNNs) represent weights, activations, or even gradients with a small number of bits, allowing for efficient fixed-point inference and reduced memory bandwidth usage; see e.g. [5, 6, 7, 8, 9, 10].”
BackgroundEnvironment-Aware Knowledge Distillation for Improved Resource-Constrained Edge Speech Recognition
Pimentel, A., Guimarães, H. R., Avila, A. R., & Falk, T. H. (2023) · Applied Sciences · doi:10.3390/app132312571 · Cited by 3
A selective quantization approach for optimizing quantized inference engine
Liu, C., & Zhang, D. (2023) · International Conference on Information Systems and Computing Technology (ISCTech) · doi:10.1109/isctech60480.2023.00024 · Cited by 1
“The most widely used mapping method is linear projection, which assumes that the distribution of the quantized discrete space is uniform, and the step between different levels is even [15] .”
MethodHIQ: One-Shot Network Quantization for Histopathological Image Classification
Chen, X., Yan, R., Wang, Y., Li, J., Cheng, J., Guan, T., & He, Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448472 · Cited by 2
“Among them, quantization is the most commonly used method as it is lightweight and is easy to achieve hardware acceleration [7, 8], which compresses the data precision of the weights and activation tensor to raise the computational speed on edge devices.”
MethodDesign of FPGA Deep Neural Network Accelerator Based on High-Level Synthesis
Fan, H. W., Wang, H., Che, K., & Wu, Z. (2023) · International Academic Exchange Conference on Science and Technology Innovation (IAECST) · doi:10.1109/iaecst60924.2023.10502749 · Cited by 1
“On the other hand, FPGAs, with their rich computational resources, low power consumption, high energy efficiency, and high flexibility, are widely used in the development of various specialized accelerators [2] , making them suitable as hardware accelerators for aircraft platforms.”
Backgrounddi Torrepadula, F. R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60796-7_15 · Cited by 1
Ser, J. D., Martínez-Seras, A., Bilbao, M. N., Lobo, J. L., Laña, I., & Herrera, F. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651338 · Cited by 3
“Within this research area, we focus our attention on the application of evolutionary algorithms to model compression, which aims at reducing the size and complexity of ML models without incurring significant losses in generalization performance [8], [9].”
MethodEfficient Latency Optimization in Batch Processing Server Systems for Multi-User Co-Inference
Ma, Y., Gao, K., & Zhang, C. (2024) · IEEE/CIC International Conference on Communications in China (ICCC) · doi:10.1109/icccworkshops62562.2024.10693813 · Cited by 2
“To enhance computational efficiency, some edge computing systems offload DNN inference tasks to edge servers and utilize hardware accelerators such as graphics processing units (GPUs) and neural processing units (NPUs) [3].”
MethodFine spatial-temporal density mapping with optimized approaches for many-core system
Wang, S., Gao, Y., Seng, B., Pei, J., Zhang, Y., & Huang, J. (2025) · Frontiers in Neuroscience · doi:10.3389/fnins.2025.1512926 · Cited by 1
Efficient Deployment of Very Wide and Very Deep Hypersparse FFNs on FPGA
Singh, P. K., & Anastasiu, D. C. (2025) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi65124.2025.11130305 · Cited by 1
“The processing cost of FFNs can be significantly reduced by pruning away unimportant weights, also known as sparsifi-cation, and weight quantization [7].”
High-fidelity compression of 3D mesh animation data for humorous cartoon animation production
Li, Y. (2025) · Computer Optics · doi:10.18287/2412-6179-co-1530 · Cited by 1
“The storage and transmission of large-scale 3D mesh data is an extremely resource-intensive task, while the direct processing of uncompressed data is not intelligent enough [6].”
abstractPIM: A Technology Backward-Compatible Compilation Flow for Processing-In-Memory
Eliahu, A., Ben-Hur, R., Ronen, R., & Kvatinsky, S. (2021) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-81641-4_16 · Cited by 1
Cross-Channel Intragroup Sparsity Neural NetworkPreprint
Yu, Z., Wang, C., Wang, X., Wu, Q., Zhao, Y., & Wu, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11971 · Cited by 1
FTBNN: Rethinking Non-linearity for 1-bit CNNs and Going BeyondPreprint
Su, Z., Fang, L., Guo, D., Hu, D., Pietikäinen, M., & Liu, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09294 · Cited by 1
“Therefore, in the past few years, numerous research effort has been devoted to developing DNN compression techniques to pursue a satisfactory tradeoff between computational efficiency and prediction accuracy (Deng et al., 2020).”
BackgroundAn Iteratively Reweighted Method for Sparse Optimization on Nonconvex $\ell_{p}$ BallPreprint
Wang, H., Yang, X., & Jiang, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.02912 · Cited by 1
Yuan, G., Liao, Z., Ma, X., Cai, Y., Kong, Z., Shen, X., Fu, J., Li, Z., Zhang, C., Peng, H., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09166 · Cited by 1
“DNN Weight pruning [8], [28]–[31], as one of the DNN model compression techniques, has also been investigated to reduce the weight storage and improve performance for ReRAM-based acceleration designs [32], [33].”
MethodModeling Accurate Human Activity Recognition for Embedded Devices Using Multi-level Distillation.Preprint
Chen, R., Luo, H., Zhao, F., Meng, X., Xie, Z., & Zhu, Y. (2021) · arXiv (Cornell University) · Cited by 4
(2021) · Annals of Computer Science and Information Systems · doi:10.15439/978-83-959183-9-1 · Cited by 1
Nerella, S., Gonzalez, K. V., Cupka, J. S., Ruppert, M. M., Loftus, T. J., Bihorac, A., & Rashidi, P. (2022) · Elsevier eBooks · doi:10.1016/b978-0-12-822548-6.00123-0 · Cited by 2
FPGA-QHAR: Throughput-Optimized for Quantized Human Action Recognition on The EdgePreprint
Alhussain, A., & Lin, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.03390 · Cited by 1
“Numerous researchers aimed to apply compression methods to reduce HAR models' computational complexity and memory footprint, making them more suitable for edge deployment [8], [9].”
MethodError Diffusion: Post Training Quantization with Block-Scaled Number Formats for Neural NetworksPreprint
Khodamoradi, A., Denolf, K., & Dellinger, E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.11203 · Cited by 1
“They are constantly growing in size (number of parameters) and complexity (layer connections and their functions) [1].”
BackgroundEfficient Hardware-aware Neural Architecture Search for Edge Computing
Benmeziane, H. (2023) · theses.fr (ABES) · doi:10.70675/12a3f1afza163z4f13zb3cfz9a3425c3d45d · Cited by 2
Efficient CNN inference acceleration on FPGAs : a pattern pruning-driven approach
Pradels, L. (2024) · theses.fr (ABES) · doi:10.70675/9b86665cz79bbz400fz9d0bz69433c967981 · Cited by 1
QuIDE: Mastering the Quantized Intelligence Trade-off via Active OptimizationPreprint
Jiang, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.10959 · Cited by 1
“Choosing the right bit-width means balancing three competing objectives: compression ratio, predictive accuracy, and inference latency Deng et al. [2020].”
Intragroup sparsity for efficient inference
Yu, Z., Wang, C., Wang, X., Zhao, Y. H., & Wu, X. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533623
“Dataflow designs differ drastically among existing accelerators so as to optimize system efficiency for specific use cases [24].”
BackgroundOptimization Tools for ConvNets on the Edge
Peluso, V., Macii, E., & Calimera, A. (2020) · IFIP/IEEE International Conference on Very Large Scale Integration · doi:10.1109/vlsi-soc46417.2020.9344075
“In practice, this can be achieved with compression methods that exploit the intrinsic resiliency of ConvNets to identify and remove those parts with less contribution to accuracy [3].”
MethodRegularization-Free Structural Pruning for GPU Inference Acceleration
Guo, C., Yang, Y., Zhang, L., Wang, S., Li, H., Long, K., Yin, X., & Zhuo, C. (2021) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed51717.2021.9424299
“In this work, we mainly consider the network acceleration on inference and thus focus on weight pruning since similar techniques can be extended to neuron pruning, which usually suffers more accuracy degradation [21].”
BackgroundMethodDesign Space Exploration of Sparse Accelerators for Deep Neural Networks.Preprint
Shin, J. H., Shafiee, A. R., Pedram, A., Abdel-Aziz, H. A., Li, L., & Hassoun, J. (2021) · arXiv (Cornell University)
Lan, W., Cheung, Y., & Lan, L. (2021) · doi:10.36227/techrxiv.17031917
2021 Index Proceedings of the IEEE Vol. 107-109
(2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3135169
Lan, W., Cheung, Y., & Lan, L. (2021) · doi:10.36227/techrxiv.17031917.v1
Hybrid Domain Convolutional Neural Network for Memory Efficient Training
Guan, B., Liu, Y., Zhang, J., Sethares, W. A., Liu, F., Xu, Q., Li, W., & Quan, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-93046-2_20
Compression of Time Series Classification Model MC-MHLF using Knowledge Distillation
Gengyo, A., & Tamura, K. (2021) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc52423.2021.9658841
“It is important for actual systems to utilize small deep learning models that can be embedded in edge devices for practical use; therefore, model compression and acceleration for deep neural networks are currently two of the most important requirements [2].”
BackgroundA New Method to Compress Neural Networks
Zhang, X. (2021) · International Conference on Advanced Mechatronic Systems · doi:10.1109/icamechs54019.2021.9661524
A Lightweight Detection Model for Typical Objects in Streetscape Dataset
Zhao, X., & Zheng, H. G. (2021) · China Automation Congress (CAC) · doi:10.1109/cac53003.2021.9727729
“In [19], [20], [21] and [22], researchers proposed some methods to compress the model such as pruning filters, weight quantification and so on.”
MethodExploring high performance deep neural networks on GPUs
Dong, S. (2020) · Digital Repository Service (Northeastern University) · doi:10.17760/d20384365 · Cited by 1
Robust 4D awareness via diffusion adaptation over Connected and Autonomated vehicles
Piperigkos, N., Nousias, S., & Lalos, A. S. (2022) · Multidimensional Signal Processing Workshop · doi:10.1109/ivmsp54334.2022.9816204
Approximate Network-on-Chips with Application to Image Classification
Chen, Y., Louri, A., Liu, S., & Lombardi, F. (2022) · IEEE International Conference on Networking, Architecture and Storage (NAS) · doi:10.1109/nas55553.2022.9925540
“Quantization and pruning methods are widely used for deep CNNs in image classification applications to reduce communication traffic and computation [20].”
MethodLightMobileNetV2: A Lightweight Model for the Classification of COVID-19 Using Chest X-Ray Images
Garg, S., & Singh, P. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-13150-9_13
Hardware/Software Co-acceleration of Progressive Learning under Feature Dimension Variation
Karn, R. R., & Elfadel, I. M. (2022) · International Conference on Electrical and Computing Technologies and Applications (ICECTA) · doi:10.1109/icecta57148.2022.9990202
Towards event camera signal recognition using a lightweight spiking neural network
刘, 昭., 吴, 金., 石, 光., & 赵, 庆. (2022) · Scientia Sinica Informationis · doi:10.1360/ssi-2022-0063
2022 Index Proceedings of the IEEE Vol. 108-110
(2022) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3234419
Compressing the Embedding Matrix by a Dictionary Screening Approach in Text Classification
Zhou, J., Jing, X., Liu, M., & Wang, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33374-3_36
Fan, S., Liu, Y., Su, J., Wu, X., & Jiang, Q. (2023) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid57682.2023.00062
Efficient Algorithms for Artificial Neural Networks and Explainable AI
Eshkiki, H. (2023) · Cronfa (Swansea University) · doi:10.23889/suthesis.63943
M-STREAM: A split model Streaming and Inferencing method for reduced end-to-end Execution Latency
Chakraborty, P., Dholariya, D., & Yip, E. H. C. (2023) · IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) · doi:10.1109/conecct57959.2023.10234760
“Each operator has many input tensors and output tensors [7].”
BackgroundJia, L., Li, G., Lu, M., Wei, X., & Yi, G. (2023) · Electronics · doi:10.3390/electronics12183747
Self-citationCross-Resiliency Analysis of Neural Networks Implementations Against Adversarial AttacksPreprint
Torca, S. D., Barbareschi, M., Barone, S., Casola, V., & Marrone, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4567884
Llanza, A., Keddous, F. E., Shvai, N., & Nakib, A. (2023) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec53210.2023.10253975
Li, T., Zhang, L. G., Ma, Y., & Endoh, T. (2023) · IEEE Transactions on Magnetics · doi:10.1109/tmag.2023.3321878
“From the perspective of algorithm improvement and circuit architecture optimization, the prevalent methods to reduce the processor’s power consumption consist of DNNs model compression [9], [10], in-memory computation [11], [12], approximate circuit [13], and so forth.”
Accelerating Safety Verification of Neural Network Dynamical Systems Using Assured Compressed Models
Shao, Z., Wang, T., & Xiang, W. (2023) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc58219.2023.10327268
“A number of neural network compression methods exist to obtain a small-scale neural network ˆΦ as referred to the comprehensive survey [14].”
BackgroundMethodSAMP: Sub-task Aware Model Pruning with Layer-Wise Channel Balancing for Person Search
Wu, Z., Chen, J., & Wang, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8549-4_17
A Sequence Model Compression Method Based on Proper Orthogonal Decomposition
Xue, J., Wu, D., & Liu, Y. (2023) · International Conference on Computer Science and Blockchain (CCSB) · doi:10.1109/ccsb60789.2023.10398824
“There is a problem that the neural network model is too massive to deploy on embedded terminal devices for the limited computing and storage capabilities [4].”
BackgroundAdaptive Methods for Tensor Data Compression
(2024) · International Conference on Communication Systems & Networks (COMSNETS) · doi:10.1109/comsnets59351.2024.10427426
Resource-Limited Automated Ki67 Index Estimation in Breast Cancer
Gliozzo, J., Marinò, G. C., Bonometti, A., Frasca, M., & Malchiodi, D. (2023) · International Conference on Bioinformatics Research and Applications (ICBRA) · doi:10.1145/3632047.3632072 · Cited by 1
“Among the vast plethora of compression techniques proposed in literature (see, e.g. [11] for a survey), we designed and exploited the most suitable techniques for the structure of PathoNet.”
MethodNetwork Pruning with Gradient SmoothingPreprint
Bohjanen, S. L., Shah, A., Hughes, I., & Manasa (2024) · Thesis Commons (OSF Preprints) · doi:10.31237/osf.io/qpfwh
Gradient Update using Layer-based Importance Ratio in Deep Neural NetworksPreprint
Shah, A., & Hughes, I. (2024) · Thesis Commons (OSF Preprints) · doi:10.31237/osf.io/4u32w
Efficient hardware acceleration of deep neural networks via arithmetic complexity reduction
Reggiani, E. (2023) · UPCommons institutional repository (Universitat Politècnica de Catalunya) · doi:10.5821/dissertation-2117-404668
Real-Time Embedded Simulation Platform for Hippocampal Traveling Waves of Electric Field Conduction
Wei, X., Ren, Z., Lu, M., & Chang, S. (2024) · Electronics · doi:10.3390/electronics13061130
DNN compression approach based on Bayesian optimization tensor ring decomposition
Zhang, M., Liu, J., Shi, C., Zhang, N., & Liu, W. (2024) · International Conference on Graphics and Image Processing (ICGIP) · doi:10.1117/12.3021187
“Classic DNNs can be classified into five categories based on different compression techniques [1] : pruning [2] , quantization [3] , knowledge distillation [4] , low-rank decomposition [5] , and network architecture search [6] .”
BackgroundCOMPRIZE: Assessing the Fusion of Quantization and Compression on DNN Hardware Accelerators
Patel, V., Shah, N., Krishna, A., Glint, T., Ronak, A. Q., & Mekie, J. (2024) · International Conference on VLSI Design · doi:10.1109/vlsid60093.2024.00048
“These accelerators must leverage refined design techniques such as compression, quantization, and pruning to attain optimal efficiency [10].”
MethodSafety or Not? A Comparative Study for Deep Learning Apps on Smartphones
Au-Yeung, J., Wang, S., Liu, Y., & Chen, Z. (2023) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom60117.2023.00036
A Method for Recovering on Unsupervised Domain Adaptation Models Compression
Wang, S., Chen, E., Yang, M., & Lee, C. (2023) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci62032.2023.00016
“Conventional compression methods include low-rank approximation, quantization, neuron pruning etc. [2] Since UDA models are trained under a special scenario, there could be some insight that compressing on UDA models can be done more efficiently.”
BackgroundImplementation of Lightweight Spacecraft Fault Diagnosis Network Based on FPGA Accelerator
Lai, J., Wang, S., Nie, S., Li, Y., & Mai, J. J. (2024) · International Conference on Computer Communication and Artificial Intelligence (CCAI) · doi:10.1109/ccai61966.2024.10603259
“While SpMxV demands less computation, its irregular nature limits the optimal performance and energy efficiency that can be achieved on hardware accelerators [12].”
BackgroundEffect of Post-Training Pruning and Quantization on Endoscopic Computer-Aided Diagnosis Models
Menu, W. J., Dehghani, N., de With, P. H. N., & van der Sommen, F. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635224
“To address the challenges posed by the use of overparameterized DNNs, various network compression techniques have been introduced in literature [3, 4, 5].”
MethodApplication-Oriented Industrial Edge Computing
Zhou, X., Ge, S., Chi, J., & Qiu, T. (2024) · Springer eBooks · doi:10.1007/978-981-97-4752-8_6
Impact of ADC Bits on CNNs in CIM Architecture
Chang, C.-H., & Cheng, W. (2024) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-taiwan62264.2024.10674453
“To address these challenges, CIM architectures emerge as a promising solution[2], offering a paradigm shift from traditional computing models.”
BackgroundIntelligent Allocation Technologies for All-Scenario KDN Resources
Liao, J., He, B., Wang, J., Wang, J., & Qi, Q. (2024) · Wireless networks · doi:10.1007/978-3-031-70606-6_7
Endriss, U., Melo, F. S., Bach, K., Bugarín-Diz, A., Alonso, J. M., Barro, S., & Heintz, F. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia392
“This approach is particularly useful for over-parameterized task-specific models such as those used for classification that can sustain significant pruning with minimal loss on performance as evidenced by numerous studies [5, 11, 17, 18, 21, 22, 26, 34, 44, 46, 47].”
Energy-Efficient Node Scheduling Method for Target Tracking in USNs
Liu, M., Zheng, R., & Zhang, S. (2024) · Springer eBooks · doi:10.1007/978-981-97-4669-9_6
Sparse loss-aware ternarization for neural networks
Zhou, R., Niu, L., & Xu, D. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121668
Wang, Y., Niu, W., & Hou, H. (2024) · IEEE Access · doi:10.1109/access.2024.3512789
S 2 NeRF: Privacy-preserving Training Framework for NeRF
Zhang, B., Zhang, Y., Zhang, Z., Yang, J., Huang, L., & Wu, J. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3690185 · Cited by 4
“The main challenges of deploying 𝑆 2 NeRF come from two aspects, one is the frequent transmission of high-dimensional features and back-propagated gradients over bandwidth-limited wireless channels, and the other is the difficulty of training deep models on resource-constrained edge networks.”
Ding, A. Y., Alwis, C. D., & Liyanage, M. (2024) · Wiley eBooks · doi:10.1002/9781119611646.refs
Explainable Process Deviance Discovery with Data-Efficient Deep Learning
Folino, F., Folino, G., Guarascio, M., & Pontieri, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-81241-5_6
Multi-Agent Pruning and Quantization in Mixed-Precision Models
Hsieh, M.-Y., Liu, A., & Chen, Z.-J. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831664
Efficient Framework with Sparse Acquisition in CMOS Image Sensors for Low-Power Edge Devices
Kisku, W., Kaur, A., & Mishra, D. (2024) · IEEE International Symposium on Smart Electronic Systems (iSES) · doi:10.1109/ises63344.2024.00057
“These challenges can be mitigated through techniques like pruning and quantization [16], which focus on network compression.”
Kang, C., Moore, J. A., Robertson, S., Wilms, M., Towlson, E. K., & Forkert, N. D. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107308
Zhang, J., Che, X., Jin, S., Pan, K., Peng, S., & Hu, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889351 · Cited by 1
“Lossy data compression methods often transform data into the frequency domain such as DCT [2] and DWT [3] or are based on neural networks [5].”
Анатольевич, Л. А., & Vozmilov, A. G. (2025) · Modern Transportation Systems and Technologies · doi:10.17816/transsyst659809 · Cited by 1
Saving Energy with Per-Variable Bitwidth Speculation
McMichen, T., Dlott, D., Wongse-ammat, P., Greiner, N., Khajanchi, H., Joseph, R., & Campanoni, S. (2025) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3676641.3716271
“While specialization in the form of a burgeoning list of fixed [9, 10, 13, 26] and programmable [22, 32, 43] accelerators can provide un-paralleled power-performance benefits for dedicated tasks, most systems still rely on general purpose CPUs to perform critical system tasks, such as data transfer…”
Model Compression in Low Performance Edge Computing
Zhang, J., Li, Y., Wen, J., Tian, K., & Zhao, M. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-86203-8_12
Sparse Reduced-Rank Fully Connected Layers with Its Applications in Detection and Classification
Fan, L., Zhou, M., Wei, X., Feng, Y., Xiang, T., Fang, B., Shang, Z., Jia, F., Zhuang, X., Pu, H., et al. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3727982
“However, it has been shown that the parameters of neural networks have sparse and low-rank structures with enormous redundancy in the parameterization process, which directly leads to increased storage space and computational cost [7,8].”
OFT: An accelerator with eager gradient prediction for attention training
Wang, M., Zhang, S., Wang, S., Zhao, Y., & Zhang, M. (2024) · doi:10.1145/3676536.3676770 · Cited by 1
SDEFL: A Lightweight Fault Detection and Localization Method for Deep Neural Networks*
Yang, B., Hu, J., & Cao, J. (2024) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec65559.2024.00043
“Through the study and localization of deep neural network model faults, the development of related technologies can be promoted, such as model compression [18], model interpretation techniques [19], robust training methods [20], etc.”
Activation functions enabling the addition of neurons and layers without altering outcomes
López-Ureña, S. (2025) · Journal of Computational and Applied Mathematics · doi:10.1016/j.cam.2025.116730
“However, this growth in complexity also poses significant challenges, particularly in optimizing and managing these extensive networks (as discussed by Deng et al. (2020), for instance).”
Wang, X., Esquivel, R. A., & Ma, C. (2024) · International Conference on Computational Modeling, Simulation and Data Analysis (CMSDA) · doi:10.1145/3727993.3728152
“Network enhancements, such as inverse residual structures and model compression, optimize performance for industrial applications [ 7 - 8 ] .”
Pearson, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5274198
Verification of AI Accelerators for Integrating Algorithm Level ValidationPreprint
Pearson, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5274271
Graham, O., & Collins, D. (2025) · Preprints.org · doi:10.20944/preprints202505.2492.v1
Energy aware computer vision algorithm deployment on heterogeneous architectures
Ali, T., Bhowmik, D., & Nicol, R. L. (2025) · Discover Electronics · doi:10.1007/s44291-025-00078-7
MAtCHelper: An Automated Tool for Guiding in the Acceleration and Compression of Neural Networks
Población-Criado, D., Quiñones, E., & García-Gasulla, D. (2025) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-96228-8_20
Bridging the Gap in Neural Network Quantization with Feature Correlation Coding
Ding, Y., Hu, L., Cheng, F., Wang, Q., Zhu, Z., & Yang, P. (2025) · Journal of Circuits Systems and Computers · doi:10.1142/s0218126625504092
Joint Compression of Multi-model Systems for Edge Devices
Vieira, J. L. P., Makarem, L., & Salzmann, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-93697-5_13
Metadata
- Authors
- Deng, B. L., Li, G., Han, S., Shi, L., & Xie, Y.
- Affiliations
- Tsinghua University (CN)
- Massachusetts Institute of Technology (US)
- University of California, Santa Barbara (US)
- Venue
- Proceedings of the IEEE
- Published
- 2020-04
- Volume
- 108
- Issue
- 4
- Pages
- 485-532
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W3012561096
- Cited by
- 996
- References
- 334
- Citations by year
- 2026: 131 · 2025: 171 · 2024: 193 · 2023: 174 · 2022: 152 · 2021: 142 · 2020: 31
- FWCI
- 46.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Proceedings of the IEEE (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- National Science Foundation (1725447)
- Beijing Academy of Artificial Intelligence (BAAI), Tsinghua University Initiative Scientific Research Program
- Institute for Guo Qiang, Tsinghua University
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- neural network compression, neural network accelerators, deep neural network, data quantization, network sparsification, tensor decomposition, compact modeling, memory efficiency, embedded systems, hardware architecture
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.