Model Compression and Acceleration for Deep Neural Networks: The Principles, Progress, and Challenges
The DOI matches its Crossref record; 61 of the 67 references it cites were found in a registry or an index.
Abstract
In recent years, deep neural networks (DNNs) have received increased attention, have been applied to different applications, and achieved dramatic accuracy improvements in many tasks. These works rely on deep networks with millions or even billions of parameters, and the availability of graphics processing units (GPUs) with very high computation capability plays a key role in their success. For example, Krizhevsky et al. achieved breakthrough results in the 2012 ImageNet Challenge using a network containing 60 million parameters with five convolutional layers and three fully connected layers. Usually, it takes two to three days to train the whole model on the ImagetNet data set with an NVIDIA K40 machine. In another example, the top face-verification results from the Labeled Faces in the Wild (LFW) data set were obtained with networks containing hundreds of millions of parameters, using a mix of convolutional, locally connected, and fully connected layers. It is also very time-consuming to train such a model to obtain a reasonable performance. In architectures that only rely on fully connected layers, the number of parameters can grow to billions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“but there is no review discussing all three categories which in future will be integrated seamlessly to design effective and efficient DNNs in the new edge intelligence era.”
From Bringing ai to edge: from deep learning’s perspective (2022) · cites this paper as [26]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“Recently, more state-of-the-art architectures are used as baseline models in many works, including network in networks [67], VGGNets [68], and ResNets [69].”
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 4×
“[1] achieved breakthrough results in the 2012 ImageNet Challenge using a network containing 60 million parameters with five convolutional layers and three fully-connected layers.”
MethodInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“Other popular standard models include LeNets [66], All-CNN-nets [67] and many others.”
MethodDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“The work in [51] introduced a KD compression framework, which eased the training of deep networks by following a student-teacher paradigm, in which the student was penalized according to a softened version of the teacher’s output.”
BackgroundDeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542 · cited 1×
“example is the top face verification results on the Labeled Faces in the Wild (LFW) dataset were obtained with networks containing hundreds of millions of parameters, using a mix of convolutional, locally-connected, and fully-connected layers [2], [3].”
MethodHan, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · neural information processing systems · Cited by 4,489
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408 · cited 2×
“works that directly train CNNs with binary weights, for instance, BinaryConnect [13], BinaryNet [14] and XNORNetworks [15].”
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · cited 2×
“Other popular standard models include LeNets [66], All-CNN-nets [67] and many others.”
Method
Cited by
Items 601–700 of 818
A model compression method based on YOLOv5s target detection
Zhang, A., Guo, R., & Liu, K. (2023) · International Conference on Cloud Computing, Big Data Application and Software Engineering (CBASE) · doi:10.1109/cbase60015.2023.10439075 · Cited by 1
Systematic Review: Face Recognition Algorithms for Photos and Real-Time Applications
Ranmale, S., & Jagdale, B. N. (2024) · International Conference on Mobile Computing and Sustainable Informatics (ICMCSI) · doi:10.1109/icmcsi61536.2024.00059 · Cited by 1
Teacher-Free Knowledge Distillation Based on Non-Progressive Meta-Learned Multi Ranking Selection
Jiang, B.-R., Ho, P.-H., He, S.-T., & Lin, A. S. (2024) · IEEE Access · doi:10.1109/access.2024.3409177 · Cited by 1
“Many methods regarding model compression [1], [2] have been proposed to solve this problem, such as pruning [3], [4], quantization [5], Knowledge Distillation (KD) The associate editor coordinating the review of this manuscript and approving it for publication was Wanqing Zhao .”
MethodVindas, Y., Roux, E., Guépié, B. K., Almar, M., & Delachartre, P. (2024) · Pattern Recognition Letters · doi:10.1016/j.patrec.2024.11.016 · Cited by 2
Forest Fire Image Classification Through Decentralized DNN Inference
Papaioannou, D., Mygdalis, V., & Pitas, I. (2024) · IEEE International Conference on Image Processing Workshops (ICIPW) · doi:10.1109/icipcw64161.2024.10769107 · Cited by 1
Lightweight CNN for Radar HRRP Recognition Using NAS-Based Pruning and Multi-Knowledge Distillation
Zhang, Z., Wu, H., Xu, Y., Zhang, L., & Wang, Y. (2024) · IEEE International Conference on Signal, Information and Data Processing (ICSIDP) · doi:10.1109/icsidp62679.2024.10868001 · Cited by 1
“Network pruning [5] could reduce the complexity of CNN and keep the performance effectively.”
Harnessing Knowledge-Distillation for Lightweight AI-Implementation on Resource-Constrained Device
Yadav, A. K., Gupta, V. K., & Kumar, B. (2024) · IEEE International Symposium on Smart Electronic Systems (iSES) · doi:10.1109/ises63344.2024.00070 · Cited by 1
PCA-based compression of deep LSTMs: a robust framework for uncertain industrial systems
Yahyaoui, Z., Hajji, M., & Bouazzi, Y. (2025) · Computing · doi:10.1007/s00607-025-01565-1 · Cited by 1
Design of a Tunable Block Floating-Point Quantizer with Fractional Exponent
Tsai, P., Yang, T.-I., Lee, C.-H., Chen, L.-M., & Lee, S.-Y. (2019) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2019.8702503 · Cited by 1
“Various model compressed techniques for neural network are illustrated in [7].”
MethodUnlabeled Data Driven Channel-Wise Bit-Width Allocation and Quantization Refinement
Yuan, Y., Chen, C., Hu, X., & Peng, S. (2019) · Cited by 1
Okuno, T., Nakata, Y., Ishii, Y., & Tsukizawa, S. (2021) · doi:10.23919/mva51890.2021.9511383 · Cited by 3
“Because edge devices typically have limited hardware resources, it is necessary to reduce the size of high computational cost models using compression methods, such as pruning and quantization [2, 3], to enable the implementation of models on edge devices.”
MethodHe-qun, Y. A. N. G., Lu, T., Guo, W., Chang, S., Liu, G., & Luo, Y. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534041 · Cited by 1
“However, the portable devices has limited computing resource and low heat dissipation performance[8,9].”
Background[Retracted] Serialized Recommendation Technology Based on Deep Neural Network
Jin, L. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/3216043 · Cited by 1
“The feedforward neural network diagram is shown in Figure 4 [14].”
MethodRetractedEmei Martial Arts Promotion Model and Properties Based on Neural Network Technology
Xing, C., Abidin, N. E. Z., & Tang, Y. (2022) · International Transactions on Electrical Energy Systems · doi:10.1155/2022/6906844 · Cited by 1
“It has been applied to different applications and achieved significant accuracy improvements in many tasks. availability of graphics processing units with very high computing power plays a key role in their success [9].”
BackgroundRetractedA Neural Network Pruning Method by the Sum of Matrix Similarity Measures
Dong, X., Song, M., Li, B., & Song, Y. (2022) · International Academic Exchange Conference on Science and Technology Innovation (IAECST) · doi:10.1109/iaecst57965.2022.10062205 · Cited by 3
“The compression methods of neural networks mainly include parameter pruning, parameter sharing, low-rank decomposition, designing compact convolution filters, and knowledge distillation [4].”
MethodHuman-grounded Evaluations of Explanation Methods for Text ClassificationPreprint
Lertvittayakumjorn, P., & Toni, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.11355 · Cited by 17
“, measuring model complexity (Bianchini and Scarselli, 2014) and model compression (Cheng et al., 2018).”
BackgroundDeep Neural Mobile NetworkingPreprint
Zhang, C. (2020) · arXiv (Cornell University) · doi:10.7488/era/356 · Cited by 3
“To address this issue, ongoing research improves existing deep learning architectures [140], such that the inference process does not violate latency or energy constraints [141, 142], nor raise any privacy concern [143].”
BackgroundMeneghetti, D. D. R., Homem, T. P. D., de Oliveira, J. H. R., Silva, I. J., Perico, D. H., & Bianchi, R. A. C. (2020) · arXiv (Cornell University) · Cited by 1
GradFreeBits: Gradient Free Bit Allocation for Dynamic Low Precision Neural NetworksPreprint
Bodner, B. J., Shalom, G. B., & Treister, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.09298 · Cited by 1
“To mitigate this issue, recent efforts have been focused on developing specialized hardware to support the computational demands (Chen et al., 2020) as well as model compression methods in order to reduce them (Cheng et al., 2018).”
BackgroundMethodA Survey on GAN Acceleration Using Memory Compression TechniquePreprint
Tantawy, D., Zahran, M., & Wassal, A. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06626 · Cited by 1
ParaDiS: Parallelly Distributable Slimmable Neural NetworksPreprint
Ozerov, A., Lambert, A., & Kumaraswamy, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02724 · Cited by 2
“Model compression schemes (Cheng et al., 2018) (e.g., pruning (LeCun et al., 1990) or knowledge distillation (Hinton et al., 2015)) allow satisfying particular memory and processing constraints, though they do only allow producing a fixed model for every particular setting.”
TP-ADMM: An Efficient Two-Stage Framework for Training Binary Neural Networks
Yuan, Y., Chen, C., Hu, X., & Peng, S. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-36808-1_63 · Cited by 3
A Note on Latency Variability of Deep Neural Networks for Mobile InferencePreprint
Yang, L., Lu, B., & Ren, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00138 · Cited by 2
“To improve performance, an additional term accounting for the latency is often factored into the loss function or constraints during DNN training/compression [8, 12].”
BackgroundTsai, P., Yang, T.-I., Lee, C.-H., Chen, L., & Lee, S.-Y. (2020) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2020.2973537
“Various model compressed techniques for neural network are illustrated in [9].”
MethodAnalysis of the Effects of Decay Coefficient and Time Resolution in SNN Backpropagation
Um, K. S., Hwang, H., Kim, H., & Heo, S. W. (2020) · International Conference on Electronics, Information, and Communication (ICEIC) · doi:10.1109/iceic49074.2020.9051056
“In recent research, researches are being actively conducted to reduce the size of the network while maintaining performance [2], and at the same time, NN that can operate at low power is being studied.”
BackgroundModeling information flow through deep convolutional neural networks
Nasiri, B. (2020) · Espace ÉTS (ETS) · Cited by 1
Identification method of airport violence sorting based on deep learning
Wang, H., Chen, Y., Hu, X., & Miao, Z. (2020) · Chinese Control Conference (CCC) · doi:10.23919/ccc50068.2020.9189657
Two-Stage Model Compression and Acceleration: Optimal Student Network for Better Performance
Tang, J., Jiang, N., Yu, W., Zhou, J., & Mai, L. (2020) · IEEE Access · doi:10.1109/access.2020.3040823
“Cheng et al. [49], Choudhary et al. [50] summarizes the existing model compression and acceleration methods to propose that how to effectively use the limited available resources to design specific compression methods for these resource-limited devices remains a challenge.”
BackgroundA Review of Structure Optimization of Convolutional Neural Networks
Liu, C., Wang, C., Yu, L., Changfeng, X., & Xiufeng, Z. (2021) · Lecture notes in electrical engineering · doi:10.1007/978-981-33-6318-2_87
Data Communication-Edge, Fog, and Cloud Computing
Pal, S. K., Mishra, D., Pal, A., Dutta, S., Chakravarty, D., & Pal, S. (2021) · Springer series in advanced manufacturing · doi:10.1007/978-3-030-81815-9_5
Deep Clustering and Deep Network Compression
Alqahtani, A. (2021) · Cronfa (Swansea University) · doi:10.23889/suthesis.57269
High Spatial Resolution Remote Sensing Classification with Lightweight CNN Using Dilated Convolution
Zhang, G., Li, W., Dong, H., & Gui, G. (2021) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-89814-4_56
Data Protection of AMI Based on Oblivious Inference and Deep Learning
Liu, T., Xiaowei, C., Fang, J. H., Shaocheng, W., Qingsong, Z., & Yuan, P. (2021) · IEEE Sustainable Power and Energy Conference (iSPEC) · doi:10.1109/ispec53008.2021.9735598
“In the existing literature, the compression methods of deep neural networks can be divided into the following five categories: parameter pruning, parameter quantization, low-rank decomposition, transfer/compression convolution filter and knowledge distillation[18, 19].”
Method(2020) · International Journal of Mechatronics and Applied Mechanics · doi:10.17683/ijomam/issue8.7
Fast Non-Uniform Quantization of Neural Networks
Gao, Y., Wang, Q., Chen, Z., & Yuan, Y. (2022) · International Conference on Cloud Computing and Big Data Analytics (ICCCBDA) · doi:10.1109/icccbda55098.2022.9778293
Robust 4D awareness via diffusion adaptation over Connected and Autonomated vehicles
Piperigkos, N., Nousias, S., & Lalos, A. S. (2022) · Multidimensional Signal Processing Workshop · doi:10.1109/ivmsp54334.2022.9816204
Dong, L., Cai, X., & Yang, Z. (2021) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads53394.2021.00070
“However, these functions rely on convolutional neural networks(CNNs) and require complex computation, which are incapable of running on resourceconstrain mobile AR devices [4].”
BackgroundBag-Of-Features-Based Knowledge Distillation For Lightweight Convolutional Neural Networks
Chariton, A., Passalis, N., & Tefas, A. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897390 · Cited by 1
“This has fueled the interest in developing more lightweight architectures [2, 3], methods for compressing existing architectures [4], as well as methods for mitigating the possible loss in accuracy due to employing lightweight architectures [5, 6], such as knowledge distillation [5], which works by…”
MethodArchitecture Design for Analyzing Hyperspectral Images
Sun, Y., Yen, G. G., & Zhang, M. (2022) · Studies in computational intelligence · doi:10.1007/978-3-031-16868-0_12
Deep Neural Architecture Compression
Sun, Y., Yen, G. G., & Zhang, M. (2022) · Studies in computational intelligence · doi:10.1007/978-3-031-16868-0_16
Knowledge Distillation for Image Signal Processing Using Only the Generator Portion of a GAN
Heo, Y., & Lee, S. (2022) · Electronics · doi:10.3390/electronics11223815
Tong, Z., Deng, X., Shao, H., & Wang, X. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-6901-0_61
Uncertainty-based Model Acceleration for Cancer Classification in Whole-Slide Images
Gao, Z., Mao, A., Wu, J., Li, Y., Wang, C., Ding, C., Gong, T., & Li, C. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995601
“Thus, they are effective in model compression, but the effect is mediocre in model acceleration ( i.e. , up to 2x) [14].”
BackgroundUnified Framework for Effective Knowledge Distillation in Single-stage Object Detectors
Saha, A., Bialkowski, A., & Khalifa, S. (2022) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta56598.2022.10034590
Toward cooperative multi-agent video streaming perception
杨, 铮., 董, 亮., & 蔡, 新. (2023) · Scientia Sinica Informationis · doi:10.1360/ssi-2021-0179
Towards Safe Machine Learning Lifecycles with ESG Model Cards
Bonnier, T., & Bosch, B. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40953-0_31 · Cited by 3
Model Pruning and Deployment Optimization for Ship Detection
Jiang, Z., Xu, C., Gu, Y., & An, K. (2023) · International Conference on Intelligent Computing and Signal Processing (ICSP) · doi:10.1109/icsp58490.2023.10248433
“However, to facilitate algorithm transplantation on embedded platforms such as edge devices, in addition to low-rank decomposition, knowledge distillation, and lightweight model design, tailoring high-precision models is also one of the commonly used methods for deployment optimization[19].”
MethodLatency-Critical Inference Serving for Deep Learning
Nigade, V. (2023) · doi:10.5463/thesis.308
Mobile Deployment Method for Defect Recognition Algorithms in Power Transmission Scenarios
Huang, H., Zhang, B., Jiang, C., & Wang, L. (2023) · International Conference on Data Science and Network Security (ICDSNS) · doi:10.1109/icdsns58469.2023.10244925
“The storage and computing resources of mobile chips are limited, which requires optimization of neural network algorithms, mainly including knowledge distillation, neural architecture search (NAS), network pruning [1-5], and model quantization [6-8].”
BackgroundAutonomous evolutionary learning for unmanned mobile platforms: Research progress and prospects
Zhang, Y., Wang, P., Zhang, L., & Yan, Q. (2023) · Chinese Science Bulletin (Chinese Version) · doi:10.1360/tb-2023-0811
Performance Analysis of Distributed Learning in Edge Computing on Handwritten Digits Dataset
Tinh, V. P., Nguyen, V. A., Nguyen, X. B. L., Dang, D. N. M., & Khoa, T. A. (2023) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-47359-3_12
Partial Feedback Control for Knowledge Distillation
Chen, S., Tang, W., Kang, Q., Chen, Z., & Liu, Z. (2023) · International Conference on Computer, Vision and Intelligent Technology (ICCVIT) · doi:10.1145/3627341.3630386
“Knowledge Distillation was initially introduced by [2] and later extended by [8], which refers to the method that helps the training process of a smaller student network under the supervision of a larger teacher network.”
MethodMedical Image Segmentation Using Improved Energy Based Model
Pathak, R., Makwana, H., & Pathik, B. (2023) · Journal of Engineering Science and Technology Review · doi:10.25103/jestr.166.09
SAA-SDM: Neural Networks Faster Learned to Segment Organ Images
Gao, C., Shi, Y., Yang, S., & Lei, B. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-023-00947-1
“In recent years, several works have proposed methods and techniques to speed up the training of, or predictions by, neural networks from various perspectives [11, 12].”
CARNet: Channel Attention Guided Compression of Residual Network
Wang, F. (2023) · International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) · doi:10.1109/icicml60161.2023.10424870
Acceleration Techniques for Video-Based Self-Recuperation Training – State-of-the-Art Review
Ekambaram, D., & Ponnusamy, V. (2023) · International Conference on Intelligent Computing and Control for Engineering and Business Systems (ICCEBS) · doi:10.1109/iccebs58601.2023.10449153
Qiu, S., Du, C., Xu, X., & Liu, J. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-1568-8_4
Qiao, Y., Ping, J., Wang, J., Xu, G., & Xu, M. (2023) · International Symposium on Artificial Intelligence for Medicine Sciences (ISAIMS) · doi:10.1145/3644116.3644170
“…assessment platform, the convolutional neural network deep learning diagnostic portal hypertension algorithm, can increase the prevalence of early screening for portal hypertension patients and improve the treatment status of 7 million portal hypertension patients in our country [ 31 - 35 ] .”
BackgroundDiffallah, Z., Ykhlef, H., & Bouarfa, H. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3660641 · Cited by 1
“It is the process by which the knowledge from one model, termed as a teacher, is transferred or distilled to another model, termed as a student [25].”
InfluentialOcular Disease Prediction Using Feature Maps with Convolutional Neural Network (CNN) Method
Kaleel, A. I., & Rajakumari, S. B. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-58604-0_8
Safety or Not? A Comparative Study for Deep Learning Apps on Smartphones
Au-Yeung, J., Wang, S., Liu, Y., & Chen, Z. (2023) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom60117.2023.00036
“With the development of model quantization [17]– [20]techniques, models can be compressed and optimised to reduce model size and computational complexity and adapt to the resource constraints of the device.”
MethodLiu, K., Wang, R., Song, X., Deng, X., & Zhu, Q. (2024) · Sensors · doi:10.3390/s24123983
“Taking the limited dataset and difficulty in obtaining high-quality remote sensing image classification as an example, in order to achieve high accuracy classification based on complex spectral features, network deployment was innovated based on CNN architecture in 2018 [37].”
MethodOptimized Deep Learning Classification Model for Intelligent Edge devices
Naveen, S., & Kounte, M. R. (2024) · Journal of Engineering Science and Technology Review · doi:10.25103/jestr.173.11 · Cited by 1
A Survey on Wireless Data Aggregation Through Over-the-Air Computation
Lee, Y., & Jung, H. (2024) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-67357-3_13
Alqemlas, D. S., & Jeragh, M. (2024) · International Conference on Multimedia and Image Processing (ICMIP) · doi:10.1145/3665026.3665046
Robust Neural Pruning with Gradient Sampling Optimization for Residual Neural Networks
Yun, J. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650301
Intelligent Allocation Technologies for All-Scenario KDN Resources
Liao, J., He, B., Wang, J., Wang, J., & Qi, Q. (2024) · Wireless networks · doi:10.1007/978-3-031-70606-6_7
Efficient Neural Network Compression Process to Facilitate Hardware Implementation
Siebert, A., Ferré, G., Gal, B. L., & Fourny, A. (2024) · International Symposium ELMAR (ELMAR) · doi:10.1109/elmar62909.2024.10694641
Strategies for Graph Optimization in Deep Learning Compilers
Zheng, Y. (2024) · International Conference on Interactive Intelligent Systems and Techniques (IIST) · doi:10.1109/iist62526.2024.00086 · Cited by 2
Unified Local-Cloud Decision-Making via Reinforcement Learning
Sengupta, K., Shangguan, Z., Bharadwaj, S., Arora, S., Ohn-Bar, E., & Mancuso, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72940-9_11 · Cited by 3
Xiong, F., Zhou, Y., Hu, X., & Duan, S. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05957-2
“On this basis, since the deployment network is usually lightweight, its learning and processing capabilities for large data are limited, and it lacks certain generalization capabilities and the ability to respond to emergencies [23], Fig.”
Ferwana, I., Chaffar, S., & Belhouari, S. B. (2024) · Advances in systems analysis, software engineering, and high performance computing book series · doi:10.4018/978-1-6684-3795-7.ch005
Deep Learning based Models for Nonlinear System Identification
Ayooghi, V. M., & Aliyari-Shoorehdeli, M. (2023) · Journal of Control · doi:10.61186/joc.17.2.1
Probing the Pitfalls: Understanding SVD’s Shortcomings in Language Model Compression
Pletenev, S. (2024) · Journal of language and Education · doi:10.17323/jle.2024.22368 · Cited by 1
“Model compression, a field that focuses on reducing the size and complexity of deep learning models, typically operates on the assumption that an existing model serves as the basis for compression techniques (Cheng et al., 2018).”
Onyekpe, U., Szkolnik, A., Palade, V., Kanarachos, S. A., & Fitzpatrick, M. E. (2025) · Neural Computing and Applications · doi:10.1007/s00521-024-10046-z
Compression Strategies for Massive Natural Language Generation Models
Dayal, A., Angara, J., Gupta, S., & Saripalle, R. S. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-77081-4_7
A Novel Fault Diagnosis Method for Multimode Hot Rolling Process Based on MGU and BLS
Zhang, C. J., Yin, W., Zhang, H., Liu, R., Zhang, Z., & Zhao, Z. (2024) · China Automation Congress (CAC) · doi:10.1109/cac63892.2024.10864790
Life Control Based on Limited Incremental learning and WE-OSELM
Wen, Y., Wang, J., & Zhang, Y. (2024) · International Conference on Computer, Vision and Intelligent Technology (ICCVIT) · doi:10.1109/iccvit63928.2024.10872540
Pruning Optimization of Deep Learning Model Based on Generalization Estimation
Li, X., Ke, W., Wen, W., Liu, S., & Sheng, X. (2024) · International Conference on Artificial Intelligence, Deep Learning and Neural Networks (AIDLNN) · doi:10.1109/aidlnn65358.2024.00039
Multi-Mode High-Precision Fault Diagnosis Control System Manage Method
Wen, Y., Wang, J., & Zhang, Y. (2024) · International Conference on Computing, Communication, Perception and Quantum Technology (CCPQT) · doi:10.1109/ccpqt64497.2024.00070
Анатольевич, Л. А., & Vozmilov, A. G. (2025) · Modern Transportation Systems and Technologies · doi:10.17816/transsyst659809 · Cited by 1
Li, Y., Yang, H., Fu, G., Xu, Y., Wang, T., & Nie, Z. (2025) · Journal of Biomechanical Engineering · doi:10.1115/1.4068422
Trembovetskyi, R., Розломій, І., & Бабенко, В. (2024) · Security of Infocommunication Systems and Internet of Things · doi:10.31861/sisiot2024.2.02009
Leblebici, M., & Çalhan, A. (2025) · Mühendislik Bilimleri ve Araştırmaları Dergisi · doi:10.46387/bjesr.1565528
Navigating Coreset Selection and Model Compression for Efficient Maritime Image Classification
Shinde, T. S., Sharma, A., Bhardwaj, S., & Vuai, A. S. (2025) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw65960.2025.00175
“Training deep neural networks (DNNs) on large-scale datasets like MASATI [7] is computationally intensive.”
Deep Networking for IoT Modules
Shnain, A. H., Patwal, H., Varshney, N., Mahajan, R., Abdullaev, J., & Mittal, V. (2024) · International Conference on Communication, Computing and Energy Efficient Technologies (I3CEET) · doi:10.1109/i3ceet61722.2024.10994075
Energy aware computer vision algorithm deployment on heterogeneous architectures
Ali, T., Bhowmik, D., & Nicol, R. L. (2025) · Discover Electronics · doi:10.1007/s44291-025-00078-7
“Various techniques have been explored such as model compression [39, 40], that make deep learning models more efficient without significantly reducing accuracy.”
Gao, J., Ma, H., Miao, D., Yao, R., & Zhang, Y. (2025) · Science Progress · doi:10.1177/00368504251357783
“76 Since the computational complexity of neural network technology is mainly determined by the number of hidden layers and neurons, rather than the dimension of the input data, it only needs to adjust the number of neurons in the input layer to adapt to data of different dimensions.”
From forward-backward sweep to backpropagation for optimal control in tumor-immune dynamics
Zhang, W., Ding, W., & Zhu, H. (2026) · Journal of Mathematical Biology · doi:10.21203/rs.3.rs-6855591/v1
Asynchronous Stochastic Gradient Descent with Model Quantization
Du, Y., & You, K. (2025) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2025.07.035
Lightweight XAI-Enhanced CNN for Fashion-MNIST Classification on Edge Devices
Çakir, S. N., & Al-Hubaishi, M. (2025) · International Symposium on Innovative Approaches in Smart Technologies (ISAS) · doi:10.1109/isas66241.2025.11101914 · Cited by 1
Kumar, B., Gaba, R., & Shaw, R. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-6385-1.ch001
Neural Network Coding Layer (NNCL): Enhancing Deep Learning Robustness Against Feature Erasure
Lee, C.-S., Khan, A., Chang, S., & Chang, H. (2025) · IEEE Access · doi:10.1109/access.2025.3610080
“Deep Compression [25] used Huffman coding to enhance compression efficiency, while other research applies ECC principles to increase robustness against quantization errors or secure reliability of compressed models [26], [27], [46].”
Data-Efficient Stream-Based Active Distillation For Scalable Edge Model Deployment
Manjah, D., Bary, T., Gérin, B., Macq, B. M. M., & Vleeschouwer, C. D. (2025) · IEEE International Conference on Image Processing Workshops (ICIPW) · doi:10.1109/icipw68931.2025.11385953
“This approach can be used to create compact models that are based on the capabilities of a more complex model, and is especially useful when deploying on edge devices, where storage and computational capacity become critical factors [8].”
Enhancing depression detection in social media through BERT model pruning
Yang, Y., Daud, S. N., & Peng, Z. (2025) · AIP conference proceedings · doi:10.1063/5.0291688
Comparative Analysis of Lightweight BERT Models and Ensemble Benefits for Extractive Q&A
Konsil, A. N. A., Chua, S., & Minoi, J.-L. (2025) · International Conference on Information Technology in Asia (CITA) · doi:10.1109/cita66455.2025.11198695
“Model compression has been widely explored in deep neural networks [3], with techniques such as knowledge distillation, parameter sharing, and factorized embedding parameterization.”
Nguyen, V.-H., & Le, T.-T. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-33971-4.00004-0
Tiremo: An End-to-End AIoT System Design to Enable Intelligence Over Connectivity
Çoban, A., Saraç, M. A., Öğütçü, E. E., Çelik, D., İsmailoğlu, İ., Yılmaz, M., Güdül, O. C., & Horsanalı, M. (2025) · International Conference on Computer Science and Engineering (UBMK) · doi:10.1109/ubmk67458.2025.11207003
FCG: High-Throughput JPEG Heterogeneous Inference with Hybrid Parallel Pipeline on Mobile Devices
Mao, Y., Kang, Z., Li, P., Chen, J., Yang, Z., & Li, Z. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755712
Edge Deep Learning for Low Capabilities Devices
Filippou, F., Foukalas, F. T., & Tsiftsis, T. A. (2025) · International Balkan Conference on Communications and Networking (BalkanCom) · doi:10.1109/balkancom65827.2025.11185953
Metadata
- Authors
- Cheng, Y., Wang, D., Zhou, P., & Zhang, T.
- Affiliations
- IBM Research - Thomas J. Watson Research Center (US)
- Harbin Institute of Technology (CN)
- Huazhong University of Science and Technology (CN)
- Tsinghua University (CN)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2018-01
- Volume
- 35
- Issue
- 1
- Pages
- 126-136
- ISSN
- 1053-5888 (ISSN-L)
- 1558-0792
- OpenAlex
- W2783538964
- Cited by
- 817
- References
- 67
- Citations by year
- 2026: 114 · 2025: 132 · 2024: 116 · 2023: 110 · 2022: 109 · 2021: 92 · 2020: 89 · 2019: 41 · 2018: 14
- FWCI
- 29.51
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- model compression, deep neural network, inference acceleration, parameter reduction, convolutional neural networks, training efficiency, GPU computing, face verification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.