The DOI matches its Crossref record; 244 of the 244 references it cites were found in a registry or an index.
A comprehensive review of model compression techniques in machine learning
Open accessAbstract
This paper critically examines model compression techniques within the machine learning (ML) domain, emphasizing their role in enhancing model efficiency for deployment in resource-constrained environments, such as mobile devices, edge computing, and Internet of Things (IoT) systems. By systematically exploring compression techniques and lightweight design architectures, it is provided a comprehensive understanding of their operational contexts and effectiveness. The synthesis of these strategies reveals a dynamic interplay between model performance and computational demand, highlighting the balance required for optimal application. As machine learning (ML) models grow increasingly complex and data-intensive, the demand for computational resources and memory has surged accordingly. This escalation presents significant challenges for the deployment of artificial intelligence (AI) systems in real-world applications, particularly where hardware capabilities are limited. Therefore, model compression techniques are not merely advantageous but essential for ensuring that these models can be utilized across various domains, maintaining high performance without prohibitive resource requirements. Furthermore, this review underscores the importance of model compression in sustainable artificial intelligence (AI) development. The introduction of hybrid methods, which combine multiple compression techniques, promises to deliver superior performance and efficiency. Additionally, the development of intelligent frameworks capable of selecting the most appropriate compression strategy based on specific application needs is crucial for advancing the field. The practical examples and engineering applications discussed demonstrate the real-world impact of these techniques. By optimizing the balance between model complexity and computational efficiency, model compression ensures that the advancements in AI technology remain sustainable and widely applicable. This comprehensive review thus contributes to the academic discourse and guides innovative solutions for efficient and responsible machine learning practices, paving the way for future advancements in the field. Graphical abstract:
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–244 of 244
Super-Resolution Model Quantized in Multi-Precision
Liu, J., Wang, Q., Zhang, D., & Shen, L. (2021) · Electronics · doi:10.3390/electronics10172176 · Cited by 9
Maestro: Uncovering Low-Rank Structures via Trainable DecompositionPreprint
Horváth, S., Laskaridis, S., Rajput, S., & Wang, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.14929 · Cited by 9
Faster R-CNN with structured sparsity learning and Ristretto for mobile environment
Nasution, M. A., Chahyati, D., & Fanany, M. I. (2017) · International Conference on Advanced Computer Science and Information Systems · doi:10.1109/icacsis.2017.8355051 · Cited by 8
Acceleration of neural network model execution on embedded systems
Chen, C.-J., Chen, K.-C., & Martin-Kuo, M.-C. (2018) · International Symposium on VLSI Design, Automation and Test · doi:10.1109/vlsi-dat.2018.8373246 · Cited by 8
Compressing Fully Connected Layers using Kronecker Tensor Decomposition
Chen, S., Sun, W., Huang, L., Yang, X., & Huang, J. (2019) · International Conference on Computer Science and Network Technology (ICCSNT) · doi:10.1109/iccsnt47585.2019.8962432 · Cited by 8
Precious, J. G., Kirubha, S. P. A., & Evangeline, I. K. (2022) · IETE Journal of Research · doi:10.1080/03772063.2022.2083027 · Cited by 8
Low Power Deep-Learning Architecture for Mobile IoT Intelligence
Elgawi, O. H., & Mutawa, A. M. (2020) · IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT) · doi:10.1109/iciot48696.2020.9089642 · Cited by 7
Iteratively Re-Weighted method for Sparsity-Inducing Norms
Nie, F., Hu, Z., Wang, X., Li, X., & Huang, H. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3179554 · Cited by 7
Post-training 4-bit Quantization of Deep Neural Networks
Tadahal, S. S., Bhogar, G., Meena, S. M., Kulkarni, U., Gurlahosur, S. V., & Vyakaranal, S. B. (2022) · International Conference for Emerging Technology (INCET) · doi:10.1109/incet54531.2022.9825213 · Cited by 7
Lee, K., Hwangbo, S., Yang, D., & Lee, G. (2023) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-023-00202-z · Cited by 7
A Novel Attention‐Based Layer Pruning Approach for Low‐Complexity Convolutional Neural Networks
Hossain, M. B., Gong, N., & Shaban, M. (2024) · Advanced Intelligent Systems · doi:10.1002/aisy.202400161 · Cited by 7
AntiDoteX: Attention-Based Dynamic Optimization for Neural Network Runtime Efficiency
Yu, F., Xu, Z., Liu, C., Stamoulis, D. S., Wang, D., Wang, Y., & Chen, X. (2022) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2022.3144616 · Cited by 6
Adaptive Tensor-Train Decomposition for Neural Network Compression
Zheng, Y., Zhou, Y., Zhao, Z., & Yu, D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69244-5_6 · Cited by 5
Waterwave: A GPU Memory Flow Engine for Concurrent DNN Training
Shi, X., Peng, X., He, L., Zhao, Y., & Jin, H. (2023) · IEEE Transactions on Computers · doi:10.1109/tc.2023.3278530 · Cited by 5
Study of performance of low-rank nonnegative tensor factorization methods
Shcherbakova, E. M., Matveev, S. A., Smirnov, A. P., et al. (2023) · Russian Journal of Numerical Analysis and Mathematical Modelling · doi:10.1515/rnam-2023-0018 · Cited by 5
Research on compression pruning methods based on deep learning
Cai, M., Su, Y., Wang, B., & Zhang, T. (2023) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2580/1/012060 · Cited by 5
Model Compression Algorithm via Reinforcement Learning and Knowledge Distillation
Liu, B., Hu, B.-B., Zhao, M., Peng, S., & Chang, J. (2023) · Mathematics · doi:10.3390/math11224589 · Cited by 5
Explainable models via compression of tree ensembles
Yan, S., Natarajan, S., Joshi, S., Khardon, R., & Tadepalli, P. V. (2023) · Machine Learning · doi:10.1007/s10994-023-06463-1 · Cited by 5
Explanation Guided Knowledge Distillation for Pre-trained Language Model Compression
Yang, Z., Zhang, Y., Sui, D., Ju, Y., Zhao, J., & Liu, K. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3639364 · Cited by 5
Ricardo Manzano, S., Naik, K., Albasir, A. A., Zaman, M., & Goel, N. (2021) · Digital Threats Research and Practice · doi:10.1145/3492327 · Cited by 4
Structured Attention Knowledge Distillation for Lightweight Networks
Gu, X., Hui, T., & Dai, Z. (2021) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc52312.2021.9601745 · Cited by 4
Pupil Detection for Augmented and Virtual Reality based on Images with Reduced Bit Depths
Fiala, G., Ye, Z., & Steger, C. (2022) · IEEE Sensors Applications Symposium (SAS) · doi:10.1109/sas54819.2022.9881378 · Cited by 4
Model Compression and Acceleration: Lip Recognition Based on Channel-Level Structured Pruning
Lu, Y., Ni, R., & Wen, J. L. (2022) · Applied Sciences · doi:10.3390/app122010468 · Cited by 4
Pruning Compact ConvNets for Efficient InferencePreprint
Ghosh, S., Prasad, K., Dai, X., Zhang, P., Wu, B., Cormode, G., & Vajda, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.04502 · Cited by 4
Online Multimodal Compression using Pruning and Knowledge Distillation for Iris Recognition
Latif, S. A., Sidek, K. A., Bakar, E. A., & Hashim, A. H. A. (2024) · Journal of Advanced Research in Applied Sciences and Engineering Technology · doi:10.37934/araset.37.2.6881 · Cited by 4
Kumar, R., Chen, G. K., Sumbul, H. E., Knag, P. C., Anders, M. A., Kaul, H., Hsu, S. K., Agarwal, A., Kar, M., Kim, S., et al. (2020) · IEEE Solid-State Circuits Letters · doi:10.1109/lssc.2020.3019349 · Cited by 3
Substituting Convolutions for Neural Network Compression
Crowley, E. J., Gray, G. D. B., Turner, J., & Storkey, A. (2021) · IEEE Access · doi:10.1109/access.2021.3086321 · Cited by 3
Cyclic Sparsely Connected Architectures: From foundations to applications
Hosseini, M., Manjunath, N. K., Kallakuri, U., Mahmoodi, H., Homayoun, H., & Mohsenin, T. (2021) · IEEE Solid-State Circuits Magazine · doi:10.1109/mssc.2021.3111431 · Cited by 3
The Compression Techniques Applied on Deep Learning Model
He, H., Huang, L., Huang, Z., & Yang, T. (2022) · Highlights in Science Engineering and Technology · doi:10.54097/hset.v4i.920 · Cited by 3
A Layer-Based Sparsification Method For Distributed DNN Training
Hu, Y., Ye, Q., Zhang, Z., & Lv, J. (2022) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc-dss-smartcity-dependsys57074.2022.00209 · Cited by 3
Quantized Graph Neural Networks for Image Classification
Xu, X., Ma, L., Zeng, T., & Huang, Q. (2023) · Mathematics · doi:10.3390/math11244927 · Cited by 3
PSQ: An Automatic Search Framework for Data-Free Quantization on PIM-based Architecture
Liu, F., Yang, N., & Jiang, L. (2023) · IEEE International Conference on Computer Design: VLSI in Computers and Processors (ICCD) · doi:10.1109/iccd58817.2023.00084 · Cited by 3
Design of low power LSTM neural network accelerator based on FPGA
Zhang, J., & Liu, X. (2023) · IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/iccc59590.2023.10507503 · Cited by 3
Quality comparison of 360° 8K images compressed by conventional and deep learning algorithms
Kufa, J., & Budac, A. (2023) · International Conference Radioelektronika · doi:10.1109/radioelektronika57919.2023.10109066 · Cited by 2
Late Breaking Results: Weight Decay is ALL You Need for Neural Network Sparsification
Chen, X., Pan, R. N., Wang, X., et al. (2023) · Proceedings - ACM IEEE Design Automation Conference · doi:10.1109/dac56929.2023.10247950 · Cited by 2
Super-resolution knowledge-distillation-based low-resolution steel defect images classification
Zhang, S., Pei, Z., & Ren, Z. (2024) · International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID) · doi:10.1117/12.3026364 · Cited by 2
COMPRESSED MOBILENET V3: AN EFFICIENT CNN FOR RESOURCE CONSTRAINED PLATFORMS
Prasad, K. P. S. P. (2021) · Purdue e-Pubs (Purdue University System) · doi:10.25394/pgs.14442710 · Cited by 2
Improving the efficiency of human action recognition using deep compression
Phan, H., Ha, C. T., & Nguyen, T. T. (2020) · International Conference on Multimedia Analysis and Pattern Recognition (MAPR) · doi:10.1109/mapr49794.2020.9237772 · Cited by 1
A Lightweight Model for Deep Frame Prediction in Video Coding
Choi, H., & Bajić, I. V. (2020) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/ieeeconf51394.2020.9443427 · Cited by 1
An 8-bit Fixed Point Quantization Method for Sparse MobileNetV2
Yu, J., Yu, X., Liu, Y., Liu, L., & Peng, X. (2021) · China Automation Congress (CAC) · doi:10.1109/cac53003.2021.9727524 · Cited by 1
Towards Understanding the Behaviors of Pretrained Compressed Convolutional Models
Zee, T., Lakshmana, M., & Nwogu, I. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956037 · Cited by 1
Wang, F., Pan, C., & Huang, J. X. (2022) · China Automation Congress (CAC) · doi:10.1109/cac57257.2022.10055501 · Cited by 1
Real-time Lightweight Hand Detection Model Combined with Network Pruning
Zhu, X., Jiang, Z., & Lou, Y. (2023) · IEEE/ACIS International Conference on Computer and Information Science (ICIS) · doi:10.1109/icis57766.2023.10210237 · Cited by 1
Industrial Image Anomaly Localization Method based on Reverse Knowledge Distillation
Zhang, T. (2024) · IEEE Advanced Information Technology, Electronic and Automation Control Conference (IAEAC) · doi:10.1109/iaeac59436.2024.10503620 · Cited by 1
Cited by
Alahmari, S., & Alghamdi, I. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.107849 · Cited by 21
Metadata
- Authors
- Dantas, P. V., Sabino da Silva, W., Cordeiro, L. C., & Carvalho, C. B.
- Venue
- Applied Intelligence
- Published
- 2024-09-02
- Volume
- 54
- Issue
- 22
- Pages
- 11804-11844
- ISSN
- 0924-669X (ISSN-L)
- 1573-7497
- OpenAlex
- W4402137675
- Cited by
- 261
- References
- 244
- FWCI
- 36.87
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.