The DOI matches its Crossref record; 104 of the 114 references it cites were found in a registry or an index.
A Survey on Model Compression for Large Language Models
Open accessAbstract
Large Language Models (LLMs) have transformed natural language processing tasks successfully. Yet, their large size and high computational needs pose challenges for practical use, especially in resource-limited settings. Model compression has emerged as a key research area to address these challenges. This paper presents a survey of model compression techniques for LLMs. We cover methods like quantization, pruning, and knowledge distillation, highlighting recent advancements. We also discuss benchmarking strategies and evaluation metrics crucial for assessing compressed LLMs. This survey offers valuable insights for researchers and practitioners, aiming to enhance efficiency and real-world applicability of LLMs while laying a foundation for future advancements.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Active efforts in the machine learning community to shrink and streamline LLMs may also help improve their efficiency.43”
From Assessment of a Zero-Shot Large Language Model in Measuring Documented Goals-of-Care Discussions (2026) · cites this paper as [43]
In the citing paper
“TLMs are typically created by employing techniques such as knowledge distillation, pruning, quantization [17,18,19,20,21,22,23], and specialized architectures [24] designed to minimize model size and memory footprint.”
From Tiny Language Models for Automation and Control: Overview, Potential Applications, and Future Research Directions (2025) · cites this paper as [18]
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,869
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,705
Pointer Sentinel Mixture ModelsPreprint
Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1609.07843 · Cited by 5,058
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
QLoRA: Efficient Finetuning of Quantized LLMs
Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023) · arXiv · doi:10.48550/arxiv.2305.14314 · Cited by 3,832
(1989) · Cited by 3,214
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
Williams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093 · cited 2×
“At the same time, Recent research [Wei et al. , 2022a; Schaeffer et al. , 2023; Zhao et al. , 2023] underscores that with the emphasis on augmenting model size, LLMs like GPT-3 (175B parameters) and PaLM (540B parameters) showcase unique behaviors when compared to smaller models like BERT (330M…”
Self-citationA Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18223 · Cited by 2,090
Self-citationLearning Structured Sparsity in Deep Neural NetworksPreprint
Wen, W. X., Wu, C., Wang, Y., Chen, Y., & Li, H. H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.03665 · Cited by 1,924
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888
Gray, R. M., & Neuhoff, D. L. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.720541 · Cited by 1,647
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
Mihaylov, T., Clark, P. E., Khot, T., & Sabharwal, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1260 · Cited by 1,602
A primer in BERTology: What we know about how BERT works
Rogers, A., Kovaleva, O., & Rumshisky, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00349 · Cited by 1,552
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
Frankle, J., & Carbin, M. (2018) · arXiv · doi:10.48550/arxiv.1803.03635 · Cited by 1,529
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495
Self-citationWang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · arXiv · doi:10.48550/arxiv.2109.00859 · Cited by 1,495
Self-citationInstructpix2pix: Learning to follow image editing instructions
Brooks, T., Holynski, A., & Efros, A. A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01764 · Cited by 1,466
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.754 · Cited by 1,243
Importance Estimation for Neural Network Pruning
Molchanov, P., Mallya, A., Tyree, S., Frosio, I., & Kautz, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01152 · Cited by 983
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Talmor, A., Herzig, J., Lourie, N., & Berant, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1421 · Cited by 919
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829
Weighted low-rank approximations
Srebro, N., & Jaakkola, T. (2003) · International Conference on Machine Learning · Cited by 715
A Survey on In-context Learning
Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Chang, B., et al. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.64 · Cited by 649
Training Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
LLM-Pruner: On the Structural Pruning of Large Language Models
Ma, X., Fang, G., & Wang, X. (2023) · neural information processing systems · doi:10.52202/075280-0950 · Cited by 434
NVIDIA A100 Tensor Core GPU: Performance and Innovation
Choquette, J., Gandhi, W., Giroux, O., Stam, N., & Krashinsky, R. (2021) · IEEE Micro · doi:10.1109/mm.2021.3061394 · Cited by 410
Opt: Open pre-trained transformer language models
Zhang, S. (2025) · TIB Data Manager · doi:10.57702/dn0pmcbs · Cited by 405
Large Language Models Are Reasoning Teachers
Ho, N., Schmid, L., & Yun, S.-Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.830 · Cited by 387
OmniQuant: Omnidirectionally Calibrated Quantization for Large Language ModelsPreprint
Shao, W., Chen, M., Zhang, Z., Xu, P., Zhao, L., Li, Z., Zhang, K., Gao, P., Qiao, Y., & Luo, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.13137 · Cited by 373
Model Compression for Deep Neural Networks: A Survey
Li, Z., Li, H., & Meng, L. (2023) · Computers · doi:10.3390/computers12030060 · Cited by 342
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight CompressionPreprint
Dettmers, T., Svirschevski, R., Egiazarian, V., Kuznedelev, D., Frantar, E., Ashkboos, S., Borzunov, A., Hoefler, T., & Alistarh, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.03078 · Cited by 338
Hsieh, C.-Y., Li, C.-L., Yeh, C.-K., Nakhost, H., Fujii, Y., Ratner, A., Krishna, R., Lee, C.-Y., & Pfister, T. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.507 · Cited by 316
SqueezeLLM: Dense-and-Sparse QuantizationPreprint
Kim, S., Hooper, C. R. C., Gholami, A., Dong, Z., Li, X., Shen, S., Mahoney, M. W., & Keutzer, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.07629 · Cited by 313
Sheared LLaMA: Accelerating Language Model Pre-training via Structured PruningPreprint
Xia, M., Gao, T., Zeng, Z., & Chen, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.06694 · Cited by 294
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers
Yao, Z., Aminabadi, R. Y., Zhang, M., Wu, X., Li, C., & He, Y. (2022) · neural information processing systems · doi:10.52202/068431-1970 · Cited by 269
SliceGPT: Compress Large Language Models by Deleting Rows and ColumnsPreprint
Ashkboos, S., Croci, M. L., do Nascimento, M. G., Hoefler, T., & Hensman, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.15024 · Cited by 261
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., & Han, S. (2022) · arXiv · doi:10.48550/arxiv.2211.10438 · Cited by 247
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
Geva, M., Khashabi, D., Segal, E., Khot, T., Roth, D., & Berant, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00370 · Cited by 242
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language ModelsPreprint
Li, Y., Yu, Y., Chen, L., He, P., Karampatziakis, N., Chen, W., & Zhao, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.08659 · Cited by 201
OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization
Guo, C., Tang, J., Hu, W., Leng, J., Zhang, C., Yang, F., Liu, Y., Guo, M., & Zhu, Y. (2023) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3579371.3589038 · Cited by 185
Self-citationWei, X., Zhang, Y., Li, Y., Zhang, X., Gong, R., Guo, J., & Liu, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.102 · Cited by 178
RPTQ: Reorder-based Post-training Quantization for Large Language ModelsPreprint
Yuan, Z., Niu, L., Liu, J., Liu, W., Wang, X., Shang, Y., Sun, G., Wu, Q., Wu, J., & Wu, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.01089 · Cited by 177
Lee, C., Jin, J., Kim, T., Kim, H., & Park, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29237 · Cited by 172
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
Liu, Z., Oğuz, B., Zhao, C., Chang, E., Stock, P., Mehdad, Y., Shi, Y., Krishnamoorthi, R., & Chandra, V. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.26 · Cited by 165
Park, G., Park, B., Kim, M., Lee, S., Kim, J.-H., Kwon, B., Kwon, S. J., Kim, B., Lee, Y., & Lee, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.09557 · Cited by 158
Wang, X., Zhu, W., Saxon, M., Mark Steyvers, B., & Wang, W. Y. (2023) · neural information processing systems · doi:10.52202/075280-0687 · Cited by 151
Self-citationShortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining MethodsPreprint
Kim, B.-K., Kim, G., Kim, T., Castells, T., Choi, S., Shin, J., & Song, H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.02834 · Cited by 145
Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation
Yao, Z., Wu, X., Li, C., Youn, S., & He, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29908 · Cited by 142
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
Frantar, E., & Alistarh, D. (2023) · arXiv · doi:10.48550/arxiv.2301.00774 · Cited by 131
Xia, H., Zhen, Z., Li, Y., Zhuang, D., Zhou, Z., Qiu, X., Li, Y., Lin, W., & Song, S. L. (2023) · Proceedings of the VLDB Endowment · doi:10.14778/3626292.3626303 · Cited by 129
Teaching Small Language Models to Reason
Magister, L. C., Mallinson, J., Adamek, J., Malmi, E., & Severyn, A. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.151 · Cited by 127
A framework for few-shot language model evaluation
Sutawika, L., Gao, L., Schoelkopf, H., Biderman, S., Tow, J., Abbasi, B., fattori, B., Lovering, C., farzanehnakhaee70, Phang, J., et al. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.5371628 · Cited by 124
SCOTT: Self-Consistent Chain-of-Thought Distillation
Wang, P., Wang, Z., Li, Z., Gao, Y., Yin, B., & Ren, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.304 · Cited by 119
Does Knowledge Distillation Really Work?Preprint
Stanton, S., Izmailov, P., Kirichenko, P., Alemi, A. A., & Wilson, A. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05945 · Cited by 110
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
Wu, M., Waheed, A., Zhang, C., Abdul-Mageed, M., & Aji, A. F. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.57 · Cited by 110
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.-M., Wang, W.-C., Xiao, G., Dang, X., Gan, C., & Han, S. (2023) · arXiv · doi:10.48550/arxiv.2306.00978 · Cited by 103
Self-citationFluctuation-Based Adaptive Structured Pruning for Large Language Models
An, Y., Xu, Z., Yu, C., Tang, M., & Wang, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.28960 · Cited by 96
A Simple and Effective Pruning Approach for Large Language Models
Sun, M., Liu, Z., Bair, A., & Kolter, J. Z. (2023) · arXiv · doi:10.48550/arxiv.2306.11695 · Cited by 92
The LAMBADA dataset: Word prediction requiring a broad discourse context
Paperno, D., Kruszewski, G., Lazaridou, A., Pham, N. Q., Bernardi, R., Pezzelle, S., Baroni, M., Boleda, G., & Fernandez, R. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1144 · Cited by 87
Lion: Adversarial Distillation of Proprietary Large Language Models
Jiang, Y., Chan, C., Chen, M., & Wang, W. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.189 · Cited by 86
Self-citationWKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains MorePreprint
Yue, Y., Yuan, Z., Duanmu, H., Zhou, S., Wu, J., & Nie, L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.12065 · Cited by 86
OneBit: Towards Extremely Low-bit Large Language Models
Xu, Y., Han, X., Yang, Z., Wang, S., Zhu, Q., Liu, Z., Liu, W., & Che, W. (2024) · neural information processing systems · doi:10.52202/079017-2122 · Cited by 85
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation
Du, D., Zhang, Y., Cao, S., Guo, J., Cao, T., Chu, X., & Xu, N. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.7 · Cited by 80
Distilling Reasoning Capabilities into Smaller Language Models
Shridhar, K., Stolfo, A., & Sachan, M. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.441 · Cited by 79
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning
Frantar, E., & Alistarh, D. (2022) · neural information processing systems · doi:10.52202/068431-0323 · Cited by 79
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
Hooper, C., Kim, S., Mohammadzadeh, H., Mahoney, M., Shao, Y., Keutzer, K., & Gholami, A. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-0040 · Cited by 69
Learning Implicitly Recurrent CNNs Through Parameter SharingPreprint
Savarese, P., & Maire, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09701 · Cited by 66
Which *BERT? A Survey Organizing Contextualized Encoders
Xia, P., Wu, S., & Durme, B. V. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.608 · Cited by 57
Specializing Smaller Language Models towards Multi-Step ReasoningPreprint
Fu, Y., Peng, H. M., Ou, L., Sabharwal, A., & Khot, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.12726 · Cited by 52
LLM-FP4: 4-Bit Floating-Point Quantized Transformers
Liu, S.-Y., Liu, Z., Huang, X., Dong, P., & Cheng, K.-T. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.39 · Cited by 50
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMsPreprint
Zhang, Y., Zhao, L., Lin, M., Sun, Y., Yao, Y., Han, X., Tanner, J., Liu, S., & Ji, R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.08915 · Cited by 50
Huang, Y., Chen, Y., Yu, Z., & McKeown, K. (2022) · arXiv.org · doi:10.48550/arxiv.2212.10670 · Cited by 45
Zhu, X., Qi, B., Zhang, K., Long, X., Lin, Z., & Zhou, B. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.142 · Cited by 45
Self-citationOn the Impact of Calibration Data in Post-training Quantization and Pruning
Williams, M., & Αλέτρας, Ν. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.544 · Cited by 40
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression
Xu, C., Zhou, W., Ge, T., Xu, K., McAuley, J., & Wei, F. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.832 · Cited by 39
Explanations from Large Language Models Make Small Reasoners BetterPreprint
Li, S., Chen, J., Shen, Y., Chen, Z., Zhang, X., Li, Z., Wang, H., Jing, Q., Peng, B., Mao, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.06726 · Cited by 39
One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models
Shao, H., Liu, B., & Qian, Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445737 · Cited by 36
Less is More: Task-aware Layer-wise Distillation for Language Model CompressionPreprint
Chen, L., Zuo, S., Zhang, Q., He, P., Chen, W., & Zhao, T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.01351 · Cited by 32
E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M SparsityPreprint
Li, Y., Niu, L., Zhang, X., Liu, K., Zhu, J., & Kang, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.15929 · Cited by 30
L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models
Jeon, H., Kim, Y., & Kim, J. J. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.99 · Cited by 30
Democratizing Reasoning Ability: Tailored Learning from Large Language Model
Wang, Z., Huang, S., Liu, Y., Wang, J., Song, M., Zhang, Z., Huang, H., Wei, F., Deng, W., Sun, F., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.120 · Cited by 22
Self-citationSelective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
Li, M., Chen, L., Chen, J., He, S., Gu, J., & Zhou, T. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.958 · Cited by 22
Li, Y., Yuan, P., Feng, S., Pan, B., Sun, B., Wang, X. X., Wang, H., & Li, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29821 · Cited by 20
PiQA: an algebra for querying protein data sets
Tata, S., & Patel, J. M. (2003) · International Conference on Scientific and Statistical Database Management · doi:10.1109/ssdm.2003.1214975 · Cited by 20
Matrix Compression via Randomized Low Rank and Low Precision Factorization
Saha, R., Srivastava, V., & Pilancı, M. (2023) · neural information processing systems · doi:10.52202/075280-0826 · Cited by 17
KIVI : Plug-and-play 2bit KV Cache Quantization with Streaming Asymmetric Quantization
Liu, Z., Yuan, J., Jin, H., Zhong, S., Xu, Z., Braverman, V., Chen, B., & Hu, X. (2023) · Unpublished · doi:10.13140/rg.2.2.28167.37282 · Cited by 6
ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
Yuan, Z., Shang, Y., Song, Y., Yang, D., Wu, Q., Yan, Y., & Sun, G. (2023) · arXiv · doi:10.48550/arxiv.2312.05821 · Cited by 4
OPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Workshop, B., :, Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A. S., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.05100
Flagged first, then by citations
10 of the 114 references couldn’t be matched to a known paper.
Research works printed but not linked: 13
- Agarwal, 2024, Generalized knowledge distillation for auto-regressive language models
- Chee, 2023, QuIP: 2-bit quantization of large language models with guarantees
- Dettmers, 2022, Gpt3.int8(): 8-bit matrix multiplication for transformers at scale
- Frantar, 2023, OPTQ: Accurate quantization for generative pre-trained transformers
- Yuxian, 2024, MiniLLM: Knowledge distillation of large language models
- Han, 2016, Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding
- Han, 2015, Learning both weights and connections for efficient neural network
- Kim, 2023, Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization
- Liu, 2024, Learning to reason with autoregressive in-context distillation
- Liu, 2024, KIVI: A tuning-free asymmetric 2bit quantization for KV cache, CoRR
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI blog, 1, 9
- Sharma, 2024, The truth is in there: Improving reasoning with layer-selective rank reduction
- Williams, 2023, How does calibration data affect the post-training pruning and quantization of large language models?, CoRR
Cited by
Sorted by citations
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257
A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations
Cheng, H., Zhang, M., & Shi, Q. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3447085 · Cited by 667
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335
Resource-efficient Algorithms and Systems of Foundation Models: A Survey
Xu, M., Cai, D., Yin, W., Wang, S., Jin, X., & Liu, X. (2024) · ACM Computing Surveys · doi:10.1145/3706418 · Cited by 255
Metadata
- Authors
- Zhu, X., Li, J., Liu, Y., Ma, C., & Wang, W.
- Affiliations
- Chinese Academy of Sciences (CN)
- Beijing Normal University (CN)
- Renmin University of China (CN)
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 12
- Pages
- 1556-1577
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W4404918643
- Semantic Scholar
- 338d8f3b199abcebc85f34016b0162ab3a9d5310
- Cited by
- 273
- References
- 114
- Citations by year
- 2026: 99 · 2025: 123 · 2024: 45
- FWCI
- 37.71
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62076234, 62106257)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- model compression, large language models, quantization, pruning, knowledge distillation, resource-constrained deployment, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
