Diet code is healthy: simplifying programs for pre-trained models of code
Open accessAbstract
Pre-trained code representation models such as CodeBERT have demonstrated superior performance in a variety of software engineering tasks, yet they are often heavy in complexity, quadratically with the length of the input sequence. Our empirical analysis of CodeBERT's attention reveals that CodeBERT pays more attention to certain types of tokens and statements such as keywords and data-relevant statements. Based on these findings, we propose DietCode, which aims at lightweight leverage of large pre-trained models for source code. DietCode simplifies the input program of CodeBERT with three strategies, namely, word dropout, frequency filtering, and an attention-based strategy that selects statements and tokens that receive the most attention weights during pre-training. Hence, it gives a substantial reduction in the computational cost without hampering the model performance. Experimental results on two downstream tasks show that DietCode provides comparable results to CodeBERT with 40% less computational cost in fine-tuning and testing.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Traditional code compressors such as DietCode and SlimCode improve syntax and structure awareness but are generally limited to function-level pruning or short code examples, leaving compression of long context for code largely unaddressed.
“Traditional code compressors such as DietCode [26] and SlimCode [27] improve syntax and structure awareness but are generally limited to function-level pruning or short code examples, leaving compression of long context for code largely unaddressed.”
From LongCodeZip: Compress Long Context for Code Language Models (2025) · cites this paper as [26]
The Introduction (p. 1) and Datasets section (p. 7) describe DietCode pruning tokens and statements from input code, with experiments on function-level code. But the Implications for Future Research section (p. 10) says CodeBERT does not recognize grammar and structural information well and treats better incorporation of code structure as future work; it does not establish the claimed improvement in syntax and structure awareness. The paper motivates pruning partly by the cost of long sequences (p. 5), but does not assess long-context code compression as a field-wide gap. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · see citing works
The Java Language Specification
Gosling, J., Joy, B., & Steele, G. L. (1996) · Cited by 3,915 · see citing works
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.139 · Cited by 2,824 · see citing works
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020) · arXiv · doi:10.48550/arxiv.2003.10555 · Cited by 1,844 · see citing works
CCFinder: a multilinguistic token-based code clone detection system for large scale source code
Kamiya, T., Kusumoto, S., & Inoue, K. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1019480 · Cited by 1,715 · see citing works
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586 · see citing works
A primer in BERTology: What we know about how BERT works
Rogers, A., Kovaleva, O., & Rumshisky, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00349 · Cited by 1,552 · see citing works
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495 · see citing works
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · arXiv · doi:10.48550/arxiv.2109.00859 · Cited by 1,495 · see citing works
code2vec: learning distributed representations of code
Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3290353 · Cited by 1,235 · see citing works
Simplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,084 · see citing works
Learning to Represent Programs with GraphsPreprint
Allamanis, M., Brockschmidt, M., & Khademi, M. (2017) · International Conference on Learning Representations · doi:10.48550/arxiv.1711.00740 · Cited by 947 · see citing works
A Survey of Model Compression and Acceleration for Deep Neural Networks
Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2017) · arXiv · doi:10.48550/arxiv.1710.09282 · Cited by 857 · see citing works
Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765695 · Cited by 817 · see citing works
Deep Unordered Composition Rivals Syntactic Methods for Text Classification
Iyyer, M., Manjunatha, V., Boyd-Graber, J. L., & Daumé, H. (2015) · doi:10.3115/v1/p15-1162 · Cited by 729 · see citing works
A Novel Neural Source Code Representation Based on Abstract Syntax Tree
Zhang, J., Wang, X., Zhang, H., Sun, H., Wang, K., & Liu, X. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00086 · Cited by 710 · see citing works
Self-citationUnified Pre-training for Program Understanding and Generation
Ahmad, W., Chakraborty, S., Ray, B., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.211 · Cited by 642 · see citing works
Deep learning code fragments for code clone detection
White, M., Tufano, M., Vendome, C., & Poshyvanyk, D. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970326 · Cited by 613 · see citing works
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536 · see citing works
CodeSearchNet Challenge: Evaluating the State of Semantic Code Search
Husain, H., Wu, H.-H., Gazit, T., Allamanis, M., & Brockschmidt, M. (2019) · arXiv · doi:10.48550/arxiv.1909.09436 · Cited by 483 · see citing works
Algorithms for Knapsack Problems
Martello, S., & Toth, P. (1987) · North-Holland mathematics studies · doi:10.1016/s0304-0208(08)73237-7 · Cited by 468 · see citing works
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
Shen, S., Dong, Z., Ye, J., Ma, L., Yao, Z., Gholami, A., Mahoney, M. W., & Keutzer, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6409 · Cited by 437 · see citing works
On the naturalness of software
Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Devanbu, P. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227135 · Cited by 425 · see citing works
0897915046/International Conference on Software Engineering
(1992) · doi:10.1145/143062 · Cited by 374 · see citing works
Supporting program comprehension with source code summarization
Haiduc, S., Aponte, J., & Marcus, A. (2010) · Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2 · doi:10.1145/1810295.1810335 · Cited by 291 · see citing works
Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks
Mastropaolo, A., Scalabrino, S., Cooper, N., Palacio, D. N., Poshyvanyk, D., Oliveto, R., & Bavota, G. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00041 · Cited by 271 · see citing works
Automatically generating commit messages from diffs using neural machine translation
Jiang, S., Armaly, A., & McMillan, C. (2017) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115626 · Cited by 267 · see citing works
Improving automated source code summarization via an eye-tracking study of programmers
Rodeghero, P., McMillan, C., McBurney, P. W., Bosch, N., & D'Mello, S. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568247 · Cited by 198 · see citing works
Query Expansion Based on Crowd Knowledge for Code Search
Nie, L., Jiang, H., Ren, Z., Sun, Z., & Li, X. (2016) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2016.2560165 · Cited by 148 · see citing works
Learned Token Pruning for Transformers
Kim, S., Shen, S., Thorsley, D., Gholami, A., Kwon, W., Hassoun, J., & Keutzer, K. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539260 · Cited by 139 · see citing works
Solving the Search for Source Code
Stolee, K. T., Elbaum, S. G., & Dobos, D. (2014) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2581377 · Cited by 122 · see citing works
NatGen: generative pre-training by “naturalizing” source code
Chakraborty, S., Ahmed, T., Ding, Y., Devanbu, P. T., & Ray, B. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549162 · Cited by 105 · see citing works
An Empirical Study on the Usage of BERT Models for Code Completion
Ciniselli, M., Cooper, N., Pascarella, L., Poshyvanyk, D., Penta, M. D., & Bavota, G. (2021) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00024 · Cited by 93 · see citing works
What do pre-trained code models know about code?
Karmakar, A., & Robbes, R. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678927 · Cited by 80 · see citing works
TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference
Ye, D., Lin, Y., Huang, Y., & Sun, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.463 · Cited by 56 · see citing works
Understanding neural code intelligence through program simplification
Rabin, M. R. I., Hellendoorn, V. J., & Alipour, M. A. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468539 · Cited by 38 · see citing works
Probing model signal-awareness via prediction-preserving input minimization
Suneja, S., Zheng, Y., Zhuang, Y., Laredo, J., & Morari, A. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468545 · Cited by 31 · see citing works
AutoFocus: Interpreting Attention-Based Neural Networks by Code Perturbation
Bui, N. D. Q., Yu, Y., & Jiang, L. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00014 · Cited by 29 · see citing works
Thinking Like a Developer? Comparing the Attention of Humans with Neural Models of Code
Paltenghi, M., & Pradel, M. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678712 · Cited by 20 · see citing works
What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source CodePreprint
Wan, Y., Zhao, W., Zhang, H., Sui, Y., Xu, G., & Jin, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.06840 · Cited by 15 · see citing works
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Research works printed but not linked: 3
- [6][6] Casey Casalnuovo, E Morgan, and P Devanbu. 2020. Does surprisal predict code comprehension difficulty. In Proceedings of the 42nd Annual Meeting of the Cognitive Science Society . Cognitive Science Society Toronto, Canada. -
- [35][35] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9. -
- [49][49] Andreas Zeller and Ralf Hildebrandt. 2002. Simplifying and isolating failureinducing input. IEEE Transactions on Software Engineering 28, 2 (2002), 183-200. -
Cited by
Sorted by citations
Towards an understanding of large language models in software engineering tasks
Zheng, Z., Ning, K., Zhong, Q., Chen, J., Chen, W., Guo, L., Wang, W., & Wang, Y. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10602-0 · Cited by 149 · see citing works
NatGen: generative pre-training by “naturalizing” source code
Chakraborty, S., Ahmed, T., Ding, Y., Devanbu, P. T., & Ray, B. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549162 · Cited by 105 · see citing works
An Empirical Comparison of Pre-Trained Models of Source Code
Niu, C., Li, C., Ng, V., Chen, D., Ge, J., & Luo, B. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00180 · Cited by 90 · see citing works
CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models
Jha, A., & Reddy, C. K. A. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26739 · Cited by 69 · see citing works
CCTEST: Testing and Repairing Code Completion Systems
Li, Z., Wang, C., Liu, Z., Wang, H., Chen, D., Wang, S., & Gao, C. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00110 · Cited by 62 · see citing works
Shi, J., Yang, Z., & Lo, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708525 · Cited by 43 · see citing works
Greening Large Language Models of Code
Shi, J., Yang, Z., Kang, H. J., Xu, B., He, J., & Lo, D. (2024) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1145/3639475.3640097 · Cited by 35 · see citing works
On the Reliability and Explainability of Language Models for Program Generation
Liu, Y., Tantithamthavorn, C. K., Liu, Y., & Li, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3641540 · Cited by 37 · see citing works
Project-specific code summarization with in-context learning
Yun, S. M., LIN, S.-J., Gu, X., & Shen, B. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112149 · Cited by 29 · see citing works
Self-citationTowards Understanding the Characteristics of Code Generation Errors Made by Large Language Models
Wang, Z., Zhou, Z., Song, D., Huang, Y., Chen, S., Ma, L., & Zhang, T. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00180 · Cited by 40 · see citing works
Li, J.-L., Zhang, Y.-F., Karas, Z., McMillan, C., Leach, K., & Huang, Y. (2024) · IEEE International Conference on Program Comprehension · doi:10.1145/3643916.3644434 · Cited by 37 · see citing works
Learning-based models for vulnerability detection: an extensive study
Ni, C., Yin, X., Shen, L., & Wang, S. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10734-x · Cited by 15 · see citing works
AI Coders Are among Us: Rethinking Programming Language Grammar towards Efficient Code Generation
Sun, Z., Du, X., Yang, Z., Li, L., & Lo, D. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680347 · Cited by 14 · see citing works
Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey
She, X., Liu, Y., Zhao, Y., He, Y., Li, L., Tantithamthavorn, C. K., Qin, Z., & Wang, H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3748647 · Cited by 23 · see citing works
From Triumph to Uncertainty: The Journey of Software Engineering in the AI Era
Mastropaolo, A., Escobar‐Velásquez, C., & Linares‐Vásquez, M. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709360 · Cited by 19 · see citing works
On-the-fly Improving Performance of Deep Code Models via Input Denoising
Tian, Z., Chen, J., & Zhang, X. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00166 · Cited by 11 · see citing works
How Effectively Do Code Language Models Understand Poor-Readability Code?
Hu, C., Chai, Y., Zhou, H., Meng, F., Zhou, J., & Gu, X. (2024) · Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3691620.3695072 · Cited by 6 · see citing works
Self-citationRepoMinCoder: Improving Repository-Level Code Generation Based on Information Loss Screening
Li, Y., Shi, E., Zheng, D., Duan, K., Chen, J., & Wang, Y. (2024) · Asia-Pacific Symposium on Internetware · doi:10.1145/3671016.3674819 · Cited by 8 · see citing works
Liu, C., Cai, Y., Lin, Y., Huang, Y.-H., Pei, Y., Jiang, B., Yang, P., Dong, J. S., & Mei, H. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652142 · Cited by 8 · see citing works
BinAug: Enhancing Binary Similarity Analysis with Low-Cost Input Repairing
Wong, W. K., Wang, H., Li, Z., & Wang, S. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623328 · Cited by 7 · see citing works
A Catalog of Data Smells for Coding Tasks
Vitale, A., Oliveto, R., & Scalabrino, S. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3707457 · Cited by 7 · see citing works
Natural Is the Best: Model-Agnostic Code Simplification for Pre-trained Large Language Models
Wang, Y., Li, X., Nguyen, T. N., Wang, S., Ni, C., & Ding, L. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3643753 · Cited by 6 · see citing works
Probing Numeracy and Logic of Language Models of Code
Baltaji, R., & Thakkar, P. (2023) · doi:10.1109/intense59150.2023.00006 · Cited by 5 · see citing works
Occlusion-Based Detection of Trojan-Triggering Inputs in Large Language Models of CodePreprint
Hussain, A., Rabin, M. R. I., Ahmed, T., Alipour, M. A., & Xu, B. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4798300 · Cited by 7 · see citing works
LongCodeZip: Compress Long Context for Code Language Models
Shi, Y., Qian, Y., Zhang, H., Shen, B., & Gu, X. (2025) · 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00020 · Cited by 7 · see citing works
“5) Code Compression Methods : We compare against the compression components from DietCode [26] and Slim-Code [27].”
Self-citationAn Adaptive Language-Agnostic Pruning Method for Greener Language Models for Code
Saad, M., López, J. A. H., Chen, B., Varró, D., & Sharma, T. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715773 · Cited by 6 · see citing works
Structural-semantics Guided Program Simplification for Understanding Neural Code Intelligence Models
Shi, C., Zhu, T., Zhang, T., Pang, J., & Pan, M. (2023) · Asia-Pacific Symposium on Internetware · doi:10.1145/3609437.3609438 · Cited by 4 · see citing works
Robustness-Enhanced Assertion Generation Method Based on Code Mutation and Attack Defense
Li, M., Chen, S., Fan, G., Zhang, L., Wu, H., Xue, X., & Feng, Z. (2024) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-54528-3_16 · Cited by 2 · see citing works
An Empirical Study of Code Simplification Methods in Code Intelligence Tasks
Shen, Z., Li, Y., Ge, J., Chen, X., Li, C., Huang, L., & Luo, B. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3720540 · Cited by 3 · see citing works
Study of Distractors in Neural Models of Code
Rabin, M. R. I., Hussain, A., Suneja, S., & Alipour, M. A. (2023) · doi:10.1109/intense59150.2023.00005 · Cited by 3 · see citing works
FeatureExtractor: A tool for extracting key input features of code intelligence models
Rabin, M. R. I., & Alipour, M. A. (2022) · Software Impacts · doi:10.1016/j.simpa.2022.100432 · Cited by 2 · see citing works
SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents
Wang, Y., Shi, Y., Yang, M., Zhang, R., He, S., Lian, H., Chen, Y., Ye, S., Cai, K., & Gu, X. (2026) · arXiv · doi:10.48550/arxiv.2601.16746 · Cited by 2 · see citing works
“Code-specific methods including DietCode (Zhang et al., 2022), SlimCode (Wang et al., 2024b), LongCodeZip (Shi et al., 2025), and CodeOCR (Shi et al., 2026) address structural concerns but are primarily evaluated on single-round proxy tasks (e.g., code search, completion) rather than multi-round…”
Self-citationLongCodeZip: Compress Long Context for Code Language Models
Shi, Y., Qian, Y., Zhang, H., Shen, B., & Gu, X. (2025) · arXiv · Cited by 7 · see citing works
“5) Code Compression Methods : We compare against the compression components from DietCode [26] and Slim-Code [27].”
Self-citationDe-duplicating Silent Compiler Bugs via Deep Semantic Representation
Chen, J., Fan, X., Yang, C., Liu, S., & Sun, J. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729375 · Cited by 1 · see citing works
CodeMark : Contextual and Natural Watermarking for Tracing Code Snippet Provenance
Li, W., Yang, B., Sun, Y., Chen, S., Chen, Y., & Xiang, L. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3594599 · Cited by 1 · see citing works
CoSTV: Accelerating Code Search with Two-Stage Paradigm and Vector Retrieval
Zheng, D., Wang, Y., Chen, W., Chen, J., & Zheng, Z. (2024) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec65559.2024.00049 · Cited by 1 · see citing works
Token Sugar: Making Source Code Sweeter for LLMs through Token-Efficient Shorthand
Sun, Z., Yang, C., Du, X., Yang, Z., Li, L., & Lo, D. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00201
Rubei, R., & Ruscio, D. D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6864772
Yang, Y., Fan, H., Lin, C., Jiang, W., Wang, S., Li, Q., Zhao, Z., Zhang, C., Guan, X., & Shen, C. (2026) · Science China Information Sciences · doi:10.1007/s11432-025-4703-6
Compiling Code LLMs into Lightweight Executables
Shi, J., He, J., Yang, Z., Yang, C., Klymenko, M. V., Hoang, T., Xu, X., Xing, Z., & Lo, D. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808196
Concord: A DSL for Generating Simplified and Scalable Graph-Based Code Representations
Saad, M., & Sharma, T. (2026) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner67736.2026.00067
BinRAG: An RAG-Based Decompilation Framework Fusing Name Prediction and Calling Context
Wong, W. K., Wu, D., Liu, Z., Wang, H., Li, Z., & Wang, S. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832245
- Authors
- Zhang, Z., Zhang, H., Shen, B., & Gu, X.
- Affiliations
- Shanghai Jiao Tong University (CN)
- University of Newcastle, Australia
- Venue
- Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2022-11-07
- Pages
- 1073-1084
- OpenAlex
- W4283751459
- Cited by
- 41
- References
- 50
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 18 · 2023: 7 · 2022: 2
- FWCI
- 8.05
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- NOVA (University of Newcastle Australia) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- NSF (National Science Foundation) (62102244)
- CCF-Baidu Open Fund (2021PP15002000)
- CCF-Tencent Rhino-Bird Young Faculty Open Research Fund
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- CodeBERT, attention analysis, program simplification, frequency filtering, computational complexity reduction, fine-tuning efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
