Large Language Models for Software Engineering: A Systematic Literature Review
The DOI matches its Crossref record; 321 of the 321 references it cites were found in a registry or an index.
Abstract
Large Language Models (LLMs) have significantly impacted numerous domains, including Software Engineering (SE). Many recent publications have explored LLMs applied to various SE tasks. Nevertheless, a comprehensive understanding of the application, effects, and possible limitations of LLMs on SE is still in its early stages. To bridge this gap, we conducted a Systematic Literature Review (SLR) on LLM4SE, with a particular focus on understanding how LLMs can be exploited to optimize processes and outcomes. We selected and analyzed 395 research articles from January 2017 to January 2024 to answer four key Research Questions (RQs). In RQ1, we categorize different LLMs that have been employed in SE tasks, characterizing their distinctive features and uses. In RQ2, we analyze the methods used in data collection, pre-processing, and application, highlighting the role of well-curated datasets for successful LLM for SE implementation. RQ3 investigates the strategies employed to optimize and evaluate the performance of LLMs in SE. Finally, RQ4 examines the specific SE tasks where LLMs have shown success to date, illustrating their practical contributions to the field. From the answers to these RQs, we discuss the current state-of-the-art and trends, identifying gaps in existing research, and highlighting promising areas for future study. Our artifacts are publicly available at https://github.com/security-pride/LLM4SE_SLR .
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To identify relevant benchmarks, we leverage a recent and comprehensive survey on LLMs for SE tasks [33], which analyzed 395 research papers from January 2017 to January 2024.”
From LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarks (2025) · cites this paper as [33]
In the citing paper
“SWE-Bench is a benchmark designed to assess the ability of LLMs to solve real-world software engineering tasks collected from 12 real-world GitHub repositories.”
From Context as a Tool: Context Management for Long-Horizon SWE-Agents (2025) · cites this paper as Hou et al., 2024
In the citing paper
“The Verified split is a high-quality subset of SWE-Bench consisting of 500 instances, which are manually curated to provide clearer problem descriptions and more reliable evaluation criteria.”
From Context as a Tool: Context Management for Long-Horizon SWE-Agents (2025) · cites this paper as Hou et al., 2024
In the citing paper
“It is increasingly acting as a community contributor-submitting issues, pull requests, code reviews, and security reports [14, 24, 73].”
From To Ban or Not to Ban? How Open Source Projects Govern GenAI Contributions (2026) · cites this paper as [24]
In the citing paper
“However, LLMs struggle with code-related tasks, particularly in terms of correctness, code quality, and other context-specific limitations [11].”
From Same Scrutiny, More Time: Eye Tracking Insights into Reviewing LLM-Labelled Code (2026) · cites this paper as [11]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254 · cited 1×
“Simultaneously, researchers have embarked on a series of research activities regarding LLMrelated works, where a number of literature reviews or survey papers have been produced [29, 58, 59, 338].”
BackgroundTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.139 · Cited by 2,824
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context LearningPreprint
Liu, H., Tam, D., Muqeeth, M., Mohta, J., Huang, T., Bansal, M., & Raffel, C. A. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2205.05638 · Cited by 1,578
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · arXiv · doi:10.48550/arxiv.2109.00859 · Cited by 1,495
Unifying Large Language Models and Knowledge Graphs: A Roadmap
Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., & Wu, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3352100 · Cited by 1,376
Clone detection using abstract syntax trees
Baxter, I. D., Yahin, A., Moura, L., Sant'Anna, M., & Bier, L. (2002) · Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272) · doi:10.1109/icsm.1998.738528 · Cited by 1,277
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.15556 · Cited by 929
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023) · arXiv · doi:10.48550/arxiv.2302.11382 · Cited by 838
Allen, F. E. (1970) · ACM SIGPLAN Notices · doi:10.1145/390013.808479 · Cited by 738
Summarizing Source Code using a Neural Attention Model
Iyer, S., Konstas, I., Cheung, A., & Zettlemoyer, L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1195 · Cited by 701
Ko, A. J., Myers, B. A., Coblenz, M. J., & Aung, H. H. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.116 · Cited by 685
Alur, R., Bodik, R., Juniwal, G., Martin, M. M. K., Raghothaman, M., Seshia, S. A., Singh, R., Solar-Lezama, A., Torlak, E., & Udupa, A. (2013) · 2013 Formal Methods in Computer-Aided Design · doi:10.1109/fmcad.2013.6679385 · Cited by 656
A Novel Neural Source Code Representation Based on Abstract Syntax Tree
Zhang, J., Wang, X., Zhang, H., Sun, H., Wang, K., & Liu, X. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00086 · Cited by 655
Self-citationLarge language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Yin, B., & Hu, X. (2023) · arXiv · doi:10.48550/arxiv.2304.13712 · Cited by 644
Unified Pre-training for Program Understanding and Generation
Ahmad, W., Chakraborty, S., Ray, B., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.211 · Cited by 642
A Syntactic Neural Model for General-Purpose Code Generation
Yin, P., & Neubig, G. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1041 · Cited by 623
Hu, X., Li, G., Xia, X., Lo, D., & Jin, Z. (2018) · Proceedings of the 26th Conference on Program Comprehension · doi:10.1145/3196321.3196334 · Cited by 621
Self-citationDefending ChatGPT against jailbreak attack via self-reminders
Xie, Y., Yi, J., Shao, J., Curl, J., Lyu, L., Chen, Q., Xie, X., & Wu, F. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00765-8 · Cited by 606
Galactica: A Large Language Model for Science
Taylor, R., Kardas, M., Cucurull, G., Scialom, T., Hartshorn, A., Saravia, E., Poulton, A., Kerkez, V., & Stojnic, R. (2022) · arXiv · doi:10.48550/arxiv.2211.09085 · Cited by 605
A Syntactic Neural Model for General-Purpose Code Generation
Yin, P., & Neubig, G. (2017) · arXiv · doi:10.48550/arxiv.1704.01696 · Cited by 585
A Deductive Approach to Program Synthesis
Manna, Z., & Waldinger, R. (1980) · ACM Transactions on Programming Languages and Systems · doi:10.1145/357084.357090 · Cited by 573
Software Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X
Zheng, Q., Xia, X., Zou, X., Dong, Y., Wang, S., Xue, Y., Shen, L., Wang, Z., Wang, A., Li, Y., et al. (2023) · Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3580305.3599790 · Cited by 563
Self-citationIdentifying relevant studies in software engineering
Zhang, H., Babar, M. A., & Tell, P. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.010 · Cited by 557
Code completion with statistical language models
Raychev, V., Vechev, M., & Yahav, E. (2014) · Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2594291.2594321 · Cited by 555
Gu, X., Zhang, H., & Kim, S. (2018) · Proceedings of the 40th International Conference on Software Engineering · doi:10.1145/3180155.3180167 · Cited by 547
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536
Large Language Models for Software Engineering: Survey and Open Problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · arXiv · doi:10.48550/arxiv.2310.03533 · Cited by 523
Gu, X., Zhang, H., Zhang, D., & Kim, S. (2016) · Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2950290.2950334 · Cited by 485
Recommender Systems in the Era of Large Language Models (LLMs)
Zhao, Z., Fan, W., Li, J., Liu, Y., Mei, X., Wang, Y., Wen, Z., Wang, F., Zhao, X., Tang, J., et al. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3392335 · Cited by 481
Self-citationPTR: Prompt Tuning with Rules for Text Classification
Han, X., Zhao, W., Ding, N., Liu, Z., & Sun, M. (2021) · arXiv · doi:10.48550/arxiv.2105.11259 · Cited by 478
CodeT5+: Open Code Large Language Models for Code Understanding and Generation
Wang, Y., Le, H., Gotmare, A., Bui, N., Li, J., & Hoi, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.68 · Cited by 459
CodeT5+: Open Code Large Language Models for Code Understanding and Generation
Wang, Y., Le, H., Gotmare, A. D., Bui, N. D. Q., Li, J., & Hoi, S. C. H. (2023) · arXiv · doi:10.48550/arxiv.2305.07922 · Cited by 459
Automated Program Repair in the Era of Large Pre-trained Language Models
Xia, C. S., Wei, Y., & Zhang, L. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00129 · Cited by 433
An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation
Schäfer, M., Nadi, S., Eghbali, A., & Tip, F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3334955 · Cited by 431
MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots
Deng, G., Liu, Y., Li, Y., Wang, K., Zhang, Y., Li, Z., Wang, H., Zhang, T., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2307.08715 · Cited by 418
Self-citationGPT-NeoX-20B: An Open-Source Autoregressive Language Model
Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022) · Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models · doi:10.18653/v1/2022.bigscience-1.9 · Cited by 404
MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation
Cassano, F., Gouwar, J., Nguyen, D., Nguyen, S., Phipps-Costin, L., Pinckney, D., Yee, M.-H., Zi, Y., Anderson, C. J., Feldman, M. Q., et al. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3267446 · Cited by 394
Improving automatic source code summarization via deep reinforcement learning
Wan, Y., Zhao, Z., Yang, M., Xu, G., Ying, H., Wu, J., & Yu, P. S. (2018) · Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering · doi:10.1145/3238147.3238206 · Cited by 385
What Makes APIs Hard to Learn? Answers from Developers
Robillard, M. P. (2009) · IEEE Software · doi:10.1109/ms.2009.193 · Cited by 382
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
Liu, H., Tam, D., Muqeeth, M., Mohta, J., Huang, T., Bansal, M., & Raffel, C. (2022) · neural information processing systems · doi:10.52202/068431-0142 · Cited by 377
0897915046/International Conference on Software Engineering
(1992) · doi:10.1145/143062 · Cited by 374
A field study of API learning obstacles
Robillard, M. P., & DeLine, R. (2011) · Empirical Software Engineering · doi:10.1007/s10664-010-9150-8 · Cited by 352
Static analysis of android apps: A systematic literature review
Li, L., Bissyandé, T. F., Papadakis, M., Rasthofer, S., Bartel, A., Octeau, D., Klein, J., & Traon, L. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2017.04.001 · Cited by 350
Self-citationTowards a Big Data Curated Benchmark of Inter-project Code Clones
Svajlenko, J., Islam, J. F., Keivanloo, I., Roy, C. K., & Mia, M. M. (2014) · 2014 IEEE International Conference on Software Maintenance and Evolution · doi:10.1109/icsme.2014.77 · Cited by 339
An Analysis of the Automatic Bug Fixing Performance of ChatGPT
Sobania, D., Briesch, M., Hanna, C., & Petke, J. (2023) · 2023 IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1109/apr59189.2023.00012 · Cited by 338
Proceedings of the 44th International Conference on Software Engineering
(2022) · doi:10.1145/3510003 · Cited by 326
Deng, Y., Xia, C. S., Peng, H., Yang, C., & Zhang, L. (2023) · Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3597926.3598067 · Cited by 326
GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow
Black, S., Leo, G., Wang, P., Leahy, C., & Biderman, S. (2021) · Zenodo · doi:10.5281/zenodo.5297715 · Cited by 324
On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilot
Mastropaolo, A., Pascarella, L., Guglielmi, E., Ciniselli, M., Scalabrino, S., Oliveto, R., & Bavota, G. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00181 · Cited by 296
White, J., Hays, S., Fu, Q., Spencer-Smith, J., & Schmidt, D. C. (2024) · Generative AI for Effective Software Development · doi:10.1007/978-3-031-55642-5_4 · Cited by 295
A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
Sahoo, P., Singh, A. K., Saha, S., Jain, V., Mondal, S., & Chadha, A. (2024) · arXiv · doi:10.48550/arxiv.2402.07927 · Cited by 288
Pierce, B. C., & Turner, D. N. (2000) · ACM Transactions on Programming Languages and Systems · doi:10.1145/345099.345100 · Cited by 287
Wei, H., & Li, M. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/423 · Cited by 280
Retrieval-based neural source code summarization
Zhang, J., Wang, X., Zhang, H., Sun, H., & Liu, X. (2020) · Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering · doi:10.1145/3377811.3380383 · Cited by 275
Risks and Benefits of Large Language Models for the Environment
Rillig, M. C., Ågerstrand, M., Bi, M., Gould, K. A., & Sauerland, U. (2023) · Environmental Science & Technology · doi:10.1021/acs.est.3c01106 · Cited by 270
From program verification to program synthesis
Srivastava, S., Gulwani, S., & Foster, J. S. (2010) · Proceedings of the 37th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages · doi:10.1145/1706299.1706337 · Cited by 267
Proceedings of the 11th Working Conference on Mining Software Repositories
(2014) · doi:10.1145/2597073 · Cited by 264
Self-collaboration code generation via ChatGPT
Dong, Y., Jiang, X., Jin, Z., & Li, G. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3672459 · Cited by 256
Self-collaboration Code Generation via ChatGPT
Dong, Y., Jiang, X., Jin, Z., & Li, G. (2023) · arXiv · doi:10.48550/arxiv.2304.07590 · Cited by 256
Sentiment analysis of commit comments in GitHub: an empirical study
Guzman, E., Azócar, D., & Li, Y. (2014) · Proceedings of the 11th Working Conference on Mining Software Repositories · doi:10.1145/2597073.2597118 · Cited by 253
Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks
Mastropaolo, A., Scalabrino, S., Cooper, N., Nader Palacio, D., Poshyvanyk, D., Oliveto, R., & Bavota, G. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00041 · Cited by 247
DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection
Chen, Y., Ding, Z., Alowain, L., Chen, X., & Wagner, D. (2023) · Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses · doi:10.1145/3607199.3607242 · Cited by 238
ChatGPT is a Remarkable Tool—For Experts
Azaria, A., Azoulay, R., & Reches, S. (2023) · Data Intelligence · doi:10.1162/dint_a_00235 · Cited by 237
Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Study
Guo, Q., Cao, J., Xie, X., Liu, S., Li, X., Chen, B., & Peng, X. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623306 · Cited by 235
Proceedings of 1993 15th International Conference on Software Engineering
(1993) · doi:10.1109/icse.1993 · Cited by 234
Gorilla: Large Language Model Connected with Massive APIs
Patil, S., Zhang, T., Wang, X., & Gonzalez, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-4020 · Cited by 226
A survey on deep learning for software engineering
Yang, Y., Xia, X., Lo, D., & Grundy, J. (2022) · ACM Computing Surveys · doi:10.1145/3505243 · Cited by 223 · cited 2×
“[309] 2022 DL General SE scope ✓ 2015-2020 250”
BackgroundSelf-citationNatural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review
Wong, M.-F., Guo, S., Hang, C. N., Ho, S., & Tan, C. W. (2023) · Entropy · doi:10.3390/e25060888 · Cited by 223
Self-Edit: Fault-Aware Code Editor for Code Generation
Zhang, K., Li, Z., Li, J., Li, G., & Jin, Z. G. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.45 · Cited by 220
SEGRESS: Software Engineering Guidelines for REporting Secondary Studies
Kitchenham, B., Madeyski, L., & Budgen, D. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3174092 · Cited by 219
How Effective Are Neural Networks for Fixing Security Vulnerabilities
Wu, Y., Jiang, N., Pham, H. V., Lutellier, T., Davis, J., Tan, L., Babkin, P., & Shah, S. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598135 · Cited by 218
Geng, M., Wang, S., Dong, D., Wang, H., Li, G., Jin, Z. G., Mao, X., & Liao, X. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3608134 · Cited by 209
Self-citationAPI method recommendation without worrying about the task-API knowledge gap
Huang, Q., Xia, X., Xing, Z., Lo, D., & Wang, X. (2018) · Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering · doi:10.1145/3238147.3238191 · Cited by 192
Self-citationLanguage Models of Code are Few-Shot Commonsense Learners
Madaan, A., Zhou, S., Alon, U., Yang, Y., & Neubig, G. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.90 · Cited by 191
Examining Zero-Shot Vulnerability Repair with Large Language Models
Pearce, H. A., Tan, B., Ahmad, B., Karri, R., & Dolan-Gavitt, B. (2023) · Proceedings - IEEE Symposium on Security and Privacy/Proceedings of the ... IEEE Symposium on Security and Privacy · doi:10.1109/sp46215.2023.10179420 · Cited by 189
Multi-task Learning based Pre-trained Language Model for Code Completion
Liu, F., Li, G., Zhao, Y., & Jin, Z. (2020) · Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3324884.3416591 · Cited by 188
Self-citationHellendoorn, V. J., Bird, C., Barr, E. T., & Allamanis, M. (2018) · Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3236024.3236051 · Cited by 181
Quality Assessment in Systematic Literature Reviews: A Software Engineering Perspective
Yang, L., Zhang, H., Shen, H., Huang, X., Zhou, X., Rong, G., & Shao, D. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106397 · Cited by 181
RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair
Wang, W., Wang, Y., Joty, S., & Hoi, S. C. H. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616256 · Cited by 180
How do professional developers comprehend software?
Roehm, T., Tiarks, R., Koschke, R., & Maalej, W. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227188 · Cited by 178
Using pre-trained models to boost code review automation
Tufano, R., Masiero, S., Mastropaolo, A., Pascarella, L., Poshyvanyk, D., & Bavota, G. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510621 · Cited by 177
Prompting Is All You Need: Automated Android Bug Replay with Large Language Models
Feng, S., & Chen, C. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3608137 · Cited by 176
CoTexT: Multi-task Learning with Code-Text Transformer
Phan, L., Tran, H., Le, D. X., Nguyen, H., Anibal, J., Peltekian, A., & Ye, Y. (2021) · doi:10.18653/v1/2021.nlp4prog-1.5 · Cited by 175
SPT-code: sequence-to-sequence pre-training for learning source code representations
Niu, C., Li, C., Ng, V., Ge, J., Huang, L., & Luo, B. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510096 · Cited by 175
Query Expansion Based on Crowd Knowledge for Code Search
Nie, L., Jiang, H., Ren, Z., Sun, Z., & Li, X. (2016) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2016.2560165 · Cited by 171
Natural Attack for Pre-trained Models of Code
Yang, Z., Shi, J., He, J., & Lo, D. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510146 · Cited by 166
Self-citationInterpretable Online Log Analysis Using Large Language Models with Prompt Strategies
Liu, Y., Tao, S., Meng, W., Wang, J., Ma, W., Chen, Y., Zhao, Y., Yang, H., & Jiang, Y. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3644408 · Cited by 164
Self-citationAPI code recommendation using statistical learning from fine-grained changes
Nguyen, A. T., Hilton, M., Codoban, M., Nguyen, H. A., Mast, L., Rademacher, E., Nguyen, T. N., & Dig, D. (2016) · Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2950290.2950333 · Cited by 163
On integrating orthogonal information retrieval methods to improve traceability recovery
Gethers, M., Oliveto, R., Poshyvanyk, D., & Lucia, A. D. (2011) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2011.6080780 · Cited by 158
Transformer-Based Language Models for Software Vulnerability Detection
Thapa, C., Jang, S. I., Ahmed, M. E., Camtepe, S. A., Pieprzyk, J., & Nepal, . (2022) · Annual Computer Security Applications Conference · doi:10.1145/3564625.3567985 · Cited by 157
Deep Learning for Android Malware Defenses: A Systematic Literature Review
Liu, Y., Tantithamthavorn, C. K., Li, L., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3544968 · Cited by 156
Self-citationSoftware documentation: the practitioners' perspective
Aghajani, E., Nagy, C., Linares‐Vásquez, M., Moreno, L., Bavota, G., Lanza, M., & Shepherd, D. C. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380405 · Cited by 153
An extensive study on pre-trained models for program understanding and generation
Zeng, Z., Tan, H., Zhang, H., Li, J., Zhang, Y., & Zhang, L. (2022) · Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3533767.3534390 · Cited by 153
PyMT5: multi-mode translation of natural language and Python code with transformers
Clement, C. B., Drain, D., Timcheck, J., Svyatkovskiy, A., & Sundaresan, N. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.728 · Cited by 148
CoditT5: Pretraining for Source Code and Natural Language Editing
Zhang, J., Panthaplackel, S., Nie, P., Li, J. J., & Gligoric, M. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556955 · Cited by 144
Getting pwn’d by AI: Penetration Testing with Large Language Models
Happe, A., & Cito, J. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3613083 · Cited by 144
Large Language Models for Test-Free Fault Localization
Yang, A. Z. H., Le Goues, C., Martins, R., & Hellendoorn, V. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3623342 · Cited by 144
Measuring Coding Challenge Competence With APPS
Hendrycks, D., Basart, S., Kadavath, S., Mazeika, M., Arora, A., Guo, E., Burns, C., Puranik, S., He, H., Song, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.09938 · Cited by 142
Cheng, L., Li, X., & Bing, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.637 · Cited by 140
Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Study
Guo, Q., Cao, J., Xie, X., Liu, S., Li, X., Chen, B., & Peng, X. (2023) · arXiv · doi:10.48550/arxiv.2309.08221 · Cited by 140
(2021) · doi:10.1145/3510003.3510203 · Cited by 139
Developing Safety-Critical Software: A Practical Guide for Aviation Software and DO-178C Compliance
Rierson, L. K. (2013) · Cited by 138
Zhang, K., Li, J., Li, G., Shi, X., & Jin, Z. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.737 · Cited by 138
Code and Named Entity Recognition in StackOverflow
Tabassum, J., Maddela, M., Xu, W. H., & Ritter, A. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.443 · Cited by 137
A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research
Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3485275 · Cited by 137
MASTERKEY: Automated Jailbreaking of Large Language Model Chatbots
Deng, G., Liu, Y., Li, Y., Wang, K., Zhang, Y., Li, Z., Wang, H., Zhang, T., & Liu, Y. (2024) · Proceedings 2024 Network and Distributed System Security Symposium · doi:10.14722/ndss.2024.24188 · Cited by 137
Self-citationLeveraging Automated Sentiment Analysis in Software Engineering
Islam, M. R., & Zibran, M. F. (2017) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2017.9 · Cited by 135
An Empirical Study on the Usage of Transformer Models for Code Completion
Ciniselli, M., Cooper, N., Pascarella, L., Mastropaolo, A., Aghajani, E., Poshyvanyk, D., Penta, M. D., & Bavota, G. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3128234 · Cited by 134
ChatGPT, An Artificial Intelligence Chatbot, Is Impacting Medical Literature
Lubowitz, J. H. (2023) · Arthroscopy · doi:10.1016/j.arthro.2023.01.015 · Cited by 132
Machine/Deep Learning for Software Engineering: A Systematic Literature Review
Wang, S., Huang, L., Gao, A., Ge, J., Zhang, T., Feng, H., Satyarth, I., Li, M., Zhang, H., & Ng, V. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3173346 · Cited by 130
Stealthy Backdoor Attack for Code Models
Yang, Z., Xu, B., Zhang, J. M., Kang, H. J., Shi, J., He, J., & Lo, D. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3361661 · Cited by 130
Self-citationSentiment Analysis for Software Engineering: How Far Can Pre-trained Transformer Models Go?
Zhang, T., Xu, B., Thung, F., Haryono, S. A., Lo, D., & Jiang, L. (2020) · 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme46990.2020.00017 · Cited by 129
Self-citationUniLog: Automatic Logging via LLM and In-Context Learning
Xu, J., Cui, Z., Zhao, Y., Zhang, X., He, S., He, P., Li, L., Kang, Y., Lin, Q., Dang, Y., et al. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623326 · Cited by 129
Self-citationNo Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT
Liu, Z., Tang, Y., Luo, X., Zhou, Y., & Zhang, L. F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3392499 · Cited by 128
Self-citationA Quantitative and Qualitative Evaluation of LLM-Based Explainable Fault Localization
Kang, S., An, G., & Yoo, S. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3660771 · Cited by 128
A survey on large language models for software engineering
Zhang, Q., Fang, C., Xie, Y., Zhang, Y., Yu, S., Sun, W., Yang, Y., & Chen, Z. (2026) · Science China Information Sciences · doi:10.1007/s11432-025-4670-0 · Cited by 124
Dataflow Analysis-Inspired Deep Learning for Efficient Vulnerability Detection
Steenhoek, B., Gao, H., & Le, W. (2022) · International Conference on Software Engineering · doi:10.1145/3597503.3623345 · Cited by 123
Explainable automated debugging via large language model-driven scientific debugging
Kang, S., Chen, B., Yoo, S., & Lou, J. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10594-x · Cited by 119
von der Mosel, J., Trautsch, A., & Herbold, S. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3178469 · Cited by 118
From query to usable code: an analysis of stack overflow code snippets
Yang, D., Hussain, A., & Lopes, C. V. (2016) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2901739.2901767 · Cited by 118
AUGER: automatically generating review comments with pre-training models
Li, L., Li, Y., Jiang, H., Yan, J., Luo, T., Hua, Z., Geng, L., & Zuo, C. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549099 · Cited by 116
Self-citationWhat do they capture? a structural analysis of pre-trained language models for source code
Wan, Y., Zhao, W., Zhang, H., Sui, Y., Xu, G., & Jin, H. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510050 · Cited by 115
Wei, Y., Xia, C. S., & Zhang, L. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616271 · Cited by 115
Is ChatGPT the Ultimate Programming Assistant -- How far is it?
Tian, H., Lu, W., Li, T. O., Tang, X., Cheung, S.-C., Klein, J., & Bissyandé, T. F. (2023) · arXiv · doi:10.48550/arxiv.2304.11938 · Cited by 114
Proceedings of the ACL 2010 Conference Short Papers
Dzikovska, M., Moore, J. D., Steinhauser, N. B., & Campbell, G. E. (2010) · The Association for Computational Linguistics · Cited by 113
Tian, H., Liu, K., Kaboré, A. K., Koyuncu, A., Li, L., Klein, J., & Bissyandé, T. F. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416532 · Cited by 113
Self-citationAutomatic Semantic Augmentation of Language Model Prompts (for Code Summarization)
Ahmed, T., Pai, K. S., Devanbu, P., & Barr, E. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639183 · Cited by 112
Tang, W., Tang, M., Ban, M., Zhao, Z., & Feng, M. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111623 · Cited by 111
Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study
Fraser, G., Staats, M., McMinn, P., Arcuri, A., & Padberg, F. (2015) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2699688 · Cited by 110
Automatic Generation of Text Descriptive Comments for Code Blocks
Liang, Y., & Zhu, K. Q. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11963 · Cited by 108
Automated handling of anaphoric ambiguity in requirements: a multi-solution study
Ezzini, S., Abualhaija, S., Arora, C., & Sabetzadeh, M. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510157 · Cited by 108
PRCBERT: Prompt Learning for Requirement Classification using BERT-based Pretrained Language Models
Luo, X., Xue, Y., Xing, Z., & Sun, J. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3560417 · Cited by 106
Self-citationCIRCLE: continual repair across programming languages
Yuan, W., Zhang, Q., He, T., Fang, C., Nguyen, Q. V. H., Hao, X., & Yin, H. (2022) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3533767.3534219 · Cited by 102
Choosing your weapons: On sentiment analysis tools for software engineering research
Jongeling, R., Datta, S., & Serebrenik, A. (2015) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2015.7332508 · Cited by 101
The FormAI Dataset: Generative AI in Software Security through the Lens of Formal Verification
Tihanyi, N., Bisztray, T., Jain, R., Ferrag, M. A., Cordeiro, L. C., & Mavroeidis, V. (2023) · International Conference on Predictive Models in Software Engineering · doi:10.1145/3617555.3617874 · Cited by 100
CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code
Zhou, S., Alon, U., Agarwal, S., & Neubig, G. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.859 · Cited by 100
Gomes, L. A. F., Torres, R. D. S., & Côrtes, M. L. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107217 · Cited by 99
Using Transfer Learning for Code-Related Tasks
Mastropaolo, A., Cooper, N., Palacio, D. N., Scalabrino, S., Poshyvanyk, D., Oliveto, R., & Bavota, G. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3183297 · Cited by 98
Flakify: A Black-Box, Language Model-Based Predictor for Flaky Tests
Fatima, S., Ghaleb, T. A., & Briand, L. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3201209 · Cited by 97
Baldur: Whole-Proof Generation and Repair with Large Language Models
First, E., Rabe, M. N., Ringer, T., & Brun, Y. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616243 · Cited by 94
Domain Knowledge Matters: Improving Prompts with Fix Templates for Repairing Python Type Errors
Peng, Y., Gao, S., Gao, C., Huo, Y., & Lyu, M. R.-T. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3608132 · Cited by 93
CodePlan: Repository-Level Coding using LLMs and Planning
Bairi, R., Sonwane, A., Kanade, A., C., V. D., Iyer, A., Parthasarathy, S., Rajamani, S., Ashok, B., & Shet, S. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3643757 · Cited by 92
GPT2SP: A Transformer-Based Agile Story Point Estimation Approach
Fu, M. C., & Tantithamthavorn, C. K. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3158252 · Cited by 90
Trustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmaps
Lo, D. (2023) · IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00010 · Cited by 90
Self-citationProceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation
Boehm, H., & Flanagan, C. (2013) · doi:10.1145/2491956 · Cited by 88
Yang, Z., Sun, Z., Yue, T. Z., Dévanbu, P., & Lo, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.07506 · Cited by 88
Self-citationCodeFill: multi-token code completion by jointly learning from structure and naming sequences
Izadi, M., Gismondi, R., & Gousios, G. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510172 · Cited by 86
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
Sheng, Y., Zheng, L., Yuan, B., Li, Z., Ryabinin, M., Fu, D. Y., Xie, Z., Chen, B., Barrett, C., Gonzalez, J. E., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.06865 · Cited by 86 · cited 1×
“We also face issues like the need for extensive labeled data [347], significant computational resources [118, 248], and avoiding potential ethical problems [5, 40, 325].”
BackgroundPlein, L., Ouédraogo, W. C., Klein, J., & Bissyandé, T. F. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3639478.3643119 · Cited by 86
CASE Tool Support for Variability Management in Software Product Lines
Bashroush, R., Garba, M., Rabiser, R., Groher, I., & Botterweck, G. (2017) · ACM Computing Surveys · doi:10.1145/3034827 · Cited by 84
Using deep learning to generate complete log statements
Mastropaolo, A., Pascarella, L., & Bavota, G. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3511561 · Cited by 81
On the transferability of pre-trained language models for low-resource programming languages
Chen, F., Fard, F. H., Lo, D., & Bryksin, T. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527917 · Cited by 81
Self-citationChatDev: Communicative Agents for Software Development
Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.07924 · Cited by 81
Developing Safety-Critical Software
Rierson, L. K. (2017) · CRC Press eBooks · doi:10.1201/9781315218168 · Cited by 80
ChatGPT as a Tool for User Story Quality Evaluation: Trustworthy Out of the Box?
Ronanki, K., Cabrero‐Daniel, B., & Berger, C. (2023) · Lecture notes in business information processing · doi:10.1007/978-3-031-48550-3_17 · Cited by 78
Fast changeset-based bug localization with BERT
Ciborowska, A., & Damevski, K. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510042 · Cited by 77
One Adapter for All Programming Languages? Adapter Tuning for Code Search and Summarization
Wang, D., Chen, B., Li, S., Luo, W., Peng, S., Dong, W., & Liao, X. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00013 · Cited by 77
On the Impact of Sample Duplication in Machine-Learning-Based Android Malware Detection
Zhao, Y., Li, L., Wang, H., Cai, H., Bissyandé, T. F., Klein, J., & Grundy, J. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3446905 · Cited by 76
Self-citationAdaptive Test Generation Using a Large Language ModelPreprint
Schäfer, M., Nadi, S., Eghbali, A., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2302.06527 · Cited by 74
A Systematic Review of Interaction in Search-Based Software Engineering
Ramírez, A., Romero, J. R., & Simons, C. L. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2803055 · Cited by 73
An automated approach to estimating code coverage measures via execution logs
Chen, B., Jian, S., Xu, P., Hu, X. Y., & Jiang, Z. M. (2018) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3238147.3238214 · Cited by 73
Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities
Ma, W., Liu, S., Zhao, M., Xie, X., Wang, W., Hu, Q., Zhang, J., & Liu, Y. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3664606 · Cited by 73
Self-citationThe 54th Annual Meeting of the Association for Computational Linguistics
Yung, F., Duh, K., Komura, T., et al. (2016) · The Association for Computational Linguistics · Cited by 71
On the importance of building high-quality training datasets for neural code search
Sun, Z., Li, L., Yan, L., Du, X., & Li, L. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510160 · Cited by 71
Self-citationProceedings of the 38th Annual Computer Security Applications Conference
(2022) · doi:10.1145/3564625 · Cited by 71
Research works printed but not linked: 271
- Mayank Agarwal Yikang Shen Bailin Wang Yoon Kim and Jie Chen. 2024. Structured code representations enable data-efficient adaptation of code language models (2024). arXiv:2401.10716. Retrieved from https://arxiv.org/abs/2401.10716 · arXiv
- Lakshya Agrawal, Aditya Kanade, Navin Goyal, Shuvendu K. Lahiri, and Sriram Rajamani. 2023. Monitor-guided decoding of code lms with static analysis of repository context. In Advances in Neural Information Processing Systems. A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 32270–32298. Retrieved from https://proceedings.neurips.cc/paper_files/paper/2023/file/662b1774ba8845fc1fa3d1fc0177ceeb-Paper-Conference.pdf
- Baleegh Ahmad Shailja Thakur Benjamin Tan Ramesh Karri and Hammond Pearce. 2023. Fixing hardware security bugs with large language models. arXiv:2302.01215. · arXiv
- Toufique Ahmed Kunal Suresh Pai Premkumar Devanbu and Earl T. Barr. 2023. Improving few-shot prompts with relevant static analysis products. arXiv:2304.06815. · arXiv
- Ali Al-Kaswan, Toufique Ahmed, Maliheh Izadi, Anand Ashok Sawant, Premkumar Devanbu, and Arie van Deursen. 2023. Extending source code pre-trained language models to summarise decompiled binarie. In Proceedings of the 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER ’23). IEEE, 260–271.
- Sven Amann, Sebastian Proksch, Sarah Nadi, and Mira Mezini. 2016. A study of visual studio usage in practice. In Proceedings of the 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER ’16), Vol. 1. IEEE, 124–134.
- Patrick Bareiß Beatriz Souza Marcelo d’Amorim and Michael Pradel. 2022. Code generation tools (almost) for free? A study of few-shot pre-trained language models on code. arXiv:2206.01335. · arXiv
- Nghi D. Q. Bui Hung Le Yue Wang Junnan Li Akhilesh Deepak Gotmare and Steven C. H. Hoi. 2023. Codetf: One-stop transformer library for state-of-the-art code LLM. arXiv:2306.00029. · arXiv
- Jialun Cao Meiziniu Li Ming Wen and Shing-chi Cheung. 2023. A study on prompt design advantages and limitations of chatgpt for deep learning program repair. arXiv:2304.08191. · arXiv
- Aaron Chan Anant Kharkar Roshanak Zilouchian Moghaddam Yevhen Mohylevskyy Alec Helyar Eslam Kamal Mohamed Elkamhawy and Neel Sundaresan. 2023. Transformer-based vulnerability detection in code at edittime: Zero-shot few-shot or fine-tuning? arXiv:2306.01754. · arXiv
- Yiannis Charalambous Norbert Tihanyi Ridhi Jain Youcheng Sun Mohamed Amine Ferrag and Lucas C. Cordeiro. 2023. A new era in software security: Towards self-healing software via large language models and formal verification. arXiv:2305.14752. · arXiv
- Angelica Chen Jérémy Scheurer Tomasz Korbak Jon Ander Campos Jun Shern Chan Samuel R. Bowman Kyunghyun Cho and Ethan Perez. 2023. Improving code generation by training with natural language feedback. arXiv:2303.16749. · arXiv
- Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman Alex Ray Raul Puri Gretchen Krueger Michael Petrov Heidy Khlaaf Girish Sastry Pamela Mishkin Brooke Chan Scott Gray Nick Ryder Mikhail Pavlov Alethea Power Lukasz Kaiser Mohammad Bavarian Clemens Winter Philippe Tillet Felipe Petroski Such Dave Cummings Matthias Plappert Fotios Chantzis Elizabeth Barnes Ariel Herbert-Voss William Hebgen Guss Alex Nichol Alex Paino Nikolas Tezak Jie Tang Igor Babuschkin Suchir Balaji Shantanu Jain William Saunders Christopher Hesse Andrew N. Carr Jan Leike Josh Achiam Vedant Misra Evan Morikawa Alec Radford Matthew Knight Miles Brundage Mira Murati Katie Mayer Peter Welinder Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever and Wojciech Zaremba. 2021. Evaluating large language models trained on code. arXiv:2107.03374. · arXiv
- Meng Chen Hongyu Zhang Chengcheng Wan Zhao Wei Yong Xu Juhong Wang and Xiaodong Gu. 2023. On the effectiveness of large language models in domain-specific code generation. arXiv:2312.01639. · arXiv
- Xinyun Chen Maxwell Lin Nathanael Schärli and Denny Zhou. 2023. Teaching large language models to self-debug. arXiv:2304.05128. · arXiv
- Xinyun Chen Chang Liu and Dawn Song. 2017. Towards synthesizing complex programs from input-output examples. arXiv:1706.01284. · arXiv
- Agnieszka Ciborowska and Kostadin Damevski. 2023. Too few bug reports? Exploring data augmentation for improved changeset-based bug localization. arXiv:2305.16430. · DOI
- Arghavan Moradi Dakhel Amin Nikanjam Vahid Majdinasab Foutse Khomh and Michel C. Desmarais. 2023. Effective test generation using pre-trained large language models and mutation testing. arXiv:2308.16557. · arXiv
- Pantazis Deligiannis Akash Lal Nikita Mehrotra and Aseem Rastogi. 2023. Fixing rust compilation errors using llms. arXiv:2308.05177. · arXiv
- Yinlin Deng Chunqiu Steven Xia Chenyuan Yang Shizhuo Dylan Zhang Shujing Yang and Lingming Zhang. 2023. Large language models are edge-case fuzzers: Testing deep learning libraries via fuzzgpt. arXiv:2304.02014. · DOI · Paper page
- Tuan Dinh, Jinman Zhao, Samson Tan, Renato Negrinho, Leonard Lausen, Sheng Zha, and George Karypis. 2024. Large language models of code fail at completing code with potential bugs. In Advances in Neural Information Processing Systems. A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 41386–41412. Retrieved from https://proceedings.neurips.cc/paper_files/paper/2023/file/819cebb05f993840e8a52d7564c5c282-Paper-Conference.pdf
- Guanting Dong Hongyi Yuan Keming Lu Chengpeng Li Mingfeng Xue Dayiheng Liu Wei Wang Zheng Yuan Chang Zhou and Jingren Zhou. 2023. How abilities in large language models are affected by supervised fine-tuning data composition. arXiv:2310.05492. · arXiv
- Yihong Dong Jiazheng Ding Xue Jiang Ge Li Zhuo Li and Zhi Jin. 2023. Codescore: Evaluating code generation by learning code execution. arXiv:2301.09043. · arXiv
- Xueying Du Mingwei Liu Juntao Li Hanlin Wang Xin Peng and Yiling Lou. 2023. Resolving crash bugs via large language models: An empirical study. arXiv:2312.10448. · arXiv
- Xueying Du Mingwei Liu Kaixin Wang Hanlin Wang Junwei Liu Yixuan Chen Jiayi Feng Chaofeng Sha Xin Peng and Yiling Lou. 2023. Classeval: A manually-crafted benchmark for evaluating llms on class-level code generation. arXiv:2308.01861. Retrieved from https://arxiv.org/abs/2308.01861 · arXiv
- Aryaz Eghbali and Michael Pradel. 2024. De-hallucinator: Iterative grounding for llm-based code completion. arXiv:2401.01701. · arXiv
- Abdelkarim El-Hajjami Nicolas Fafin and Camille Salinesi. 2023. Which ai technique is better to classify requirements? An experiment with SVM LSTM and ChatGPT. arXiv:2311.11547. · arXiv
- Madeline Endres Sarah Fakhoury Saikat Chakraborty and Shuvendu K. Lahiri. 2023. Formalizing natural language intent into program specifications via large language models. arXiv:2310.01831. Retrieved from https://arxiv.org/abs/2310.01831 · arXiv
- Sarah Fakhoury Saikat Chakraborty Madan Musuvathi and Shuvendu K. Lahiri. 2023. Towards generating functionally correct code edits from natural language issue descriptions. arXiv:2304.03816. Retrieved from https://arxiv.org.abs/2304.03816 · arXiv
- Zhiyu Fan Xiang Gao Abhik Roychoudhury and Shin Hwei Tan. 2022. Automated repair of programs from large language models. arXiv:2205.10583. Retrieved from https://arxiv.org/abs/2205.10583 · arXiv
- Mohamed Amine Ferrag Ammar Battah Norbert Tihanyi Merouane Debbah Thierry Lestable and Lucas C. Cordeiro. 2023. Securefalcon: The next cyber reasoning system for cyber security. arXiv:2307.06616. Retrieved from https://arxiv.org/abs/2307.06616 · arXiv
- Daniel Fried Armen Aghajanyan Jessy Lin Sida Wang Eric Wallace Freda Shi Ruiqi Zhong Wen-tau Yih Luke Zettlemoyer and Mike Lewis. 2022. Incoder: A generative model for code infilling and synthesis. arXiv:2204.05999. Retrieved from https://arxiv.org/abs/2204.05999 · arXiv
- Apurva Gandhi Thong Q. Nguyen Huitian Jiao Robert Steen and Ameya Bhatawdekar. 2023. Natural language commanding via program synthesis. arXiv:2306.03460. Retrieved from https://arxiv.org/abs/2306.03460 · arXiv
- Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe Charles Foster Jason Phang Horace He Anish Thite Noa Nabeshima Shawn Presser and Connor Leahy. 2020. The pile: An 800gb dataset of diverse text for language modeling. arXiv:2101.00027. Retrieved from https://arxiv.org/abs/2101.00027 · arXiv
- Shuzheng Gao Wenxin Mao Cuiyun Gao Li Li Xing Hu Xin Xia and Michael R. Lyu. 2024. Learning in the wild: Towards leveraging unlabeled data for effectively tuning pre-trained code models. arXiv:2401.01060. Retrieved from https://arxiv.org/abs/2401.01060 · arXiv
- Shuzheng Gao Xin-Cheng Wen Cuiyun Gao Wenxuan Wang and Michael R. Lyu. 2023. Constructing effective in-context demonstration for code intelligence tasks: An empirical study. arXiv:2304.07575. Retrieved from https://arxiv.org/abs/2304.07575 · arXiv
- Zeyu Gao Hao Wang Yuchen Zhou Wenyu Zhu and Chao Zhang. 2023. How far have we gone in vulnerability detection using large language models. arXiv:2311.12420. Retrieved from https://arxiv.org/abs/2311.12420 · arXiv
- Jian Gu, Pasquale Salza, and Harald C. Gall. 2022. Assemble foundation models for automatic code summarization. In Proceedings of the 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER ’22). IEEE, 935–946.
- Daya Guo Shuo Ren Shuai Lu Zhangyin Feng Duyu Tang Shujie Liu Long Zhou Nan Duan Alexey Svyatkovskiy Shengyu Fu Michele Tufano Shao Kun Deng Colin Clement Dawn Drain Neel Sundaresan Jian Yin Daxin Jiang and Ming Zhou. 2020. Graphcodebert: Pre-training code representations with data flow. arXiv:2009.08366. Retrieved from https://arxiv.org/abs/2009.08366 · arXiv
- Daya Guo Canwen Xu Nan Duan Jian Yin and Julian McAuley. 2023. Longcoder: A long-range pre-trained language model for code completion. arXiv:2306.14893. Retrieved from https://arxiv.org/abs/2306.14893 · arXiv
- Daya Guo Qihao Zhu Dejian Yang Zhenda Xie Kai Dong Wentao Zhang Guanting Chen Xiao Bi Y. Wu Y. K. Li Fuli Luo Yingfei Xiong and Wenfeng Liang. 2024. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv:2401.14196. Retrieved from https://arxiv.org/abs/2401.14196 · arXiv
- Priyanshu Gupta Avishree Khare Yasharth Bajpai Saikat Chakraborty Sumit Gulwani Aditya Kanade Arjun Radhakrishna Gustavo Soares and Ashish Tiwari. 2023. Grace: Generation using associated code edits. arXiv:2305.14129. Retrieved from https://arxiv.org/abs/2305.14129 · arXiv
- Patrick Hajali and Ignas Budvytis. 2023. Function-constrained program synthesis. arXiv:2311.15500. Retrieved from https://arxiv.org/abs/2311.15500 · arXiv
- Yu Hao Weiteng Chen Ziqiao Zhou and Weidong Cui. 2023. E & v: Prompting large language models to perform static analysis by pseudo-code execution and verification. arXiv:2312.08477. Retrieved from https://arxiv.org/abs/2312.08477 · arXiv
- Junda He Zhou Xin Bowen Xu Ting Zhang Kisub Kim Zhou Yang Ferdian Thung Ivana Irsan and David Lo. 2023. Representation learning for stack overflow posts: How far are we? arXiv:2303.06853. Retrieved from https://arxiv.org/abs/2303.06853 · arXiv
- Jordan Henkel, Denini Silva, Leopoldo Teixeira, Marcelo d’Amorim, and Thomas Reps. 2021. Shipwright: A human-in-the-loop system for dockerfile repair. In Proceedings of the 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE ’21). IEEE, 1148–1160.
- Tobias Hey, Jan Keim, Anne Koziolek, and Walter F. Tichy. 2020. Norbert: Transfer learning for requirements classification. In Proceedings of the 2020 IEEE 28th International Requirements Engineering Conference (RE ’20). IEEE, 169–179.
- Sirui Hong Xiawu Zheng Jonathan Chen Yuheng Cheng Jinlin Wang Ceyao Zhang Zili Wang Steven Ka Shing Yau Zijuan Lin Liyang Zhou Chenyu Ran Lingfeng Xiao Chenglin Wu and Jürgen Schmidhuber. 2023. METAGPT: Meta programming for multi-agent collaborative framework. arXiv:2308.00352. Retrieved from https://arxiv.org/abs/2308.00352 · arXiv
- Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the International Conference on Machine Learning. PMLR, 2790–2799.
- Jie Hu Qian Zhang and Heng Yin. 2023. Augmenting greybox fuzzing with generative AI. arXiv:2306.06782. Retrieved from https://arxiv.org/abs/2306.06782 · arXiv
- Xueyu Hu Kun Kuang Jiankai Sun Hongxia Yang and Fei Wu. 2024. Leveraging print debugging to improve code generation in large language models. arXiv:2401.05319. Retrieved from https://arxiv.org/abs/2401.05319 · arXiv
- Dong Huang Qingwen Bu and Heming Cui. 2023. Codecot and beyond: Learning to program and test like a developer. arXiv: 2308.08784. Retrieved from https://arxiv.org/abs/2308.08784 · arXiv
- Dong Huang Qingwen Bu Jie M. Zhang Michael Luck and Heming Cui. 2023. Agentcoder: Multi-agent-based code generation with iterative testing and optimisation. arXiv: 2312.13010. Retrieved from https://arxiv.org/abs/2312.13010 · arXiv
- Di Huang Ziyuan Nan Xing Hu Pengwei Jin Shaohui Peng Yuanbo Wen Rui Zhang Zidong Du Qi Guo Yewen Pu and Yunji Chen. 2023. anpl: Compiling natural programs with interactive decomposition. arXiv:2305.18498. Retrieved from https://arxiv.org/abs/2305.18498 · arXiv
- Qing Huang Yishun Wu Zhenchang Xing He Jiang Yu Cheng and Huan Jin. 2023. Adaptive intellect unleashed: The feasibility of knowledge transfer in large language models. arXiv:2308.04788. Retrieved from https://arxiv.org/abs/2308.04788 · arXiv
- Qing Huang Jiahui Zhu Zhenchang Xing Huan Jin Changjing Wang and Xiwei Xu. 2023. A chain of ai-based solutions for resolving fqns and fixing syntax errors in partial code. arXiv:2306.11981. Retrieved from https://arxiv.org/abs/2306.11981 · arXiv
- Qing Huang Zhou Zou Zhenchang Xing Zhenkang Zuo Xiwei Xu and Qinghua Lu. 2023. Ai chain on large language model for unsupervised control flow graph generation for statically-typed partial code. arXiv: 2306.00757. Retrieved from https://arxiv.org/abs/2306.00757 · arXiv
- Ali Reza Ibrahimzada Yang Chen Ryan Rong and Reyhaneh Jabbarvand. 2023. Automated bug generation in the era of large language models. arXiv: 2310.02407. Retrieved from https://arxiv.org/abs/2310.02407 · arXiv
- Nafis Tanveer Islam and Peyman Najafirad. 2024. Code security vulnerability repair using reinforcement learning with large language models. arXiv:2401.07031. Retrieved from https://arxiv.org/abs/2401.07031 · arXiv
- Abhinav Jain Chima Adiole Thomas Reps Swarat Chaudhuri and Chris Jermaine. 2024. Coarse-tuning models of code with reinforcement learning feedback. Retrieved from https://openreview.net/forum?id=vLqkCvjHRD
- Naman Jain Tianjun Zhang Wei-Lin Chiang Joseph E Gonzalez Koushik Sen and Ion Stoica. 2023. LLM-assisted code cleaning for training accurate code generators. arXiv:2311.14904. Retrieved from https://arxiv.org/abs/2311.14904 · arXiv
- Prithwish Jana Piyush Jha Haoyang Ju Gautham Kishore Aryan Mahajan and Vijay Ganesh. 2023. Attention compilation and solver-based symbolic analysis are all you need. arXiv:2306.06755. Retrieved from https://arxiv.org/abs/2306.06755 · arXiv
- Zhenlan Ji Pingchuan Ma Zongjie Li and Shuai Wang. 2023. Benchmarking and explaining large language model-based code generation: A causality-centric approach. arXiv:2310.06680. Retrieved from https://arxiv.org/abs/2310.06680 · arXiv
- Nan Jiang Kevin Liu Thibaud Lutellier and Lin Tan. 2023. Impact of code language models on automated program repair. arXiv:2302.05020. Retrieved from https://arxiv.org/abs/2302.05020 · arXiv
- Nan Jiang Chengxiao Wang Kevin Liu Xiangzhe Xu Lin Tan and Xiangyu Zhang. 2023. Nova \({}^{+}\) : Generative language models for binaries. arXiv:2311.13721. Retrieved from https://arxiv.org/abs/2311.13721 · arXiv
- Shuyang Jiang Yuhao Wang and Yu Wang. 2023. Selfevolve: A code evolution framework via large language models. arXiv:2306.02907. Retrieved from https://arxiv.org/abs/2306.02907 · arXiv
- Xue Jiang Yihong Dong Lecheng Wang Qiwei Shang and Ge Li. 2023. Self-planning code generation with large language model. arXiv:2303.06689. Retrieved from https://arxiv.org/abs/2303.06689 · arXiv
- Carlos E. Jimenez John Yang Alexander Wettig Shunyu Yao Kexin Pei Ofir Press and Karthik Narasimhan. 2023. Swe-bench: Can language models resolve real-world GitHub issues? arXiv:2310.06770. Retrieved from https://arxiv.org/abs/2310.06770 · arXiv
- Matthew Jin Syed Shahriar Michele Tufano Xin Shi Shuai Lu Neel Sundaresan and Alexey Svyatkovskiy. 2023. Inferfix: End-to-end program repair with LLMS. arXiv:2303.07263. Retrieved from https://arxiv.org/abs/2303.07263 · arXiv
- Xin Jin Jonathan Larson Weiwei Yang and Zhiqiang Lin. 2023. Binary code summarization: Benchmarking CHATGPT/GPT-4 and other large language models. arXiv:2312.09601. Retrieved from https://arxiv.org/abs/2312.09601 · arXiv
- Md Mahir Asef Kabir Sk Adnan Hassan Xiaoyin Wang Ying Wang Hai Yu and Na Meng. 2023. An empirical study of ChatGPT-3.5 on question answering and code maintenance. arXiv:2310.02104. Retrieved from https://arxiv.org/abs/2310.02104 · arXiv
- Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. 2020. Learning and evaluating contextual embedding of source code. In Proceedings of the International Conference on Machine Learning. PMLR, 5110–5121.
- Sungmin Kang Gabin An and Shin Yoo. 2023. A preliminary evaluation of LLM-based fault localization. arXiv:2308.05487. Retrieved from https://arxiv.org/abs/2308.05487 · arXiv
- Sungmin Kang Juyeon Yoon Nargiz Askarbekkyzy and Shin Yoo. 2023. Evaluating diverse large language models for automatic and general bug reproduction. arXiv:2311.04532. Retrieved from https://arxiv.org/abs/2311.04532 · arXiv
- Sungmin Kang Juyeon Yoon and Shin Yoo. 2022. Large language models are few-shot testers: Exploring LLM-based general bug reproduction. arXiv:2209.11515. Retrieved from https://arxiv.org/abs/2209.11515 · arXiv
- Jai Kannan. 2023. Can llms configure software tools. arXiv:2312.06121. Retrieved from https://arxiv.org/abs/2312.06121 · arXiv
- Rafael-Michael Karampatsis and Charles Sutton. 2020. Scelmo: Source code embeddings from language models. arXiv:2004.13214. Retrieved from https://arxiv.org/abs/2004.13214 · arXiv
- Adam Khakhar Stephen Mell and Osbert Bastani. 2023. PAC prediction sets for large language models of code. arXiv:2302.08703. Retrieved from https://arxiv.org/abs/2302.08703 · arXiv
- Junaed Younus Khan, Md Tawkat Islam Khondaker, Gias Uddin, and Anindya Iqbal. 2021. Automatic detection of five API documentation smells: Practitioners’ perspectives. In Proceedings of the 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER ’21). IEEE, 318–329.
- Junaed Younus Khan and Gias Uddin. 2022. Automatic detection and analysis of technical debts in peer-review documentation of r packages. In Proceedings of the 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER ’22). IEEE, 765–776.
- Mohammad Abdullah Matin Khan M. Saiful Bari Xuan Long Do Weishi Wang Md Rizwan Parvez and Shafiq Joty. 2023. Xcodeeval: A large scale multilingual multitask benchmark for code understanding generation translation and retrieval. arXiv:2303.03004. Retrieved from https://arxiv.org/abs/2303.03004 · arXiv
- Muhammad Fawad Akbar Khan Max Ramsdell Erik Falor and Hamid Karimi. 2023. Assessing the promise and pitfalls of chatgpt for automated code generation. arXiv:2311.02640. Retrieved from https://arxiv.org/abs/2311.02640 · arXiv
- Hiroyuki Kirinuki and Haruto Tanno. 2024. Chatgpt and human synergy in black-box testing: A comparative analysis. arXiv:2401.13924. Retrieved from https://arxiv.org/abs/2401.13924 · arXiv
- Barbara Ann Kitchenham and Stuart Charters. 2007. Guidelines for performing systematic literature reviews in software engineering. In Technical Report EBSE 2007-001. Keele University and Durham University Joint Report. Retrieved from https://www.elsevier.com/__data/promis_misc/525444systematicreviewsguide.pdf
- 10.5555/1987360.1987364 · doi:10.5555/1987360.1987364
- Takashi Koide Naoki Fukushi Hiroki Nakano and Daiki Chiba. 2023. Detecting phishing sites using chatgpt. arXiv:2306.05816. Retrieved from https://arxiv.org/abs/2306.05816 · arXiv
- Bonan Kou Muhao Chen and Tianyi Zhang. 2023. Automated summarization of stack overflow posts. arXiv:2305.16680. Retrieved from https://arxiv.org/abs/2305.16680 · arXiv
- Bonan Kou Shengmai Chen Zhijie Wang Lei Ma and Tianyi Zhang. 2023. Is model attention aligned with human attention? An empirical study on large language models for code generation. arXiv:2306.01220. Retrieved from https://arxiv.org/abs/2306.01220 · arXiv
- Shuvendu K Lahiri Aaditya Naik Georgios Sakkas Piali Choudhury Curtis von Veh Madanlal Musuvathi Jeevana Priya Inala Chenglong Wang and Jianfeng Gao. 2022. Interactive code generation via test-driven user-intent formalization. arXiv:2208.05950. Retrieved from https://arxiv.org/abs/2208.05950 · arXiv
- Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2023. Ds-1000: A natural and reliable benchmark for data science code generation. In Proceedings of the International Conference on Machine Learning. PMLR, 18319–18345.
- Md Tahmid Rahman Laskar M. Saiful Bari Mizanur Rahman Md Amran Hossen Bhuiyan Shafiq Joty and Jimmy Xiangji Huang. 2023. A systematic study and comprehensive evaluation of ChatGPT on benchmark datasets. arXiv:2305.18486. Retrieved from https://arxiv.org/abs/2305.18486 · arXiv
- Hung Le Hailin Chen Amrita Saha Akash Gokul Doyen Sahoo and Shafiq Joty. 2023. Codechain: Towards modular code generation through chain of self-revisions with representative sub-modules. arXiv:2310.08992. Retrieved from https://arxiv.org/abs/2310.08992 · arXiv
- Dong Li Yelong Shen Ruoming Jin Yi Mao Kuan Wang and Weizhu Chen. 2022. Generation-augmented query expansion for code retrieval. arXiv:2212.10692. Retrieved from https://arxiv.org/abs/2212.10692 · arXiv
- Feng-Lin Li, Jennifer Horkoff, John Mylopoulos, Renata S. S. Guizzardi, Giancarlo Guizzardi, Alexander Borgida, and Lin Liu. 2014. Non-functional requirements as qualities, with a spice of ontology. In Proceedings of the 2014 IEEE 22nd International Requirements Engineering Conference (RE ’14). IEEE, 293–302.
- Haochen Li Xin Zhou and Zhiqi Shen. 2024. Rewriting the code: A simple method for large language model augmented code search. arXiv:2401.04514. Retrieved from https://arxiv.org/abs/2401.04514 · arXiv
- Jingyao Li Pengguang Chen and Jiaya Jia. 2023. Motcoder: Elevating large language models with modular of thought for challenging programming tasks. arXiv:2312.15960. Retrieved from https://arxiv.org/abs/2312.15960 · arXiv
- Jingxuan Li, Rui Huang, Wei Li, Kai Yao, and Weiguo Tan. 2021. Toward less hidden cost of code completion with acceptance and ranking models. In Proceedings of the 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME ’21). IEEE, 195–205.
- Jia Li Ge Li Yongmin Li and Zhi Jin. 2023. Enabling programming thinking in large language models toward code generation. arXiv:2305.06599. Retrieved from https://arxiv.org/abs/2305.06599 · arXiv
- Jia Li Ge Li Yongmin Li and Zhi Jin. 2023. Structured chain-of-thought prompting for code generation. arXiv:2305.06599. Retrieved from https://arxiv.org/abs/2305.06599 · arXiv
- Jia Li Ge Li Yunfei Zhao Yongmin Li Zhi Jin Hao Zhu Huanyu Liu Kaibo Liu Lecheng Wang Zheng Fang Lanshen Wang Jiazheng Ding Xuanming Zhang Yihong Dong Yuqi Zhu Bin Gu and Mengfei Yang. 2024. Deveval: Evaluating code generation in practical software projects. arXiv:2401.06401. Retrieved from https://arxiv.org/abs/2401.06401 · arXiv
- Peng Li Tianxiang Sun Qiong Tang Hang Yan Yuanbin Wu Xuanjing Huang and Xipeng Qiu. 2023. Codeie: Large code generation models are better few-shot information extractors. arXiv:2305.05711. Retrieved from https://arxiv.org/abs/2305.05711 · arXiv
- Tsz-On Li Wenxi Zong Yibo Wang Haoye Tian Ying Wang and Shing-Chi Cheung. 2023. Finding failure-inducing test cases with ChatGPT. arXiv:2304.11686. Retrieved from https://arxiv.org/abs/2304.11686 · arXiv
- Tsz-On Li, Wenxi Zong, Yibo Wang, Haoye Tian, Ying Wang, Shing-Chi Cheung, and Jeff Kramer. 2023. Nuances are the key: Unlocking chatgpt to find failure-inducing tests with differential prompting. In Proceedings of the 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE ’23). IEEE, 14–26.
- Xin-Ye Li Jiang-Tian Xue Zheng Xie and Ming Li. 2023. Think outside the code: Brainstorming boosts large language models in code generation. arXiv:2305.10679. Retrieved from https://arxiv.org/abs/2305.10679 · arXiv
- Yichen Li Yintong Huo Zhihan Jiang Renyi Zhong Pinjia He Yuxin Su and Michael R. Lyu. 2023. Exploring the effectiveness of LLMS in automated logging generation: An empirical study. arXiv:2307.05950. Retrieved from https://arxiv.org/abs/2307.05950 · arXiv
- Youjia Li Jianjun Shi and Zheng Zhang. 2023. A novel approach for rapid development based on chatgpt and prompt engineering. arXiv:2312.13115. Retrieved from https://arxiv.org/abs/2312.13115 · arXiv
- Yao Li Tao Zhang Xiapu Luo Haipeng Cai Sen Fang and Dawei Yuan. 2022. Do pre-trained language models indeed understand software engineering tasks? arXiv:2211.10623. Retrieved from https://arxiv.org/abs/2211.10623 · arXiv
- Zongjie Li, Chaozheng Wang, Zhibo Liu, Haoxuan Wang, Dong Chen, Shuai Wang, and Cuiyun Gao. 2023. Cctest: Testing and repairing code completion systems. In Proceedings of the 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1238–1250.
- Zongjie Li Chaozheng Wang Zhibo Liu Haoxuan Wang Shuai Wang and Cuiyun Gao. 2022. Cctest: Testing and repairing code completion systems. arXiv:2208.08289. Retrieved from https://arxiv.org/abs/2208.08289 · arXiv
- Jinfeng Lin, Yalin Liu, Qingkai Zeng, Meng Jiang, and Jane Cleland-Huang. 2021. Traceability transformed: Generating more accurate links with pre-trained bert models. In Proceedings of the 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE ’21). IEEE, 324–335.
- Yu-Chen Lin Akhilesh Kumar Wen-Liang Zhang Norman Chang Muhammad Zakir Rucha Apte Chao Wang and Jyh-Shing Roger Jang. 2023. Applications of large language models in data processing: Innovative approaches to segmenting and renewing information. arXiv:2311.16267. Retrieved from https://arxiv.org/abs/2311.16267 · arXiv
- Chao Liu Xuanlin Bao Hongyu Zhang Neng Zhang Haibo Hu Xiaohong Zhang and Meng Yan. 2023. Improving ChatGPT prompt for code generation. arXiv:2305.08360. Retrieved from https://arxiv.org/abs/2305.08360 · arXiv
- Jinrun Liu Xinyu Tang Linlin Li Panpan Chen and Yepang Liu. 2023. Which is a better programming assistant? A comparative study between chatgpt and stack overflow. arXiv:2308.13851. Retrieved from https://arxiv.org/abs/2308.13851 · arXiv
- Jiawei Liu Chunqiu Steven Xia Yuyao Wang and Lingming Zhang. 2023. Is your code generated by chatgpt really correct? Rigorous evaluation of large language models for code generation. arXiv:2305.01210. Retrieved from https://arxiv.org/abs/2305.01210 · arXiv
- Puzhuo Liu Chengnian Sun Yaowen Zheng Xuan Feng Chuan Qin Yuncheng Wang Zhi Li and Limin Sun. 2023. Harnessing the power of llm to support binary taint analysis. arXiv:2310.08275. Retrieved from https://arxiv.org/abs/2310.08275 · arXiv
- Shangqing Liu Bozhi Wu Xiaofei Xie Guozhu Meng and Yang Liu. 2023. Contrabert: Enhancing code pre-trained models via contrastive learning. arXiv:2301.09072. Retrieved from https://arxiv.org/abs/2301.09072 · arXiv
- Tianyang Liu Canwen Xu and Julian McAuley. 2023. Repobench: Benchmarking repository-level code auto-completion systems. arXiv:2306.03091. Retrieved from https://arxiv.org/abs/2306.03091 · arXiv
- Yi Liu Gelei Deng Yuekang Li Kailong Wang Tianwei Zhang Yepang Liu Haoyu Wang Yan Zheng and Yang Liu. 2023. Prompt injection attack against LLM-integrated applications. arXiv:2306.05499. Retrieved from https://arxiv.org/abs/2306.05499 · arXiv
- Yue Liu Thanh Le-Cong Ratnadira Widyasari Chakkrit Tantithamthavorn Li Li Xuan-Bach D. Le and David Lo. 2023. Refining ChatGPT-generated code: Characterizing and mitigating code quality issues. arXiv:2307.12596. Retrieved from https://arxiv.org/abs/2307.12596 · arXiv
- Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692. Retrieved from https://arxiv.org/abs/1907.11692 · arXiv
- Zhe Liu, Chunyang Chen, Junjie Wang, Xing Che, Yuekai Huang, Jun Hu, and Qing Wang. 2023. Fill in the blank: Context-aware automated text input generation for mobile GUI testing. In Proceedings of the 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE ’23). IEEE, 1355–1367.
- Zhe Liu Chunyang Chen Junjie Wang Mengzhuo Chen Boyu Wu Xing Che Dandan Wang and Qing Wang. 2023. Testing the limits: Unusual text inputs generation for mobile app crash detection with large language model. arXiv:2310.15657. Retrieved from https://arxiv.org/abs/2310.15657 · arXiv
- Junyi Lu, Lei Yu, Xiaojia Li, Li Yang, and Chun Zuo. 2023. Llama-reviewer: Advancing code review automation with large language models through parameter-efficient fine-tuning. In Proceedings of the 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE ’23). IEEE, 647–658.
- Dipeeka Luitel Shabnam Hassani and Mehrdad Sabetzadeh. 2023. Improving requirements completeness: Automated assistance through large language models. arXiv:2308.03784. Retrieved from https://arxiv.org/abs/2308.03784 · arXiv
- Ziyang Luo Can Xu Pu Zhao Qingfeng Sun Xiubo Geng Wenxiang Hu Chongyang Tao Jing Ma Qingwei Lin and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct. arXiv:2306.08568. Retrieved from https://arxiv.org/abs/2306.08568 · arXiv
- Lezhi Ma Shangqing Liu Yi Li Xiaofei Xie and Lei Bu. 2024. SpecGen: Automated generation of formal program specifications via large language models. arXiv:2401.08807. Retrieved from https://arxiv.org/abs/2401.08807 · arXiv
- Wei Ma Shangqing Liu Wenhan Wang Qiang Hu Ye Liu Cen Zhang Liming Nie and Yang Liu. 2023. The scope of chatgpt in software engineering: A thorough investigation. arXiv:2305.12138. Retrieved from https://arxiv.org/abs/2305.12138 · arXiv
- Shantanu Mandal Adhrik Chethan Vahid Janfaza S. M. Mahmud Todd A Anderson Javier Turek Jesmin Jahan Tithi and Abdullah Muzahid. 2023. Large language models based automatic synthesis of software specifications. arXiv:2304.09181. Retrieved from https://arxiv.org/abs/2304.09181 · arXiv
- Antonio Mastropaolo, Massimiliano Di Penta, and Gabriele Bavota. 2023. Towards automatically addressing self-admitted technical debt: How far are we?. In Proceedings of the 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE ’23). IEEE, 585–597.
- Mohammad Mahdi Mohajer Reem Aleithan Nima Shiri Harzevili Moshi Wei Alvine Boaye Belle Hung Viet Pham and Song Wang. 2023. Skipanalyzer: An embodied agent for code analysis with large language models. arXiv:2310.18532. Retrieved from https://arxiv.org/abs/2310.18532 · arXiv
- Seungjun Moon Yongho Song Hyungjoo Chae Dongjin Kang Taeyoon Kwon Kai Tzu-iunn Ong Seung-won Hwang and Jinyoung Yeo. 2023. Coffee: Boost your code LLMS by fixing bugs with feedback. arXiv:2311.07215. Retrieved from https://arxiv.org/abs/2311.07215 · arXiv
- 10.5555/1858842.1858883 · doi:10.5555/1858842.1858883
- Fangwen Mu Lin Shi Song Wang Zhuohao Yu Binquan Zhang Chenxue Wang Shichao Liu and Qing Wang. 2023. ClarifyGPT: Empowering LLM-based code generation with intention clarification. arXiv:2310.10996. Retrieved from https://arxiv.org/abs/2310.10996 · arXiv
- Manisha Mukherjee and Vincent J Hellendoorn. 2023. Stack over-flowing with results: The case for domain-specific pre-training over one-size-fits-all models. arXiv:2306.03268. Retrieved from https://arxiv.org/abs/2306.03268 · arXiv
- Daye Nam Andrew Macvean Vincent Hellendoorn Bogdan Vasilescu and Brad Myers. 2023. In-IDE generation-based information support with a large language model. arXiv:2307.08177. Retrieved from https://arxiv.org/abs/2307.08177 · arXiv
- Nathalia Nascimento Paulo Alencar and Donald Cowan. 2023. Comparing software developers with ChatGPT: An empirical investigation. arXiv:2305.11837. Retrieved from https://arxiv.org/abs/2305.11837 · arXiv
- Phuong T. Nguyen Juri Di Rocco Claudio Di Sipio Riccardo Rubei Davide Di Ruscio and Massimiliano Di Penta. 2023. Is this snippet written by chatgpt? An empirical study with a codebert-based classifier. arXiv:2307.09381. Retrieved from https://arxiv.org/abs/2307.09381 · arXiv
- Ansong Ni, Srini Iyer, Dragomir Radev, Veselin Stoyanov, Wen-tau Yih, Sida Wang, and Xi Victoria Lin. 2023. Lever: Learning to verify language-to-code generation with execution. In Proceedings of the International Conference on Machine Learning. PMLR, 26106–26128.
- Ansong Ni Pengcheng Yin Yilun Zhao Martin Riddell Troy Feng Rui Shen Stephen Yin Ye Liu Semih Yavuz Caiming Xiong Shafiq Joty Yingbo Zhou Dragomir Radev and Arman Cohan. 2023. L2ceval: Evaluating language-to-code generation capabilities of large language models. arXiv:2309.17446. Retrieved from https://arxiv.org/abs/2309.17446 · arXiv
- Erik Nijkamp Hiroaki Hayashi Caiming Xiong Silvio Savarese and Yingbo Zhou. 2023. CodeGen2: Lessons for training llms on programming and natural languages. arXiv:2305.02309. Retrieved from https://arxiv.org/abs/2305.02309 · arXiv
- Erik Nijkamp Bo Pang Hiroaki Hayashi Lifu Tu Huan Wang Yingbo Zhou Silvio Savarese and Caiming Xiong. 2022. CodeGen: An open large language model for code with multi-turn program synthesis. arXiv:2203.13474. Retrieved from https://arxiv.org/abs/2203.13474 · arXiv
- Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. CodeGen: An open large language model for code with multi-turn program synthesis. In Proceedings of the International Conference on Learning Representations. Retrieved from https://api.semanticscholar.org/CorpusID:252668917
- David Noever. 2023. Can large language models find and fix vulnerable software? arXiv:2308.10345. Retrieved from https://arxiv.org/abs/2308.10345 · arXiv
- Marcel Ochs, Krishna Narasimhan, and Mira Mezini. 2023. Evaluating and improving transformers pre-trained on ASTS for code completion. In Proceedings of the 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER ’16). IEEE, 834–844.
- OpenAI. 2023. GPT-4 technical report. arXiv:2303.08774. Retrieved from https://arxiv.org/abs/2303.08774 · arXiv
- Shuyin Ouyang Jie M. Zhang Mark Harman and Meng Wang. 2023. Llm is like a box of chocolates: The non-determinism of chatgpt in code generation. arXiv:2308.02828. Retrieved from https://arxiv.org/abs/2308.02828 · arXiv
- Jialing Pan Adrien Sadé Jin Kim Eric Soriano Guillem Sole and Sylvain Flamant. 2023. Stelocoder: A decoder-only LLM for multi-language to python code translation. arXiv:2310.15539. Retrieved from https://arxiv.org/abs/2310.15539 · arXiv
- Rangeet Pan Ali Reza Ibrahimzada Rahul Krishna Divya Sankar Lambert Pouguem Wassi Michele Merler Boris Sobolev Raju Pavuluri Saurabh Sinha and Reyhaneh Jabbarvand. 2023. Understanding the effectiveness of large language models in code translation. arXiv:2308.03109. Retrieved from https://arxiv.org/abs/2308.03109 · arXiv
- Bhargavi Paranjape Scott Lundberg Sameer Singh Hannaneh Hajishirzi Luke Zettlemoyer and Marco Tulio Ribeiro. 2023. Art: Automatic multi-step reasoning and tool-use for large language models. arXiv:2303.09014. Retrieved from https://arxiv.org/abs/2303.09014 · arXiv
- Emilio Parisotto Abdel-rahman Mohamed Rishabh Singh Lihong Li Dengyong Zhou and Pushmeet Kohli. 2016. Neuro-symbolic program synthesis. arXiv:1611.01855. Retrieved from https://arxiv.org/abs/1611.01855 · arXiv
- Arkil Patel Siva Reddy Dzmitry Bahdanau and Pradeep Dasigi. 2023. Evaluating in-context learning of libraries for code generation. arXiv:2311.09635. Retrieved from https://arxiv.org/abs/2311.09635 · arXiv
- Rishov Paul Md Mohib Hossain Masum Hasan and Anindya Iqbal. 2023. Automated program repair based on code review: How do pre-trained transformer models perform? arXiv:2304.07840. Retrieved from https://arxiv.org/abs/2304.07840 · arXiv
- Rishov Paul Md. Mohib Hossain Mohammed Latif Siddiq Masum Hasan Anindya Iqbal and Joanna C. S. Santos. 2023. Enhancing automated program repair through fine-tuning and prompt engineering. arXiv:2304.07840. Retrieved from https://arxiv.org/abs/2304.07840 · arXiv
- Tommaso Pegolotti Elias Frantar Dan Alistarh and Markus Püschel. 2023. QIGen: Generating efficient kernels for quantized inference on large language models. arXiv:2307.03738. Retrieved from https://arxiv.org/abs/2307.03738 · arXiv
- Kexin Pei, David Bieber, Kensen Shi, Charles Sutton, and Pengcheng Yin. 2023. Can large language models reason about program invariants? In Proceedings of the 40th International Conference on Machine Learning (ICML’23). Vol. 202, 27496–27520.
- Sanyogita Piya and Allison Sullivan. 2023. LLM4TDD: Best practices for test driven development using large language models. arXiv:2312.04687. Retrieved from https://arxiv.org/abs/2312.04687 · arXiv
- Amrit Poudel Jinfeng Lin and Jane Cleland-Huang. 2023. Leveraging transformer-based language models to automate requirements satisfaction assessment. arXiv:2312.04463. Retrieved from https://arxiv.org/abs/2312.04463 · arXiv
- Rohith Pudari and Neil A. Ernst. 2023. From copilot to pilot: Towards AI supported software development. arXiv:2303.04142. Retrieved from https://arxiv.org/abs/2303.04142 · arXiv
- Mengnan Qi Yufan Huang Maoquan Wang Yongqiang Yao Zihan Liu Bin Gu Colin Clement and Neel Sundaresan. 2023. Sut: Active defects probing for transcompiler models. arXiv:2310.14209. Retrieved from https://arxiv.org/abs/2310.14209 · arXiv
- Vu Le Anh Quan Chau Thuan Phat Kiet Van Nguyen Phan The Duy and Van-Hau Pham. 2023. XGV-BERT: Leveraging contextualized language model and graph neural network for efficient software vulnerability detection. arXiv:2309.14677. Retrieved from https://arxiv.org/abs/2309.14677 · arXiv
- Alec Radford and Karthik Narasimhan. 2018. Improving language understanding by generative pre-training. Retrieved from https://api.semanticscholar.org/CorpusID:49313245
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.
- 10.5555/3455716.3455856 · doi:10.5555/3455716.3455856
- Sajjad Rahmani AmirHossein Naghshzan and Latifa Guerrouj. 2023. Improving code example recommendations on informal documentation using bert and query-aware LSH: A comparative study. arXiv:2305.03017. Retrieved from https://arxiv.org/abs/2305.03017 · arXiv
- Abhinav Rao Sachin Vashistha Atharva Naik Somak Aditya and Monojit Choudhury. 2023. Tricking llms into disobedience: Understanding analyzing and preventing jailbreaks. arXiv:2305.14965. Retrieved from https://arxiv.org/abs/2305.14965 · arXiv
- Nikitha Rao Jason Tsay Kiran Kate Vincent J. Hellendoorn and Martin Hirzel. 2023. AI for low-code for AI. arXiv:2305.20015. Retrieved from https://arxiv.org/abs/2305.20015 · arXiv
- Xiaoxue Ren, Xinyuan Ye, Dehai Zhao, Zhenchang Xing, and Xiaohu Yang. 2023. From misuse to mastery: Enhancing code generation with knowledge-driven ai chaining. In Proceedings of the 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE ’23). IEEE, 976–987.
- Tal Ridnik Dedy Kredo and Itamar Friedman. 2024. Code generation with alphacodium: From prompt engineering to flow engineering. arXiv:2401.08500. Retrieved from https://arxiv.org/abs/2401.08500 · arXiv
- Baptiste Roziere Marie-Anne Lachaux Marc Szafraniec and Guillaume Lample. 2021. DOBF: A deobfuscation pre-training objective for programming languages. arXiv:2102.07492. Retrieved from https://arxiv.org/abs/2102.07492 · arXiv
- Fernando Vallecillos Ruiz Anastasiia Grishina Max Hort and Leon Moonen. 2024. A novel approach for automatic program repair using round-trip translation with large language models. arXiv:2401.07994. Retrieved from https://arxiv.org/abs/2401.07994 · arXiv
- Iman Saberi Fatemeh Fard and Fuxiang Chen. 2023. Multilingual adapter-based knowledge aggregation on code summarization for low-resource languages. arXiv:2307.07854. Retrieved from https://arxiv.org/abs/2307.07854 · arXiv
- Ahmed Sadik Antonello Ceravola Frank Joublin and Jibesh Patra. 2023. Analysis of ChatGPT on source code. arXiv:2306.00597. Retrieved from https://arxiv.org/abs/2306.00597 · arXiv
- Anthony Saieva Saikat Chakraborty and Gail Kaiser. 2023. On contrastive learning of semantic similarity forcode to code search. arXiv:2305.03843. Retrieved from https://arxiv.org/abs/2305.03843 · arXiv
- Fardin Ahsan Sakib Saadat Hasan Khan and A. H. M. Karim. 2023. Extending the frontier of ChatGPT: Code generation and debugging. arXiv:2307.08260. Retrieved from https://arxiv.org/abs/2307.08260 · arXiv
- Teven Le Scao Angela Fan Christopher Akiki Ellie Pavlick Suzana Ilić Daniel Hesslow Roman Castagné Alexandra Sasha Luccioni François Yvon Matthias Gallé Jonathan Tow Alexander M. Rush Stella Biderman Albert Webson Pawan Sasanka Ammanamanchi Thomas Wang Benoît Sagot Niklas Muennighoff Albert Villanova del Moral Olatunji Ruwase Rachel Bawden Stas Bekman Angelina McMillan-Major Iz Beltagy Huu Nguyen L ucile Saulnier Samson Tan Pedro Ortiz Suarez Victor Sanh Hugo Laurençon Yacine Jernite Julien Launay Margaret Mitchell Colin Raffel Aaron Gokaslan Adi Simhi Aitor Soroa Alham Fikri Aji Amit Alfassy Anna Rogers Ariel Kreisberg Nitzav Canwen Xu Chenghao Mou Chris Emezue Christopher Klamm Colin Leong Daniel van Strien David Ifeoluwa Adelani Dragomir Radev Eduardo González Ponferrada Efrat Levkovizh Ethan Kim Eyal Bar Natan Francesco De Toni Gérard Dupont Germán Kruszewski Giada Pistilli Hady Elsahar Hamza Benyamina Hieu Tran Ian Yu Idris Abdulmumin Isaac Johnson Itziar Gonzalez-Dios Javier de la Rosa Jenny Chim Jesse Dodge Jian Zhu Jonathan Chang Jörg Frohberg Joseph Tobing Joydeep Bhattacharjee Khalid Almubarak Kimbo Chen Kyle Lo Leandro Von Werra Leon Weber Long Phan Loubna Ben allal Ludovic Tanguy Manan Dey Manuel Romero Muñoz Maraim Masoud María Grandury Mario Šaško Max Huang Maximin Coavoux Mayank Singh Mike Tian-Jian Jiang Minh Chien Vu Mohammad A. Jauhar Mustafa Ghaleb Nishant Subramani Nora Kassner Nurulaqilla Khamis Olivier Nguyen Omar Espejel Ona de Gibert Paulo Villegas et al. 2022. BLOOM: A 176B-parameter open-access multilingual language model. arXiv:2211.05100. Retrieved from https://arxiv.org/abs/2211.05100 · arXiv
- Imanol Schlag Sainbayar Sukhbaatar Asli Celikyilmaz Wen tau Yih Jason Weston Jürgen Schmidhuber and Xian Li. 2023. Large language model programs. arXiv:2305.05364. Retrieved from https://arxiv.org/abs/2305.05364 · arXiv
- Martin Schroder. 2023. AutoScrum: Automating project planning using large language models. arXiv:2306.03197. Retrieved from https://arxiv.org/abs/2306.03197 · arXiv
- Oussama Ben Sghaier and Houari Sahraoui. 2023. A multi-step learning approach to assist code review. In Proceedings of the 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER ’23). IEEE, 450–460.
- Murray Shanahan. 2022. Talking about large language models. arXiv:2212.03551. Retrieved from https://arxiv.org/abs/2212.03551 · arXiv
- Anton Shapkin Denis Litvinov and Timofey Bryksin. 2023. Entity-augmented code generation. arXiv:2312.08976. Retrieved from https://arxiv.org/abs/2312.08976 · arXiv
- Alexey Shestov Anton Cheshkov Rodion Levichev Ravil Mussabayev Pavel Zadorozhny Evgeny Maslov Chibirev Vadim and Egor Bulychev. 2024. Finetuning large language models for vulnerability detection. arXiv:2401.17010. Retrieved from https://arxiv.org/abs/2401.17010 · arXiv
- Ensheng Shi Yanlin Wang Hongyu Zhang Lun Du Shi Han Dongmei Zhang and Hongbin Sun. 2023. Towards efficient fine-tuning of pre-trained code models: An experimental study and beyond. arXiv:2304.05216. Retrieved from https://arxiv.org/abs/2304.05216 · arXiv
- Ensheng Shi Fengji Zhang Yanlin Wang Bei Chen Lun Du Hongyu Zhang Shi Han Dongmei Zhang and Hongbin Sun. 2023. SoTaNa: The open-source software development assistant. arXiv:2308.13416. Retrieved from https://arxiv.org/abs/2308.13416 · arXiv
- Zejian Shi, Yun Xiong, Xiaolong Zhang, Yao Zhang, Shanshan Li, and Yangyong Zhu. 2022. Cross-modal contrastive learning for code search. In Proceedings of the 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME ’22). IEEE, 94–105.
- Jiho Shin Sepehr Hashtroudi Hadi Hemmati and Song Wang. 2023. Domain adaptation for deep unit test case generation. arXiv:2308.08033. Retrieved from https://arxiv.org/abs/2308.08033 · arXiv
- Jiho Shin Clark Tang Tahmineh Mohati Maleknaz Nayebi Song Wang and Hadi Hemmati. 2023. Prompt engineering or fine tuning: An empirical assessment of large language models in automated software engineering tasks. arXiv:2310.10508. Retrieved from https://arxiv.org/abs/2310.10508 · arXiv
- Atsushi Shirafuji Yutaka Watanobe Takumi Ito Makoto Morishita Yuki Nakamura Yusuke Oda and Jun Suzuki. 2023. Exploring the robustness of large language models for solving programming problems. arXiv:2306.14583. Retrieved from https://arxiv.org/abs/2306.14583 · arXiv
- Alexander Shypula Aman Madaan Yimeng Zeng Uri Alon Jacob Gardner Milad Hashemi Graham Neubig Parthasarathy Ranganathan Osbert Bastani and Amir Yazdanbakhsh. 2023. Learning performance-improving code edits. arXiv:2302.07867. Retrieved from https://arxiv.org/abs/2302.07867 · arXiv
- Mohammed Latif Siddiq Beatrice Casey and Joanna Santos. 2023. A lightweight framework for high-quality code generation. arXiv:2307.08220. Retrieved from https://arxiv.org/abs/2307.08220 · arXiv
- Mohammed Latif Siddiq Joanna Santos Ridwanul Hasan Tanvir Noshin Ulfat Fahmid Al Rifat and Vinicius Carvalho Lopes. 2023. Exploring the effectiveness of large language models in generating unit tests. arXiv:2305.00418. Retrieved from https://arxiv.org/abs/2305.00418 · arXiv
- André Silva Sen Fang and Martin Monperrus. 2023. Repairllama: Efficient representations and fine-tuned adapters for program repair. arXiv:2312.15698. Retrieved from https://arxiv.org/abs/2312.15698 · arXiv
- Benjamin Steenhoek Michele Tufano Neel Sundaresan and Alexey Svyatkovskiy. 2023. Reinforcement learning from automatic feedback for high-quality unit test generation. arXiv:2310.02368. Retrieved from https://arxiv.org/abs/2310.02368 · arXiv
- Hongjin Su Jungo Kasai Chen Henry Wu Weijia Shi Tianlu Wang Jiayi Xin Rui Zhang Mari Ostendorf Luke Zettlemoyer Noah A. Smith and Tao Yu. 2022. Selective annotation makes language models better few-shot learners. arXiv:2209.01975. Retrieved from https://arxiv.org/abs/2209.01975 · arXiv
- Jiamou Sun Zhenchang Xing Qinghua Lu Xiwei Xu Liming Zhu Thong Hoang and Dehai Zhao. 2023. Silent vulnerable dependency alert prediction with vulnerability key aspect explanation. arXiv:2302.07445. Retrieved from https://arxiv.org/abs/2302.07445 · arXiv
- Weisong Sun Chunrong Fang Yudu You Yuchen Chen Yi Liu Chong Wang Jian Zhang Quanjun Zhang Hanwei Qian Wei Zhao Yang Liu and Zhenyu Chen. 2023. A prompt learning framework for source code summarization. arXiv:2312.16066. Retrieved from https://arxiv.org/abs/2312.16066 · arXiv
- Weisong Sun Chunrong Fang Yudu You Yun Miao Yi Liu Yuekang Li Gelei Deng Shenghan Huang Yuchen Chen Quanjun Zhang Hanwei Qian Yang Liu and Zhenyu Chen. 2023. Automatic code summarization via ChatGPT: How far are we? arXiv:2305.12865. Retrieved from https://arxiv.org/abs/2305.12865 · arXiv
- Yuqiang Sun Daoyuan Wu Yue Xue Han Liu Wei Ma Lyuye Zhang Miaolei Shi and Yang Liu. 2024. LLM4Vuln: A unified evaluation framework for decoupling and enhancing LLMS’ vulnerability reasoning. arXiv:2401.16185. Retrieved from https://arxiv.org/abs/2401.16185 · arXiv
- Yuqiang Sun Daoyuan Wu Yue Xue Han Liu Haijun Wang Zhengzi Xu Xiaofei Xie and Yang Liu. 2023. When GPT meets program analysis: Towards intelligent detection of smart contract logic vulnerabilities in gptscan. arXiv:2308.03314. Retrieved from https://arxiv.org/abs/2308.03314 · arXiv
- Zhensu Sun Xiaoning Du Fu Song Shangwen Wang and Li Li. 2024. When neural code completion models size up the situation: Attaining cheaper and faster completion through dynamic model inference. arXiv:2401.09964. Retrieved from https://arxiv.org/abs/2401.09964 · arXiv
- Chee Wei Tan Shangxin Guo Man Fai Wong and Ching Nam Hang. 2023. Copilot for xcode: Exploring AI-assisted programming by prompting cloud-based large language models. arXiv:2307.14349. Retrieved from https://arxiv.org/abs/2307.14349 · arXiv
200 of 271 shown
Cited by
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
“For example, Hou et al. [14] provided an excellent recent SLR covering 229 research papers from 2017 to 2023 reporting SE tasks tackled, data collection and preprocessing techniques, and strategies for optimising LLM performance (such as prompt engineering).”
BackgroundSoftware Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573
Large Language Models for Software Engineering: Survey and Open Problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · arXiv · doi:10.48550/arxiv.2310.03533 · Cited by 523
MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots
Deng, G., Liu, Y., Li, Y., Wang, K., Zhang, Y., Li, Z., Wang, H., Zhang, T., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2307.08715 · Cited by 418
Self-citation
Metadata
- Authors
- Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H.
- Affiliations
- Huazhong University of Science and Technology (CN)
- Monash University (AU)
- Singapore Management University (SG)
- Beihang University (CN)
- Hong Kong Polytechnic University (HK)
- ORCID
- Venue
- ACM Transactions on Software Engineering and Methodology
- Published
- 2024-11-30
- Volume
- 33
- Issue
- 8
- Pages
- 1-79
- ISSN
- 1049-331X (ISSN-L)
- 1557-7392
- OpenAlex
- W4402665833
- Cited by
- 1,029
- References
- 321
- Citations by year
- 2026: 469 · 2025: 417 · 2024: 135
- FWCI
- 58.78
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Software Engineering and Methodology (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Funders
- National Natural Science Foundation of China (62072046)
- Key R&D Program of Hubei (2023BAB017, 2023BAB079)
- National Research Foundation, under its Investigatorship (NRF-NRFI08-2022-0002)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, software engineering tasks, LLM evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.