Large language models for code completion: A systematic literature review
Open accessAbstract
Code completion serves as a fundamental aspect of modern software development, improving developers' coding processes. Integrating code completion tools into an Integrated Development Environment (IDE) or code editor enhances the coding process and boosts productivity by reducing errors and speeding up code writing while reducing cognitive load. This is achieved by predicting subsequent tokens, such as keywords, variable names, types, function names, operators, and more. Different techniques can achieve code completion, and recent research has focused on Deep Learning methods, particularly Large Language Models (LLMs) utilizing Transformer algorithms. While several research papers have focused on the use of LLMs for code completion, these studies are fragmented, and there is no systematic overview of the use of LLMs for code completion. Therefore, we aimed to perform a Systematic Literature Review (SLR) study to investigate how LLMs have been applied for code completion so far. We have formulated several research questions to address how LLMs have been integrated for code completion-related tasks and to assess the efficacy of these LLMs in the context of code completion. To achieve this, we retrieved 244 papers from scientific databases using auto-search and specific keywords, finally selecting 23 primary studies based on an SLR methodology for in-depth analysis. This SLR study categorizes the granularity levels of code completion achieved by utilizing LLMs in IDEs, explores the existing issues in current code completion systems, how LLMs address these challenges, and the pre-training and fine-tuning methods employed. Additionally, this study identifies open research problems and outlines future research directions. Our analysis reveals that LLMs significantly enhance code completion performance across several programming languages and contexts, and their capability to predict relevant code snippets based on context and partial input boosts developer productivity substantially.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Risks of LLM-based code synthesis identified in primary studies include generation of incorrect code, introduction of security vulnerabilities, unnecessarily complex code completions, and reduced code maintainability.
“Risks include the generation of incorrect code, the introduc -tion of security vulnerabilities, unnecessarily complex code completions, and reduced code maintainability.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Husein et al. (2025)
Cites
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · see citing works
5th International Conference on Learning Representations (ICLR 17)
Havrylov, S., & Titov, I. (2017) · Cited by 3,503 · see citing works
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 2,842 · see citing works
Lin, T., Wang, Y., Liu, X., & Qiu, X. (2022) · AI Open · doi:10.1016/j.aiopen.2022.10.001 · Cited by 1,422 · see citing works
Proceedings of the 2013 International Conference on Software Engineering
Notkin, D. S., Cheng, B. H. C., & Pohl, K. (2013) · Cited by 875 · see citing works
Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI'12)
Hoffmann, J., & Selman, B. (2012) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 718 · see citing works
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540 · see citing works
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335 · see citing works
Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey
Ling, C., Zhao, X., Lu, J., Deng, C., Zheng, C., Wang, J., Chowdhury, T., Li, Y., Cui, H., Zhang, X., et al. (2025) · ACM Computing Surveys · doi:10.1145/3764579 · Cited by 263 · see citing works
Machine learning applications in production lines: A systematic literature review
Kang, Z., Catal, C., & Teki̇nerdoğan, B. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.106773 · Cited by 244 · see citing works
Self-citationCode Prediction by Feeding Trees to Transformers
Kim, S., Zhao, J., Tian, Y., & Chandra, S. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00026 · Cited by 209 · see citing works
Multi-task Learning based Pre-trained Language Model for Code Completion
Liu, F., Li, G., Zhao, Y., & Jin, Z. (2020) · Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3324884.3416591 · Cited by 188 · see citing works
A survey on machine learning techniques applied to source code
Sharma, T., Kechagia, M., Georgiou, S., Tiwari, R., Vats, I., Moazen, H., & Sarro, F. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111934 · Cited by 130 · see citing works
A Review on Code Generation with LLMs: Application and Evaluation
Wang, J., & Chen, Y. (2023) · IEEE International Conference on Medical Artificial Intelligence (MedAI) · doi:10.1109/medai59581.2023.00044 · Cited by 122 · see citing works
Breaking the Silence: the Threats of Using LLMs in Software Engineering
Sallou, J., Durieux, T., & Panichella, A. (2024) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1145/3639476.3639764 · Cited by 115 · see citing works
An Empirical Study on the Usage of Transformer Models for Code Completion
Ciniselli, M., Cooper, N., Pascarella, L., Mastropaolo, A., Aghajani, E., Poshyvanyk, D., Penta, M. D., & Bavota, G. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3128234 · Cited by 101 · see citing works
ReACC: A Retrieval-Augmented Code Completion Framework
Lu, S., Duan, N., Han, H., Guo, D., Hwang, S.-W., & Svyatkovskiy, A. (2022) · arXiv · doi:10.48550/arxiv.2203.07722 · Cited by 96 · see citing works
An Empirical Study on the Usage of BERT Models for Code Completion
Ciniselli, M., Cooper, N., Pascarella, L., Poshyvanyk, D., Penta, M. D., & Bavota, G. (2021) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00024 · Cited by 93 · see citing works
Cruz-Benito, J., Vishwakarma, S., Martín-Fernández, F., & Faro, I. (2021) · AI · doi:10.3390/ai2010001 · Cited by 45 · see citing works
Domain Adaptive Code Completion via Language Models and Decoupled Domain Databases
Tang, Z. J., Ge, J., Liu, S., Zhu, T., Xu, T., Huang, L., & Luo, B. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00076 · Cited by 40 · see citing works
Representation Learning for Stack Overflow Posts: How Far Are We?
He, J., Zhou, X., Xu, B., Zhang, T., Kim, K., Yang, Z., Thung, F., Irsan, I. C., & Lo, D. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3635711 · Cited by 32 · see citing works
A unified multi-task learning model for AST-level and token-level code completion
Liu, F., Li, G., Wei, B., Xia, X., Fu, Z., & Jin, Z. G. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10140-7 · Cited by 27 · see citing works
Espejel, J. L., Alassan, M. S. Y., Chouham, E. M., Dahhane, W., & Ettifouri, E. H. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100013 · Cited by 23 · see citing works
Non-Autoregressive Line-Level Code Completion
Liu, F., Fu, Z., Li, G., Jin, Z., Liu, H., Hao, Y., & Zhang, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3649594 · Cited by 22 · see citing works
Automating Code-Related Tasks Through Transformers: The Impact of Pre-training
Tufano, R., Pascarella, L., & Bavota, G. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00203 · Cited by 20 · see citing works
CodeBERT-nt: Code Naturalness via CodeBERT
Khanfir, A., Jimenez, M., Papadakis, M., & Traon, Y. L. (2022) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs57517.2022.00098 · Cited by 18 · see citing works
Syntax-aware on-the-fly code completion
Takerngsaksiri, W., Tantithamthavorn, C. K., & Li, Y.-F. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107336 · Cited by 14 · see citing works
Benchmarking Causal Study to Interpret Large Language Models for Source Code
Rodriguez-Cardenas, D., Palacio, D. N., Khati, D., Burke, H., & Poshyvanyk, D. (2023) · 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme58846.2023.00040 · Cited by 14 · see citing works
DCServCG: A data-centric service code generation using deep learning
Alizadehsani, Z., Ghaemi, H., Shahraki, A., González‐Briones, A., & Corchado, J. M. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106304 · Cited by 12 · see citing works
Boosting source code suggestion with self-supervised Transformer Gated Highway
Hussain, Y., Huang, Z., Zhou, Y., & Wang, S. (2022) · Journal of Systems and Software · doi:10.1016/j.jss.2022.111553 · Cited by 10 · see citing works
Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion
Pei, H., Zhao, J., Lausen, L., Zha, S., & Karypis, G. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25653 · Cited by 10 · see citing works
CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming SequencesPreprint
Izadi, M., Gismondi, R., & Gousios, G. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.06689 · Cited by 9 · see citing works
On the Applicability of Language Models to Block-Based Programs
Griebl, E., Fein, B., Obermüller, F., Fraser, G., & Just, R. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00199 · Cited by 8 · see citing works
Neural language models for code quality identification
Sengamedu, S. H., & Zhao, H. (2022) · doi:10.1145/3549034.3561175 · Cited by 7 · see citing works
VSCuda: LLM based CUDA extension for Visual Studio Code
Chen, B., Mustakin, N., Hoang, A., Fuad, S., & Wong, D. (2023) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3624062.3624064 · Cited by 6 · see citing works
Source code auto-completion using various deep learning models under limited computing resources
Sharma, M., Mishra, T. K., & Kumar, A. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00708-7 · Cited by 5 · see citing works
Next Syntactic-Unit Code Completion and Applications
Nguyen, A. T., Yadavally, A., & Nguyen, T. N. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3559544 · Cited by 5 · see citing works
Diera, A., Dahou, A. H., Galke, L. P. A., Karl, F., Sihler, F., & Scherp, A. (2023) · GenBench Workshop on Generalisation (Benchmarking) in NLP (GenBench) · doi:10.18653/v1/2023.genbench-1.2 · Cited by 5 · see citing works
A methodology for refined evaluation of neural code completion approaches
Le, K. T., Rashidi, G., & Andrzejak, A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00866-9 · Cited by 4 · see citing works
AdaComplete: improve DL-based code completion method’s domain adaptability
Wang, Z., Liu, F., Hao, Y., & Jin, Z. G. (2023) · Automated Software Engineering · doi:10.1007/s10515-023-00376-y · Cited by 3 · see citing works
Enhancing Code Completion with Implicit Feedback
Jin, H., Zhou, Y., & Hussain, Y. (2023) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs60937.2023.00030 · Cited by 2 · see citing works
Enhancing intelligent IoT services development by integrated multi-token code completion
Xia, Y., Liang, T., Min, W., Kuang, L., & Gao, H. (2023) · Computer Communications · doi:10.1016/j.comcom.2023.10.014 · Cited by 1 · see citing works
An Empirical Study on the Performance of Large Pre-trained Models for Automatic Line Completion
Anonymous (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.6718872 · Cited by 1 · see citing works
Flagged first, then by citations
4 of the 50 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
LLMs: A game-changer for software engineers?
Haque, M. A. (2025) · BenchCouncil Transactions on Benchmarks Standards and Evaluations · doi:10.1016/j.tbench.2025.100204 · Cited by 29 · see citing works
The dawn of a new era: can machine learning and large language models reshape QSP modeling?
Androulakis, I. P., Cucurull-Sanchez, L., Kondic, A., Mehta, K., Pichardo, C., Pryor, M., & Renardy, M. (2025) · Journal of Pharmacokinetics and Pharmacodynamics · doi:10.1007/s10928-025-09984-5 · Cited by 22 · see citing works
Towards accessible website design through artificial intelligence: A systematic literature review
Vera-Amaro, G., & Rojano-Cáceres, J. R. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107821 · Cited by 21 · see citing works
Rehan, S., Al‐Bander, B., & Ahmad, A. A.-S. (2025) · Electronics · doi:10.3390/electronics14071463 · Cited by 19 · see citing works
Lyu, W., Wang, Y., Sun, Y., & Zhang, Y. (2025) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3698205.3729544 · Cited by 19 · see citing works
Prompt engineering and its implications on the energy consumption of Large Language Models
Rubei, R., Moussaid, A., Sipio, C. D., & Ruscio, D. D. (2025) · IEEE/ACM International Workshop on Green and Sustainable Software (GREENS) · doi:10.1109/greens66463.2025.00014 · Cited by 13 · see citing works
Is LLM-Generated Code More Maintainable & Reliable Than Human-Written Code?
Molison, A. S., Moraes, M. C., Melo, G., Santos, F., & Assunção, W. K. G. (2025) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem64174.2025.00036 · Cited by 12 · see citing works
Methods and Techniques of Agentic Software Engineering: A Systematic Literature Review
Otoum, N. A., & El-Khalili, N. H. (2026) · IEEE Access · doi:10.1109/access.2026.3652325 · Cited by 11 · see citing works
Test case generation using large language models: a systematic literature review
Tasarsu, M., Tokmak, A. V., & Catal, C. (2026) · Cluster Computing · doi:10.1007/s10586-026-06021-z · Cited by 10 · see citing works
Self-citationSoftware refactoring research with large language models: A systematic literature review
Martinez, S., Xu, L., Elnaggar, M., & AlOmar, E. A. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112762 · Cited by 8 · see citing works
Desolda, G., Esposito, A., Greco, F., Tucci, C., Buono, P., & Piccinno, A. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103648 · Cited by 7 · see citing works
Díaz-Arrieta, R., Monroy, B. L. D., Heredia, L. J. C., & Cobos, A. G. V. (2025) · IEEE Access · doi:10.1109/access.2025.3594657 · Cited by 5 · see citing works
Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective
Gong, J., Giavrimis, R., Brookes, P., Voskanyan, V., Wu, F., Ashiga, M., Truscott, M., Basios, M., Kanthan, L., Xu, J., et al. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00295 · Cited by 5 · see citing works
Li, R., Liang, P., Wang, Y., Cai, Y., Sun, W., & Li, Z. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3798163 · Cited by 4 · see citing works
Leveraging Large Language Models in the Software Development Lifecycle: Opportunities and Challenges
Bhalla, J. S., & Jodhka, M. K. (2025) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2025.0161105 · Cited by 3 · see citing works
Light and shadows of smart contract development with LLMs
Napoli, E. A., Romani, N., Gatteschi, V., & Schifanella, C. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129011 · Cited by 3 · see citing works
Zhang, W., Jiang, B., Yuhong, F., Koziolek, A., Hebig, R., & Strüber, D. (2026) · Software & Systems Modeling · doi:10.1007/s10270-026-01402-9 · Cited by 3 · see citing works
SOLAR: Illuminating LLM performance in API discovery and service ranking for edge AI and IoT
Al‐Masri, E., & Subramanian, I. N. (2025) · Internet of Things · doi:10.1016/j.iot.2025.101630 · Cited by 3 · see citing works
Abdelsalam, Y., Peitek, N., Maurer, A.-M., Toneva, M., & Apel, S. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773219 · Cited by 3 · see citing works
Ru, Q., Wang, Y., Huang, C., & Luo, H. (2025) · International Conference on Computer Science and Technologies in Education (CSTE) · doi:10.1109/cste64638.2025.11092232 · Cited by 2 · see citing works
Evaluating Psychological Competency via Chinese Q&A in Large Language Models
Gao, F., He, Y., Chen, Q., & Liu, F. (2025) · Applied Sciences · doi:10.3390/app15169089 · Cited by 2 · see citing works
The fools are certain; the wise are doubtful: Exploring LLM confidence in code completion
Kotti, Z., Dritsa, K., Spinellis, D. D., & Λουρίδας, Π. (2026) · Automated Software Engineering · doi:10.21203/rs.3.rs-7302683/v1 · Cited by 2 · see citing works
AskGraph: A Dependency-Aware Code Assistant Powered by Code Graphs and LLM-Generated Cypher Queries
Yang, N., Reynolds, J., Prast, L., & Corvino, R. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00065 · Cited by 2 · see citing works
Zou, H., Li, X., Yang, Y., Yao, K., Kong, X., Shao, Z., & Gao, F. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.104372 · Cited by 2 · see citing works
Vera-Amaro, G., Vera-Amaro, R., Mata-Rivera, M.-F., & Rojano-Cáceres, J. R. (2026) · Technologies · doi:10.3390/technologies14050287 · Cited by 2 · see citing works
Is your prompt poisoning code? Defect induction rates and security mitigation strategies
Wang, B., Zhong, Y., Wan, M., Yu, W., Ouyang, Y., Huang, Y., Wu, H., & Li, H. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10866-8 · Cited by 2 · see citing works
Li, J., Deng, H., Zhang, Y., Zhang, K., Shao, T., Zhao, T., Wang, W., Jin, Z. G., Li, G., Liu, Y., et al. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797106 · Cited by 2 · see citing works
An Evaluation of the Impact of Code Generation Tools on Software Development
Osório, L. F. M., Neto, P. D. A. D. S., Avelino, G. A., & Lira, W. A. L. (2025) · Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI) · doi:10.5753/sbsi.2025.246605 · Cited by 2 · see citing works
ATHENA: A customizable LLM-based Code Completion Tool for Visual Studio Code
Desolda, G., Esposito, A., Greco, F., Tucci, C., Buono, P., & Piccinno, A. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708557.3716357 · Cited by 2 · see citing works
LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Review ⋆Preprint
Rasheed, Z., Muhammad, W., Kemell, K., Saari, M., & Abrahamsson, P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6332149 · Cited by 2 · see citing works
Sundaram, G., Venktesh, B., V, & Sundharakumar, K. B. (2025) · International Conference on Emerging Technologies in Computing and Communication (ETCC) · doi:10.1109/etcc65847.2025.11108633 · Cited by 1 · see citing works
A Survey on AI-Generated Text Detection: Methods, Challenges, and Future Directions
Ismail, A. A., Ahmed, C., & ElKorany, A. (2025) · Future International Conference on Artificial Intelligence and Cybersecurity (FICAC) · doi:10.1109/ficac65757.2025.11341850 · Cited by 1 · see citing works
Preventing Accidental Sharing of Misinformation Using Large Language Models
Franco, M., Grimm, V., & Herder, E. (2025) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3748699.3749798 · Cited by 1 · see citing works
Hou, S., Jiao, H., Liu, Z., Liang, J., Xie, L., Guan, X., Gui, Z., & Wu, H. (2026) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2026.2700394 · Cited by 1 · see citing works
Beyond More Context: How Granularity and Order Drive Code Completion Quality
Yusuf, U., Caumartin, G., & Costa, D. E. (2025) · IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) · doi:10.1109/asew67777.2025.00075 · Cited by 1 · see citing works
Cao, J., & Lu, Y. (2025) · Qeios · doi:10.32388/mlaotg · Cited by 1 · see citing works
Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability
Borg, M., Hewett, D., Hagatulah, N., Couderc, N., Söderberg, E., Graham, D., Kini, U., & Farley, D. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10889-1 · Cited by 4 · see citing works
Indonesia's Policies and Strategies for Embracing Inclusive and Environmentally Friendly Recovery
Purniyati, A. S. (2025) · Journal Of World Science · doi:10.58344/jws.v4i1.1276
Making Smart Contracts Easier To Understand
Napoli, E. A., Gangemi, L., Meneguzzo, S., Romani, N., & Gatteschi, V. (2025) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac65507.2025.00238
Evaluating Method and Field Relocation with LLMs and Prompt TuningPreprint
Morell, Z. S., & Ferreira, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5498414
Enabling Grounded Answers Through Knowledge Graphs and Retrieval Augmented Generation
Hoang, D., Gorsich, D., Castanier, M. P., & Imani, F. (2025) · SAE technical papers on CD-ROM/SAE technical paper series · doi:10.4271/2025-01-0488
Designing Human-Centered Code Completion Tools in the Age of Generative AI
Desolda, G., Esposito, A., Greco, F., Tucci, C., Buono, P., & Piccinno, A. (2025) · Biannual Conference of the Italian SIGCHI Chapter (CHItaly) · doi:10.1145/3750069.3755957
Advancements in AI-Powered Code Generation: A Comparative Analysis of Modern LLMs
Nimisha, P., & Eldho, K. J. (2025) · International Conference on Innovative Mechanisms for Industry Applications · doi:10.1109/icimia67127.2025.11200714
ChatGPT Based Best Practices for Pair Programming
Gökalp, M. S., & Kolukısa, A. (2025) · International Conference on Computer Science and Engineering (UBMK) · doi:10.1109/ubmk67458.2025.11206910
Evaluating Large Language Models Security and Resilience: A Practical Testing Framework
Nitescu, G., Ouatu, A., & Țurcanu, D. (2025) · RoEduNet Conference: Networking in Education and Research (RoEduNet) · doi:10.1109/roedunet68395.2025.11208478
Code2Req: Using Generative AI to generate requirements from source codePreprint
Persson, E., Alégroth, E., & Gorschek, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5845431
Osório, L. F. M., Neto, P. D. A. D. S., Avelino, G. A., & Lira, W. A. L. (2025) · Simpósio Brasileiro de Qualidade de Software (SBQS) · doi:10.5753/sbqs.2025.13845
Code2Req: Using Generative AI to generate requirements from source codePreprint
Persson, E., Alégroth, E., & Gorschek, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5845438
Atomizer: An LLM-based Collaborative Multi-Agent Framework for Intent-Driven Commit Untangling
Zhu, K., Tian, Z., Wang, S., Leng, M., & Mao, X. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3787773
A Fully Automated Agent for End-to-End Code Translation and Validation
Erer, E., Bozanta, A., Aytac, T., & Basar, A. (2025) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem64174.2025.00058
A Fully Automated Agent for End-to-End Code Translation and Validation
Erer, E., Bozanta, A., Aytac, T., & Basar, A. (2025) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1109/cascon66301.2025.00128
LLM-Driven Cloud-Based Infrastructure Design
Sepielli, A., Calamo, M., Bianchini, F., Luzi, F. D., Marinacci, M., Monti, F., Rossi, J., Mecella, M., Tamla, P., Krause, T., et al. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356715
Circuit explained: How does a transformer perform compositional generalization
Tang, C., Lake, B. M., & Jazayeri, M. (2026) · PLoS ONE · doi:10.1371/journal.pone.0340088
A Survey and Prospects of Source Code Processing Techniques Based on Large Language Models
Wang, L., & Yang, Z. (2025) · IEEE International Conference on Software Engineering and Service Science (ICSESS) · doi:10.1109/icsess67729.2025.11380277
IMECA method of risk-based assessment and ensuring cybersecurity of Large Language Models
Neretin, O., & Kharchenko, V. S. (2026) · Herald of Advanced Information Technology · doi:10.15276/hait.09.2026.05
Building a Vietnamese Query Completion System Using Statistical Language Model
Pham, D.-H. (2026) · International Conference on Mobile Computing and Sustainable Informatics (ICMCSI) · doi:10.1109/icmcsi67283.2026.11412887
The Impact of Artificial Intelligence on Web UI Development
Bezhentsev, Y. (2026) · Universal library of engineering technology. · doi:10.70315/uloap.ulete.2026.0301010
An Adaptive Approach to Reverse Code Completion
Torres, N. (2025) · International Conference of the Chilean Computer Science Society (SCCC) · doi:10.1109/sccc67219.2025.11420381
Prabakar, D., Bhardwaj, R., S, S. C., Tiwari, M., & Manikandan, G. (2026) · Journal of Computer Languages · doi:10.1016/j.cola.2026.101396
Albayrak, A., Albayrak, M., & Kaynaklı, M. (2026) · Bitlis Eren Üniversitesi Fen Bilimleri Dergisi · doi:10.17798/bitlisfen.1787094
LLM-based multi-agent systems for code generation: A multi-vocal literature review
Rasheeda, Z., Waseema, M., Kemella, K.-K., Saari, M., & Abrahamsson, P. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113142
Proactive Software Fault Prediction with LLMs, Balanced ML, and Postmortem Analysis
Yadav, M., Kumar, L., & Passricha, V. (2026) · Innovations in Software Engineering Conference (ISEC) · doi:10.1145/3796563.3796605
Entropy-Aware Watermarking for Code Generation Models
Shi, Y., Bao, S., Wu, H., Ye, J., & Zhang, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7820-7_13
Beyond the hype: Enabling informed LLM adoption in industry through systematic evaluation
Pizarro, E., Azanza, M., & Lamancha, B. P. (2026) · Science of Computer Programming · doi:10.1016/j.scico.2026.103530
Cui, M., Liu, Y., Lu, Z., Hu, Y., & Zhu, P. (2026) · Automated Software Engineering · doi:10.1007/s10515-026-00667-0
Anh, L. H. N., Luong, H. V. D., & Vũ, V. A. (2026) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3795095.3805199
Bui, T., Zhang, T., Thung, F., Xiong, Y., Jiang, P., Zhou, X., & Lo, D. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-5737-9_5
Pérez, R. J. P., Castillo, N. A. V., & Martínez, R. S. V. (2026) · Research Club Scientia · doi:10.65415/xfce7x48
Muthumaniraja, S. (2026) · ShodhAI Journal of Artificial Intelligence · doi:10.29121/shodhai.v3.i2.2026.97
Flores, F. D. L. C., Flores, P. A. D. L. C., Garcia, T., & Frausto, H. R. (2026) · Journal of Integrative Science and Societal Impact · doi:10.29121/jissi.v2.i2.2026.35
Maxim, B. R., Ghammam, A., Roy, M., Gandi, S., & Gujarathi, A. (2026) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--60806
Rahaman, M. F., Xueyuan, L., Sunny, M. N. M., Amjad, M., Gao, X., & Hasan, S. (2026) · Sensors · doi:10.3390/s26206663
Zekai, W., Qasem, A. G., Lu, L., Ozaydin, B., & Ahmed, A. (2026) · Healthcare · doi:10.3390/healthcare14172767
ALIoTh: Orchestrating Multi-Agent LLMs for Autonomous IoT Penetration Testing
Pironti, F. A., Arena, L., Blefari, F., Lupinacci, M., Romeo, F., Maresca, A., & Furfaro, A. (2026) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms69089.2026.11668188
Shirzadi, N., Lopez, P., & Stylianou, M. (2026) · Energy and AI · doi:10.1016/j.egyai.2026.100896
Jain, S., Purohit, P. K., Somani, V., Suwalka, A. K., Singh, N., & Kumar, A. (2026) · International Conference Computational Vision and Bio Inspired Computing · doi:10.1109/iccvbic71195.2026.11688826
Prompts Matter: On the Impact of Code Smell Remediation with Large Language Models
Hagatulah, N., Borg, M., & Söderberg, E. (2026) · doi:10.1145/3843779.3844637
Evaluating Large Language Models in conceptual modeling: An experimental design
Guimarães, P., Vieira, A., & Santos, M. Y. (2026) · Data & Knowledge Engineering · doi:10.1016/j.datak.2026.102657
LLMs for Model-driven Engineering: A Systematic Literature Review
Zhang, M., Li, Y., & Yue, T. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3859883
Wahyudin, D., & Iskandar, N. N. R. (2026) · International Journal of Engineering Science and Information Technology · doi:10.52088/ijesty.v6i4.1971
Pandini, G., Göknil, A., Shivashankar, K., Tverdal, S., Nguyen, P., Videsjorden, A. N., Fontana, F. A., & Martini, A. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10939-8
- Authors
- Husein, R. A., Aburajouh, H., & Catal, C.
- Affiliations
- Qatar University (QA)
- Corresponding author
- Catal, Cagatay
- ORCID
- Catal, Cagatay
- Venue
- Computer Standards & Interfaces
- Publisher
- Elsevier BV
- Published
- 2025-03
- Volume
- 92
- Pages
- 103917
- ISSN
- 0920-5489 (ISSN-L)
- OpenAlex
- W4401671212
- Cited by
- 72
- References
- 50
- Citations by year
- 2026: 34 · 2025: 38
- FWCI
- 4.17
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Computer Standards & Interfaces (publishedVersion, cc-by)
- Figshare (submittedVersion, cc-by)
- Qatar University QSpace (Qatar University) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- code completion, large language models, integrated development environment, code editor, token prediction, developer productivity, transformer models, pre-training and fine-tuning, programming languages, open problems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
