DebugBench: Evaluating Debugging Capability of Large Language Models

Large Language Models for Software Engineering: A Systematic Literature Review

Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029 · see citing works

When LLMs meet cybersecurity: a systematic literature review

Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 220 · see citing works

Look Before You Leap: An Exploratory Study of Uncertainty Analysis for Large Language Models

Huang, Y., Song, J., Wang, Z., Zhao, S., Chen, H., Juefei-Xu, F., & Ma, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3519464 · Cited by 132 · see citing works

Generative AI and LLMs for Critical Infrastructure Protection: Evaluation Benchmarks, Agentic AI, Challenges, and Opportunities

Yigit, Y., Ferrag, M. A., Ghanem, M. C., Sarker, I. H., Maglaras, L. A., Chrysoulas, C., Moradpoor, N., Tihanyi, N., & Janicke, H. (2025) · Sensors · doi:10.3390/s25061666 · Cited by 69 · see citing works

A Comprehensive Overview and Analysis of Large Language Models: Trends and Challenges

Mohammed, A., & Kora, R. (2025) · IEEE Access · doi:10.1109/access.2025.3573955 · Cited by 38 · see citing works

Cognitive Agents Powered by Large Language Models for Agile Software Project Management

Cinkusz, K., Chudziak, J. A., & Niewiadomska‐Szynkiewicz, E. (2024) · Electronics · doi:10.3390/electronics14010087 · Cited by 36 · see citing works

AutoVerus: Automated Proof Generation for Rust Code

Yang, C., Li, X., Misu, M. R. H., Yao, J., Cui, W., Gong, Y., Hawblitzel, C., Lahiri, S., Lorch, J. R., Lu, S., et al. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763174 · Cited by 20 · see citing works

Let’s Fix this Together: Conversational Debugging with GitHub Copilot

Bajpai, Y., Chopra, B., Biyani, P., Aslan, C., Coleman, D., Gulwani, S., Parnin, C., Radhakrishna, A., & Soares, G. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00011 · Cited by 18 · see citing works

Aligning the Objective of LLM-Based Program Repair

Xu, J., Fu, Y., Tan, S. H., & He, P. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00169 · Cited by 27 · see citing works

Debugging with Open-Source Large Language Models: An Evaluation

Majdoub, Y., & Charrada, E. B. (2024) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3674805.3690758 · Cited by 13 · see citing works

Improving Retrieval-Augmented Deep Assertion Generation via Joint Training

Zhang, Q., Fang, C., Zheng, Y., Qian, R., Yu, S., Zhao, Y., Zhou, J., Yang, Y., Zheng, T., & Chen, Z. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3545970 · Cited by 11 · see citing works

Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-Trained Language Models

Zhang, Q., Fang, C., Zheng, Y., Zhang, Y., Zhao, Y. F., Huang, R., Zhou, J., Yang, Y. J., Zheng, T., & Chen, Z. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3721128 · Cited by 11 · see citing works

A Taxonomy of Inefficiencies in LLM-Generated Python Code

Abbassi, A. A., Silva, L. D., Nikanjam, A., & Khomh, F. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00043 · Cited by 11 · see citing works

Large Language Models are Catalyzing Chemistry EducationPreprint

Du, Y., Duan, C., Bran, A. M., Sotnikova, A., Qu, Y., Kulik, H. J., Bosselut, A., Xu, J., & Schwaller, P. (2024) · ChemRxiv · doi:10.26434/chemrxiv-2024-h722v · Cited by 10 · see citing works

An LLM-Enabled Multi-Agent Autonomous Mechatronics Design Framework

Wang, Z., Lo, F. P.-W., Chen, Q., Zhang, Y., Chen, L., Chen, X., Yu, Z., Thompson, A. J., Yeatman, E. M., & Lo, B. P. L. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00404 · Cited by 7 · see citing works

Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling

Kavathekar, I., Donakanti, R., Kumaraguru, P., & Vaidhyanathan, K. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3756681.3757001 · Cited by 8 · see citing works

Prospects for using artificial intelligence to understand intrinsic kinetics of heterogeneous catalytic reactions

Medford, A. J., Whittaker, T. N., Kreitz, B., Flaherty, D. W., & Kitchin, J. (2026) · Current Opinion in Chemical Engineering · doi:10.1016/j.coche.2026.101232 · Cited by 6 · see citing works

Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Unified Approach for Elevating Benchmark Quality

Koohestani, R., de Bekker, P., Koç, B., & Izadi, M. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3644183 · Cited by 5 · see citing works

The Digital Cybersecurity Expert: How Far Have We Come?

Wang, D., Zhou, G., Li, X., Bai, Y., Chen, L., Qin, T., Sun, J. Q., & Li, D. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00198 · Cited by 5 · see citing works

LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarks

Zhou, X., Weyssow, M., Widyasari, R., Zhang, T., He, J., Lyu, Y., Chang, J., Zhang, B., Huang, D., & Lo, D. (2025) · arXiv · doi:10.48550/arxiv.2502.06215 · Cited by 3 · see citing works

Automated summarization of software documents: an LLM-based multi-agent approach

Nguyen, D. S. H., Nguyen, M. T., Nguyen, P. T., Rocco, J. D., & Ruscio, D. D. (2026) · Automated Software Engineering · doi:10.1007/s10515-025-00588-4 · Cited by 3 · see citing works

Risk Assessment Framework for Code LLMs via Leveraging Internal States

Huang, Y., Ma, L., Nishikino, K., & Akazaki, T. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728566 · Cited by 3 · see citing works

Defects4C: Benchmarking Large Language Model Repair Capability with C/C++ Bugs

Wang, J., Xie, X., Hu, Q., Liu, S., Yu, J., Kong, J., & Li, Y. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00029 · Cited by 2 · see citing works

Exploring the Impacts and Challenges of Vibe Coding Paradigm to Children's Programming Learning and Practices

Si, J. J., Li, D., Wang, Q., Liu, A. W., Lin, X., Zhu, Y., Zhou, X., Wang, A. Y., & Wang, K. Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790366 · Cited by 3 · see citing works

SpeedGen: Enhancing Code Efficiency through Large Language Model-Based Performance Optimization

Purschke, N., Kirchner, S., & Knoll, A. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00045 · Cited by 2 · see citing works

Towards Adaptive Software Agents for Debugging

Majdoub, Y., Charrada, E. B., & Touati, H. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728514 · Cited by 1 · see citing works

Characterizing User-Reported Risks across LLM Chatbots

Li, L., Ma, R., Xue, Z., & Xiong, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791505 · Cited by 1 · see citing works

Acoda: Adversarial Code Obfuscation for Defending against LLM-based Analysis

Rao, H., Dong, Z., Zhao, Y., Li, H., & Wang, H. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3774748.3793624 · Cited by 1 · see citing works

An Automated Methodology for Generating Labeled Datasets of Semantic Errors in Code

Kassem, M., Ribeiro, F., & Nadi, S. (2026) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1145/3786181.3788730 · Cited by 1 · see citing works

Large Language Models in Electrical and Computer Engineering Education: Opportunities, Challenges and the Road Ahead

Alwarafy, A. (2026) · European Journal of Education · doi:10.1111/ejed.70761 · Cited by 1 · see citing works

Effective Code Generation Using Program Synthesis And CodeLLM

Sadasivam, G. S., & Dharun Kanna, N. (2025) · International Conference on Computational Innovations and Engineering Sustainability (ICCIES) · doi:10.1109/iccies63851.2025.11032972

Semantic Analysis of ChatGPT’s Behavior in Contextual Code Correction for Python

Alhalangy, A. (2025) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.3419

Debugmate: an AI agent for efficient on-call debugging in complex production systems

Modi, R., Reddy, N., & Kodur, S. S. (2025) · Discover Data · doi:10.1007/s44248-025-00074-y

Analysis of Programming Capability of LLMs in the Context of Computer Science I

Qu, J., Bai, S., Jeff, B. A., & Khosravi, E. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08649-5_7

Bugs in AI-Generated Code - Understanding Bug Patterns and Possible Fix Strategies

Gao, R. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00098

Code Redteaming: Probing Ethical Sensitivity of LLMs Through Natural Language Embedded in Code

Park, C., Yoon, J., & Lim, H. (2026) · Mathematics · doi:10.3390/math14010189

AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation

Zhang, T., Mao, X.-J., Wang, S., Zhao, Y.-X., Lu, Y., Zhang, J., Zhang, Z., Yang, K., & Yu, Y. (2025) · International Conference on Automated Software Engineering · doi:10.1109/ase63991.2025.00103 · Cited by 2 · see citing works

Walking on Thin Ice?: Defending RAG-based Code Recommender Systems Against Poisoning AttacksPreprint

Nguyen, P. T., Ngo, Q. M., Vu, D. H., Bui, A. M. T., Ruscio, D. D., & Penta, M. D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6193142

Large Language Models-Driven Bug Localization Framework for Automated Debugging

Dhanalakshmi, J., Naik, D. A., Evangeline, C., Shanthi, A. V. K., Eapen, M., & Dhanraj, J. A. (2025) · International Conference on Artificial Intelligence and Knowledge Discovery in Concurrent Engineering (ICECONF) · doi:10.1109/iceconf65644.2025.11379560

AI-Driven Intrusion Detection Systems for Protecting Critical Infrastructure in Cybersecurity

Khobragade, P., Dhone, M., Chakole, N., Ghumde, R., Dhankar, P. K., & Shobhane, P. D. (2025) · IEEE International Conference on Smart Innovations in Systems, Infrastructure, Mechanical, Power, AI and Computing Technologies (SISIMPACT) · doi:10.1109/sisimpact67725.2025.11439790

SimStep: Human-in-the-Loop Authoring of Interactive Educational Simulations Through Task-Level Abstractions

Kaputa, Z. G., Rajaram, A., Feliciano, V. A., Lyu, Z., Agrawala, M., & Subramonyam, H. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791514

JavaTutor: An Integrated Large Language Model Tutoring Platform for Learning Java

Dhongade, A. M., & Wu, C.-S. M. (2026) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa67686.2026.11468147

Llm-Based Framework for Improving Code Quality in Oilfield Data Acquisition Systems

Yang, W., Chen, J., & Estevez, C. (2026) · IEEE SoutheastCon · doi:10.1109/southeastcon63549.2026.11476591

PAMF: An LLM-driven framework for automated mesh generation in mechanical simulation and CAE workflows

Pan, F., Yin, C., & Dou, G. (2026) · Journal of Mechanical Science and Technology · doi:10.1007/s12206-026-0334-6

Coding in a Bubble? Evaluating LLMs in Resolving Context Adaptation Bugs during Code Adaptation

Zhang, T., Mao, X., Wang, S., Zhao, Y., Lu, Y., Tang, Z., Xu, W., Sun, L., Xie, C., Yang, K., et al. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797148

Not All RAGs Are Created Equal: A Component-Wise Empirical Study for Software Engineering Tasks

Ke, Q., Zhao, Y., Leng, H., Zhao, S., & Wang, H. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808190

Design and Implementation of Data Acquisition and Analysis System for Programming Debugging Process Based On VS Code Plug-In

Liu, B. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.05.276

Closing the Loop: Code Generation and Intelligent Debugging with Large Language Models

Lyu, N. (2026) · International Conference on Computational Intelligence and Communication Networks (CICN) · doi:10.1109/cicn70047.2026.11594236

DOTA: Intelligent Debugging with D elta o f T houghts A gents

Yang, R., Gupta, R., Zhang, Q., & Kundu, A. (2026) · ACM SIGOPS Operating Systems Review · doi:10.1145/3830422.3830424

Walking on thin ice? Defending RAG-based code recommender systems against poisoning attacks

Vu, D. H., Ngo, Q. M., Bui, A. M. T., Nguyen, P. T., Ruscio, D. D., & Penta, M. D. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113064

AI-Powered Python Code Reviewer using Streamlit and Google Gemini

Rajput, D., Dixit, K., Sehrawat, A., Jain, A., Rohit, & Malik, B. (2026) · International Conference on NextGen Data Science and Analytics (ICNDSA) · doi:10.1109/icndsa68777.2026.11652287

Synthetic Repo-level Bug Dataset for Training Automated Program Repair Models

Minh, P. V. T., Phan, H. N., Phan, N. H., Le, C. A., Nguyen, T. N., & Bui, N. D. Q. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3764562

Toward Agentic Runtime Healing

Sun, Z., Zhu, H., Xu, B., Du, X., Li, L., & Lo, D. (2026) · Communications of the ACM · doi:10.1145/3800691 · Cited by 2 · see citing works

SWE-PDB: Teaching LLMs to Leverage Debugging Tools via Agentic Training

Liu, J., Hu, X., & Xia, X. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832117

TraceGate: A Framework for Policy-Controlled Failure Evidence in LLM-Assisted Program Repair

Schuler, N., Scotti, V., & Mirandola, R. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834596

RepoDebug: Repository-Level Multi-Task and Multi-Language Debugging Evaluation of Large Language Models

Liu, J., Liu, Z., Cheng, Z., He, M., Shi, X., Guo, Y., Zhu, X., Guo, Y., Wang, Y., & Wang, H. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.1294

MdEval: Massively Multilingual Code Debugging

Liu, S., Chai, L., Yang, J., Shi, J., Zhu, H., Wang, L., Jin, K., Zhang, W., Zhu, H., Guo, S., et al. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.1041 · Cited by 1 · see citing works

LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models

Parmar, M., Patel, N., Varshney, N., Nakamura, M., Luo, M., Mashetty, S., Mitra, A., & Baral, C. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2024.acl-long.739 · Cited by 108 · see citing works

T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step

Chen, Z., Du, W., Zhang, W., Liu, K., Liu, J., Zheng, M., Zhuo, J., Zhang, S., Lin, D., Chen, K., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.515 · Cited by 32 · see citing works

AI Slop: A Failure Mode of Large Language Model Reasoning in Software Engineering Debugging

Esendemir, K. (2026) · doi:10.13140/rg.2.2.20243.13605