Methods and Techniques of Agentic Software Engineering: A Systematic Literature Review
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
Ooi, K.-B., Tan, G. W.-H., Al-Emran, M., Al-Sharafi, M. A., Capatina, A., Chakraborty, A., Dwivedi, Y. K., Huang, T.-L., Kar, A. K., Lee, V.-H., et al. (2025) · Journal of Computer Information Systems · doi:10.1080/08874417.2023.2261010 · Cited by 746 · cited 1×
“Cross-disciplinary studies can help to enhance agentic practice with diverse views and methods, thus creating stronger and more functional systems [56].”
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650 · cited 2×
“Models such as OpenAI’s Codex, GPT-4, and specialized models like AlphaCode have demonstrated capabilities that approach or sometimes exceed human-level performance on specific programming tasks [4], [8].”
Large Language Models for Software Engineering: Survey and Open Problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · arXiv · doi:10.48550/arxiv.2310.03533 · Cited by 523
Automated Program Repair in the Era of Large Pre-trained Language Models
Xia, C. S., Wei, Y., & Zhang, L. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00129 · Cited by 433
An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation
Schäfer, M., Nadi, S., Eghbali, A., & Tip, F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3334955 · Cited by 431 · cited 1×
“Test-driven development agents take this further by generating both the tests and the implementation code in lockstep, ensuring that the requirements, tests and final program stay aligned [18], [32], [42].”
A survey on LLM-based multi-agent systems: Workflow, infrastructure, and challenges
Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y. (2024) · Vicinagearth · doi:10.1007/s44336-024-00009-2 · Cited by 404 · cited 6×
“…of the most urgent requirements is the development of advanced coordination mechanisms of agents, as they require advanced coordination frameworks that allow agents to work on complex software tasks and need to collaborate and share an understanding over a protracted period of time [1], [12], [57].”
InfluentialGenerative AI for Software Practitioners
Ebert, C., & Louridas, P. (2023) · IEEE Software · doi:10.1109/ms.2023.3265877 · Cited by 293 · cited 1×
“…emerging from our analysis include improved developer productivity, as agentic approaches can amplify developer capabilities by automating routine tasks, suggesting candidate solutions, and handling parts of complex implementations that would otherwise require substantial human effort [62].”
Automated Repair of Programs from Large Language Models
Fan, Z., Gao, X., Mirchev, M., Roychoudhury, A., & Tan, S. H. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00128 · Cited by 256
Research works printed but not linked: 45
- Jin, 2024, From LLMs to LLMbased agents for software engineering: A survey of current, challenges and future, arXiv:2408.02479 · arXiv
- 10.3390/e25060888 · DOI · Paper page
- 10.1007/s10515-024-00426-z · DOI · Paper page
- 10.20537/2076-7633-2024-16-7-1715-1726 · DOI
- Liu, 2024, Large language model-based agents for software engineering: A survey, arXiv:2409.02977 · arXiv
- Yehudai, 2025, Survey on evaluation of LLM-based agents, arXiv:2503.16416 · arXiv
- Hong, MetaGPT: Meta programming for a multi-agent collaborative framework, 23247
- 10.1145/3719351 · DOI
- Yang, SWE-agent: Agent-computer interfaces enable automated software engineering, 37, 50528
- Esposito, 2025, Generative AI for software architecture. Applications, trends, challenges, and future directions, arXiv:2503.13310v2 · arXiv
- Chen, 2022, CodeT: Code generation with generated tests, arXiv:2207.10397 · arXiv
- Bucaioni, 2024, Programming with ChatGPT: How far can we go?, Mach. Learn. With Appl., 15
- Shinn, 2023, Reflexion: Language agents with verbal reinforcement learning, arXiv:2303.11366 · arXiv
- Zhou, 2023, Language agent tree search unifies reasoning acting and planning in language models, arXiv:2310.04406 · arXiv
- Wrona, 2023, Overview of software agent platforms available in 2023, Information, 14, 348, 10.3390/info14060348 · DOI
- 10.1609/aaai.v39i1.32007 · DOI
- 10.18653/v1/2024.emnlp-main.632 · DOI
- Huang, 2023, CodeCoT: Tackling code syntax errors in CoT reasoning for code generation, arXiv:2308.08784 · arXiv
- 10.1109/tse.2024.3440503 · DOI
- 10.1145/3690635 · DOI · Paper page
- Zhang, 2023, Planning with large language models for code generation, arXiv:2303.05510 · arXiv
- Huang, 2023, AgentCoder: Multi-Agent-based code generation with iterative testing and optimisation, arXiv:2312.13010 · arXiv
- Kang, 2022, Large language models are fewshot testers: Exploring LLM-based general bug reproduction, arXiv:2209.11515 · arXiv
- Ni, LEVER: Learning to verify language-to-code generation with execution, 202, 26106
- Chen, 2023, Teaching large language models to self-debug, arXiv:2304.05128 · arXiv
- Coello, 2024, Effectiveness of ChatGPT in coding: A comparative analysis of popular large language models, Digital, 4, 114, 10.3390/digital4010005 · DOI · Paper page
- 10.1007/s10664-024-10594-x · DOI · Paper page
- Zhang, 2022, Coder reviewer reranking for code generation, arXiv:2211.16490 · arXiv
- Christakis, 2025, Evaluating large language models in code generation: INFINITE methodology for defining the inference index, Appl. Sci., 15, 3784, 10.3390/app15073784 · DOI
- 10.1109/icse48619.2023.00181 · DOI · Paper page
- 10.3897/jucs.134739 · DOI
- 10.1145/3708522 · DOI · Paper page
- 10.1109/tai.2023.3318183 · DOI · Paper page
- 10.1017/dsj.2024.8 · DOI
- Hussein, 2024, Usability of LLMs for Assisting Software Engineering: A Literature Review
- 10.3389/frai.2024.1436350 · DOI
- Zubair, 2025, The use of large language models for program repair, Comput. Standards Interface, 93, 10.1016/j.csi.2024.103951 · DOI
- 10.3389/fcomp.2025.1519437 · DOI
- Rasheed, 2024, CodePori: Large-scale system for autonomous software development using multi-agent technology, arXiv:2402.01411 · arXiv
- Wang, 2024, OpenHands: An open platform for AI software developers as generalist agents, arXiv:2407.16741 · arXiv
- Wu, 2024, OScopilot: Towards generalist computer agents with self-improvement, arXiv:2402.07456 · arXiv
- 10.1007/s10515-025-00544-2 · DOI
- Murphy, 2024, Combining LLM code generation with formal specifications and reactive program synthesis, arXiv:2410.19736 · arXiv
- 10.1080/08874417.2025.2482571 · DOI
- 10.1007/s10796-025-10581-7 · DOI
Metadata
- Authors
- Otoum, N., & Elkhalili, N.
- Venue
- IEEE Access
- Published
- 2026
- Volume
- 14
- Pages
- 7443-7465
- ISSN
- 2169-3536
- Cited by
- 10
- References
- 23
- Funders
- Deanship of Scientific Research and Graduate Studies at the University of Petra, Jordan
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Methods and Techniques of Agentic Software Engineering: A Systematic Literature Review. https://app.cochat.ai/papers/10.1109/access.2026.3652325
- The paper
- Otoum, N., & Elkhalili, N. (2026). Methods and Techniques of Agentic Software Engineering: A Systematic Literature Review. IEEE Access. https://doi.org/10.1109/access.2026.3652325
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.