The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Human-In-The-Loop Software Development Agents
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Human oversight remains essential in high-stakes or uncertainty-prone settings (Mozannar et al., 2025;Takerngsaksiri et al., 2025).”
From The landscape of agentic reinforcement learning for LLMs: a survey (2025) · cites this paper as Takerngsaksiri et al., 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
Self-citationDemystifying LLM-Based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715754 · Cited by 849
Zhou, J., Zhang, H., & Lo, D. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227210 · Cited by 507
AutoCodeRover: Autonomous Program Improvement
Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A. (2024) · Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3650212.3680384 · Cited by 202
AutoCodeRover: Autonomous Program Improvement
Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A. (2024) · arXiv · doi:10.48550/arxiv.2404.05427 · Cited by 202
Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues
Liu, Y., Le-Cong, T., Widyasari, R., Tantithamthavorn, C., Li, L., Le, X.-B. D., & Lo, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3643674 · Cited by 149
Self-citationMeasuring Coding Challenge Competence With APPS
Hendrycks, D., Basart, S., Kadavath, S., Mazeika, M., Arora, A., Guo, E., Burns, C., Puranik, S., He, H., Song, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.09938 · Cited by 142
CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code
Zhou, S., Alon, U., Agarwal, S., & Neubig, G. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.859 · Cited by 100
CodeR: Issue Resolving with Multi-Agent and Task Graphs
Chen, D., Lin, S., Zeng, M., Zan, D., Wang, J.-G., Cheshkov, A., Sun, J., Yu, H., Dong, G., Aliev, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.01304 · Cited by 8
Research works printed but not linked: 35
- Liu, 2024, Large language model-based agents for software engineering: A survey, arXiv preprint arXiv
- 10.1145/3695988 · DOI · Paper page
- 10.18653/v1/2023.acl-long.411 · DOI
- Yang, 2024, SWE-Agent: Agent-computer interfaces enable automated software engineering, arXiv preprint arXiv
- Tao, 2024, Magis: Llm-based multi-agent framework for github issue resolution, arXiv preprint arXiv
- Arora, 2024, Masai: Modular architecture for softwareengineering ai agents, arXiv preprint arXiv
- Ma, 2024, How to understand whole software repository?, arXiv preprint arXiv
- 10.1109/SNPD.2013.92 · DOI
- 10.1016/j.infsof.2018.06.001 · DOI
- 10.1016/j.infsof.2023.107336 · DOI
- 10.1109/COMPSAC61105.2024.00252 · DOI · Paper page
- Takerngsaksiri, 2024, PyTester: Deep reinforcement learning for text-to-testcase generation, arXiv preprint arXiv
- 10.1016/j.infsof.2024.107565 · DOI
- Li, 2022, CodeReviewer: Pre-training for automating code review activities, arXiv preprint arXiv
- 10.1016/j.infsof.2024.107523 · DOI
- 10.1145/3510003.3510067 · DOI
- 10.1145/3540250.3549119 · DOI
- 10.1145/3524842.3528452 · DOI · Paper page
- Li, 2024, Llm-assisted static analysis for detecting security vulnerabilities, arXiv preprint arXiv
- 10.1007/s10664-023-10346-3 · DOI
- 10.1145/3540250.3549098 · DOI
- 10.1109/ICSE-FoSE59343.2023.00010 · DOI
- 10.1109/MS.2023.3268725 · DOI
- 10.1109/ICSE.2012.6227210 · DOI · Paper page
- 10.1109/ICSME52107.2021.00017 · DOI
- 2024, Introducing swe-bench verified
- Jimenez, 2023, Swe-bench: Can language models resolve real-world github issues?, arXiv preprint arXiv
- Zhang, 2019, Bertscore: Evaluating text generation with bert, arXiv preprint arXiv
- Zheng, 2023, Judging llm-as-a-judge with mt-bench and chatbot arena, Advances in Neural Information Processing Systems, 36, 46595
- 10.7551/mitpress/9837.003.0013 · DOI
- Spencer, 2009, Card sorting: Designing usable categories
- 10.11613/BM.2012.031 · DOI · Paper page
- Austin, 2021, Program synthesis with large language models, arXiv preprint arXiv
- Chen, 2021, Evaluating large language models trained on code, arXiv preprint arXiv
- 10.1145/3748647 · DOI
Cited by
The landscape of agentic reinforcement learning for LLMs: a survey
Zhang, G., Geng, H., Yu, X., Yin, Z., Zhang, Z., Tan, Z., Zhou, H., Li, Z., Xue, X., Li, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2509.02547 · Cited by 229
Stray, V., Barbala, A., & Wivestad, V. T. (2025) · Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering · doi:10.1145/3696630.3730566 · Cited by 19
Guiding Human Validation of LLM-Generated Code via Verifiable Literate Programming
Yuan, Z., Lu, W., Wu, H., Jin, D., & Wu, C. (2026) · arXiv · doi:10.48550/arxiv.2607.02333
Metadata
- Authors
- Takerngsaksiri, W., Pasuksmit, J., Thongtanunam, P., Tantithamthavorn, C., Zhang, R., Jiang, F., Li, J., Cook, E., Chen, K., & Wu, M.
- Affiliations
- Monash University,Australia
- Atlassian,Australia
- The University of Melbourne,Australia
- Venue
- 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)
- Publisher
- IEEE
- Published
- 2025-04-27
- Pages
- 342-352
- Cited by
- 36
- References
- 9
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.