Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures
Open accessAbstract
LLM-based coding agents can localize bugs, generate patches, and run tests with diminishing human oversight, yet the scaffolding code that surrounds the language model (the control loop, tool definitions, state management, and context strategy) remains poorly understood. Existing surveys classify agents by abstract capabilities (tool use, planning, reflection) that cannot distinguish between architecturally distinct systems, and trajectory studies observe what agents do without examining the scaffold code that determines why. This paper presents a source-code-level architectural taxonomy derived from analysis of 13 open-source coding agent scaffolds at pinned commit hashes. Each agent is characterized across 12 dimensions organized into three layers: control architecture, tool and environment interface, and resource management. The analysis reveals that scaffold architectures resist discrete classification: control strategies range from fixed pipelines to Monte Carlo Tree Search, tool counts range from 0 to 37, and context compaction spans seven distinct strategies. Five loop primitives (ReAct, generate-test-repair, plan-execute, multi-attempt retry, tree search) function as composable building blocks that agents layer in different combinations; 11 of 13 agents compose multiple primitives rather than relying on a single control structure. Dimensions converge where external constraints dominate (tool capability categories, edit formats, execution isolation) and diverge where open design questions remain (context compaction, state management, multi-model routing). All taxonomic claims are grounded in file paths and line numbers, providing a reusable reference for researchers studying agent behavior and practitioners designing new scaffolds.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The existing literature reaches neither the goal of architectural grounding nor the goal of asking where the category is heading: benchmark surveys report scores without touching architecture, conceptual taxonomies describe patterns without implementations, and the one concurrent source-level taxonomy excludes provider-native systems.
“The existing literature reaches neither goal. Benchmark surveys report scores (typically SWEBench [1]) without touching architecture; conceptual taxonomies [5, 6] describe patterns without implementations; the one concurrent source-level taxonomy [34] excludes the provider-native systems that define the field's frontier.”
From Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems (2026) · cites this paper as [34]
Split into 2 atoms — 0 confirmed, 0 partially supported, 2 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredRombaut's source-code taxonomy of thirteen open-source scaffolds is the closest methodological relative of the present study.
“The closest methodological relative of this study is Rombaut's Inside the Scaffold [34]”
From Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems (2026) · cites this paper as [34]
Most production loops compose primitives (ReAct, generate-test-repair, plan-execute, retry) rather than implementing one.
“most production loops compose primitives (ReAct, generate-test-repair, plan-execute, retry) rather than implementing one”
From Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems (2026) · cites this paper as [34]
Rombaut's corpus is open-source-only, skews toward benchmark-oriented pipelines, and explicitly excludes Claude Code for lack of published source.
“Rombaut's corpus is open-source-only, skews toward benchmark-oriented pipelines (AutoCodeRover, Agentless, Moatless Tools, DARS, Prometheus, SWE-agent), and explicitly excludes Claude Code for lack of published source.”
From Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems (2026) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey
Masterman, T., Besen, S., Sawtell, M., & Chao, A. (2024) · arXiv · doi:10.48550/arxiv.2404.11584 · Cited by 35 · cited 4× · see citing works
“The initial pilot began with six candidate dimensions drawn from the conceptual agent architecture literature [Masterman et al., 2024]: control loop, tool interface, state management, context retrieval, execution isolation, and multi-model routing.”
Influential
Flagged first, then by citations
Research works printed but not linked: 41
- Rui Abreu, Peter Zoeteweij, and Arjan J. C. Van Gemund. On the accuracy of spectrum-based fault localization. In Testing: Academic and Industrial Conference Practice and Research Techniques (TAIC PART), pages 89-98. IEEE, 2007.
- Vaibhav Aggarwal, Ojasv Kamal, Abhinav Japesh, Zhijing Jin, and Bernhard Schölkopf. DARS: Dynamic action re-sampling to enhance coding agent performance by adaptive tree traversal. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), 2025.
- Daman Arora, Atharv Sonwane, Nalin Wadhwa, Abhav Mehrotra, Saiteja Utpala, Ramakrishna Bairi, Aditya Kanade, and Nagarajan Natarajan. MASAI: Modular architecture for software- engineering AI agents. arXiv preprint arXiv:2406.11638, 2024. · arXiv
- Hudson Borges and Marco Tulio Valente. What's in a GitHub star? understanding repository starring practices in a social coding platform. Journal of Systems and Software, 146:112-129, 2018.
- Islem Bouzenia and Michael Pradel. Understanding software engineering agents: A study of thought-action-result trajectories. In Proceedings of the 40th IEEE/ACM International Con- ference on Automated Software Engineering (ASE), 2025. arXiv preprint arXiv:2506.18824. · arXiv
- Islem Bouzenia, Premkumar Devanbu, and Michael Pradel. RepairAgent: An autonomous, LLM- based agent for program repair. In Proceedings of the 47th IEEE/ACM International Conference on Software Engineering (ICSE), pages 2188-2200, 2025.
- Sergey Brin and Lawrence Page. The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems, 30(1-7):107-117, 1998.
- Cameron B. Browne, Edward Powley, Daniel Whitehouse, et al. A survey of Monte Carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in Games, 4(1):1-43, 2012.
- Nghi D. Q. Bui. Building AI coding agents for the terminal: Scaffolding, harness, context engi- neering, and lessons learned. arXiv preprint arXiv:2603.05344, 2026. · arXiv
- Ira Ceka, Saurabh Pujar, Shyam Ramji, Luca Buratti, Gail Kaiser, and Baishakhi Ray. Under- standing software engineering agents through the lens of traceability: An empirical study. arXiv preprint arXiv:2506.08311, 2025. · arXiv
- Zhi Chen, Zhensu Sun, Yuling Shi, Chao Peng, Xiaodong Gu, David Lo, and Lingxiao Jiang. Rethinking the value of agent-generated tests for LLM-based software engineering agents. arXiv preprint arXiv:2602.07900, 2026. · arXiv
- Xiang Deng, Jeff Da, Edwin Pan, et al. SWE-Bench Pro: Can AI agents solve long-horizon software engineering tasks? arXiv preprint arXiv:2509.16941, 2025. · arXiv
- Zhiyu Fan, Kirill Vasilevski, Dayi Lin, Boyuan Chen, Yihao Chen, Zhiqing Zhong, Jie M. Zhang, Pinjia He, and Ahmed E. Hassan. SWE-Effi: Re-evaluating software AI agent system effectiveness under resource constraints. arXiv preprint arXiv:2509.09853, 2025. · arXiv
- Matthias Galster, Seyedmoein Mohsenimofidi, Jai Lal Lulla, Muhammad Auwal Abubakar, Christoph Treude, and Sebastian Baltes. Configuring agentic AI coding tools: An exploratory study. arXiv preprint arXiv:2602.14690, 2026. · arXiv
- Spandan Garg, Benjamin Steenhoek, and Yufan Huang. Saving SWE-Bench: A benchmark muta- tion approach for realistic agent evaluation. In Proceedings of the 5th IEEE/ACM International Conference on AI Engineering -Software Engineering for AI (CAIN), 2026. arXiv preprint arXiv:2510.08996. · arXiv
- Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, et al. Gemma: Open models based on Gemini research and technology. arXiv preprint arXiv:2403.08295, 2024. · arXiv
- Sirui Hong, Mingchen Zhuge, Jonathan Chen, et al. MetaGPT: Meta programming for a multi- agent collaborative framework. In International Conference on Learning Representations (ICLR), 2024.
- Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues? In Interna- tional Conference on Learning Representations (ICLR), 2024.
- Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535-547, 2021.
- Thibault Le Sellier De Chezelles, Maxime Gasse, Alexandre Drouin, Massimo Caccia, Léo Boisvert, Megh Thakkar, Tom Marty, Rim Assouel, Sahar Omidi Shayegan, Lawrence Keunho Jang, et al. The BrowserGym ecosystem for web agent research. arXiv preprint arXiv:2412.05467, 2024. · arXiv
- Oorja Majgaonkar, Zhiwei Fei, Xiang Li, Federica Sarro, and He Ye. Understanding code agent behaviour: An empirical study of success and failure trajectories. In Proceedings of the 48th IEEE/ACM International Conference on Software Engineering (ICSE), 2026. arXiv preprint arXiv:2511.00197. · arXiv
- S lawomir Nowaczyk. Architectures for building agentic AI. arXiv preprint arXiv:2512.09458, 2025. · arXiv
- Yue Pan, Zimin Chen, Siyu Lu, Zhaoyang Chu, Xiang Li, Han Li, Yang Feng, Claire Le Goues, Federica Sarro, Martin Monperrus, and He Ye. Prometheus: Towards long-horizon codebase navigation for repository-level problem solving. arXiv preprint arXiv:2507.19942, 2026. · arXiv
- Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large language model connected with massive APIs. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
- Per Runeson and Martin Höst. Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering, 14(2):131-164, 2009.
- Johnny Saldaña. The Coding Manual for Qualitative Researchers. SAGE Publications, 4th edition, 2021.
- Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
- Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflex- ion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
- David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George Van Den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mas- tering the game of Go with deep neural networks and tree search. Nature, 529(7587):484-489, 2016.
- Débora Souza and Patrícia Machado. Toward architecture-aware evaluation metrics for LLM agents. In Proceedings of the 5th IEEE/ACM International Conference on AI Engineering -Software Engineering for AI (CAIN), 2026. arXiv preprint arXiv:2601.19583. · arXiv
- Anselm Strauss and Juliet Corbin. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE Publications, 2nd edition, 1998.
- Xingyao Wang, Boxuan Li, Yufan Song, et al. OpenHands: An open platform for AI software developers as generalist agents. In International Conference on Learning Representations (ICLR), 2025. arXiv preprint arXiv:2407.16741. · arXiv
- W. Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. A survey on software fault localization. IEEE Transactions on Software Engineering, 42(8):707-740, 2016.
- Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Demystifying LLM-based software engineering agents. Proceedings of the ACM on Software Engineering, 2:801-824, 2025. FSE 2025. ACM SIGSOFT Distinguished Paper Award.
- Jingxuan Xu, Ken Deng, Weihao Li, et al. SWE-Compass: Towards unified evaluation of agentic coding abilities for large language models. arXiv preprint arXiv:2511.05459, 2025. · arXiv
- John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. SWE-agent: Agent-computer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
- Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
- Robert K. Yin. Case Study Research and Applications: Design and Methods. SAGE Publications, 6th edition, 2018.
- Quanjun Zhang, Chunrong Fang, Yuxiang Ma, Weisong Sun, and Zhenyu Chen. A survey of learning-based automated program repair. ACM Transactions on Software Engineering and Methodology, 33(2):1-69, 2024a.
- Quanjun Zhang, Chunrong Fang, Yang Xie, Yaxin Zhang, Yun Yang, Weisong Sun, Shengcheng Yu, and Zhenyu Chen. A survey on large language models for software engineering. Science China Information Sciences, 69:141102, 2026. doi: 10.1007/s11432-025-4670-0. · DOI · Paper page
- Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. AutoCodeRover: Au- tonomous program improvement. In Proceedings of the 33rd ACM SIGSOFT International Sym- posium on Software Testing and Analysis (ISSTA), 2024b.
Cited by
Sorted by citations
When Does Restricting a Coding Agent to execute_code Help? A Regime $\times$ Agent-Design Ablation
Yang, H., Yu, Q., & Desell, T. (2026) · arXiv · doi:10.48550/arxiv.2607.10569
“Descriptive surveys of Claude Code’s primitive surface [6, 19, 29] and component-evolution work on coding-agent harnesses [18, 38] do not swap the entire surface either.”
An Empirical Study of Harness Design for Coding Agents
Fan, R.-Z., Zhang, Z., Ma, S., Hu, Y., Wang, S., Song, K., Liu, F., Zamani, H., & Wang, X. (2026) · arXiv · doi:10.48550/arxiv.2609.20804
“Later systems add long-term memory for the harness (Wong et al., 2025) or add task graphs and modular sub-agents (Chen et al., 2024; Arora et al., 2024), and recent surveys catalogue the design space (Rombaut, 2026).”
Barbaste, P., Darrigol, T., Vu, G., & Wiltberger, T. (2026) · arXiv · doi:10.48550/arxiv.2609.00006
- Authors
- Rombaut, B.
- Affiliations
- Huawei Technologies (Canada) (CA)
- Corresponding author
- Rombaut, Benjamin
- ORCID
- Rombaut, Benjamin
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2604.03515
- OpenAlex
- W7151236611
- Cited by
- 0
- References
- 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- coding agents, LLM agents, agent architecture, control loops, tool use, state management, React
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
