Architecting Resilient LLM Agents: A Guide to Secure Plan-then-Execute Implementations
Open accessAbstract
As Large Language Model (LLM) agents become increasingly capable of automating complex, multi-step tasks, the need for robust, secure, and predictable architectural patterns is paramount. This paper provides a comprehensive guide to the ``Plan-then-Execute'' (P-t-E) pattern, an agentic design that separates strategic planning from tactical execution. We explore the foundational principles of P-t-E, detailing its core components - the Planner and the Executor - and its architectural advantages in predictability, cost-efficiency, and reasoning quality over reactive patterns like ReAct (Reason + Act). A central focus is placed on the security implications of this design, particularly its inherent resilience to indirect prompt injection attacks by establishing control-flow integrity. We argue that while P-t-E provides a strong foundation, a defense-in-depth strategy is necessary, and we detail essential complementary controls such as the Principle of Least Privilege, task-scoped tool access, and sandboxed code execution. To make these principles actionable, this guide provides detailed implementation blueprints and working code references for three leading agentic frameworks: LangChain (via LangGraph), CrewAI, and AutoGen. Each framework's approach to implementing the P-t-E pattern is analyzed, highlighting unique features like LangGraph's stateful graphs for re-planning, CrewAI's declarative tool scoping for security, and AutoGen's built-in Docker sandboxing. Finally, we discuss advanced patterns, including dynamic re-planning loops, parallel execution with Directed Acyclic Graphs (DAGs), and the critical role of Human-in-the-Loop (HITL) verification, to offer a complete strategic blueprint for architects, developers, and security engineers aiming to build production-grade, resilient, and trustworthy LLM agents.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Plan-then-Execute architectures create distinct security boundaries between planning and execution stages, a challenge that becomes significantly more complex when planning and execution are distributed across multiple agents.
“Such containment has been explored in Plan-then-Execute architectures (Del Rosario et al., 2025), where the separation between planning and execution stages creates distinct security boundaries, a challenge that becomes significantly more complex when planning and execution are distributed across multiple agents.”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Del Rosario et al., 2025
Section 2.1 (p. 5) says separating planning from execution protects control flow by fixing the plan before execution encounters untrusted data, supporting the security-boundary point. Section 7.4.2 (p. 18) discusses distributing planning among agents, but presents this as improving scalability and failure containment; it does not say that distributing planning and execution across multiple agents makes the security challenge significantly more complex. (LLM, conf=0.88)
Experimental — accuracy not yet measuredStatic sandboxing and hardware-enforced trusted execution environments provide baseline guarantees but are often too brittle for agents capable of self-generated code execution.
“Static sandboxing and hardware-enforced trusted execution environments provide baseline guarantees (Costan & Devadas, 2016;Del Rosario et al., 2025), but are often too brittle for agents capable of self-generated code execution (Wölflein et al., 2025;Li et al., 2025a).”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Del Rosario et al., 2025
Isolated tool use and hardware-enforced trusted execution environments provide baseline defenses, but static sandboxing is too brittle for agents capable of self-generated code execution.
“Isolated tool use and hardware-enforced trusted execution environments provide baseline defenses (Costan & Devadas, 2016;Del Rosario et al., 2025), but static sandboxing is too brittle for agents capable of selfgenerated code execution (Wölflein et al., 2025;Li et al., 2025a).”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Del Rosario et al., 2025
Hardware-based trusted execution environments provide hardware-enforced memory isolation for sensitive agent components, preventing even a compromised host OS or hypervisor from accessing protected data.
“Hardware-based trusted execution environments (TEEs) provide hardware-enforced memory isolation for sensitive agent components, preventing even a compromised host OS or hypervisor from accessing protected data, while network partitioning, for example using ideas from software-defined networking (Kreutz et al., 2015), and sandboxed deployment can limit the blast radius of compromised agents.”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Del Rosario et al., 2025
Section 3.1 (pp. 7–8) discusses Docker-based sandboxing and limiting the blast radius of malicious agent code. Searches for trusted execution environments, TEEs, secure enclaves, and hardware memory isolation found no passage addressing protection from a compromised host OS or hypervisor. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Still being checked.
Research works printed but not linked: 4
- Engineer's Guide to Data & AI/ML. LangGraph Simplified: Understanding Conditional edge using Hotel Guest Check-In Process, June 2025.
- Gaole He, Gianluca Demartini, and Ujwal Gadiraju. Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems , number 414, pp. 1-22. Association for Computing Machinery, New York, NY, USA, April 2025. ISBN 979-8-4007-1394-1.
- Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning. In The Thirteenth International Conference on Learning Representations , October 2024.
- Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. HuggingGPT: Solving AI tasks with chatgpt and its friends in hugging face. In Proceedings of the 37th International Conference on Neural Information Processing Systems , NIPS '23, pp. 38154-38180, Red Hook, NY, USA, December 2023. Curran Associates Inc.
Cited by
Sorted by citations
Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
de Witt, C. S., Krawiecka, K., Krawczuk, I., Hagag, B., Anderson, W. L., Belcak, P., Bucknall, B., Cai, X., Chopra, A., Cohen, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.02077 · Cited by 2 · see citing works
Self-citationSmartWalkCoach: An AI Companion for End-to-End Walking Guidance, Motivation, and Reflection
Zhang, X., Hu, M., Xue, B., Purwanto, E., Selig, T., & Yonto, D. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789170
- Authors
- Del Rosario, R. F., Krawiecka, K., & de Witt, C. S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2509.08646
- OpenAlex
- W4417070646
- Cited by
- 1
- References
- 0
- Citations by year
- 2026: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- LLM agents, agent security, prompt injection, control-flow integrity, defense-in-depth, principle of least privilege, LangChain, LangGraph, CrewAI, AutoGen, replanning, directed acyclic graphs, human-in-the-loop verification, agent architecture
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
