PreprintCited by 4Security and Verification in ComputingAdversarial Robustness in Machine LearningAccess Control and Trust

Progent: Securing AI Agents with Privilege Control

Open access
Shi, Tianneng, He, Jingxuan, Wang, Zhun, Li, Hongwei, Wu, Linyu, Guo, Wenbo, Song, Dawn·2025·arXiv

Abstract

AI agents interact with external environments through tool calls, exposing them to attacks like indirect prompt injection that can trigger unauthorized actions. Securing these agents is challenging: they behave autonomously and probabilistically, security requirements evolve depending on the user's task and execution state, and there is an inherent tradeofff between security and utility. In this work, we introduce Progent, a novel framework that secures AI agents via privilege control. Progent represents privilege as a security policy consisting of symbolic rules over tool names and arguments. These rules specify which tool calls are allowed for task completion and which unnecessary ones are blocked for security. Every tool call is checked against such a policy through a deterministic procedure, enforcing the principle of least privilege. To handle diverse user tasks and evolving execution contexts, an LLM automatically generates the initial policy from the user's task and updates it during execution as new information arrives. Each proposed update is determined by an SMT solver to be either a narrowing (applied automatically) or an expansion (requiring explicit approval), ensuring that the agent's effective action space can only shrink without approval (monotonic confinement). This deterministic update mechanism preserves utility and prevents silent privilege escalation, even when adversarial inputs are present. Our evaluation on popular benchmarks (i.e., AgentDojo and ASB) shows that Progent significantly reduces attack success rates while maintaining high utility. We further validate Progent's practicality by showcasing its effectiveness in real-world agent frameworks such as LangChain and OpenAI Agents SDK.

AI snapshot

Written from the paper’s full text.

TL;DR

Progent is a privilege-control framework for AI agents that enforces symbolic policies over tool calls deterministically, using an LLM to generate and update policies while an SMT solver allows only narrowing updates automatically and requires approval for expansions. On AgentDojo and ASB it substantially reduces attack success rates while maintaining utility, and it integrates with real-world agent frameworks.

Key findings

  • On AgentDojo, Progent reduces attack success rate (ASR) from 39.9% with no defense to 1.0%, with utility 76.3 without attack and 61.3 under attack; on ASB, it reduces ASR from 70.3% to 3.9%, with utility 78.2 without attack and 69.7 under attack.
  • The initial LLM-generated policy alone reduces ASR from 39.9% to 2.5% while maintaining utility; after the SMT-based expansion check, only 6% of policy updates required approval in the AgentDojo experiments.
  • Ablations show Auto-Deny achieves the highest security among fully automated configurations, Auto-Approve improves utility and security over Disable Update but has slightly higher ASR, and Manual Approval reduces ASR to 0%; across policy LLMs, ASR is below 5% and reaches 1% with the best model.
  • Real-world integrations with LangChain, OpenAI Agents SDK, and OpenHands consistently reduce ASR to around 1% while maintaining strong utility; in an AutoGen multi-agent setup, Progent defends against more than 95% of successful attacks, and adaptive attacks only marginally increase ASR (normal 1.0, if-then-else 0.5, allow attack tool call 4.2, AgentVigil 0.9).

Methods

  • Progent inserts a policy check between the agent and its tools; policies are JSON Schema rules over tool names and arguments with allow/forbid effects and fallback actions, enforced deterministically at runtime with calls matching no rule blocked by default.
  • An LLM generates the initial policy from the user task and proposes updates during execution; a Z3 SMT solver compares policies to classify each update as a narrowing (applied automatically) or an expansion (requiring approval), guaranteeing monotonic confinement.
  • Evaluation uses AgentDojo (Banking, Slack, Travel, Workspace) and ASB (ten agent scenarios, five attack templates), with gpt-4o as both agent and policy LLM; main experiments use an Auto-Approve worst-case approver, compare against prior defenses, and include ablations over approver configurations and model choices, plus an MCP-based benchmark and integrations with LangChain, OpenAI Agents SDK, OpenHands, and AutoGen.

Limitations

  • Stated: user mistakes such as incorrectly approving widening updates or giving ambiguous initial tasks may permit policies exceeding least privilege; Progent cannot defend attacks that operate within least privilege (e.g., preference manipulation) or attacks targeting only text outputs; proxy mode cannot protect built-in tools that bypass external MCP interfaces; and the current scope is text-only, not multimodal agents.
  • Stated/evident: ASB failure cases occur when attack tool names and descriptions closely resemble benign tools and appear related to user tasks, making them hard to identify even for humans without prior trust knowledge; manually defined policies with additional human insight can reduce ASR to 0%, indicating the automated pipeline may miss such cases.
  • Evident from the text: the main experiments assume all expansion requests are auto-approved, simulating a user with limited security awareness, and Manual Approval is needed to reach 0% ASR, so the strongest security guarantee depends on approval behavior.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2504.11703

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 20 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Progent is effective in real-world agent frameworks such as LangChain and OpenAI Agents SDK.

    “We further validate Progent's practicality by showcasing its effectiveness in real-world agent frameworks such as LangChain and OpenAI Agents SDK.”
    • Also stated in: § Implementation and Evaluation. · boosted
      “Finally, we extend Agent-Dojo into an MCP-based benchmark and connect it to various realworld agent frameworks (e.g., LangChain and OpenAI Agents SDK), where we demonstrate the effectiveness of Progent on protecting both single-and multi-agent systems.”
    Quote in the paperSubstantial claim
  • Findingown data

    In the experiments for AgentDojo, only 6% of updates required approval.

    “In our experiments for AgentDojo (Section 8), only 6% of updates required approval.”
    Quote in the paperSubstantial claim
  • Findingown data

    LLMs are capable of effectively managing Progent's policies.

    “We found that LLMs are capable of effectively managing Progent's policies, likely due to their strong security capabilities.”
    Quote in the paperSubstantial claim
  • Findingown data

    The three prompt injection detectors (transformers_pi_detector, DataSentinel, and LlamaPromptGuard2) are ineffective on AgentDojo.

    “We also observe that the three prompt injection detectors (transformers_pi_detector, Da taSentinel, and LlamaPromptGuard2) are ineffective.”
    Quote in the paperSubstantial claim
  • Findingnone given

    In Disable Update mode, compared to no-defense baseline, ASR is effectively reduced.

    “In the Disable Update mode, compared to the nodefense baseline, the ASR is effectively reduced.”
    Quote in the paperSubstantial claim
  • May combine two claims. 62
  • Classified as probably not a substantial claim. 47
  • The quote only partly states this claim. 1
  • May combine two claims.

    Findingown data

    Progent reduces the attack success rate (ASR) from 39.9% to 1.0% on AgentDojo, and from 70.3% to 3.9% on ASB, while maintaining the agent's utility in both cases.

    “Progent reduces the attack success rate (ASR) from 39.9% to 1.0% on Agent-Dojo, and from 70.3% to 3.9% on ASB, while maintaining the agent's utility in both cases.”
    • Also stated in: § Abstract · plain
      “Our evaluation on popular benchmarks (i.e., AgentDojo and ASB) shows that Progent significantly reduces attack success rates while maintaining high utility.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [11]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [57]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    Manual Approval mode reduces ASR to 0% with human review, demonstrating that human oversight can capture complex security requirements that the automatic pipeline may miss.

    “Finally, the Manual Approval mode reduces the ASR to 0% with human review, demonstrating that human oversight can capture complex security requirements that the automatic pipeline may miss.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingreasoning

    The LLM-generated policies can successfully identify the necessary tools for the user task and allow their use.

    “Specifically, the LLM-generated policies can successfully identify the necessary tools for the user task and allow their use.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingreasoning

    Auto-Approve mode has slightly higher ASR than Auto-Deny mode because widened policies may inadvertently permit malicious actions.

    “However, the ASR is slightly higher than in the Auto-Deny mode because widened policies may inadvertently permit malicious actions, which is why explicit approval is required in practice for such updates.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingreasoning

    Auto-Deny mode achieves the highest level of security among fully automated configurations, as all updates are strictly bounded by invariant checks.

    “When the policy update mechanism is enabled, the Auto-Deny mode achieves the highest level of security among fully automated configurations, as all updates are strictly bounded by invariant checks.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The initial LLM-generated policy reduces the attack success rate from 39.9% to 2.5% while maintaining utility.

    “Specifically, it reduces the attack success rate from 39.9% to 2.5% while maintaining utility.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    In the running example (Figure 1), when send_email is blocked at step 4, the fallback message allows the agent to continue and successfully complete the benign task at step 5.

    “For instance, in the running example (Figure 1), when send_email is blocked at step 4, the fallback message allows the agent to continue and successfully complete the benign task at step 5.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Progent reduces ASR from 39.9% with no defense to 1.0% on AgentDojo while maintaining utility scores in both no-attack and under-attack scenarios.

    “Progent significantly reduces ASR from 39.9% with the no defense baseline to 1.0%, while maintaining utility scores in both no-attack and under-attack scenarios.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The user configures the approver to always accept policy-expansion requests.
  • May combine two claims.

    Findingcitation

    Prior defenses are ineffective in reducing ASR on ASB, consistent with the original paper of ASB.

    “The prior defenses are ineffective in reducing ASR, a result consistent with the original paper of ASB [57].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [57]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    Auto-Approve mode allows policies to expand beyond their initial invariants, which improves both utility and security compared to Disable Update mode.

    “In contrast, the Auto-Approve mode allows policies to expand beyond their initial invariants, which improves both utility and security compared to the Disable Update mode.”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim.

    Findingnone given

    Manually defined policies enhanced with human insights can provably reduce the ASR to 0%.

    “These manually defined policies can provably reduce the ASR to 0%.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    With Progent applied, ASR is reduced to 0.5% and 0.3% for claude-sonnet-4 and Meta-SecAlign-70B, defending about 90% of attacks.

    “With Progent applied, the ASR is even reduced further to 0.5% and 0.3%, defending about 90% of attacks.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    In the multi-agent setting, both utility and ASR are slightly lower than in single-agent systems.

    “Figure 8 shows that in the multi-agent setting, both utility and ASR are slightly lower than in single-agent systems.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    After the SMT-based expansion check, only 6% of policy updates are expansions that require approval; the rest are narrowings handled automatically.

    “We also observe that, after the SMT-based expansion check, only 6% of policy updates are expansions that require approval; the rest are narrowings handled automatically.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Progent either maintains utility under no attack or introduces marginal reduction.

    “It either maintains utility under no attack or introduces marginal reduction.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Progent significantly reduces attack success rates while preserving high utility.

    “Our evaluation on AgentDojo and ASB demonstrates that Progent significantly reduces attack success rates while preserving high utility”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Progent is effective with LLMs for policy generation and update, reducing ASR below 5% across all models and to 1% with the best performing LLM.

    “As shown in Figure 6, Progent is effective with LLMs for policy generation and update, reducing ASR below 5% across all models and to 1% with the best performing LLM.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    After applying Progent, it maintains similar utility while significantly improving security, defending against more than 95% of successful attacks in the multi-agent setting.

    “After applying Progent, it maintains similar utility while significantly improving security, defending against more than 95% of successful attacks.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Under attacks, Progent improves utility and significantly reduces ASR across different models.

    “Under attacks, it improves the utility and significantly reduces the ASR across different models.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Progent substantially reduces ASR and retains meaningful protection against user-introduced mistakes using an adaptive attack.

    “In Appendix E, we empirically evaluate Progent against user-introduced mistakes using an adaptive attack, showing that Progent substantially reduces ASR and retains meaningful protection despite possible human misjudgment.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Progent consistently reduces ASR to around 1% while maintaining strong utility on real-world agents.

    “As shown in Figure 8, Progent consistently reduces the ASR to around 1% while maintaining strong utility.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    claude-sonnet-4 and Meta-SecAlign-70B achieve ASR of only 6.8% and 4.8% without any defense applied.

    “We also find that claude-sonnet-4 and Meta-SecAlign-70B, themselves already have strong safety mechanisms, achieving a remarkable ASR of only 6.8% and 4.8% without any defense applied.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Progent represents privilege as a security policy consisting of symbolic rules over tool names and arguments, which specify which tool calls are allowed and which should be blocked.

    “Progent represents privilege as a security policy consisting of symbolic rules over tool names and arguments.”
    • Also stated in: § Challenges for Securing AI Agents. · plain
      “Progent leverages a privilege control policy consisting of symbolic rules over tool names and arguments.These rules specify which tool calls are allowed and which should be blocked.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    Distinguishing legitimate from adversarial expansions, without sacrificing security or utility, is a fundamental challenge.

    “Distinguishing legitimate from adversarial expansions, without sacrificing security or utility, is a fundamental challenge.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Agents must sometimes expand their privileges to complete benign tasks.

    “Finally, agents must sometimes expand their privileges to complete benign tasks.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Premisereasoning

    There are two kinds of context for an agent: task context and execution context.

    “We identify two kinds of context for the agent.The first is task context: the same agent can process different tasks, such as summarizing emails, booking a flight, or reconciling a budget, each authorizing a different set of tools with different tool arguments.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    Progent uses an LLM to automatically generate the initial policy from the user's task and updates it during execution as new information arrives.

    “To handle diverse user tasks and evolving execution contexts, an LLM automatically generates the initial policy from the user's task and updates it during execution as new information arrives.”
    • Also stated in: § Challenges for Securing AI Agents. · plain
      “To handle context-dependent security requirements, Progent utilizes an LLM to automatically generate the initial policy from the user's task and updates it as new information arrives during execution.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    All external effects of an agent occur through tool calls, and they expose a structured interface (tool names and typed arguments), unlike the agent's unstructured reasoning.

    “We focus on tool calls because all external effects of an agent occur through them, and they expose a structured interface (tool names and typed arguments), unlike the agent's unstructured reasoning.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisereasoning

    The attack is subtle: the user's original task is completed while the malicious action is carried out, potentially leaving the user unaware.

    “The attack is subtle: the user's original task is completed while the malicious action is carried out, potentially leaving the user unaware.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Premisenone given

    Involving the user in every security decision is impractical and negates the benefit of an autonomous agent.

    “A defense must track these changes automatically, since involving the user in every security decision is impractical and negates the benefit of an autonomous agent.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisereasoning

    Introducing an LLM for policy construction introduces non-determinism, creating a fundamental tension between automation and deterministic enforcement.

    “However, introducing an LLM for policy construction introduces non-determinism, creating a fundamental tension between automation and deterministic enforcement.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given

    Different tasks require different tools and arguments, and the relevant context evolves at runtime as the agent interacts with the environment.

    “Different tasks require different tools and arguments, and the relevant context evolves at runtime as the agent interacts with the environment.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Adversaries also seek to exploit the same mechanism to induce harmful actions, and both legitimate and adversarial expansion requests arrive through the same channel.

    “Yet, adversaries also seek to exploit the same mechanism to induce harmful actions, and both legitimate and adversarial expansion requests arrive through the same channel.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    A policy permissive enough for every task is too broad to block attacks, while one designed for a single task is too narrow for any other.

    “A policy permissive enough for every task is too broad to block attacks, while one designed for a single task is too narrow for any other.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    Security requires deterministic enforcement that holds regardless of how the LLM reasons, even under adversarial influence.

    “In contrast, security requires deterministic enforcement that holds regardless of how the LLM reasons, even under adversarial influence.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    Progent can be integrated by inserting a separate module between the agent and its tools, requiring no changes to the agent's internal architecture, making integration non-intrusive and generalizable across diverse agent frameworks.

    “Since Progent operates at the tool-call level, it can be integrated by inserting a separate module between the agent and its tools.This requires no changes to the agent's internal architecture, making integration non-intrusive and generalizable across diverse agent frameworks.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Under the benign user query threat model, the initialized policy P is expected to accurately identify and restrict tool usage and arguments in accordance with the principle of least privilege.

    “Under our threat model, where the user query is benign, the initialized 𝑃 is expected to accurately identify and restrict tool usage and arguments in accordance with the principle of least privilege.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The user query is benign.
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    The same channel can carry adversarial content, making it challenging to balance security and utility.

    “However, the same channel can carry adversarial content, making it challenging to balance security and utility.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    The three prompt injection detectors fail to generalize to AgentDojo, exhibiting high rates of false positives and negatives.

    “While they may perform well on datasets similar to their training distributions, they fail to generalize to AgentDojo, exhibiting high rates of false positives and negatives.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    Progent successfully enforces the principle of least privilege, allowing tool calls necessary for completing the user task while blocking malicious tool calls.

    “This means Progent successfully enforces the principle of least privilege, allowing tool calls necessary for completing the user task while blocking malicious tool calls.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    The only requirement for Progent's formulation is that the agent can call tools within T.

    “The only requirement is that the agent can call tools within T .”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductreasoning

    Higher-priority generic policies cannot be overridden by lower-priority ones, and no lower-priority or task-specific policy can weaken stronger generic guarantees.

    “Higher-priority policies are enforced first and cannot be overridden by lower-priority ones, while lower-priority policies may further restrict the remaining action space, ensuring that no lower-priority or task-specific policy can weaken stronger generic guarantees.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    Both requests arrive through the same channel, a tool return that shifts the agent's plan, and the adversarial case is especially hard to detect because the injected instruction arrives inside the same tool return as the legitimate email content the agent needs to summarize.

    “Both requests arrive through the same channel, a tool return that shifts the agent's plan, and the adversarial case is especially hard to detect because the injected instruction arrives inside the same tool return as the legitimate email content the agent needs to summarize.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Study design and conductreasoning

    The deterministic SMT check ensures that policy updates can only narrow privileges without approval, whereas any expansion requires explicit approval.

    “This deterministic check ensures that policy updates can only narrow privileges without approval, whereas any expansion requires explicit approval.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    No attack can silently escalate the agent's privileges; only expansion requests require approval, and all other security decisions are handled deterministically.

    “This guarantees that no attack can silently escalate the agent's privileges: only expansion requests require approval, and all other security decisions are handled deterministically.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Rules can specify concrete values, wildcards, and regular-expression patterns, making the policy expressive enough to capture fine-grained constraints.

    “Rules can specify concrete values (such as 20), wildcards (*), and regular-expression patterns, making the policy expressive enough to capture fine-grained constraints.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    Expansion is sensitive: it may be necessary for utility but can also be caused by an attack.

    “Expansion, however, is sensitive: it may be necessary for utility (as in 𝑃 1 → 𝑃 2 , where the task requires more tools) but can also be caused by an attack (as in step 3's injection).”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Interpretationnone given

    Progent's runtime enforcement is a deterministic procedure that checks every tool call against the current policy.

    “Progent's runtime enforcement is a deterministic procedure that checks every tool call against the current policy, as illustrated in Algorithm 2.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The function P(c) creates a policy-governed tool call that behaves like the original tool call when allowed and switches to the fallback function when not.

    “The function 𝑃 (𝑐) creates a policy-governed tool call.It behaves just like the original tool call 𝑐 when the policy 𝑃 allows it, and it automatically switches to the fallback function when it does not.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    Manual Approval mode delivers the strongest security, as all policies are explicitly reviewed by the user, but it demands manual effort and is most suitable for security-critical environments.

    “Finally, the Manual Approval mode delivers the strongest security, as all policies are explicitly reviewed by the user, but it demands manual effort and is most suitable for security-critical environments.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    Even when the LLM is manipulated or hallucinates, the deterministic judgment ensures that no new permission is granted without explicit approval, so utility-driven expansions are permitted while adversarial ones are caught.

    “Even when the LLM is manipulated or hallucinates, the deterministic judgment ensures that no new permission is granted without explicit approval, so utility-driven expansions are permitted while adversarial ones are caught.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Disable Update mode provides deterministic stability by fixing policies after initialization, though it may reduce adaptability to dynamic contexts.

    “The Disable Update mode provides deterministic stability by fixing policies after initialization, though it may reduce adaptability to dynamic contexts.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Given the vast range of possible outcomes from tool calls, the attacker could cause a variety of downstream damages.

    “Given the vast range of possible outcomes from tool calls, the attacker could cause a variety of downstream damages.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    If no rule in P targets the tool or the tool call's arguments do not trigger any rule, Progent blocks the tool call by default for security purposes.

    “Finally, at Line 8, if no rule in P targets the tool or the tool call's arguments do not trigger any rule, we block the tool call by default for security purposes.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    The development of AI agents is complex, involving various modules, strategic architectural decisions, and sophisticated implementation.

    “The development of AI agents is complex, involving various modules, strategic architectural decisions, and sophisticated implementation [46].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [46]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    When the agent issues a tool call, Progent matches it against the current policy's rules and deterministically allows or blocks it; calls that match no rule are blocked by default.

    “When the agent issues a tool call, Progent matches it against the current policy's rules and deterministically allows or blocks it; calls that match no rule are blocked by default.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    Because Progent's policies are symbolic, both relations are decidable by an SMT solver, giving a fully deterministic judgment.

    “Because Progent's policies are symbolic, both relations are decidable by an SMT solver, giving a fully deterministic judgment (details in Section 4.3).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Premisecitation

    State-of-the-art LLM service providers use JSON to format tool calls, making JSON Schema a natural choice for validating tool calls against policy rules.

    “State-of-the-art LLM service providers (e.g., OpenAI API [35]) use JSON to format tool calls, making JSON Schema a natural choice for validating tool calls against policy rules.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [35]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    tool_filter suffers from higher utility reduction and ASR because its coarse-grained approach of ignoring tool arguments either blocks an entire tool, harming utility, or allows it completely, causing attack success.

    “tool_filter suffers from higher utility reduction and ASR because its coarse-grained approach of ignoring tool arguments either blocks an entire tool, harming utility, or allows it completely, causing attack success.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    When the agent retrieves attacker-manipulated data via tool calls, the injected command can alter the agent's behavior.

    “When the agent retrieves such data via tool calls, the injected command can alter the agent's behavior.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The attacker can manipulate the agent's external data source in the environment E, such as an email, to embed malicious commands.
  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Progent places forbid rules before allow rules such that the forbid ones take effect first.

    “Then, at Line 3, we take a conservative approach to place forbid rules before allow rules such that the forbid ones take effect first.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Auto-Deny offers a conservative balance between security and automation, ensuring that all expansions are rejected.

    “In contrast, Auto-Deny offers a conservative balance between security and automation, ensuring that all expansions are rejected.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Because agents execute autonomously, they must adapt to new information that may legitimately require additional privileges.

    “Because agents execute autonomously, they must adapt to new information that may legitimately require additional privileges.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: AI agents execute autonomously, and the context changes as the agent gathers new information at runtime.
  • May combine two claims.

    Premisecitation

    The attacker could induce dangerous data erasure operations and unauthorized financial transactions.

    “For instance, as shown in [11,57], the attacker could induce dangerous data erasure operations and unauthorized financial transactions.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [11]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [57]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Since the agent interacts with the external environment via tool calls, dangerous behaviors exhibit as malicious tool calls at Line 3 of Algorithm 1.

    “Since the agent interacts with the external environment via tool calls, such dangerous behaviors exhibit as malicious tool calls at Line 3 of Algorithm 1.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    Because this check is purely symbolic over structured policy rules, every security decision is deterministic, directly addressing Challenge I.

    “Because this check is purely symbolic over structured policy rules, every security decision is deterministic, directly addressing Challenge I.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Study design and conductnone given

    Progent introduces a privilege control policy consisting of symbolic rules over tool names and arguments, enforced deterministically with no LLM in the decision loop.

    “Progent introduces a privilege control policy consisting of symbolic rules over tool names and arguments, enforced deterministically with no LLM in the decision loop.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    Progent serves as an effective defense-in-depth layer that complements other defenses while requiring minimal code changes.

    “For agents, Progent serves as an effective defense-in-depth layer that complements other defenses while requiring minimal code changes.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    Multi-agent systems naturally introduce isolation between sub-agents, which can reduce coordination efficiency but also limit attack propagation, improving security.

    “This is likely because multi-agent systems naturally introduce isolation between sub-agents, which can reduce coordination efficiency but also limit attack propagation, improving security.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Many other system-level defenses require substantial architectural modifications.

    “whereas many other system-level defenses require substantial architectural modifications.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    There are no policies or licensing restrictions preventing the authors from making the artifacts publicly available.

    “There are no policies or licensing restrictions preventing us from making the artifacts publicly available.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationnone given

    The proxy mode offers out-of-the-box deployability and requires no modification to the underlying agent, but it cannot protect agent functionalities implemented as built-in tools that bypass external MCP interfaces.

    “Besides, the proxy mode offers out-of-the-box deployability and requires no modification to the underlying agent, but it cannot protect agent functionalities implemented as built-in tools that bypass external MCP interfaces.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    Progent complements existing defense mechanisms and serves as an effective defense-in-depth layer in practice.

    “These results demonstrate that Progent complements existing defense mechanisms and serves as an effective defense-in-depth layer in practice.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The marginal variations in utility and security across different agents are likely due to differences in system prompts and retry mechanisms among these agents.

    “The marginal variations in utility and security across different agents are likely due to differences in system prompts and retry mechanisms among these agents.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    Progent is effective and robust when deployed with real-world agents under realistic attack scenarios.

    “These results demonstrate Progent's effectiveness and robustness when deployed with real-world agents under realistic attack scenarios.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Progent can be applied separately for each sub-agent, allowing customized policies tailored to the role and functionality of each agent.

    “Progent can also be applied separately for each sub-agent, allowing customized policies tailored to the role and functionality of each agent.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The datasets and benchmarks used in the evaluation have been made publicly available by their authors.

    “The datasets and benchmarks used in the evaluation have been made publicly available by their authors.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    The MCP-based benchmark is fully reusable and can be applied to future agent frameworks or products as long as they support MCP.

    “It is fully reusable and can be applied to future agent frameworks or products as long as they support MCP, making it a practical and extensible testbed for real-world agent security research.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Progent naturally supports multi-agent systems through its modular design.

    “Progent naturally supports multi-agent systems through its modular design.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Two injection tasks in the travel suite are preference attacks, which mislead the agent into choosing another legitimate hotel rather than the target one.

    “Two injection tasks in the travel suite are preference attacks, which mislead the agent into choosing another legitimate hotel rather than the target one.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Study design and conductnone given

    All datasets used in the experiments are publicly available and do not contain any private or sensitive data.

    “All datasets used in the experiments are publicly available and do not contain any private or sensitive data.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Existing policy languages are not suited to the dynamic behavior of AI agents.

    “yet existing policy languages are not suited to the dynamic behavior of AI agents.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Progent's modular design enables integration into existing agent implementations with minimal changes, potentially lowering adoption barriers.

    “Its modular design enables integration into existing agent implementations with minimal changes, potentially lowering adoption barriers”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    An attacker who is aware of the LLM-based policy update mechanism and the user with limited security awareness could construct adaptive attacks to circumvent Progent's defense effect.

    “Considering that the LLM-based policy update mechanism utilizes external information and the user with limited security awareness who may make risky decisions in practice, an attacker who is aware of this could construct adaptive attacks to circumvent Progent's defense effect.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The LLM-based policy update mechanism utilizes external information and the user may have limited security awareness and make risky decisions in practice.
    • Rests on an assumption: An attacker can be aware of the LLM-based policy update mechanism and the user's limited security awareness.
  • May combine two claims.

    Interpretationreasoning

    In Progent, even if the tool is blocked, it still exists in the trace with a blocking message and it would be wrongly classified by the AgentDojo success check.

    “In our method, even if the tool is blocked, it still exists in the trace with a blocking message and it would be wrongly classified.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    The library mode requires only a few lines of code changes yet can secure built-in tools as well.

    “In contrast, the library mode requires only a few lines of code changes yet can secure built-in tools as well.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    For another injection task in the slack suite, the AgentDojo implementation directly looks for the attack tool call in the execution trace to determine whether the attack is successful regardless of whether the tool call succeeds or not.

    “For another injection task in the slack suite, the AgentDojo implementation directly looks for the attack tool call in the execution trace to determine whether the attack is successful regardless of whether the tool call succeeds or not.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    All experiments are done in a local and sandboxed environment which will not leak any attack prompt to real-world applications.

    “All experiments are done in a local and sandboxed environment which will not leak any attack prompt to the real-world applications.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Study design and conductreasoning

    Treating the entire multi-agent system as a unified entity and enforcing a single layer of privilege control ensures consistent security and prevents cross-agent privilege escalation.

    “This global context and policy management ensures consistent security and prevents cross-agent privilege escalation.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Progent provides deterministic security enforcement, which guardrail models cannot.

    “Another key distinction is that Progent provides deterministic security enforcement, which guardrail models cannot.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductnone given

    An LLM automatically generates and updates the policy, while a deterministic SMT-based check determines whether each update is a narrowing (applied automatically) or an expansion (requiring approval), guaranteeing monotonic confinement.

    “To handle diverse tasks and evolving execution contexts, an LLM automatically generates and updates the policy, while a deterministic SMT-based check determines whether each update is a narrowing (applied automatically) or an expansion (requiring approval), guaranteeing monotonic confinement.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Text-to-text attacks are more aligned with model-level risks that have been extensively studied by existing model security works.

    “Such text-to-text attacks are more aligned with model-level risks that have been extensively studied by existing model security works.”
    Quote in the paperNot a substantial claim
92 of 92
  • The paper relies on this without support.

    Asking the user to approve every tool call requires too much effort and removes the benefit of using an agent.

    “since asking the user to approve every tool call requires too much effort and removes the benefit of using an agent.”
  • The paper relies on this without support.

    Agent execution involves different kinds of contexts, and automatically tracking them to enforce security is challenging.

    “Agent execution involves different kinds of contexts, and automatically tracking them to enforce security is challenging.”
  • The paper relies on this without support.

    AI agents execute autonomously, and the context changes as the agent gathers new information at runtime.

    “AI agents execute autonomously, and the context changes as the agent gathers new information at runtime.”
  • The paper relies on this without support.

    Execution-gathered information is security-critical.

    “Such execution-gathered information is security-critical”
  • The paper relies on this without support.

    An attacker seeks to expand the agent's tool usage for malicious purposes.

    “an attacker also seeks to expand the agent's tool usage for malicious purposes.”
  • The paper relies on this without support.

    All agent actions with external effects flow through tool calls.

    “all agent actions with external effects flow through them.”
  • The paper relies on this without support.

    The tool-call interface, consisting of a tool name and typed arguments, is structured, unlike the unstructured text the agent reasons over.

    “the tool-call interface, consisting of a tool name and typed arguments, is structured, unlike the unstructured text the agent reasons over.”
  • The paper relies on this without support.

    Agent security depends on context.

    “agent security depends on context”
  • The paper relies on this without support.

    Both A and E are stateful, so prior interactions may influence A and E.

    “Both A and E are stateful, so prior interactions may influence A (𝑜 𝑖 -1 ) and E (𝑐 𝑖 ).”
  • The paper relies on this without support.

    AI agents gain enhanced task-solving capabilities through access to diverse tools.

    “Compared with standalone models, AI agents gain enhanced task-solving capabilities through access to diverse tools in T , such as email clients, file browsers, and financial applications.”
  • The authors state this as an assumption.

    The attacker's goal is to disrupt the agent's task-solving flow, leading to unauthorized actions that benefit the attacker.

    “The attacker's goal is to disrupt the agent's tasksolving flow, leading to the agent performing unauthorized actions that benefit the attacker in some way.”
  • The authors state this as an assumption.

    The attacker can manipulate the agent's external data source in the environment E, such as an email, to embed malicious commands.

    “We assume the attacker can manipulate the agent's external data source in the environment E, such as an email, to embed malicious commands.”
  • The authors state this as an assumption.

    The user U is benign, and the user's input query is always benign.

    “However, we assume the user U is benign, and as such, the user's input query is always benign.”
  • The authors state this as an assumption.

    The user query o0 is benign and any observation oi (i > 0) can be controlled by the attacker.

    “In other words, in terms of Algorithm 1, we assume that the user query 𝑜 0 is benign and any observation 𝑜 𝑖 (𝑖 > 0) can be controlled by the attacker.”
  • The authors state this as an assumption.

    The attacker cannot modify the agent's internals, such as changing the model or its system prompt.

    “However, the attacker cannot modify the agent's internals, such as changing the model or its system prompt.”
  • The paper relies on this without support.

    A tool call's safety often depends on the specific arguments it receives.

    “These conditions are critical because a tool call's safety often depends on the specific arguments it receives.”
  • The authors state this as an assumption.

    The user query is benign.

    “Under our threat model, where the user query is benign”
  • The authors state this as an assumption.

    The initial context o0 used during policy initialization is trusted.

    “While the initial context 𝑜 0 used during policy initialization is trusted”
  • The authors state this as an assumption.

    Contextual information obtained from the environment at runtime cannot always be considered trustworthy.

    “the contextual information obtained from the environment at runtime cannot always be considered trustworthy.”
  • The paper relies on this without support.

    JSON is widely represented in LLM training data, so LLMs can generate syntactically correct policies in JSON format without any fine-tuning.

    “Because JSON is widely represented in LLM training data, LLMs can generate syntactically correct policies in JSON format without any fine-tuning”
  • The authors state this as an assumption.

    The user configures the approver to always accept policy-expansion requests.

    “we assume the user configures the approver to always accept policy-expansion requests”
  • The authors state this as an assumption.

    The attacker injects malicious prompts in the environment.

    “The attacker injects malicious prompts in the environment”
  • The authors state this as an assumption.

    For the Manual Approval setting in Section 8.2, the user has strong security awareness: they would not grant unrestricted access to highly sensitive tools such as send_money (i.e., with no constraint on the recipient), and would reject expansions that add a malicious address to the recipient allow list.

    “For the Manual Approval setting in Section 8.2, we assume the user has strong security awareness: they would not grant unrestricted access to highly sensitive tools such as send_money (i.e., with no constraint on the recipient), and would reject expansions that add a malicious address to the recipient allow list.”
  • The paper relies on this without support.

    The LLM-based policy update mechanism utilizes external information and the user may have limited security awareness and make risky decisions in practice.

    “Considering that the LLM-based policy update mechanism utilizes external information and the user with limited security awareness who may make risky decisions in practice”
  • The paper relies on this without support.

    An attacker can be aware of the LLM-based policy update mechanism and the user's limited security awareness.

    “an attacker who is aware of this could construct adaptive attacks to circumvent Progent's defense effect.”
  • The paper relies on this without support.

    The LLM-generated policy update can be manipulated by adversarial inputs.

    “even if the LLM-generated policy update is manipulated by adversarial inputs”
26 of 26

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Expansion has two incompatible definitions

    Minor

    A policy update can add some allowed calls while removing others. Such an update is not a strictly larger set of allowed calls under the first definition, but it is an expansion under the later definition because it permits at least one new call.

1 of 1

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 5

  1. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. In ICLR.
  2. Xun Li, Vineeth Kashyap, Jason K Oberg, Mohit Tiwari, Vasanth Ram Rajarathi- nam, Ryan Kastner, Timothy Sherwood, Ben Hardekopf, and Frederic T Chong. 2014. Sapper: A language for hardware-level security policy enforcement. In Proceedings of the 19th international conference on Architectural support for pro- gramming languages and operating systems. 97-112.
  3. Chen Henry Wu, Rishi Rajesh Shah, Jing Yu Koh, Russ Salakhutdinov, Daniel Fried, and Aditi Raghunathan. 2024. Dissecting Adversarial Robustness of Multimodal LM Agents. In NeurIPS 2024 Workshop on Open-World Agents.
  4. Yuhao Wu, Franziska Roesner, Tadayoshi Kohno, Ning Zhang, and Umar Iqbal. 2025. IsolateGPT: An Execution Isolation Architecture for LLM-Based Systems. In Network and Distributed System Security Symposium (NDSS).
  5. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In ICLR.

Cited by

Sorted by citations

Authors
Shi, T., He, J., Wang, Z., Li, H., Wu, L., Guo, W., & Song, D.
Affiliations
  • University of California, Berkeley (US)
  • University of California, Santa Barbara (US)
  • National University of Singapore (SG)
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
OpenAlex
W4416541023
Cited by
3
References
0
Citations by year
2026: 2 · 2025: 1
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Social Sciences › Sociology and Political Science › Access Control and Trust
Keywords
AI agent security, principle of least privilege, prompt injection, security policy generation, privilege escalation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-24.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-24.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review