CoChat
Sign Up
Cited by 15Adversarial Robustness in Machine LearningSafety Systems Engineering in AutonomyEthics and Social Impacts of AI

AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents

Open access
Wang, Haoyu, Poskitt, Christopher M., Sun, Jun·2026·Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering
DOI 10.1145/3744916.3764546
Open this paper in CoChat — free account

Abstract

Agents built on LLMs are increasingly deployed across diverse domains, automating complex decision-making and task execution. However, their autonomy introduces safety risks, including security vulnerabilities, legal violations, and unintended harmful actions. Existing mitigation methods, such as model-based safeguards and early enforcement strategies, fall short in robustness, interpretability, and adaptability. To address these challenges, we propose AgentSpec, a lightweight domain-specific language for specifying and enforcing runtime constraints on LLM agents. With AgentSpec, users define structured rules that incorporate triggers, predicates, and enforcement mechanisms, ensuring agents operate within predefined safety boundaries. We implement AgentSpec across multiple domains, including code execution, embodied agents, and autonomous driving, demonstrating its adaptability and effectiveness. Our evaluation shows that AgentSpec successfully prevents unsafe executions in over 90% of code agent cases, eliminates all hazardous actions in embodied agent tasks, and enforces 100% compliance by autonomous vehicles (AVs). Despite its strong safety guarantees, AgentSpec remains computationally lightweight, with overheads in milliseconds. By combining interpretability, modularity, and efficiency, AgentSpec provides a practical and scalable solution for enforcing LLM agent safety across diverse applications. We also automate the generation of rules using LLMs and assess their effectiveness. Our evaluation shows that the rules generated by OpenAI o1 achieve a precision of 95.56% and recall of 70.96% for embodied agents, successfully identify 87.26% of the risky code, and prevent AVs from breaking laws in 5 out of 8 scenarios.

Verified 2026-10-24 · 11 references not checked yet

AI snapshot

Written from the paper’s full text.

TL;DR

AgentSpec is a domain-specific language for specifying and enforcing runtime safety constraints on LLM agents through triggers, predicates, and enforcement mechanisms. Across code, embodied, and autonomous-driving domains, it substantially reduces unsafe behavior with millisecond-level overhead, though LLM-generated rules remain imperfect.

Key findings

  • AgentSpec prevented unsafe executions in over 90% of code agent cases; in the RedCode-Exec evaluation, the code agent attempted risky invocations in 24 of 25 categories, and AgentSpec’s checks flagged most risky programs, with some categories allowed after user inspection.
  • For embodied agents, AgentSpec eliminated hazardous actions in all 10 unsafe categories (0% hazard with AgentSpec), while safe-task fulfillment dropped only from 58.62% to 54.26%.
  • For autonomous vehicles, AgentSpec enforced 100% compliance across the tested FixDrive law-violation scenarios in Table 5.
  • LLM-generated rules using OpenAI o1 achieved 95.56% precision and 70.96% recall for embodied agents, identified 87.26% of risky code, and prevented AV lawbreaking in 5 out of 8 scenarios; runtime overhead was milliseconds (parsing ~1.42 ms, predicate evaluation 2.83 ms for code and 1.11 ms for embodied agents).

Methods

  • AgentSpec is a DSL where rules consist of a trigger, check predicates, and enforcement actions; it was implemented on LangChain 0.3.13 by intercepting `iter_next_step` at `AgentAction`, `AgentStep`, and `AgentFinish`, and rules are parsed with ANTLR4.
  • Evaluation used three agents and datasets: a CodeAct code agent on RedCode-Exec (25 vulnerability types, 30 test cases per category), an embodied agent on SafeAgentBench (10 unsafe categories plus safe tasks), and an Apollo-based autonomous driving agent on FixDrive law-breaking scenarios.
  • The study evaluated manually written rules and automatically generated rules from OpenAI o1, measuring enforcement rates, precision/recall, compliance, and runtime overhead.

Limitations

  • The paper states that AgentSpec performs deterministic enforcement at discrete checkpoints and lacks trajectory-based safety analysis for long-term consequences; future work proposes probabilistic enforcement such as learning a DTMC.
  • The paper states threats to validity including rule overfitting, mitigated by a 1:9 dataset split, and human involvement in manual validation, mitigated by structured evaluation, cross-validation, and blind protocols where applicable.
  • The text also makes evident that LLM-generated rules can overfit examples, miss broader unsafe patterns, produce false negatives/positives, and fail in some AV scenarios (e.g., Law38 and Law46), leaving room for improvement compared with manually implemented rules.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3744916.3764546

Cites

44 of 69 references verified0 citation errors · 14 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

6 of 19 cited sources checked · 4 supported · 0 flagged · accuracy not yet measured

  • Premisecitation
    § 1 Introduction

    LLM-powered risk evaluation systems such as ToolEmu simulate potential outcomes using an LLM sandbox before execution.

    “LLM-powered risk evaluation systems, such as ToolEmu [32], simulate potential outcomes using an LLM sandbox before execution.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [32] [32] Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J. Maddison, and Tatsunori Hashimoto. 2024. Identifying the Risks of LM Agents with an LM-Emulated Sandbox. In ICLR . OpenReview.net. - source
  • Interpretationcitation
    § 6.1 Comparison to Prior Work

    In contrast to frameworks like GuardAgent, which rely on LLMs to interpret and apply safety constraints, AgentSpec adopts an external, developer-defined enforcement model.

    “In contrast to frameworks like GuardAgent [44], which rely on LLMs to interpret and apply safety constraints, AgentSpec adopts an external, developer-defined enforcement model .”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [44] [44] Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, Dawn Song, and Bo Li. 2024. GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning. CoRR abs/2406.09187 (2024). - source
  • Findingown data
    § Abstract

    AgentSpec eliminates all hazardous actions in embodied agent tasks.

    “eliminates all hazardous actions in embodied agent tasks”
    • Also stated in: § 1 Introduction · plain
      “eliminated all hazardous actions in 10 categories of embodied agent tasks”
    • Also stated in: § 1 Introduction · plain
      “eliminates hazardous actions in embodied agent tasks”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The three datasets (RedCode-Exec, SafeAgentBench, FixDrive) target critical safety challenges representative of real-world risks.
  • Findingown data
    § Abstract

    AgentSpec prevents unsafe executions in over 90% of code agent cases.

    “AgentSpec successfully prevents unsafe executions in over 90% of code agent cases”
    • Also stated in: § 1 Introduction · boosted
      “Our evaluation demonstrates that AgentSpec reduced unsafe executions in code agents by detecting and intercepting risks in over 90% of cases”
    • Also stated in: § 1 Introduction · boosted
      “An experimental evaluation showing that AgentSpec prevents over 90% of unsafe code executions”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The three datasets (RedCode-Exec, SafeAgentBench, FixDrive) target critical safety challenges representative of real-world risks.
  • Findingnone given
    § Abstract

    AgentSpec has overheads in milliseconds.

    “overheads in milliseconds”
    • Also stated in: § 1 Introduction · plain
      “AgentSpec imposes minimal overhead in milliseconds”
    • Also stated in: § 1 Introduction · plain
      “operates with millisecond-level overhead”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The three datasets (RedCode-Exec, SafeAgentBench, FixDrive) target critical safety challenges representative of real-world risks.
  • May combine two claims. 28
  • Classified as probably not a substantial claim. 22

  • May combine two claims.

    Premisecitation
    § 1 Introduction

    GuardAgent enforces safety policies via autogenerated guardrails but requires manual implementation for each agent instance, making adoptability a challenge.

    “GuardAgent [44], for example, enforces safety policies via autogenerated guardrails but requires manual implementation for each agent instance, making adoptability a challenge.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [44] [44] Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, Dawn Song, and Bo Li. 2024. GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning. CoRR abs/2406.09187 (2024). - source

      The Introduction (p. 2) and Framework overview (p. 5) support the claim that GuardAgent generates executable guardrail code to enforce safety requests. Deployment requires agent-specific specifications and safety requests, but the paper does not say that guardrails must be manually implemented for each agent instance. Instead, the Framework (p. 5) describes GuardAgent as non-invasive and adaptable to new agents, while the Abstract (p. 1) claims low operational overhead; the asserted adoptability challenge is not supported. (LLM, conf=0.87)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § 1 Introduction

    LLM-powered risk evaluation systems lack interpretability and offer no mechanism for safety enforcement, making them susceptible to adversarial manipulation.

    “While effective in identifying risks, these methods lack interpretability and offer no mechanism for safety enforcement, making them susceptible to adversarial manipulation.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [32] [32] Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J. Maddison, and Tatsunori Hashimoto. 2024. Identifying the Risks of LM Agents with an LM-Emulated Sandbox. In ICLR . OpenReview.net. - source

      The Introduction reports that the LM-based evaluator identified 73.1% of failures identified by human annotators, supporting its ability to identify risks. Limitations & Future Directions reports that emulators overlooked constraints more often in complex or adversarial scenarios, but it does not say risk evaluators lack interpretability or safety-enforcement mechanisms, or that they are susceptible to adversarial manipulation. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation
    § 1 Introduction

    FixDrive evaluates autonomous driving agents in law-breaking scenarios.

    “FixDrive [37] evaluates autonomous driving agents in law-breaking scenarios.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [37] [37] Yang Sun, Christopher M. Poskitt, Kun Wang, and Jun Sun. 2025. FixDrive: Automatically Repairing Autonomous Vehicle Driving Behaviour for $0.08 per Violation. In ICSE . IEEE, 1921-1933. - source

  • Classified as probably not a substantial claim.

    Premisecitation
    § 1 Introduction

    SafeAgentBench assesses the ability of embodied agents to avoid hazards like fire and electrical shocks.

    “SafeAgentBench [49] assesses the ability of embodied agents to avoid hazards like fire and electrical shocks”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [49] [49] Sheng Yin, Xianghe Pang, Yuanzhuo Ding, Menglan Chen, Yutong Bi, Yichen Xiong, Wenhao Huang, Zhen Xiang, Jing Shao, and Siheng Chen. 2024. SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents. CoRR abs/2412.13178 (2024). AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents - source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning
    § 5.5 RQ4: Runtime Overhead

    For the stop action, execution time is negligible as it immediately halts the process.

    “For the stop action, the execution time is negligible as it immediately halts the process.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5.4 RQ3: AgentSpec Rule Generalizability

    In the autonomous vehicle context, 6 rules address 8 distinct scenarios, reflecting that each legal requirement yielded only 1-2 violation scenarios.

    “Finally, in the autonomous vehicle context, 6 rules were created to address 8 distinct scenarios, reflecting the fact that each legal requirement yielded only 1-2 violation scenarios.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5.2 RQ1: Expressiveness and Effectiveness

    In the 'Miss default case' and 'Duplicate key in the associative list' categories, although the agent invoked the Python code, it did not follow the instructions.

    “Upon manual inspection, it was found that although the agent invoked the Python code, it did not follow the instructions (i.e., there was no missing default case or duplicate key in the Python code planned by agent, respectively).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § 5.2 RQ1: Expressiveness and Effectiveness

    In the 'Unsafe deserialization' scenario, the generated code was deemed harmless and allowed to proceed.

    “Upon user inspection, the generated code was deemed harmless and allowed to proceed.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given
    § 5.4 RQ3: AgentSpec Rule Generalizability

    One AV rule (Figure 8) aims to prevent collisions and extends its coverage beyond the scenarios explicitly identified in the dataset.

    “Notably, one of these rules (Figure 8) aims to prevent collisions and extends its coverage beyond the scenarios explicitly identified in the dataset.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5.5 RQ4: Runtime Overhead

    On average, evaluating predicates requires 2.83 milliseconds for code-based scenarios and 1.11 milliseconds for embodied agents.

    “On average, evaluating predicates requires 2.83 milliseconds for code-based scenarios and 1.11 milliseconds for embodied agents.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 5.2 RQ1: Expressiveness and Effectiveness

    The program shown in Figure 6 satisfies both predicates, identifying it as risky.

    “The program shown in Figure 6 satisfies both predicates, identifying it as risky.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5.3 RQ2: Effectiveness of LLM-Generated Rules

    Enforcing LLM-generated rules in the AV experiments prevented law-breaking in five out of eight scenarios, covering no collision, Law44, Law53, and finish journey.

    “Nevertheless, by enforcing these LLM-generated rules, AgentSpec successfully prevented law-breaking in five out of eight scenarios, covering no collision , Law44 , Law53 , and fi nish journey .”
    • Also stated in: § 5.4 RQ3: AgentSpec Rule Generalizability · plain
      “For AVs, rules generated from the laws in a zero-shot setting prevented 5 out of 8 law-breaking scenarios.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.4 RQ3: AgentSpec Rule Generalizability

    Despite only 10% of risky scenarios being used as in-context examples, the generated rules detected 87.26% of risky cases in the code agent and 95.56% in the embodied agent on unseen scenarios in the remaining dataset.

    “Despite this limited exposure, the generated rules successfully detected 87.26% of risky cases in the code agent and 95.56% in the embodied agent when applied to unseen scenarios in the remaining dataset.”
    • Also stated in: § 5.4 RQ3: AgentSpec Rule Generalizability · plain
      “For code and embodied agents, only 10% of the risky scenarios in the dataset were used as in-context examples to learn the safety rules.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 5.3 RQ2: Effectiveness of LLM-Generated Rules

    When evaluating risks such as deleting critical configuration files, the LLM may only recognize explicitly mentioned files and directories as sensitive, and consequently fails to flag similar but unlisted cases.

    “For instance, when evaluating risks such as deleting critical configuration files (e.g., those in /etc/ ), the LLM may only recognize explicitly mentioned files and directories as sensitive. Consequently, it fails to flag similar but unlisted cases, leading to gaps in detection.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3 RQ2: Effectiveness of LLM-Generated Rules

    The overall precision of the LLM-generated rules for the embodied agent is acceptable, successfully enforcing 95.56% of the risky cases.

    “Our evaluation on the agent demonstrates that the overall precision of the LLM-generated rules is acceptable, successfully enforcing 95.56% of the risky cases.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingnone given
    § 5.5 RQ4: Runtime Overhead

    Code-based agents exhibit an average execution time of 25.4 seconds, while embodied agents operate with an average runtime of 9.82 seconds.

    “Code-based agents exhibit an average execution time of 25.4 seconds, while embodied agents operate with an average runtime of 9.82 seconds.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 8 Conclusion

    Rules can be generated both manually and automatically, with LLMs achieving high accuracy in specifying and enforcing safety conditions.

    “Furthermore, we demonstrate that rules can be generated both manually and automatically, with LLMs achieving high accuracy in specifying and enforcing safety conditions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 8 Conclusion

    Empirical evaluations show that AgentSpec prevents unsafe code executions, avoids hazardous actions in embodied agents, and ensures lawful decision-making for autonomous driving, all with minimal runtime overhead.

    “Empirical evaluations show that AgentSpec prevents unsafe code executions, avoids hazardous actions in embodied agents, and ensures lawful decision-making for autonomous driving, all with minimal runtime overhead.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction

    RedCode-Exec tests code execution safety with prompts covering 25 vulnerability types across eight domains.

    “RedCode-Exec [13] tests code execution safety with prompts covering 25 vulnerability types across eight domains.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] [13] ChengquanGuo,XunLiu,ChulinXie, Andy Zhou, Yi Zeng, Zinan Lin, Dawn Song, and Bo Li. 2024. RedCode: Risky Code Execution and Generation Benchmark for Code Agents. In NeurIPS . -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § 3.2 Triggers, Checks, and Enforcements

    The event-based framework is not limited to these domains; it is inherently extensible.

    “The event-based framework is not limited to these domains; it is inherently extensible.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 1 Introduction

    AgentSpec is framework-agnostic and can be adapted to other ecosystems, such as Microsoft's AutoGen and autonomous vehicle systems like Apollo.

    “While LangChain serves as the primary integration example, AgentSpec remains framework-agnostic and can be adapted to other ecosystems, such as Microsoft's AutoGen [25] and autonomous vehicle systems like Apollo [3].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [25] [25] Microsoft. 2025. AutoGen: A framework for building AI agents and applications. https://microsoft.github.io/autogen/stable//index.html. Accessed: 2025-01-14. -

      Not checked in the free run.

    • cites [3] [3] Baidu Apollo. 2025. Apollo Self-Driving. https://www.apollo.auto/apollo-selfdriving. Accessed: 2025-02-11. -

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data
    § 5.3 RQ2: Effectiveness of LLM-Generated Rules

    For each agent, the risky dataset was split into examples and a test set, with the test set used to evaluate the enforcement rate.

    “For each agent, we split the corresponding risky dataset (containing #Scenario cases), into examples (containing #Example cases) and a test set (rest of the dataset).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 7 Related Work

    AgentSpec can specify properties to defend against attacks by providing a framework for defining rules to safeguard LLM agents.

    “Our work, AgentSpec, can specify properties to defend against these attacks by providing a framework for defining rules to safeguard LLM agents.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 5.3 RQ2: Effectiveness of LLM-Generated Rules

    The LLM-generated rules effectively identify unsafe actions with high accuracy, but occasionally fail to capture certain unsafe scenarios.

    “This indicates that while the LLM-generated rules effectively identify unsafe actions with high accuracy, they occasionally fail to capture certain unsafe scenarios.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning
    § 6.2 Expressiveness of AgentSpec

    This decoupling avoids brittleness from prompt engineering or fine-tuning and enables transparent inspection and auditing of enforced rules.

    “This decoupling avoids brittleness from prompt engineering or fine-tuning and enables transparent inspection and auditing of enforced rules.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning
    § 6.2 Expressiveness of AgentSpec

    These capabilities make AgentSpec well-suited for privacy-sensitive domains such as enterprise automation, legal assistance, and healthcare.

    “These capabilities make AgentSpec well-suited for privacy-sensitive domains such as enterprise automation, legal assistance, and healthcare.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 5.3 RQ2: Effectiveness of LLM-Generated Rules

    The former is part of the prompt provided as in-context demonstration for the LLM, and the latter is used when evaluating the enforcement rate, i.e., the percentage of unseen risky scenarios in which the LLM-generated rules are successfully applied.

    “The former is part of the prompt provided as in-context demonstration for the LLM, and the latter is used when evaluating the enforcement rate, i.e., the percentage of unseen risky scenarios in which the LLM-generated rules are successfully applied.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 5.2 RQ1: Expressiveness and Effectiveness

    The rule-based enforcement mechanism ensures that the agent resists executing all risky tasks while maintaining high fulfillment ability on safe tasks.

    “The rule-based enforcement mechanism ensures that the agent resists executing all risky tasks while maintaining high fulfillment ability on safe tasks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 5.5 RQ4: Runtime Overhead

    AgentSpec's computational overhead is on the order of milliseconds, so it remains lightweight and does not impose a significant performance burden on the overall system.

    “Given that the computational overhead introduced by AgentSpec is on the order of milliseconds, it remains lightweight and does not impose a significant performance burden on the overall system.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 5.2 RQ1: Expressiveness and Effectiveness

    A program was allowed to proceed only if it was determined to pose no harm.

    “A program was allowed to proceed only if it was determined to pose no harm.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 5.2 RQ1: Expressiveness and Effectiveness

    AgentSpec ensures that embodied agents adhere to safety constraints while remaining functional in executing benign instructions.

    “Through this mechanism, AgentSpec ensures that embodied agents adhere to safety constraints while remaining functional in executing benign instructions, as illustrated in Figure 7.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation
    § 5.2 RQ1: Expressiveness and Effectiveness

    AgentSpec can translate and enforce all rules defined in FixDrive, ensuring AVs comply with traffic laws and avoid unsafe behaviors.

    “This allows AgentSpec to seamlessly translate and enforce all rules defined in FixDrive [37], ensuring AVs comply with traffic laws and avoid unsafe behaviors.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites FixDrive [37]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 6.1 Comparison to Prior Work

    By decoupling constraint interpretation from LLM internals, AgentSpec enhances transparency, auditability, and confidence in the system's adherence to safety specifications.

    “By decoupling constraint interpretation from LLM internals, AgentSpec enhances transparency, auditability, and confidence in the system's adherence to safety specifications.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation
    § 7 Related Work

    AgentSpec aligns with the vision of integrating formal methods with LLMs by embedding symbolic rule enforcement into agent execution.

    “AgentSpec aligns with this vision by embedding symbolic rule enforcement into agent execution.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [53] [53] Yedi Zhang, Yufan Cai, Xinyue Zuo, Xiaokun Luan, Kailong Wang, Zhe Hou, Yifan Zhang, Zhiyuan Wei, Meng Sun, Jun Sun, Jing Sun, and Jin Song Dong. 2025. Position: Trustworthy AI Agents Require the Integration of Large Language Models and Formal Methods. In ICML . OpenReview.net. -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning
    § 3.2 Triggers, Checks, and Enforcements

    By combining predefined and customizable actions, the enforcement framework ensures adaptive, safe, and efficient decision-making across diverse domains while safeguarding against potential risks.

    “By combining predefined and customizable actions, this enforcement framework ensures adaptive, safe, and efficient decision-making across diverse domains while safeguarding against potential risks.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 4 AgentSpec Implementation

    AgentSpec can be extended to complex autonomous systems such as Apollo.

    “Additionally, AgentSpec can be extended to complex autonomous systems such as Apollo [3], an autonomous driving software stack.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Apollo [3] [3] Baidu Apollo. 2025. Apollo Self-Driving. https://www.apollo.auto/apollo-selfdriving. Accessed: 2025-02-11. -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning
    § 5.4 RQ3: AgentSpec Rule Generalizability

    Because different risky behaviors often involve unique features, additional rules are still needed for novel risks.

    “Because different risky behaviors often involve unique features, additional rules are still needed for novel risks.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 6.1 Comparison to Prior Work

    Compared to Nemo, which applies natural language constraints at the dialogue level, AgentSpec enforces rules at execution-critical junctures, enabling finer-grained and more reliable interventions.

    “Compared to Nemo [27], which applies natural language constraints at the dialogue level, AgentSpec enforces rules at execution-critical junctures-such as immediately before invoking high-impact or potentially unsafe actions-enabling finer-grained and more reliable interventions.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] [27] NVIDIA. 2025. NeMo: A scalable generative AI framework. https://github.com/ NVIDIA/NeMo. -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning
    § 5.5 RQ4: Runtime Overhead

    To reduce the risk from human involvement in experimental evaluations, structured evaluation methodologies were implemented, including cross-validation by multiple authors, predefined assessment criteria, and blind evaluation protocols where applicable.

    “To reduce this risk, we implement structured evaluation methodologies, including cross-validation by multiple authors, predefined assessment criteria, and blind evaluation protocols where applicable.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § 5.5 RQ4: Runtime Overhead

    To mitigate overfitting risk, the risky dataset was split into two parts at a 1:9 ratio, using the smaller portion as demonstrations for rule development and the larger portion for testing.

    “To mitigate this, we split the risky dataset into two parts at a 1:9 ratio, using the smaller portion as demonstrations for rule development and the larger portion for testing.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 5.5 RQ4: Runtime Overhead

    Predicate evaluation is computationally lightweight and does not introduce significant latency in system execution.

    “These results indicate that predicate evaluation is computationally lightweight and does not introduce significant latency in system execution.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 4 AgentSpec Implementation

    AgentSpec can be integrated into Microsoft's AutoGen.

    “For instance, AgentSpec can be integrated into Microsoft's AutoGen [25], which also supports multi-step reasoning and agent-tool interactions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites AutoGen [25]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 6.3 Limitations and Future Work

    This approach offers high reliability and interpretability but does not reason about the long-term consequences of current actions.

    “While this approach offers high reliability and interpretability, it does not reason about the long-term consequences of current actions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 7 Related Work

    AgentSpec differs by offering a flexible and generalizable rule-based enforcement framework that can be configured to address LLM threats.

    “Our work, AgentSpec, differs by offering a flexible and generalizable rule-based enforcement framework that can be configured to address above LLM threats.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 6.2 Expressiveness of AgentSpec

    Rule enforcement is declarative and externalized from the LLM, ensuring consistent behavior across runs, environments, and model versions.

    “Rule enforcement is declarative and externalized from the LLM, ensuring consistent behavior across runs, environments, and model versions.”
    Quote in the paperSubstantial claim
45 of 45

  • The paper relies on this without support.

    Organizations that use LLM agents may exhibit varying levels of risk tolerance.

    “organizations that use them may exhibit varying levels of risk tolerance.”

  • The paper relies on this without support.

    The three datasets (RedCode-Exec, SafeAgentBench, FixDrive) target critical safety challenges representative of real-world risks.

    “AgentSpec is evaluated using three datasets targeting critical safety challenges.”

  • The paper relies on this without support.

    LLM agents are increasingly deployed across diverse domains, automating complex decision-making and task execution.

    “Agents built on LLMs are increasingly deployed across diverse domains, automating complex decision-making and task execution.”

  • The paper relies on this without support.

    Increasing the generalizability of AgentSpec rules enhances their reusability and reduces the overall effort required, which in turn facilitates the maintenance of a comprehensive rule repository.

    “Increasing the generalizability of these rules enhances their reusability and reduces the overall effort required, which in turn facilitates the maintenance of a comprehensive rule repository.”
4 of 4

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Embodied-agent detection rate conflicts with reported recall

    Major

    The paper defines the enforcement rate as the percentage of unseen risky scenarios successfully covered, then says the generated rules detected 95.56% of risky embodied-agent cases. That conflicts with its reported 70.96% recall for detecting violations in those cases; the 95.56% figure is also labeled precision elsewhere.

1 of 1

Cites

Flagged first, then by citations

24 of the 57 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Wang, H., Poskitt, C. M., & Sun, J.
Affiliations
Singapore Management University (SG)
ORCID
Venue
Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering
Publisher
ACM
Published
2026-04-12
Pages
2938-2950
DOI
10.1145/3744916.3764546
OpenAlex
W6947676652
Semantic Scholar
a929afcf3c78db99d73a5532926ff2484514d037
Cited by
14
References
57
Citations by year
2026: 14
FWCI
122.50
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE/ACM International Conference on Software Engineering (ICSE) (publishedVersion, cc-by)
  • Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (cc-by)
Funders
Ministry of Education - Singapore (Tier 3, MOET32020-0004)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
LLM agents, runtime enforcement, safety constraints, domain-specific languages, coding agents, embodied agents, autonomous driving, rule generation, LLM safety, security vulnerabilities, legal compliance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-24.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-24.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review