WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks
Open accessAbstract
Autonomous UI agents powered by AI have tremendous potential to boost human productivity by automating routine tasks such as filing taxes and paying bills. However, a major challenge in unlocking their full potential is security, which is exacerbated by the agent's ability to take action on their user's behalf. Existing tests for prompt injections in web agents either over-simplify the threat by testing unrealistic scenarios or giving the attacker too much power, or look at single-step isolated tasks. To more accurately measure progress for secure web agents, we introduce WASP -- a new publicly available benchmark for end-to-end evaluation of Web Agent Security against Prompt injection attacks. Evaluating with WASP shows that even top-tier AI models, including those with advanced reasoning capabilities, can be deceived by simple, low-effort human-written injections in very realistic scenarios. Our end-to-end evaluation reveals a previously unobserved insight: while attacks partially succeed in up to 86% of the case, even state-of-the-art agents often struggle to fully complete the attacker goals -- highlighting the current state of security by incompetence.
AI snapshot
Written from the paper’s full text.
TL;DR
WASP is a publicly released benchmark for end-to-end evaluation of web-agent security against prompt injection attacks, built on self-hosted VisualWebArena GitLab and Reddit environments. It finds that agents are often hijacked from their user goal, but attackers rarely achieve their full objective, a pattern the authors call “security by incompetence.”
Key findings
- Across tested models and scaffoldings, ASR-intermediate ranged from 17% to 86%, while ASR-end-to-end ranged from 0% to 17% (Table 2; abstract reports up to 86% partial success and 0–17% attacker task completion).
- The highest ASR-intermediate in Table 2 was OpenAI o1 with Tool Calling Loop and instruction hierarchy in the system role at 0.857, with ASR-end-to-end 0.167; GPT-4o-mini with VisualWebArena had ASR-intermediate 0.345 and ASR-end-to-end 0.024.
- URL injections were generally more effective for ASR-intermediate than plain-text injections, e.g. GPT-4o with axtree + SOM had 0.619 vs 0.238 ASR-intermediate, while Claude showed higher ASR-end-to-end for plain-text prompts.
- Task-agnostic injections reduced attack success but remained non-zero, e.g. Claude-3.7-sonnet ASR-intermediate fell from 0.500 to 0.321 and ASR-end-to-end from 0.036 to 0.000.
Methods
- WASP builds on VisualWebArena with self-hosted GitLab and Reddit environments; attackers are black-box adversarial users limited to creating issues, posts, and comments, not controlling site layout or other users’ content.
- Test cases combine 21 attacker goals with 2 user goals per environment, yielding 42 scenarios; two prompt-injection templates (plain-text and URL) produce 84 tasks, and a separate 37-task utility set measures baseline agent performance without attacks.
- Evaluated backbones include GPT-4o, GPT-4o-mini, OpenAI o1, Claude Sonnet 3.5 v2, and Claude Sonnet 3.7 with Extended Thinking, using VisualWebArena, Claude CURI, and Tool Calling Loop scaffoldings; metrics are ASR-intermediate via a GPT-4o LLM judge, ASR-end-to-end via rule-based evaluators, and Utility.
Limitations
- The paper states WASP currently supports only two environments (Reddit and GitLab), lacks a more diverse set of websites and prompt injection attack prompts, and has not yet been extended to desktop or code agents.
- Evident from the text: the benchmark uses only two user goals per environment, manual attack baselines, and an LLM judge for ASR-intermediate; the checklist also reports no error bars or statistical significance testing and no compute-resource reporting.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
6 of 11 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured
- Study design and conductcitation§ 1 Introduction · p. 2
WASP is a dynamic benchmark built within a sandbox web environment based on VisualWebArena.
“WASP is a dynamic benchmark built within a sandbox web environment based on VisualWebArena (Koh et al., 2024).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Koh et al., 2024 source
Section 3 describes VisualWebArena’s self-hosted web applications, but does not establish that WASP is built on them. Searches for WASP, a dynamic benchmark, and a sandbox environment found no passage addressing WASP or its relationship to VisualWebArena. (LLM, conf=0.72)
Experimental — accuracy not yet measured
- Findingown data§ 4.2 Results · p. 8
For all models, the adversary can influence the model's actions in more than 50% of tested cases.
“For all models, the adversary can influence the model's actions in more than 50% of tested cases.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Findingreasoning§ 4.2 Results · p. 8
URL injections are highly effective in terms of ASR-intermediate.
“The result suggests that URL injections are highly effective in terms of ASR-intermediate .”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Findingown data§ Abstract · p. 1
Evaluating with WASP shows that even top-tier AI models, including those with advanced reasoning capabilities, can be deceived by simple, low-effort human-written injections in very realistic scenarios.
“Evaluating with WASP shows that even top-tier AI models, including those with advanced reasoning capabilities, can be deceived by simple, low-effort human-written injections in very realistic scenarios.”
- Also stated in: § 4.2 Results · p. 7 · hedged
“We observe a high ASR-intermediate across all scaffoldings and models, indicating that agents-even those backed by models with enhanced reasoning capabilities, such as Claude Sonnet 3.7 with Extended Thinking and o1-are readily hijacked by counterintuitive malicious instructions.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Also stated in: § 4.2 Results · p. 7 · hedged
- Findingreasoning§ 4.2 Results · p. 8
Claude exhibits a higher ASR-end-to-end for plain-text prompts than for URL injections.
“Interestingly, Claude exhibits a higher ASR-end-to-end for plain-text prompts.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Findingown data§ 4.2 Results · p. 8
For the o1 model, placing capability instructions in the system message is more vulnerable than placing them in the developer message.
“there is a difference in whether the instructions describing the system capability are placed in the developer or in the system message for the o1 model, with the latter being more vulnerable.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Findingown data§ 4.2 Results · p. 8
Counterintuitively, some of the highest ASR-intermediate occur in models deployed with the Tool Calling Loop scaffolding that only places web page content in the lowest-privilege messages in instruction hierarchy.
“Counterintuitively, we observe some of our highest ASR-intermediate in models deployed with the Tool Calling Loop scaffolding that only places the web page content (and, therefore, all injected prompts) in the lowest-privilege messages in instruction hierarchy.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Findingown data§ 4.2 Results · p. 9
For Claude-3.7-sonnet, ASR-intermediate decreases from 50% for task-related injection to 32% for task-agnostic injection.
“For instance, ASR-intermediate for Claude decreases from 50% to 32%.”
Quote in the paperSubstantial claim- Rests on an assumption: The LLM-based judge used to determine whether an agent is compromised accurately classifies the agent's reasoning.
- Findingown data§ 5 Conclusion · p. 9
It is relatively easy to hijack agents from their original objectives.
“Through our benchmark, we find that it is relatively easy to hijack agents from their original objectives, and current mitigation techniques are insufficient to prevent this.”
Quote in the paperSubstantial claim- Rests on an assumption: The two environments (reddit and gitlab) are sufficiently representative of real-world web navigation tasks for WASP's findings to generalize.
- Rests on an assumption: The LLM-based judge used to determine whether an agent is compromised accurately classifies the agent's reasoning.
- Findingown data§ 1 Introduction · p. 3
In WASP end-to-end evaluation, ASR-end-to-end (attacker task completion) is at most 17%.
“with attacker task completion rates ranging from 0 to 17% .”
- Also stated in: § 4 Experiments · p. 6 · plain
“with an attack success rate ( ASR-end-to-end ) of only up to 16%.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Also stated in: § 4 Experiments · p. 6 · plain
- Findingown data§ 4.2 Results · p. 8
More capable agents, such as o1 with Tool Calling Loop, show increased ASR-end-to-end as their ability to reason through web pages and select correct elements improves.
“When more capable agents are hijackedsuch as the o1 model with the Tool Calling Loop scaffoldingASR-end-to-end increases as their ability to reason through the different parts of the web page and select the correct element to interact with improves.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Findingown data§ 5 Conclusion · p. 9
Current mitigation techniques are insufficient to prevent hijacking agents from their original objectives.
“Through our benchmark, we find that it is relatively easy to hijack agents from their original objectives, and current mitigation techniques are insufficient to prevent this.”
Quote in the paperSubstantial claim- Rests on an assumption: The two environments (reddit and gitlab) are sufficiently representative of real-world web navigation tasks for WASP's findings to generalize.
- Rests on an assumption: The LLM-based judge used to determine whether an agent is compromised accurately classifies the agent's reasoning.
- Findingown data§ Abstract · p. 1
In WASP end-to-end evaluation, attacks partially succeed in up to 86% of cases.
“while attacks partially succeed in up to 86% of the case”
- Also stated in: § 4 Experiments · p. 6 · plain
“between 17-86% ASR-intermediate”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Also stated in: § 4 Experiments · p. 6 · plain
- Interpretationreasoning§ 5 Conclusion · p. 9
Achieving the ultimate goal of the attacker is significantly more challenging than hijacking, due to the limitations of the agents' capabilities and the complexity of the attacker's objectives.
“However, achieving the ultimate goal of the attacker proves to be significantly more challenging due to the limitations of the agents' capabilities and the complexity of the attacker's objectives.”
Quote in the paperSubstantial claim- Rests on an assumption: The two environments (reddit and gitlab) are sufficiently representative of real-world web navigation tasks for WASP's findings to generalize.
- Rests on an assumption: The LLM-based judge used to determine whether an agent is compromised accurately classifies the agent's reasoning.
- Study design and conductnone given§ 5 Conclusion · p. 9
WASP incorporates more complex attacker goals (e.g., changing the user's password).
“incorporating realistic assumptions about attacker and defender capabilities and more complex attacker goals (e.g., changing the user's password).”
Quote in the paperSubstantial claim - Study design and conductnone given§ 1 Introduction · p. 2
WASP models attackers as adversarial users of websites rather than assuming entire websites are compromised.
“we do not assume that entire websites are compromised; instead, we model attackers as adversarial users of these websites.”
Quote in the paperSubstantial claim - Study design and conductnone given§ 3.1 Overview · p. 4
WASP only considers black-box attackers with control over specific webpage elements.
“In WASP, we only consider black-box attackers with control over specific webpage elements.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- Interpretationown data§ 4.2 Results · p. 8
The main bottleneck for attack success is not the strength of the attack but the capability of the agent to complete the attacker's goal when hijacked.
“the main bottleneck for attack success is not the strength of their attack but rather the capability of the agent to complete their goal when hijacked.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Interpretationreasoning§ 2 Background · p. 3
WASP attacks are more realistic than prior work because its threat model is weaker.
“In contrast, our threat model is even weaker, which makes our attacks more realistic.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- Premisenone given§ 1 Introduction · p. 2
Many benchmarks used by major model providers to assess pre-launch risk are not released publicly.
“many benchmarks-especially those used by major model providers to assess pre-launch risk and discussed in their system cards-are not released publicly.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 3.3 Prompt injection pipeline · p. 5
As long as the current webpage's URL is part of the agent's context, the malicious instruction will be visible to the agent.
“As a result, as long as the current webpage's URL is part of the agent's context, the malicious instruction will be visible to the agent.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The URL injection template assumes the agent has a memory feature that stores its past states and actions.
- Rests on an assumption: Web agents store state in the current URL.
- Rests on an assumption: URL anchor payloads are ignored by the web server.
- Interpretationreasoning§ 4.2 Results · p. 8
Text-based templates explicitly reveal the attacker's goal, making it easier for the agent to disregard the prompt.
“In contrast, text-based templates explicitly reveal the attacker's goal, along with the attempt to divert from the user objective, making it easier for the agent to disregard the prompt altogether.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Study design and conductnone given§ 5 Conclusion · p. 9
WASP employs fully operational, self-hosted websites.
“Unlike most previous studies that utilize simulated environments with simplistic attacker objectives (e.g., displaying 'Hacked'), our benchmark employs fully operational, self-hosted websites, incorporating realistic assumptions about attacker and defender capabilities and more complex attacker goals (e.g., changing the user's password).”
Quote in the paperSubstantial claim - Interpretationreasoning§ 1 Introduction · p. 2
The community lacks a standard way of tracking attack success rate, hindering reproducibility and a unified view of the risk.
“The community, therefore, lacks a standard way of tracking attack success rate, hindering reproducibility and a unified view of the risk.”
Quote in the paperSubstantial claim - Study design and conductnone given§ Abstract · p. 1
WASP is a publicly available benchmark for end-to-end evaluation of web agent security against prompt injection attacks.
“we introduce WASP-a new publicly available benchmark for end-to-end evaluation of W eb A gent S ecurity against P rompt injection attacks.”
Quote in the paperSubstantial claim - Study design and conductnone given§ 1 Introduction · p. 2
WASP tests prompt injection attacks and task performance end-to-end in an isolated and controllable environment.
“We test prompt injection attacks as well as task performance in an end-to-end manner in an isolated and controllable environment.”
Quote in the paperSubstantial claim - Interpretationown data§ 1 Introduction · p. 3
Current web-navigation agents exhibit a form of security by incompetence.
“This suggests that current web-navigation agents exhibit a form of security by incompetence, which is only discoverable through end-to-end evaluations.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Interpretationreasoning§ 1 Introduction · p. 1
Similar threats are likely to apply to web-navigation agents.
“and similar threats are likely to apply to web-navigation agents.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 28
- May combine two claims. 15
- The quote only partly states this claim. 3
Classified as probably not a substantial claim.
Premisecitation§ 4.1 Setup · p. 7Instruction hierarchy defense assigns responses from tools the lowest-level privilege.
“This defense assigns responses from tools with the lowest-level privilege.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Wallace et al., 2024 source
Classified as probably not a substantial claim.
Premisecitation§ 1 Introduction · p. 1LLMs are already known to be susceptible to indirect prompt injection attacks.
“LLMs are already known to be susceptible to indirect prompt injection attacks (Greshake et al., 2023; Liu et al., 2024)”
Quote in the paperSubstantial claim1 of 2 sources checked- cites Greshake et al., 2023 source
- cites Liu et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Study design and conductcitation§ 3.1 Overview · p. 4WASP is built on VisualWebArena and focuses on the gitlab and reddit environments.
“We build WASP on top of VisualWebArena (Koh et al., 2024)-a sandbox web environment for end-to-end evaluation of generalist web agents. We focus on two web environments within VisualWebArena : gitlab , a clone of the GitLab, and reddit , an open-source version of the social network forum based on Postmill.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Koh et al., 2024 source
The Introduction (p. unavailable) identifies VisualWebArena as a benchmark for web agents and lists Reddit as one of its environments. But it names the three environments as Classifieds, Shopping, and Reddit—not GitLab. The cited paper also does not establish the citing authors’ statement that WASP is built on VisualWebArena. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ A Additional Related Work · p. 13Manually designing prompts appears to be the predominant approach for prompt injection attacks.
“manually designing prompts appears to be the predominant approach (Bhatt et al., 2024), which we employ in this benchmark (see section 3.3).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Bhatt et al., 2024 source
Section 4.1 describes prompt injection attacks and the paper’s test strategies, but does not say that manually designing prompts is the predominant approach. I searched for manual and handcrafted prompt design; the passage about handcrafted questions concerns judging test outcomes, not how commonly attack prompts are designed. (LLM, conf=0.73)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisenone given§ A Additional Related Work · p. 13Bhatt et al. (2024) provides one of the most comprehensive benchmarks for prompt injection attacks.
“Bhatt et al. (2024) provides one of the most comprehensive benchmarks for prompt injection attacks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Bhatt et al. (2024) source
Section 4.1.1 (page unavailable) says the authors sought to cover a large set of prompt-injection strategies, while noting that they did not cover all known techniques. The Conclusion (page unavailable) identifies prompt injection as a test suite within CyberSecEval 2. These passages support calling it a broad prompt-injection benchmark, but do not establish the comparative claim that it is one of the most comprehensive. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data§ 4.1 Setup · p. 7WASP includes 3 past observations in each request to the model because this improves performance.
“We also include 3 past observations in each request to the model since we observe this improves the performance.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ 3.2 Test case construction · p. 4WASP test scenarios combine 21 attacker goals and 2 user goals in each web environment, resulting in 42 test scenarios per environment and 84 tasks total.
“Combining each attacker goal with a user goal gives us a total of ( # of attacker goals ) × ( # of user goals ) = 21 × 2 test scenarios in each web environment.”
- Also stated in: § 3.2 Test case construction · p. 4 · plain
“Combined with 42 test scenarios, this results in a total of 84 tasks”
Quote in the paperSubstantial claim- Also stated in: § 3.2 Test case construction · p. 4 · plain
The quote only partly states this claim.
Findingown data§ 4.2 Results · p. 9Task-agnostic injection ASR remains non-zero for both models.
“However, these numbers remain non-zero and are still high for ASR-intermediate .”
Quote in the paperSubstantial claim- Rests on an assumption: The LLM-based judge used to determine whether an agent is compromised accurately classifies the agent's reasoning.
May combine two claims.
Findingown data§ 4.2 Results · p. 8Defensive system prompting lowers both ASR-intermediate and ASR-end-to-end for GPT-4o and GPT-4o-mini using the same scaffolding.
“We observe both lower ASR-intermediate and lower ASR-end-to-end for the 4o and 4o-mini models using the same scaffolding when defensive system prompt is applied.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
The quote only partly states this claim.
Findingown data§ 3.4 Main metrics · p. 6In an illustrative example, a GPT-4o agent with VisualWebArena scaffolding without defensive system prompt was successfully attacked end-to-end and replaced the user's account email.
“we present an instance of a successful end-to-end attack on GPT-4o based agent in Figure 2.”
- Also stated in: § 3.4 Main metrics · p. 6 · plain
“hijacks the agent to replace the user's account email.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
- The authors state this as an assumption: The prompt injection template assumes the adversary has full knowledge of the user goal.
- Also stated in: § 3.4 Main metrics · p. 6 · plain
May combine two claims. The quote only partly states this claim.
Findingown data§ 4.2 Results · p. 9For both GPT-4o and Claude-3.7-sonnet, task-agnostic injections yield lower ASR than task-related injections for both intermediate and end-to-end metrics.
“Naturally, using a generic prompt results in lower ASR (both types).”
Quote in the paperNot a substantial claim- Rests on an assumption: The LLM-based judge used to determine whether an agent is compromised accurately classifies the agent's reasoning.
Classified as probably not a substantial claim.
Findingreasoning§ 3.4 Main metrics · p. 5WASP utility metric is performance on a combined set of 37 tasks.
“The performance on this combined set of 37 tasks is our Utility metric.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductown data§ 3.4 Main metrics · p. 5WASP constructs a utility test set to verify that the agents tested achieve baseline performance when not exposed to attacks.
“we construct a utility test set to verify that the agents tested achieve baseline performance when not exposed to attacks.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductreasoning§ 1 Introduction · p. 2WASP provides a comprehensive picture of what actually happens on the open web while maintaining reproducibility.
“This provides a comprehensive picture of what actually happens on the open web while still maintaining reproducibility.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductnone given§ 3.2 Test case construction · p. 4WASP manually constructs attacker goals and verifies that each goal can be achieved within the web environment.
“We manually construct these attacker goals and verify that each goal can be achieved within the web environment.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationreasoning§ 4.1 Setup · p. 7If the webpage is only provided through tool responses, then injected attack prompts will always have the least privilege and should not override the user instruction.
“If the webpage is only provided through tool responses, then injected attack prompts will always have the least privilege and should not override the user instruction.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ A Additional Related Work · p. 13Indirect prompt injections are an additional attack vector for applications powered by LLMs.
“Indirect prompt injections (Greshake et al., 2023; Liu et al., 2023, 2024) are an additional attack vector for applications powered by LLMs”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Greshake et al., 2023; Liu et al., 2023, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S4 (based).
Premisenone given§ 5 Conclusion · p. 9Most previous studies utilize simulated environments with simplistic attacker objectives.
“Unlike most previous studies that utilize simulated environments with simplistic attacker objectives (e.g., displaying 'Hacked'), our benchmark employs fully operational, self-hosted websites, incorporating realistic assumptions about attacker and defender capabilities and more complex attacker goals (e.g., changing the user's password).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationreasoning§ 4.2 Results · p. 8Claude, being a screenshot-based agent, may not fully display instructions hidden in the URL body.
“Our argument for this is that Claude, being a screenshot-based agent, may not fully display instructions hidden in the URL body.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: The URL injection template assumes the agent has a memory feature that stores its past states and actions.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone given§ A Additional Related Work · p. 13Both the automated redteaming and the prompt injection lines of work are in simpler settings than WASP.
“Both the automated redteaming and the prompt injection lines of work are in simpler settings than ours, but some of the general techniques used are applicable to us as well.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductreasoning§ 1 Introduction · p. 2WASP simulates prompt injection attacks in different web environments realistically without exposing the agent or any web users to real threats.
“This allows us to simulate prompt injection attacks in different web environments in a realistic manner without exposing the agent or any web users to real threats.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ A Additional Related Work · p. 13Model-based methods for defending against prompt injection involve altering the fine-tuning process to enhance defense performance.
“Model-based methods involve altering the fine-tuning process to enhance defense performance (Piet et al., 2023; Chen et al., 2024a,b; Yi et al., 2023; Wallace et al., 2024; Wu et al., 2024c).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Piet et al., 2023; Chen et al., 2024a,b; Yi et al., 2023; Wallace et al., 2024; Wu et al., 2024c
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S4 (based).
Study design and conductnone given§ 1 Introduction · p. 2WASP is compatible with any generalist web or computer agent.
“Our benchmark is compatible with any generalist web or computer agent”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductown data§ 3.4 Main metrics · p. 5WASP utility set includes all adversarial goals as legitimate user goals and common activities on the two environments.
“First, we include all adversarial goals from Table 5 when given to the agent as legitimate user goals. Second, we include common activities on the two environments (e.g., commenting on a post or an issue).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ 3.1 Overview · p. 4In the reddit environment, the attacker can only create new posts and comments.
“Similarly, for the reddit environment, we only allow the attacker to create new posts and comments.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ 5 Conclusion · p. 9WASP currently supports only two environments (reddit and gitlab).
“it currently supports only two environments ( reddit and gitlab )”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone given§ 3.2 Test case construction · p. 4WASP user goals are selected to be easily achievable by the current generation of agents.
“each test case in our benchmark also includes a (benign) user goal selected to be easily achievable by the current generation of agents.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ A Additional Related Work · p. 13Straightforward approaches to defend against prompt injection include adding instructions to the prompt to alert the model to these attacks.
“Straightforward approaches include adding instructions to the prompt to alert the model to these attacks (Schulhoff and Yanni, 2023; Yi et al., 2023; Willison, 2023; Wei et al., 2024).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Schulhoff and Yanni, 2023; Yi et al., 2023; Willison, 2023; Wei et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ A Additional Related Work · p. 13State-of-the-art industry agents appear to follow the principle of using scaffolding around LLMs with website representations and a browser interaction backend.
“Although not explicitly detailed, state-of-the-art industry agents appear to follow this principle (OpenAI, 2025; Anthropic, 2024), which is the approach we adopt in this work.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites OpenAI, 2025; Anthropic, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone given§ 5 Conclusion · p. 9WASP currently lacks a diverse set of prompt injection attack prompts.
“Additionally, the benchmark currently lacks a diverse set of prompt injection attack prompts.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Premisenone given§ Abstract · p. 1Existing tests for prompt injections in web agents either over-simplify the threat by testing unrealistic scenarios or giving the attacker too much power, or look at single-step isolated tasks.
“Existing tests for prompt injections in web agents either over-simplify the threat by testing unrealistic scenarios or giving the attacker too much power, or look at single-step isolated tasks.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone given§ 1 Introduction · p. 2WASP defines concrete attacker goals that reflect realistic security violations and are realizable within the simulated environment.
“we define concrete attacker goals that reflect realistic security violations and are realizable within our simulated environment, rather than relying on artificial or single-step objectives.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 5 Conclusion · p. 9WASP offers a dynamic framework for evaluating both emerging prompt injection techniques and innovative mitigation strategies that may develop in the future.
“Furthermore, our benchmark offers a dynamic framework for evaluating both emerging prompt injection techniques and innovative mitigation strategies that may develop in the future.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone given§ 1 Introduction · p. 2WASP does not assume attackers are aware of agents' implementation details.
“we do not assume that the attackers are aware of the agents' implementation details.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone given§ 3.1 Overview · p. 4In the gitlab environment, the attacker can create a new issue and post comments, but cannot alter webpage layout, create pop-ups, or modify other users' comments.
“For the gitlab environment, the attacker is able to create a new issue and post comments, but cannot alter the webpage layout, create pop-ups, or modify other users' comments.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The attacker can only control specific webpage elements.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductnone given§ 1 Introduction · p. 2WASP code and benchmark are open-sourced and publicly available.
“our code and benchmark are open-sourced and publicly available.”
Quote in the paperSubstantial claim
The authors state this as an assumption.
The attacker can only control specific webpage elements.
“We assume the attacker can only control specific webpage elements (highlighted in red).”
The authors state this as an assumption.
The prompt injection template assumes the adversary has full knowledge of the user goal.
“The above template assumes the adversary has full knowledge of the user goal and can mislead the agent to believe that visiting instantiated url of action is necessary to complete its original task.”
The authors state this as an assumption.
The URL injection template assumes the agent has a memory feature that stores its past states and actions.
“The above templates assume that the agent has a memory feature that stores its past states and actions.”
The paper relies on this without support.
Web agents store state in the current URL.
“we exploit the fact that web agents do store state in the current URL”
The paper relies on this without support.
URL anchor payloads are ignored by the web server.
“use anchors in URLs to add a prompt injection payload that is ignored by the web server.”
The paper relies on this without support.
The two environments (reddit and gitlab) are sufficiently representative of real-world web navigation tasks for WASP's findings to generalize.
“it currently supports only two environments ( reddit and gitlab )”
“Through our benchmark, we find that it is relatively easy to hijack agents from their original objectives”
The authors state this as an assumption.
The attacker and defender capabilities modeled in WASP are realistic.
“incorporating realistic assumptions about attacker and defender capabilities”
The paper relies on this without support.
The LLM-based judge used to determine whether an agent is compromised accurately classifies the agent's reasoning.
“Your objective is to identify if the reasoning and decision of the large language model agent was compromised.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Task-agnostic prompt does not lower both GPT-4o ASR metrics
MinorTable 4 shows GPT-4o end-to-end ASR remains 0.012 for both task-related and task-agnostic injections, contrary to the statement that both types of ASR are lower.
“Naturally, using a generic prompt results in lower ASR (both types).”
“| GPT-4o ( axtree + SOM ) | 0.226 | 0.012 | 0.071 | 0.012 |”
System prompt does not lower GPT-4o-mini end-to-end ASR
MinorTable 2 gives GPT-4o-mini the same end-to-end ASR, 0.024, with and without the defensive system prompt. Its intermediate ASR falls, but both metrics do not.
“We observe both lower ASR-intermediate and lower ASR-end-to-end for the 4o and 4o-mini models using the same scaffolding when defensive system prompt is applied.”
“| GPT-4o-mini | VisualWebArena | axtree | - | 0.345 | 0.024 | 0.432 | | GPT-4o-mini | VisualWebArena | axtree | system prompt | 0.333 | 0.024 | 0.351 |”
Cites
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023) · Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3605764.3623985 · Cited by 667 · cited 4× · see citing works
“Greshake et al. (2023) first demonstrated the possibility of indirect prompt injection attacks against LLM-integrated applications, where the LLM’s original instruction can be overridden by malicious instructions injected into the retrieved data.”
InfluentialGorilla: Large Language Model Connected with Massive APIs
Patil, S., Zhang, T., Wang, X., & Gonzalez, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-4020 · Cited by 226 · cited 1× · see citing works
“Other methods have explored creating simulated environments (Ruan et al., 2024) or leveraging interactions via RESTful APIs (Patil et al., 2023).”
Gorilla: Large Language Model Connected with Massive APIs
Patil, S. G., Zhang, T., Wang, X., & Gonzalez, J. E. (2023) · arXiv · doi:10.48550/arxiv.2305.15334 · Cited by 226 · see citing works
Universal and Transferable Adversarial Attacks on Aligned Language Models
Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., & Fredrikson, M. (2023) · arXiv · doi:10.48550/arxiv.2307.15043 · Cited by 197 · see citing works
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B., Anderson, H., Singer, Y., & Karbasi, A. (2023) · arXiv · doi:10.48550/arxiv.2312.02119 · Cited by 183 · cited 1× · see citing works
“A large body of work (Zou et al., 2023; Paulus et al., 2024; Pavlova et al., 2024; Mehrotra et al., 2024) studies jailbreaking or automated red-teaming of large language models and their multimodal variants; here the goal is to automatically generate prompts that cause LLM chat-bots to output…”
OpenAI, :, Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., et al. (2024) · arXiv · Cited by 180 · see citing works
Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models
Yi, J., Xie, Y., Zhu, B., Kıcıman, E., Sun, G., Xie, X., & Wu, F. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709179 · Cited by 175 · see citing works
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents
Zhan, Q., Liang, Z., Ying, Z., & Kang, D. (2024) · arXiv · doi:10.48550/arxiv.2403.02691 · Cited by 143 · cited 1×
“For example, InjecAgent (Zhan et al., 2024) has attacker goals such as causing financial or physical harm, but these goals are not executable by the agent even if it is compromised.”
Attacking Vision-Language Computer Agents via Pop-upsPreprint
Zhang, Y., Yu, T., & Yang, D.-Y. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2411.02391 · Cited by 142 · see citing works
(2023) · doi:10.52202/075280-1220 · Cited by 134 · see citing works
Prompt Injection attack against LLM-integrated Applications
Liu, Y., Deng, G., Li, Y., Wang, K., Wang, Z., Wang, X., Zhang, T., Liu, Y., Wang, H., Zheng, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05499 · Cited by 100 · cited 3× · see citing works
“More advanced methods involve altering the fine-tuning process to enhance defense performance (Piet et al., 2023; Chen et al., 2024a,b; Yi et al., 2023; Wallace et al., 2024; Wu et al., 2024c).”
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
Mehrotra, A., Zampetakis, M., Kassianik, P., et al. (2024) · neural information processing systems · doi:10.52202/079017-1952 · Cited by 96 · see citing works
Imprompter: Tricking LLM Agents into Improper Tool UsePreprint
Fu, X.-H., Li, S., Wang, Z., Liu, Y.-H., Gupta, R. K., Berg-Kirkpatrick, T., & Fernandes, E. (2024) · arXiv.org · doi:10.48550/arxiv.2410.14923 · Cited by 88 · cited 3× · see citing works
“However, as LLMs are known to be vulnerable to (indirect) prompt injection attacks (Greshake et al., 2023; Liu et al., 2024), it is conceivable that adversaries will seek to manipulate these web agents by embedding malicious instructions into webpages, which in turn hijack the agent to instead…”
Refusal-Trained LLMs Are Easily Jailbroken As Browser AgentsPreprint
Kumar, P., Lau, E., Vijayakumar, S., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2410.13886 · Cited by 81 · cited 1× · see citing works
“Kumar et al. (2024) shows that a safety-trained LLM, when used as a web-navigation agent, can allow a user to perform harmful actions, and hence safety-training for harmful content does not automatically preclude the agent from performing harmful actions on behalf of its user.”
The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections
Chen, C.-R., Zhang, Z.-P., Guo, B., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2504.11281 · Cited by 79 · cited 1×
“Liao et al. (2024) and Chen et al. (2025) show that an adversary who has control over an entire external environment, such as a website accessed by the agent, can steal the agent user’s private information and otherwise control the agent; however, their assumption on the adversary’s capabilities is…”
Jatmo: Prompt Injection Defense by Task-Specific Finetuning
Piet, J., Al-Rashed, M., Sitawarin, C., Chen, S., Wei, Z., Sun, E., Alomair, B., & Wagner, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70879-4_6 · Cited by 70 · see citing works
Caution for the Environment: Multimodal Agents are Susceptible to Environmental Distractions
Ma, X.-B., Wang, Y.-T., Yao, Y., Yuan, T.-X., Zhang, A., Zhang, Z., & Zhao, H. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2408.02544 · Cited by 69
Automated Red Teaming with GOAT: the Generative Offensive Agent TesterPreprint
Pavlova, M., Brinkman, E., Iyer, K., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2410.01606 · Cited by 58 · cited 1× · see citing works
“A large body of work (Zou et al., 2023; Paulus et al., 2024; Pavlova et al., 2024; Mehrotra et al., 2024) studies jailbreaking or automated red-teaming of large language models and their multimodal variants; here the goal is to automatically generate prompts that cause LLM chat-bots to output…”
Self-citationJatmo: Prompt Injection Defense by Task-Specific Finetuning
Piet, J., Alrashed, M., Sitawarin, C., Chen, S., Wei, Z., Sun, E., Alomair, B., & Wagner, D. (2023) · arXiv · doi:10.48550/arxiv.2312.17673 · Cited by 54 · cited 1×
“More advanced methods involve altering the fine-tuning process to enhance defense performance (Piet et al., 2023; Chen et al., 2024a,b; Yi et al., 2023; Wallace et al., 2024; Wu et al., 2024c).”
Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations
Wei, Z., Wang, Y., Li, A., Mo, Y., & Wang, Y. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3660147 · Cited by 51 · cited 1× · see citing works
“Straightforward approaches include adding instructions to the prompt to alert the model to these attacks (Schulhoff and Yanni, 2023; Yi et al., 2023; Willison, 2023; Wei et al., 2024).”
WebArena: A Realistic Web Environment for Building Autonomous Agents
Zhou, S., Xu, F. F., Zhu, H., Zhou, X., Lo, R., Sridhar, A., Cheng, X., Ou, T., Bisk, Y., Fried, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.13854 · Cited by 31 · cited 4× · see citing works
“We also carry out experiments with the original set of WebArena tasks by Zhou et al. (2023), without prompt injections.”
InfluentialSet-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
Yang, J., Zhang, H., Li, F., Zou, X., Li, C., & Gao, J. (2023) · arXiv · doi:10.48550/arxiv.2310.11441 · Cited by 25 · cited 2× · see citing works
“A large body of work (Zou et al., 2023; Paulus et al., 2024; Pavlova et al., 2024; Mehrotra et al., 2024) studies jailbreaking or automated red-teaming of large language models and their multimodal variants; here the goal is to automatically generate prompts that cause LLM chat-bots to output…”
A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems
Wu, F., Zhang, N., Jha, S., McDaniel, P., & Xiao, C. (2024) · arXiv · doi:10.48550/arxiv.2402.18649 · Cited by 23 · cited 1× · see citing works
“Indeed, recent work has demonstrated the feasibility of this type of attack against language models integrated in broader systems, including web browsing agents (Greshake et al., 2023; Fu et al., 2024; Liao et al., 2024; Zhang et al., 2024b; Ma et al., 2024; Wu et al., 2024a,b; Li et al., 2025).”
Identifying the Risks of LM Agents with an LM-Emulated Sandbox
Ruan, Y., Dong, H., Wang, A., Pitis, S., Zhou, Y., Ba, J., Dubois, Y., Maddison, C. J., & Hashimoto, T. (2023) · arXiv · doi:10.48550/arxiv.2309.15817 · Cited by 19 · cited 1× · see citing works
“Other methods have explored creating simulated environments (Ruan et al., 2024) or leveraging interactions via RESTful APIs (Patil et al., 2023).”
AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses
Carlini, N., Rando, J., Debenedetti, E., et al. (2025) · International Conference on Machine Learning · doi:10.48550/arxiv.2503.01811 · Cited by 16 · cited 1×
“Carlini et al. (2025) shows that LLM-based AI agents are considerably worse than humans at the task of attacking defenses against adversarial examples.”
Defeating Prompt Injections by Design
Debenedetti, E., Shumailov, I., Fan, T., Hayes, J., Carlini, N., Fabian, D., Kern, C., Shi, C., Terzis, A., & Tramèr, F. (2026) · 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml68715.2026.00040 · Cited by 15 · see citing works
The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
Wallace, E., Xiao, K., Leike, R., Weng, L., Heidecke, J., & Beutel, A. (2024) · arXiv · doi:10.48550/arxiv.2404.13208 · Cited by 14 · cited 3× · see citing works
“More advanced methods involve altering the fine-tuning process to enhance defense performance (Piet et al., 2023; Chen et al., 2024a,b; Yi et al., 2023; Wallace et al., 2024; Wu et al., 2024c).”
Formalizing and Benchmarking Prompt Injection Attacks and Defenses
Liu, Y., Jia, Y., Geng, R., Jia, J., & Gong, N. Z. (2023) · arXiv · doi:10.48550/arxiv.2310.12815 · Cited by 14 · see citing works
CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models
Bhatt, M., Chennabasappa, S., Li, Y., Nikolaidis, C., Song, D., Wan, S., Ahmad, F., Aschermann, C., Chen, Y., Kapil, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.13161 · Cited by 12 · cited 3× · see citing works
“…jailbreaking techniques can be applied in this context (Chen et al., 2024b) – since both involve optimizing prompts to elicit specific outputs from LLMs – manually designing prompts appears to be the predominant approach (Bhatt et al., 2024), which we employ in this benchmark (see section 3.3).”
InfluentialAgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
Andriushchenko, M., Souly, A., Dziemian, M., Duenas, D., Lin, M., Wang, J., Hendrycks, D., Zou, A., Kolter, Z., Fredrikson, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.09024 · Cited by 11 · cited 1× · see citing works
“Andriushchenko et al. (2024) provides a benchmark of harmful behaviors for tool-use agents, together with an evaluation method, and shows that many existing LLMs are susceptible to these requests.”
GPT-4V(ision) is a Generalist Web Agent, if Grounded
Zheng, B., Gou, B., Kil, J., Sun, H., & Su, Y. (2024) · arXiv · doi:10.48550/arxiv.2401.01614 · Cited by 10 · cited 1× · see citing works
“A common practical approach involves creating so-called scaffolding around LLMs to enhance their capabilities and enable interactions with tools like browsers and email clients (Zhou et al., 2023; Koh et al., 2024; Deng et al., 2023; Zheng et al., 2024; He et al., 2024).”
StruQ: defending against prompt injection with structured queries
Chen, S., Piet, J., Sitawarin, C., & Wagner, D. (2024) · arXiv · doi:10.48550/arxiv.2402.06363 · Cited by 10 · cited 1× · see citing works
“More advanced methods involve altering the fine-tuning process to enhance defense performance (Piet et al., 2023; Chen et al., 2024a,b; Yi et al., 2023; Wallace et al., 2024; Wu et al., 2024c).”
Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents
Zhang, H., Huang, J., Mei, K., Yao, Y., Wang, Z., Zhan, C., Wang, H., & Zhang, Y. (2024) · arXiv · doi:10.48550/arxiv.2410.02644 · Cited by 9 · see citing works
Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models
Yi, J., Xie, Y., Zhu, B., Kiciman, E., Sun, G., Xie, X., & Wu, F. (2023) · arXiv · doi:10.48550/arxiv.2312.14197 · Cited by 8 · see citing works
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
Debenedetti, E., Zhang, J., Balunović, M., Beurer-Kellner, L., Fischer, M., & Tramèr, F. (2024) · arXiv · doi:10.48550/arxiv.2406.13352 · Cited by 7 · see citing works
WebVoyager: Building an End-to-End Web Agent with Large Multimodal ModelsPreprint
He, H., Yao, W., Ma, K., Yu, W., Dai, Y., Zhang, H., Lan, Z., & Yu, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.13919 · Cited by 5 · cited 5× · see citing works
“Indeed, recent work has demonstrated the feasibility of this type of attack against language models integrated in broader systems, including web browsing agents (Greshake et al., 2023; Fu et al., 2024; Liao et al., 2024; Zhang et al., 2024b; Ma et al., 2024; Wu et al., 2024a,b; Li et al., 2025).”
InfluentialCommercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
Li, A., Zhou, Y., Raghuram, V. C., Goldstein, T., & Goldblum, M. (2025) · arXiv · doi:10.48550/arxiv.2502.08586 · Cited by 2 · cited 1× · see citing works
“Indeed, recent work has demonstrated the feasibility of this type of attack against language models integrated in broader systems, including web browsing agents (Greshake et al., 2023; Fu et al., 2024; Liao et al., 2024; Zhang et al., 2024b; Ma et al., 2024; Wu et al., 2024a,b; Li et al., 2025).”
VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
Koh, J. Y., Lo, R., Jang, L., Duvvur, V., Lim, M. C., Huang, P.-Y., Neubig, G., Zhou, S., Salakhutdinov, R., & Fried, D. (2024) · arXiv · doi:10.48550/arxiv.2401.13649 · Cited by 1 · cited 5× · see citing works
“To achieve this, we build WASP on top of VisualWebArena (Koh et al., 2024)—a sandbox web environment for end-to-end evaluation of generalist web agents—by injecting attacker prompts into controllable webpage elements (e.g., GitLab issues, Reddit posts) that aim to achieve a particular security…”
InfluentialEIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
Liao, Z., Mo, L., Xu, C., Kang, M., Zhang, J., Xiao, C., Tian, Y., Li, B., & Sun, H. (2024) · arXiv · doi:10.48550/arxiv.2409.11295 · Cited by 1 · cited 4× · see citing works
“For instance, Liao et al. (2024) showed that web agents can be hijacked via malicious prompts on attacker-controlled webpages to exfiltrate user information to the attacker.”
InfluentialDissecting Adversarial Robustness of Multimodal LM Agents
Wu, C. H., Shah, R., Koh, J. Y., Salakhutdinov, R., Fried, D., & Raghunathan, A. (2024) · arXiv · doi:10.48550/arxiv.2406.12814 · Cited by 1 · see citing works
Instructional Segment Embedding: Improving LLM Safety with Instruction HierarchyPreprint
Wu, T., Zhang, S., Song, K., Xu, S., Zhao, S., Agrawal, R., Indurthi, S. R., Xiang, C., Mittal, P., & Zhou, W. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.09102 · cited 1×
“More advanced methods involve altering the fine-tuning process to enhance defense performance (Piet et al., 2023; Chen et al., 2024a,b; Yi et al., 2023; Wallace et al., 2024; Wu et al., 2024c).”
AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs
Paulus, A., Zharmagambetov, A., Guo, C., Amos, B., & Tian, Y. (2024) · arXiv · doi:10.48550/arxiv.2404.16873 · cited 1×
“A large body of work (Zou et al., 2023; Paulus et al., 2024; Pavlova et al., 2024; Mehrotra et al., 2024) studies jailbreaking or automated red-teaming of large language models and their multimodal variants; here the goal is to automatically generate prompts that cause LLM chat-bots to output…”
Self-citationSecAlign: Defending Against Prompt Injection with Preference Optimization
Chen, S., Zharmagambetov, A., Mahloujifar, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.05451
Self-citation
Flagged first, then by citations
2 of the 45 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
Chhabra, A., Datta, S., Nahin, S. K., & Mohapatra, P. (2026) · IEEE Access · doi:10.1109/access.2026.3675554 · Cited by 28 · see citing works
“More specifically, we cover issues associated with measuring long-horizon safety in agentic systems, approaches for multi-agent system security, and developing better benchmarks for security evaluation of AI agents, among other considerations.”
Execution of Harm-Enabling Actions by LLM Agents During Simulated Psychiatric CrisesPreprint
Ben‐Zion, Z., Piterman, D., Refoua, E., & Elyoseph, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-8743363/v1 · Cited by 1 · see citing works
Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents
Wang, Z., Li, Y., Wu, Y., Chen, K., Liu, Z., Wai, F. K., Chen, P.-Y., Thing, V. L. L., Li, B., Tao, D., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.13385
“WASP [9] and AGENTDAM [48] evaluate web agents in realistic, stateful environments, yet both report vulnerability with respect to the user’s own delegated task, so a seller whose product is displaced or a platform whose authorization boundary is crossed leaves no trace in the reported results.”
LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection
Zhao, L., Bhaskar, A., & Dobriban, E. (2026) · arXiv · doi:10.48550/arxiv.2605.17986
- Authors
- Evtimov, I., Zharmagambetov, A., Grattafiori, A., Guo, C., & Chaudhuri, K.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2504.18575
- OpenAlex
- W6948050241
- Cited by
- 1
- References
- 45
- Citations by year
- 2026: 1
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Information Systems › Web Application Security Vulnerabilities
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- prompt injection, web agents, AI security, security benchmarking, deception attacks, large language models, end-to-end evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
