AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability
Open accessAbstract
Scaling adversarial evaluation of large language models requires both a method for generating hard inputs and a reliable way to confirm that resulting failures are real. We present AdversaBench, an end-to-end red-teaming pipeline that mutates seed prompts with five structured operators, queries a target model, and confirms failures through a three-judge panel with a meta-judge tiebreaker. We report experiments on 45 seeds across three categories: reasoning, instruction-following, and tool use. Every seed produced a confirmed failure. Four findings stand out. First, operator effectiveness varies sharply by category: inject_distractor scores 0.00 mean reward on instruction-following seeds but 0.80-0.83 on reasoning and tool-use. Second, binary failure rate hides difficulty: instruction-following seeds required 2.4 attacker iterations on average versus 1.1 for other categories, a gap visible in survival curves. Third, pairwise judge agreement of 80-87% coexists with near-zero Cohen's kappa due to label skew; category-level disagreement rates are more informative. Fourth, adversarial prompts generated against Llama 3.1 8B transfer zero-shot to Llama 3.3 70B, suggesting the mutations exploit general behavioral patterns rather than model-specific weaknesses. Code, dataset, and analysis scripts are available at https://github.com/khanak0509/AdversaBench .
AI snapshot
Written from the paper’s full text.
TL;DR
AdversaBench is an automated LLM red-teaming pipeline that mutates seed prompts, queries a weak target, and confirms failures with a three-judge panel plus meta-judge tiebreaker. Across 45 seeds, every seed produced a confirmed failure, but operator effectiveness, iteration cost, judge disagreement, and cross-model transferability varied sharply by category.
Key findings
- All 45 seeds produced confirmed failures; instruction-following seeds required 2.4 attacker iterations on average versus 1.1 for reasoning and tool-use, with 60% of instruction seeds still unbroken after iteration 1 compared to 10% in other categories.
- Operator effectiveness was category-dependent: `inject_distractor` scored 0.33 mean reward on instruction-following seeds but 1.00 on reasoning and tool-use seeds.
- Pairwise judge agreement was 80–87%, but Cohen’s κ was near zero or negative (−0.05 to −0.11) because failures made up 90–97% of verdicts; category-level disagreement was 33% for instruction-following versus 0% for reasoning.
- A zero-shot transferability test on 15 verified adversarial prompts found that attacks generated against Llama 3.1 8B transferred to Llama 3.3 70B, with constraint_add and inject_distractor mutations showing the highest transfer rate and jailbreak_wrap less consistent.
Methods
- The pipeline uses five structured mutation operators (`rephrase`, `inject_distractor`, `role_flip`, `constraint_add`, `jailbreak_wrap`) with epsilon-greedy selection (ε = 0.2), attacker escalation from Llama 3.3 70B to GPT-4o-mini, and up to five iterations per seed.
- Data: 45 seed prompts, 15 each in reasoning, instruction-following, and tool-use, expanded from an initial pilot of 30; each seed has a base prompt, `expected_behavior`, and `reference_answer`. The target is Llama 3.1 8B Instant on Groq.
- Failures are confirmed by three judges—Llama 3.3 70B, Cerebras GPT-OSS 120B, and Qwen3 32B—with GPT-4o-mini as meta-judge for splits or structured-output errors; reliability statistics were computed post hoc from saved verdicts, including leniency, pairwise agreement, Cohen’s κ, category disagreement, operator ablation, Markov-chain transition analysis, and transferability.
Limitations
- Stated: the study uses 45 seeds with an intentionally weak target, so the 45/45 break rate is by design; the author-constructed `expected_behavior` specifications have not been independently validated by human raters; ε = 0.2 was not ablated; and the transferability result is based on preliminary manual review of 15 prompts rather than systematic annotation.
- Evident from the text: the pipeline is evaluated on a single weak target model, and the paper leaves larger seed sets, stronger targets, independent spec validation, and human annotation of transfer results to future work.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
3 of 4 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured
- Findingown data
Instruction-following took more than twice the attacker effort (mean iterations) compared to other categories in this run.
“Instruction-following took more than twice the attacker effort in this run, a gap that is invisible if you only report whether the model broke.”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
- Findingown data
Reasoning showed 0% panel splits.
“instruction-following showing 33% panel splits versus 0% for reasoning”
Quote in the paperNot a substantial claim - Findingreasoning
Adversarial prompts generated against a weak model transfer to stronger ones.
“adversarial prompts generated against a weak model transfer to stronger ones”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
- Findingown data
60% of instruction seeds were still unbroken after iteration 1.
“a survival curve showing 60% of instruction seeds still unbroken after iteration 1 compared to 10% in other categories”
Quote in the paperSubstantial claim - Findingown data
Reasoning and tool-use needed 1.1 iterations on average.
“Reasoning and tool-use needed 1.1.”
Quote in the paperNot a substantial claim
- Findingown data
Tool-use seeds needed 1.1 attacker iterations on average.
“instruction-following seeds needed 2.4 attacker iterations on average versus 1.1 for reasoning and tool-use”
Quote in the paperSubstantial claim - Findingown data
Failures made up 90-97% of all verdicts.
“because failures make up 90-97% of all verdicts”
Quote in the paperSubstantial claim - Findingown data
rephrase is the most common finisher on instruction-following (4/15).
“rephrase is the most common finisher on instructionfollowing (4/15)”
Quote in the paperSubstantial claim - Findingown data
role_flip achieved 0.615 mean reward (n = 13).
“The workload was carried by constraint_add (0.636, n = 11), role_flip (0.615, n = 13), and inject_distractor (0.562, n = 16), all of which maintained strong success rates despite high selection volume.”
Quote in the paperNot a substantial claim - Findingown data
constraint_add achieved 0.636 mean reward (n = 11).
“The workload was carried by constraint_add (0.636, n = 11), role_flip (0.615, n = 13), and inject_distractor (0.562, n = 16), all of which maintained strong success rates despite high selection volume.”
Quote in the paperNot a substantial claim - Findingown data
Llama 3.3 70B passed only one seed (3% leniency).
“Llama 3.3 70B passed only one seed (3% leniency).”
Quote in the paperNot a substantial claim - Findingown data
An independent audit scored clean-tier rows on a 1-5 quality scale; all received 5/5.
“An independent audit scored clean-tier rows on a 1-5 quality scale; all received 5/5.”
Quote in the paperNot a substantial claim - Findingown data
With marginal fail rates around 90-97%, chance agreement is already high.
“With marginal fail rates around 90-97%, chance agreement is already high.”
Quote in the paperSubstantial claim - Findingown data
Seeds involving ambiguous or conflicting constraints (primarily constraint_add and inject_distractor mutations) showed the highest transfer rate.
“Seeds involving ambiguous or conflicting constraints (primarily constraint_add and inject_distractor mutations) showed the highest transfer rate”
Quote in the paperSubstantial claim - Findingown data
inject_distractor scores 1.00 mean reward on tool-use seeds.
“inject_distractor scores 0.33 mean reward on instruction-following seeds but 1.00 on reasoning and tool-use”
Quote in the paperSubstantial claim- Rests on an assumption: Observed failures are caused by the prompt mutations rather than random model variation.
- Findingown data
Cohen's κ was between -0.05 and -0.11.
“pairwise judge agreement of 80-87% coexists with near-zero Cohen's κ (-0.05 to -0.11)”
Quote in the paperSubstantial claim - Findingown data
Preliminary manual review indicates that a substantial fraction of the logical traps, constraint additions, and distractors that fooled the 8B model successfully transferred to the 70B model.
“Preliminary manual review of transferability_results.json indicates that a substantial fraction of the logical traps, constraint additions, and distractors that fooled the 8B model successfully transferred to the 70B model.”
Quote in the paperSubstantial claim - Findingown data
A single judge would have missed failures that Cerebras or Qwen3 passed.
“A single judge would have missed failures that Cerebras or Qwen3 passed, and would have hidden the fact that a third of instruction seeds split the panel.”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
- Rests on an assumption: The GPT-4o-mini meta-judge correctly resolves judge splits.
- Findingown data
All 45 seeds produced confirmed failures.
“All 45 seeds produced confirmed failures.”
Quote in the paperSubstantial claim - Findingown data
The clean tier (unanimous 3/3 judge consensus) and verified tier (clean + meta-judge arbitration) together cover every seed.
“The clean tier (unanimous 3/3 judge consensus) and verified tier (clean + meta-judge arbitration) together cover every seed.”
Quote in the paperSubstantial claim - Findingown data
Instruction-following showed 33% panel splits.
“instruction-following showing 33% panel splits versus 0% for reasoning”
Quote in the paperSubstantial claim - Findingown data
Instruction-following seeds remain at 60% survival after iteration 1.
“instruction-following seeds remain at 60% survival after iteration 1”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
- Findingown data
Every seed produced a confirmed failure.
“Every seed produced a confirmed failure.”
Quote in the paperSubstantial claim- Rests on an assumption: The three-judge panel with a meta-judge tiebreaker reliably confirms true failures.
- Rests on an assumption: The expected_behavior specifications correctly define the ground-truth correct behavior for each seed.
- Findingnone given
Reasoning failures are clear cut: every row was a unanimous fail.
“Reasoning failures are clear cut: every row was a unanimous fail.”
Quote in the paperSubstantial claim - Findingown data
Instruction-following seeds require up to 5 iterations to fully break.
“require up to 5 iterations to fully break”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
- Findingown data
Reasoning seeds needed 1.1 attacker iterations on average.
“instruction-following seeds needed 2.4 attacker iterations on average versus 1.1 for reasoning and tool-use”
Quote in the paperNot a substantial claim - Findingown data
10% of seeds in other categories were still unbroken after iteration 1.
“a survival curve showing 60% of instruction seeds still unbroken after iteration 1 compared to 10% in other categories”
Quote in the paperSubstantial claim - Findingown data
Instruction-following seeds needed 2.4 iterations on average.
“Instruction-following seeds needed 2.4 iterations on average.”
Quote in the paperNot a substantial claim - Findingown data
A zero-shot transferability test on 15 verified adversarial prompts shows that attacks generated against Llama 3.1 8B transfer to Llama 3.3 70B.
“a zero-shot transferability test on 15 verified adversarial prompts shows that attacks generated against Llama 3.1 8B transfer to Llama 3.3 70B”
Quote in the paperSubstantial claim- Rests on an assumption: Llama 3.1 8B is a weak target model.
- Rests on an assumption: Llama 3.3 70B is substantially stronger than Llama 3.1 8B.
- Rests on an assumption: The 15 verified adversarial prompts used in the transferability test are representative of the generated attacks.
- Findingown data
inject_distractor scores 0.33 mean reward on instruction-following seeds.
“inject_distractor scores 0.33 mean reward on instruction-following seeds but 1.00 on reasoning and tool-use”
Quote in the paperSubstantial claim- Rests on an assumption: Observed failures are caused by the prompt mutations rather than random model variation.
- Findingown data
Both jailbreak_wrap successes were instruction-following seeds.
“both jailbreak_wrap successes were instruction-following seeds.”
Quote in the paperSubstantial claim - Findingown data
A single judge would have hidden the fact that a third of instruction seeds split the panel.
“A single judge would have missed failures that Cerebras or Qwen3 passed, and would have hidden the fact that a third of instruction seeds split the panel.”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
- Rests on an assumption: The GPT-4o-mini meta-judge correctly resolves judge splits.
- Findingown data
Instruction-following seeds needed 2.4 attacker iterations on average.
“instruction-following seeds needed 2.4 attacker iterations on average versus 1.1 for reasoning and tool-use”
Quote in the paperSubstantial claim - Findingown data
Reasoning and tool-use seeds drop to 10% survival after iteration 1.
“Reasoning and tool-use seeds drop to 10% survival after iteration 1”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
- Findingown data
Pairwise judge agreement was 80-87%.
“pairwise judge agreement of 80-87% coexists with near-zero Cohen's κ (-0.05 to -0.11)”
Quote in the paperSubstantial claim - Findingown data
inject_distractor scores 1.00 mean reward on reasoning seeds.
“inject_distractor scores 0.33 mean reward on instruction-following seeds but 1.00 on reasoning and tool-use”
Quote in the paperSubstantial claim- Rests on an assumption: Observed failures are caused by the prompt mutations rather than random model variation.
- Findingown data
Reasoning failures are unanimous across all seeds.
“Reasoning failures are unanimous across all seeds”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
- Rests on an assumption: The GPT-4o-mini meta-judge correctly resolves judge splits.
- Findingown data
Instruction-following produces partial disagreement in 33% of cases.
“instructionfollowing produces partial disagreement in 33% of cases”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
- Rests on an assumption: The GPT-4o-mini meta-judge correctly resolves judge splits.
- Findingown data
inject_distractor achieved 0.562 mean reward (n = 16).
“The workload was carried by constraint_add (0.636, n = 11), role_flip (0.615, n = 13), and inject_distractor (0.562, n = 16), all of which maintained strong success rates despite high selection volume.”
Quote in the paperNot a substantial claim - Findingown data
jailbreak_wrap mutations were less consistent across model scales.
“jailbreak_wrap mutations were less consistent across model scales.”
Quote in the paperNot a substantial claim - Findingown data
Instruction-following is where the panel actually splits.
“Instruction-following is where the panel actually splits.”
Quote in the paperSubstantial claim - Interpretationreasoning
The near-zero κ is not a bug in the code.
“This is not a bug in the code.”
Quote in the paperNot a substantial claim - Interpretationown data
The epsilon-greedy policy discovered a reasonable escalation ordering.
“These patterns suggest that the epsilon-greedy policy discovered a reasonable escalation ordering”
Quote in the paperNot a substantial claim - Interpretationreasoning
Mean iterations-to-failure discriminates when pass/fail rate does not.
“Mean iterations-to-failure does.”
Quote in the paperSubstantial claim - Interpretationown data
The mutations expose general behavioral patterns (difficulty resolving conflicting constraints, susceptibility to misleading context) rather than weaknesses specific to a small model.
“This result suggests that the mutations expose general behavioral patterns -difficulty resolving conflicting constraints, susceptibility to misleading context -rather than weaknesses specific to a small model.”
Quote in the paperSubstantial claim - Interpretationnone given
The 80-87% judge agreement numbers are mostly base-rate agreement.
“They are mostly base-rate agreement.”
Quote in the paperSubstantial claim - Interpretationreasoning
A single-judge pipeline provides no way to detect when a judge is being lenient or strict.
“A single-judge pipeline gives you no way to see when either is happening.”
Quote in the paperSubstantial claim - Interpretationreasoning
The numerator in Equation 1 is therefore close to zero.
“The numerator in Equation 1 is therefore close to zero.”
Quote in the paperSubstantial claim - Interpretationnone given
Cohen's κ can look meaningless under heavy class imbalance even when raw disagreement rates show exactly where judges diverge.
“Cohen's κ can look meaningless under heavy class imbalance even when raw disagreement rates show exactly where judges diverge”
Quote in the paperSubstantial claim - Interpretationown data
Iteration cost reveals difficulty that binary failure rate hides.
“iteration cost reveals difficulty that binary failure rate hides”
Quote in the paperSubstantial claim - Interpretationreasoning
When every seed eventually fails, pass/fail rate stops discriminating.
“When every seed eventually fails, pass/fail rate stops discriminating.”
Quote in the paperSubstantial claim - Interpretationreasoning
Adversarial evaluation with a weak target is an extreme version of the skewed prevalence setting.
“adversarial evaluation with a weak target is an extreme version of that setting.”
Quote in the paperNot a substantial claim- Rests on an assumption: Llama 3.1 8B is a weak target model.
- Interpretationown data
The mutations exploit general behavioral patterns rather than weaknesses specific to a small model.
“suggesting the mutations exploit general behavioral patterns rather than weaknesses specific to a small model.”
Quote in the paperSubstantial claim- Rests on an assumption: Llama 3.1 8B is a weak target model.
- Rests on an assumption: Llama 3.3 70B is substantially stronger than Llama 3.1 8B.
- Rests on an assumption: The 15 verified adversarial prompts used in the transferability test are representative of the generated attacks.
- Interpretationown data
The meta-judge is most active on the hardest category.
“confirming that the meta-judge is most active on the hardest category”
Quote in the paperNot a substantial claim- Rests on an assumption: The GPT-4o-mini meta-judge correctly resolves judge splits.
- Interpretationown data
The near-zero Cohen's κ despite high raw agreement is because failures make up 90-97% of all verdicts.
“near-zero Cohen's κ (-0.05 to -0.11) because failures make up 90-97% of all verdicts”
Quote in the paperSubstantial claim - Premisecitation
The high-agreement, low-κ paradox shows up whenever prevalence is skewed.
“It shows up whenever prevalence is skewed”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Feinstein and Cicchetti [1990] source
- Interpretationown data
High raw agreement can mask near-zero κ when failures dominate the label distribution.
“show that high raw agreement can mask near-zero κ when failures dominate the label distribution.”
Quote in the paperSubstantial claim - Interpretationreasoning
The mutations capture general behavioral patterns.
“suggesting the mutations capture general behavioral patterns”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 13
- May combine two claims. 17
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationZheng et al. [2023] showed that GPT-4 judgments align with human preferences at scale.
“Zheng et al. [2023] introduced MT-Bench and Chatbot Arena, showing that GPT-4 judgments align with human preferences at scale.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Zheng et al., 2023 source
Classified as probably not a substantial claim.
PremisecitationZheng et al. [2023] showed that careful rubric prompting can match human agreement rates above 80%.
“Zheng et al. [2023] showed that careful rubric prompting can match human agreement rates above 80%.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Zheng et al., 2023 source
Section 4.2 (p. 7) reports 85% agreement between GPT-4 and humans on MT-bench when tie votes are excluded, compared with 81% agreement among humans. The paper describes judge prompts and says GPT-4 has a relatively stable internal rubric, but it does not show that careful rubric prompting caused the above-80% agreement. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPerez et al. [2022] showed that one language model can generate adversarial inputs for another at scale.
“Perez et al. [2022] showed that one language model can generate adversarial inputs for another at scale.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Perez et al., 2022 source
May combine two claims.
Findingown dataMarkov-chain analysis of operator transitions shows three high-confidence escalation paths: constraint_add → jailbreak_wrap (100% success), role_flip → role_flip (100% success on repeated application), and inject_distractor → constraint_add (100% success).
“Markov-chain analysis of operator transitions shows three high-confidence escalation paths: constraint_add → jailbreak_wrap (100% success), role_flip → role_flip (100% success on repeated application), and inject_distractor → constraint_add (100% success).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataFor Llama and Cerebras, expected agreement P_e ≈ 0.873, while observed agreement P_o ≈ 0.867.
“For Llama and Cerebras specifically, P e = p 1,fail p 2,fail + p 1,pass p 2,pass ≈ 0.873, while observed agreement is P o ≈ 0.867.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown datarephrase and jailbreak_wrap each achieved 1.00 mean reward but were selected infrequently (n = 4 and n = 2 respectively).
“rephrase and jailbreak_wrap each achieved 1.00 mean reward but were selected infrequently (n = 4 and n = 2 respectively).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOnly 4/10 original instruction seeds broke on the first attempt, compared with 9/10 in the other two categories.
“Only 4/10 original instruction seeds broke on the first attempt, compared with 9/10 in the other two categories.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown datainject_distractor achieves 1.00 mean reward on reasoning and tool-use seeds but 0.33 on instruction-following.
“inject_distractor achieves 1.00 mean reward on reasoning and tool-use seeds but 0.33 on instruction-following”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataSeed instruction-002 took five iterations (inject_distractor, then role_flip, then three constraint_add attempts) before judges unanimously confirmed the break.
“Seed instruction-002 illustrates the stacking behavior: it took five iterations (inject_distractor, then role_flip, then three constraint_add attempts) before judges unanimously confirmed the break.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown datainject_distractor accounts for half of tool-use breaks but achieves a mean reward of 0.33 on instruction-following seeds.
“inject_distractor accounts for half of tool-use breaks but achieves a mean reward of 0.33 on instruction-following seeds (see Figure 2).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataCerebras and Qwen3 each passed three seeds (10% leniency).
“Cerebras and Qwen3 each passed three (10%).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAgreement is high (80-87%), but κ is near zero or slightly negative.
“Agreement is high (80-87%), but κ is near zero or slightly negative.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataReasoning and tool-use lines drop steeply after the first iteration; the instruction-following line decays slowly across all five iterations.
“Reasoning and tool-use lines drop steeply after the first iteration; the instruction-following line decays slowly across all five iterations.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOperator effectiveness varies by task category, so no single operator dominates across all task types.
“Operator selection is category-dependent.”
- Also stated in: § 5. Discussion · boosted
“The heatmap in Figure 2 makes it clear that no single operator dominates across all task types.”
- Also stated in: § 6. Conclusion · boosted
“operator effectiveness varies sharply by task category”
Quote in the paperSubstantial claim- Rests on an assumption: The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
- Also stated in: § 5. Discussion · boosted
May combine two claims.
Findingown dataEvery category eventually broke, so binary failure rate is 15/15 across the board.
“Every category eventually broke, so binary failure rate is 15/15 across the board.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenA single judge cannot tell when it is being lenient and cannot resolve close calls on its own.
“A single judge is fast, but it cannot tell you when it is being lenient, and it cannot resolve close calls on its own.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationnone givenAdversaBench is a reproducible red-teaming methodology with structured mutations, multi-judge confirmation, tiered dataset export, and cross-model transferability testing.
“AdversaBench is a reproducible red-teaming methodology with structured mutations, multi-judge confirmation, tiered dataset export, and cross-model transferability testing.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
InterpretationreasoningThe meta-judge is doing exactly what it was added for: resolving cases where reasonable judges disagree on genuinely ambiguous failures.
“The meta-judge is doing exactly what it was added for: resolving cases where reasonable judges disagree on genuinely ambiguous failures.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductnone givenGround truth is fixed before any judge runs, so verdicts can be checked against an objective standard rather than panel opinion alone.
“Ground truth is fixed before any judge runs, so verdicts can be checked against an objective standard rather than panel opinion alone.”
Quote in the paperSubstantial claim- Rests on an assumption: The expected_behavior specifications correctly define the ground-truth correct behavior for each seed.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThis is a study on 45 seeds with an intentionally weak target, so a 45/45 break rate is by design.
“This is a study on 45 seeds with an intentionally weak target, so a 45/45 break rate is by design.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe transferability result is based on preliminary manual review of 15 prompts rather than systematic annotation.
“The transferability result is based on preliminary manual review of 15 prompts rather than systematic annotation.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone givenFailures are confirmed by a three-judge panel with a meta-judge tiebreaker.
“Failures are confirmed by a three-judge panel with a meta-judge tiebreaker.”
Quote in the paperSubstantial claim- Rests on an assumption: The three-judge panel with a meta-judge tiebreaker reliably confirms true failures.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe ε = 0.2 value was not ablated in this study.
“The ε = 0.2 value was not ablated in this study”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductnone givenThe ground-truth expected_behavior specifications are author-constructed and have not been independently validated by human raters; verdicts are only as reliable as the specifications themselves.
“The ground-truth expected_behavior specifications are author-constructed and have not been independently validated by human raters; verdicts are only as reliable as the specifications themselves.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductreasoningThe expected_behavior specifications were written to be unambiguous enough for a judge to verify without subjective interpretation.
“The expected_behavior specifications were written to be unambiguous enough for a judge to verify without subjective interpretation”
Quote in the paperSubstantial claim- Rests on an assumption: The expected_behavior specifications correctly define the ground-truth correct behavior for each seed.
May combine two claims.
Interpretationown dataThe best operator depends strongly on the task type, so aggregate operator counts are misleading.
“This categoryoperator interaction is the main argument against reporting aggregate operator counts: the aggregate hides the fact that the best operator depends strongly on the task type.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe ε = 0.2 selection parameter was not ablated.
“The ε = 0.2 selection parameter was not ablated.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationown dataA pipeline that selects operators uniformly or based only on aggregate reward is leaving efficiency on the table.
“A pipeline that selects operators uniformly or based only on aggregate reward is leaving efficiency on the table.”
Quote in the paperSubstantial claim
The paper relies on this without support.
Structured mutations make it easier to attribute why a prompt broke the target.
“because operators make it easier to attribute why a prompt broke the target.”
The paper relies on this without support.
The three-judge panel with a meta-judge tiebreaker reliably confirms true failures.
“Failures are confirmed by a three-judge panel with a meta-judge tiebreaker.”
The paper relies on this without support.
Llama 3.1 8B is a weak target model.
“queries a weak target model”
The paper relies on this without support.
Llama 3.3 70B is substantially stronger than Llama 3.1 8B.
“substantially stronger 70B model”
The paper relies on this without support.
The 15 verified adversarial prompts used in the transferability test are representative of the generated attacks.
“a zero-shot transferability test on 15 verified adversarial prompts”
The paper relies on this without support.
The expected_behavior specifications correctly define the ground-truth correct behavior for each seed.
“Ground truth is fixed before any judge runs”
The paper relies on this without support.
The meta-judge resolves judge disagreements correctly.
“disagreements go to a meta-judge”
The paper relies on this without support.
Observed failures are caused by the prompt mutations rather than random model variation.
“operator effectiveness varies sharply by category”
The paper relies on this without support.
Human pairwise evaluation does not scale to thousands of synthetic examples per day.
“Human pairwise evaluation remains the reference standard, but it does not scale to thousands of synthetic examples per day.”
The paper relies on this without support.
A weak target keeps failure prevalence high, which stress-tests the judge metrics.
“A weak target also keeps failure prevalence high, which stress-tests the judge metrics in instructive ways.”
The paper relies on this without support.
The author-constructed expected_behavior specifications are valid ground truth for evaluating model behavior.
“The ground-truth expected_behavior specifications are author-constructed and have not been independently validated by human raters; verdicts are only as reliable as the specifications themselves.”
The paper relies on this without support.
The GPT-4o-mini meta-judge correctly resolves judge splits.
“splits are resolved by a GPT-4o-mini meta-judge.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Final-operator table accounts for 30 of 45 seeds
MajorThe operator counts sum to 10 in each category, or 30 overall, although the table says it covers 45 seeds. Fifteen confirmed breaks have no final operator in the table.
Reported 45 · recomputed 30
“Table: 1 Table 1 : Final operator by task category (45 seeds). | Category | inj_dist role constr reph jail | | | | | | --- | --- | --- | --- | --- | --- | | Reasoning | 4 | 3 | 3 | 0 | 0 | | Instruction-following | 0 | 2 | 2 | 4 | 2 | | Tool-use | 5 | 3 | 2 | 0 | 0 |”
Pairwise agreement is incompatible with the reported vote totals
MajorWith only one Llama pass and three Cerebras or Qwen3 passes among 45 seeds, either Llama pair must agree on at least 41/45 seeds (91%), not 87%. With three passes each, Cerebras and Qwen3 must agree on at least 39/45 (87%), not 80%; the reported κ values consequently cannot describe those same 45 verdicts.
“| Llama 70B × Cerebras 120B | 87% | -0.053 | weak | | Llama 70B × Qwen3 32B | 87% | -0.053 | weak | | Cerebras 120B × Qwen3 32B | 80% | -0.111 | weak |”
“Table: 3 Table 3 : Judge leniency across 45 seeds. | Judge | Pass Fail L j | | | | --- | --- | --- | --- | | Llama 3.3 70B | 1 | 44 | 3% | | Cerebras GPT-OSS 120B | 3 | 42 10% | | | Qwen3 32B | 3 | 42 10% | |”
Stated chance agreement does not match the 45-seed marginals
MajorThe stated formula applied to Llama's 44 fail/1 pass and Cerebras's 42 fail/3 pass gives (44×42 + 1×3)/452 ≈ 0.914, not 0.873. The quoted observed agreement of 0.867 is also incompatible with those vote totals.
“For Llama and Cerebras specifically, P e = p 1,fail p 2,fail + p 1,pass p 2,pass ≈ 0.873, while observed agreement is P o ≈ 0.867.”
“| Llama 3.3 70B | 1 | 44 | 3% | | Cerebras GPT-OSS 120B | 3 | 42 10% | |”
Llama leniency uses the wrong denominator
MinorOne pass among 45 verdicts is 2.22%, not the reported 3%.
Reported 3 · recomputed 2.222222
“Llama 3.3 70B passed only one seed (3% leniency).”
Cerebras and Qwen3 leniency percentages are too high
MinorEach judge passed three of 45 seeds, which is 6.67%, not the reported 10%.
Reported 10 · recomputed 6.666667
“Cerebras and Qwen3 each passed three (10%).”
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · see citing works
High agreement but low Kappa: I. the problems of two paradoxes
Feinstein, A. R., & Cicchetti, D. V. (1990) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(90)90158-l · Cited by 3,034 · cited 1× · see citing works
“Feinstein and Cicchetti [1990] called this the high-agreement, low-κ paradox.”
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · cited 4× · see citing works
“Strong models can stand in for human annotators when the task is preference ranking, and Zheng et al. [2023] showed that careful rubric prompting can match human agreement rates above 80%.”
InfluentialRed Teaming Language Models with Language ModelsPreprint
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2202.03286 · Cited by 1,430 · see citing works
A StrongREJECT for Empty Jailbreaks
Souly, A., Lu, Q.-Y., Bowen, D., et al. (2024) · Neural Information Processing Systems · doi:10.48550/arxiv.2402.10260 · Cited by 455 · cited 1×
“Recent jailbreak and safety benchmarks such as HarmBench [Mazeika et al., 2024 JailbreakBench [Chao et al., 2024], and StrongREJECT [Souly et al., 2024] standardize attack suites and automated grading at larger scale; our work complements these efforts with a focused study on multi-judge…”
Red Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363 · cited 1× · see citing works
“Perez et al. [2022] showed that one language model can generate adversarial inputs for another at scale.”
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
Chao, P., Debenedetti, E., Robey, A., Andriushchenko, M., Croce, F., Sehwag, V., Dobriban, E., Flammarion, N., Pappas, G. J., Tramer, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.01318 · Cited by 116 · cited 1× · see citing works
“Recent jailbreak and safety benchmarks such as HarmBench [Mazeika et al., 2024 JailbreakBench [Chao et al., 2024], and StrongREJECT [Souly et al., 2024] standardize attack suites and automated grading at larger scale; our work complements these efforts with a focused study on multi-judge…”
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Mazeika, M., Phan, L., Yin, X., Zou, A., Wang, Z., Mu, N., Sakhaee, E., Li, N., Basart, S., Li, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2402.04249 · Cited by 28 · cited 1×
“Recent jailbreak and safety benchmarks such as HarmBench [Mazeika et al., 2024 JailbreakBench [Chao et al., 2024], and StrongREJECT [Souly et al., 2024] standardize attack suites and automated grading at larger scale; our work complements these efforts with a focused study on multi-judge…”
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
Debenedetti, E., Zhang, J., Balunović, M., Beurer-Kellner, L., Fischer, M., & Tramèr, F. (2024) · arXiv · doi:10.48550/arxiv.2406.13352 · Cited by 7 · cited 1× · see citing works
“AgentDojo [Debenedetti et al., 2024] evaluates prompt-injection attacks in tool-using agents.”
Flagged first, then by citations
1 of the 10 references couldn’t be matched to a known paper.
- Authors
- Khandelwal, K.
- Affiliations
- Indian Institute of Technology Jodhpur (IN)
- Corresponding author
- Khandelwal, Khanak
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2606.24589
- OpenAlex
- W7165802700
- Cited by
- 0
- References
- 10
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- large language models, adversarial evaluation, red teaming, cross-model transfer, zero-shot transfer, instruction following, tool use, reasoning, label skew, Cohen's kappa, adversarial prompts
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
