CoChat
Sign Up
PreprintCited by 0Topic ModelingMobile Crowdsensing and CrowdsourcingEthics and Social Impacts of AI

Adversarial Review: Structured Disagreement for Grounded Agentic Code Review

Open access
Qiu, Eric S., Gill, Joyce·2026·arXiv
DOI 10.48550/arxiv.2608.18167arXiv 2608.18167
Open this paper in CoChat — free accountOpen PDF

Abstract

Early multi-agent LLM systems often used role-separated teams, yet scaling agent count yields diminishing returns on repository-level coding tasks. Recent alternatives treat agents as passive tools (subagents), yet this removes the benefits of agent interaction entirely. We study whether a subagent paradigm can support a middle ground: minimal agentic cooperation without the overhead of large multi-agent teams. We introduce Adversarial Review (AR), a minimal cooperative code-review protocol in which a main coding agent works with a reviewer and a critic agent. The reviewer evaluates code, while the critic audits the review through structured disagreement before the main agent edits. On LiveCodeBench, AR achieves the highest pass rate among tested methods, outperforming a five-agent baseline while using only three agents. On SWE-PRBench, naive AR exposes a false-consensus failure mode, where agents converge on agreement without sufficient evidence, but a single prompt iteration that adds disagreement explicitly achieves the highest F1 among tested methods. On SWE-bench Verified, AR also shows improvements over the baselines on repository-level coding tasks. Together, AR demonstrates that cooperative code review does not require many agents or complex communication structures: it requires that disagreement be minimal, structured, and evidence-grounded.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2608.18167

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

3 of 7 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured

  • Interpretationreasoning

    This design gives AR some benefits of role-separated cooperation without requiring a large round-table debate like MAD.

    “This design gives AR some benefits of role-separated cooperation without requiring a large round-table debate like MAD (Du et al., 2024;Smit et al., 2023;Wang et al., 2025).”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites Du et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Smit et al., 2023 source

      The Multi-Agent Debate and Experiments sections describe agents with distinct roles and multi-round debate, while the Conclusions say MAD typically requires more API calls. But the paper does not establish that MAD requires a large round-table: its Multi-Persona testbed uses only two agents. It also does not assess AR's benefits. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites Wang et al., 2025 source
  • Findingown data

    On SWE-PRBench, naive AR exposes a false-consensus failure mode, where agents converge on agreement without sufficient evidence.

    “On SWE-PRBench, naive AR exposes a false-consensus failure mode, where agents converge on agreement without sufficient evidence”
    Quote in the paperSubstantial claim
  • Findingown data

    On SWE-bench Verified, AR shows improvements over the baselines on repository-level coding tasks.

    “On SWE-bench Verified, AR also shows improvements over the baselines on repository-level coding tasks.”
    Quote in the paperSubstantial claim
  • Findingown data

    AR uses more tokens than Zero-shot on all three benchmarks.

    “AR uses more tokens than Zero-shot on all three benchmarks.”
    • Also stated in: § 7. Discussion and limitations · plain
      “It uses more tokens than Zero-shot on all three benchmarks.”
    Quote in the paperSubstantial claim
  • Findingown data

    AR is on the cost-quality Pareto frontier of all three benchmarks evaluated.

    “AR is on the cost-quality Pareto frontier of all three benchmarks we evaluate.”
    Quote in the paperSubstantial claim
  • May combine two claims. 35
  • The quote only partly states this claim. 2
  • Classified as probably not a substantial claim. 21

  • May combine two claims.

    Premisecitation

    Scaling the number of peer agents can yield diminishing, and sometimes negative, returns on complex repository-level coding tasks, partly because unconstrained communication introduces coordination overhead and failure modes of its own.

    “recent work suggests that scaling the number of peer agents can yield diminishing, and sometimes negative, returns on complex repository-level coding tasks, partly because unconstrained communication introduces coordination overhead and failure modes of its own (Smit et al., 2023;Kim et al., 2025).”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Smit et al., 2023 source

      The Experiments section reports that additional computing does not guarantee better results and that one debate setup reduced accuracy because a second agent contradicted correct answers. The Conclusions also note higher API and token costs for multi-agent debate. The paper evaluates question-answering and reasoning datasets, however, not repository-level coding tasks, and it does not establish unconstrained communication or coordination overhead as the cause. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites Kim et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    The four methods (Zero-shot, Self-Refine, Single-reviewer, Two-reviewers) cluster at the same level, and naive verification cannot push past this level.

    “This is still within noise of Singlereviewer, Self-Refine, and Zero-shot.The four-methods cluster at the same level, and naive verification cannot push past this level.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    AR with text constraint reaches F1 = 0.533, the highest in the four-method subset on SWE-PRBench.

    “The iterated protocol is AR with text constraint and reaches F1 = 0.533, the highest in the subset.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The GPT-5.2 judge accurately matches the agent's comments against the actual PR feedback in SWE-PRBench.

  • May combine two claims.

    Findingown data

    On Case B task, AR has F1 = 0.286, while MARS catches the same bug at F1 = 0.667.

    “On this task, AR has F1 = 0.286, while MARS catches the same bug at F1 = 0.667.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The GPT-5.2 judge accurately matches the agent's comments against the actual PR feedback in SWE-PRBench.

  • May combine two claims.

    Findingown data

    On LCB, Single-reviewer pass-rate is 77% and pass-on-hard is 36/57.

    “On LCB, pass-rate is 77%.Pass-on-hard is 36/57.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On LiveCodeBench, AR achieves the highest pass rate among tested methods, outperforming a five-agent baseline while using only three agents.

    “On LiveCodeBench, AR achieves the highest pass rate among tested methods, outperforming a five-agent baseline while using only three agents.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In Case A, the judge marks 3 of 5 comments as fabricated because the comments are too thin or too speculative; on this task AR has F1 = 0.250.

    “The judge marks 3 of 5 comments as fabricated, because the comments are too thin or too speculative.The mechanism is that the R-C loop adds findings without filtering them.R hedges often.C confirms most of R's hedges.So the result is too many thin comments, which the judge marks as fabricated.On this task AR has F1 = 0.250.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The GPT-5.2 judge accurately matches the agent's comments against the actual PR feedback in SWE-PRBench.

  • May combine two claims.

    Findingown data

    On LCB, AR has the highest pass-rate of all six methods tested, with pass-rate 87% and pass-on-hard 43/57.

    “On LCB, AR has the highest pass-rate of all six methods we test (pass-rate is 87%.Pass-on-hard is 43/57).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    On LCB, Zero-shot pass-rate is 77% and Self-Refine pass-rate is also 77%.

    “Zero-shot pass-rate is 77%.Self-Refine pass-rate is also 77%.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    AR reaches 75.2% pass-rate on SWE-bench Verified (N=500), Zero-shot reaches 71.6%, and MARS reaches 72.6%.

    “AR reaches 75.2% pass-rate on N=500.Zero-shot on the same Claude Code scaffold reaches 71.6%.MARS reaches 72.6%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On LCB, Two-reviewers pass-rate is 75% and pass-on-hard is 34/57.

    “On LCB, pass-rate is 75%.Pass-on-hard is 34/57.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On LCB, MARS pass-rate is 85% (pass/105) and pass-on-hard is 43/57, which is clearly higher than the four-method cluster.

    “On LCB, MARS is the first method whose pass-rate is clearly higher than Zero-shot, Self-Refine, Single-reviewer, and Two-reviewers.MARS gains +5pp on pass/105 (85%) over the four-method cluster and +3 hard tasks (43/57) over Single-reviewer.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    AR scores +5pp over MARS on LCB while using fewer reviewing roles: 2 (R and C) versus MARS's 4.

    “AR scores +5pp over MARS while using fewer reviewing roles: 2 (R and C) versus MARS's 4 (three reviewers plus one meta-reviewer).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The naive AR protocol gives F1 = 0.457 on SWE-PRBench, the lowest of the four methods in the subset; the other three subset methods all reach F1 around 0.50.

    “The naive AR protocol gives F1 = 0.457 on SWE-PRBench.This is the lowest of the four methods in the subset.The other three subset methods all reach F1 around 0.50.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The GPT-5.2 judge accurately matches the agent's comments against the actual PR feedback in SWE-PRBench.

  • May combine two claims.

    Findingown data

    AR achieves the highest measured score among the methods tested on all three benchmarks, while remaining a compact interactive structure in the main-agent-with-subagents paradigm.

    “AR achieves the highest measured score among the methods we test on all three benchmarks, while remaining a compact interactive structure in the main-agent-withsubagents paradigm.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    MARS requires five agents (one author, three reviewers, and one meta-reviewer) to obtain a smaller gain than AR.

    “MARS requires five agents (one author, three reviewers, and one meta-reviewer) to obtain a smaller gain than AR.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    On LiveCodeBench, the first four methods (Zero-shot, Self-Refine, Single-reviewer, Two-reviewers) cluster at the same pass-rate (75-77%).

    “The first four methods cluster at the same pass-rate.”
    • Also stated in: § Matched-model directive (applies to every Task tool call) This is a matched-model experiment.The main agent and every sub-agent (reviewer, critic, meta-reviewer, executor) MUST run on Sonnet 4.5.When you spawn a sub-agent via the Task tool, pass: · plain
      “| Zero-shot | 77% | 35/57 (61%) | 1 |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Two prompt variants of AR are reported: AR (baseline; F1 = 0.457) and AR with text constraint (F1 = 0.533).

    “Two prompt variants of AR are reported: AR (baseline; F1 = 0.457) and AR with text constraint (F1 = 0.533; Section 4.4).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    AR performs best on LCB but worst among the review-centered methods on SWE-PRBench.

    “AR performs best on LCB but worst among the review-centered methods on SWE-PRBench.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The reviewer-critic loop helps the main agent inspect one frame deeper in the call chain, moving from symptom repair to root-cause repair.

    “We interpret this as a case where the reviewer-critic loop helps the main agent inspect one frame deeper in the call chain, moving from symptom repair to root-cause repair.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Structured disagreement can reliably fix false consensus, improving both code generation and review quality across three benchmarks.

    “we show that structured disagreement can reliably fix false consensus, improving both code generation and review quality across three benchmarks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The protocol pushes R and C to agree, so C tends to agree; R can end the disagreement by writing a confident-sounding rebuttal, even when R is wrong, turning structured disagreement into false agreement.

    “Case B reveals a structural mechanism: the protocol pushes R and C to agree, so C tends to agree.R can end the disagreement by writing a confident-sounding rebuttal, even when R is wrong.What looks like”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Cooperative code review does not require many agents or complex communication structures; it requires that disagreement be minimal, structured, and evidence-grounded.

    “Together, AR demonstrates that cooperative code review does not require many agents or complex communication structures: it requires that disagreement be minimal, structured, and evidence-grounded.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    When two LLM agents are asked to agree on a joint output, they tend to agree with each other and do not always find the truth; therefore the protocol must include explicit pushback.

    “The case studies show that when two LLM agents are asked to agree on a joint output, they tend to agree with each other.They do not always find the truth.So the protocol must include explicit pushback.Without pushback, what looks like”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    On LCB and SWE-PRBench, a Python orchestrator enforces strict-controlled comparisons across methods.

    “On LCB and SWE-PRBench, a Python orchestrator enforces strictcontrolled comparisons across methods.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    SWE-bench Verified measures whether the final patch passes tests, but it does not fully measure maintainability, minimality, or long-term code quality.

    “Similarly, SWE-bench Verified measures whether the final patch passes tests, but it does not fully measure maintainability, minimality, or long-term code quality.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: LCB measures competitive-programming style correctness, SWE-PRBench measures similarity to human pull-request feedback, and SWE-bench Verified measures repository-level repair through hidden tests.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    The AR row of Table 3 uses the baseline AR (two-verdict) protocol; the AR with text constraint upgrade of Section 4.4 is not back-ported to it.

    “The AR row of Table 3 uses the baseline AR (two-verdict) protocol; the AR with text constraint upgrade of Section 4.4 is not back-ported here.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The MARS body used on SWE-bench Verified is the same protocol as in Section 3.4 and Section 4, expressed as an analogous SKILL-style document so the agent can follow it autonomously.

    “The MARS body used here is the same protocol as in Section 3.4 and Section 4, expressed as an analogous SKILL-style document so the agent can follow it autonomously”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    The comparison on SWE-PRBench is restricted to four methods (AR, MARS, Two-reviewers, Single-reviewer), excluding Zero-shot and Self-Refine because they do not apply to a review-only benchmark.

    “We restrict the comparison to four methods: AR, MARS, Two-reviewers, and Single-reviewer.We exclude Zero-shot and Self-Refine, because they do not apply to a review-only benchmark.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    SWE-PRBench may penalize valid comments that differ from the human review or reward comments that match surface form without improving the patch.

    “This makes it useful for comparing review methods, but it may penalize valid comments that differ from the human review or reward comments that match surface form without improving the patch.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: SWE-PRBench depends on an LLM judge that matches generated comments to human review comments.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The within-panel claim in Figure 2 is the within-panel trade-off, not a cross-panel comparison.

    “The within-panel claim is the within-panel trade-off, not a cross-panel comparison.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    The textual constraint forces an agent to discretize its disagreement into exactly one of three types, which makes agent disagreement clear-cut, explicit, and transparent.

    “Our textual constraint forces an agent to discretize its disagreement into exactly one of three types, which makes agent disagreement clear-cut, explicit, and transparent.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Current LLMs are still prone to forgetting or ignoring instructions, especially as the context window approaches full and past full.

  • May combine two claims.

    Interpretationown data

    On LiveCodeBench, AR scores highest with the fewest agents among the methods that broke out of the cluster.

    “AR scores highest with the fewest agents among the methods that broke out.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    AR addresses false consensus by explicitly showing disagreement, which forces evidence-grounded pushback before any flag can be settled.

    “AR addresses this by explicitly showing disagreement, which forces evidence-grounded pushback before any flag can be settled.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The conclusions are based on three empirical probes: constructive progression on LCB, inner-loop review quality on SWE-PRBench, and skills-form repository repair on SWE-bench Verified.

    “Our conclusions are based on three empirical probes: constructive progression on LCB, inner-loop review quality on SWE-PRBench, and skills-form repository repair on SWE-bench Verified.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Study design and conductnone given

    LCB token medians are estimated from per-method call counts, while SWE-PRBench and SWE-bench Verified token medians are computed from per-task logs.

    “LCB token medians are estimated from per-method call counts; SWE-PRBench and SWE-bench Verified token medians are computed from per-task logs.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Only three methods are evaluated on SWE-bench Verified because full benchmark runs estimate over 300 hours for each method.

    “We evaluate only three methods on SWE-bench Verified because full benchmark runs estimate over 300 hours for each method.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The SWE-PRBench metric is F1 over matched comments.

    “The metric is F1 over matched comments.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    These settings are complementary, but they do not cover all coding tasks or all agent scaffolds.

    “These settings are complementary, but they do not cover all coding tasks or all agent scaffolds.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The case studies partly address the gap by examining mechanism, but larger and more comprehensive evaluations would be needed to validate AR as a deployed review assistant.

    “The case studies partly address this gap by examining mechanism, but larger and more comprehensive evaluations would be needed to validate AR as a deployed review assistant.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § Multi-Agent Review System (MARS)

    The structured output schema, in which each role returns labelled fields, lets the Author act on the outputs deterministically.

    “Structured output schema: each role returns labelled fields, so the Author can act on them deterministically.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Every review-having prompt (AR and MARS) has the same three-part shape: a shared SWE-bench wrapper, a method-specific body, and a shared SWE-bench overrides block.

    “Every review-having prompt (AR and MARS) has the same three-part shape:”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The text-only inner loop keeps the artifact stable while reviewers determine what, if anything, is actually wrong.

    “Text-only inner loop keeps the artifact stable while reviewers figure out what (if anything) is actually wrong.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § Multi-Agent Review System (MARS)

    Using the Meta-Reviewer as decision authority prevents simple vote counting and weighs evidence over numbers.

    “Meta-Reviewer as decision authority: prevents simple vote counting; weighs evidence over numbers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § Multi-Agent Review System (MARS)

    Independent review without inter-reviewer communication avoids debate-style overhead while preserving review diversity through independent sampling.

    “Independent review without inter-reviewer communication: avoids debate-style overhead while preserving review diversity through independent sampling.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    First-pass termination guarantees quality: if the reviewer had to think twice or the critic had to push back at all, the artifact is not yet ready.

    “First-pass termination guarantees quality: if the reviewer had to think twice, or the critic had to push back at all, the artifact isn't yet at a”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    The MARS row of Table 3 uses the same MARS protocol as in Section 3.4 and Section 4: three independent reviewers, one meta-reviewer, and two outer rounds.

    “The MARS row uses the same MARS protocol as in Section 3.4 and Section 4: three independent reviewers, one meta-reviewer, two outer rounds.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The prompts are inlined rather than loaded via the agent's native skill mechanism, so that every method's protocol is explicit and reproducible.

    “The prompts are inlined (rather than loaded via the agent's native skill mechanism) so that every method's protocol is explicit and reproducible.”
    Quote in the paperNot a substantial claim
48 of 48

  • The paper relies on this without support.

    The GPT-5.2 judge accurately matches the agent's comments against the actual PR feedback in SWE-PRBench.

    “A GPT-5.2 judge matches the agent's comments against the actual”

  • The paper relies on this without support.

    Production coding agents often follow a mainagent-plus-tools pattern, where subagents are invoked as tools.

    “Production coding agents often follow a mainagent-plus-tools pattern, where subagents are invoked as tools.”

  • The paper relies on this without support.

    Earlier multi-agent coding systems often follow a team pattern, where many role-separated agents generate and aggregate outputs.

    “Earlier multi-agent coding systems often follow a team pattern, where many role-separated agents generate and aggregate outputs.”

  • The paper relies on this without support.

    Current LLMs are still prone to forgetting or ignoring instructions, especially as the context window approaches full and past full.

    “current LLMs are still prone to forgetting or ignoring instructions, especially as the context window approaches full and past full.”

  • The paper relies on this without support.

    SWE-PRBench depends on an LLM judge that matches generated comments to human review comments.

    “SWE-PRBench depends on an LLM judge that matches generated comments to human review comments.”

  • The paper relies on this without support.

    LCB measures competitive-programming style correctness, SWE-PRBench measures similarity to human pull-request feedback, and SWE-bench Verified measures repository-level repair through hidden tests.

    “LCB measures competitive-programming style correctness, SWE-PRBench measures similarity to human pull-request feedback, and SWE-bench Verified measures repository-level repair through hidden tests.”

  • The paper relies on this without support.

    Parallel Task tool calls do not share state, so reviewers spawned in parallel are independent.

    “Sequential dispatch loses the independence guarantee if any state leaks across calls.”
7 of 7

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • MARS overall pass rate differs from Table 1

    Minor

    The text reports an 85% LiveCodeBench pass rate for MARS, while Table 1 reports 82%. This changes the comparison with AR's reported 87% pass rate.

  • MARS hard-task count differs from Table 1

    Minor

    The text gives MARS 43 hard-task passes, but Table 1 gives it 39. The table's count is consistent with the stated gain of three over Single-reviewer's 36.

2 of 2

Cites

Flagged first, then by citations

11 of the 32 references couldn’t be matched to a known paper.

Authors
Qiu, E. S., & Gill, J.
Affiliations
  • Cornell University (US)
  • Stanford University (US)
ORCID
Gill, Joyce
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2608.18167
arXiv
2608.18167
OpenAlex
W7203945251
Cited by
0
References
32
Type
posted content
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
adversarial review, code review, multi-agent LLM systems, agent cooperation, false consensus effect, LiveCodeBench, SWE-bench Verified, prompt engineering

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review