CoChat
Sign Up
PreprintCited by 36Topic ModelingNatural Language Processing TechniquesAdversarial Robustness in Machine Learning

Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models

Open access
Verga, Pat, Hofstatter, Sebastian, Althammer, Sophia, Su, Yixuan, Piktus, Aleksandra, Arkhangorodsky, Arkady, Xu, Minjie, White, Naomi +1 more·2024·arXiv
DOI 10.48550/arxiv.2404.18796arXiv 2404.18796
Open this paper in CoChat — free accountOpen PDF

Abstract

As Large Language Models (LLMs) have become more advanced, they have outpaced our abilities to accurately evaluate their quality. Not only is finding data to adequately probe particular model properties difficult, but evaluating the correctness of a model's freeform generation alone is a challenge. To address this, many evaluations now rely on using LLMs themselves as judges to score the quality of outputs from other LLMs. Evaluations most commonly use a single large model like GPT4. While this method has grown in popularity, it is costly, has been shown to introduce intramodel bias, and in this work, we find that very large models are often unnecessary. We propose instead to evaluate models using a Panel of LLm evaluators (PoLL). Across three distinct judge settings and spanning six different datasets, we find that using a PoLL composed of a larger number of smaller models outperforms a single large judge, exhibits less intra-model bias due to its composition of disjoint model families, and does so while being over seven times less expensive.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes evaluating LLM generations with a Panel of LLM evaluators (PoLL) composed of smaller models from different families instead of a single large judge such as GPT-4. Across three judge settings and six datasets, PoLL correlates better with human judgments and is over seven times cheaper than a single GPT-4 judge.

Key findings

  • On single-hop QA datasets from KILT, PoLL achieves the strongest Cohen’s κ correlation with human judgments: NQ 0.763, TQA 0.906, HPQA 0.867, compared with GPT-4 at 0.627, 0.841, and 0.830 respectively (Table 1).
  • On Chatbot Arena Hard, PoLL has the best correlation with the human ELO ranking: Pearson 0.917 and Kendall Tau 0.778, versus GPT-4 at 0.817 and 0.667 (Table 2).
  • GPT-4 is a relatively weak and prompt-sensitive judge on KILT NQ: κ is 0.518 zero-shot, 0.627 with the few-shot standard prompt, and 0.725 with a “don’t overthink” instruction, still below Command-R (0.734), Haiku (0.749), and PoLL (0.763) (Tables 1 and 3).
  • PoLL shows the smallest spread in multi-hop judge accuracy deltas, with a standard deviation of 2.2, while GPT-3.5 has the highest spread at 6.1; individual models tend to receive their highest positive delta when judged by themselves. PoLL costs $1.25/input + $4.25/output versus GPT-4 Turbo at $10/input + $30/output, making it seven to eight times less expensive.

Methods

  • The authors evaluate three judge settings—single-hop QA, multi-hop QA, and Chatbot Arena Hard—spanning six datasets: KILT Natural Questions, TriviaQA, HotpotQA (single-hop), Bamboogle, HotpotQA (multi-hop), and Chatbot Arena Hard.
  • The PoLL instantiation uses three models from disparate families: Command R, Haiku, and GPT-3.5. QA judgments use max voting over binary correctness, while Chatbot Arena uses average pooling over 1–5 scores.
  • Human reference judgments come from Cohere’s internal annotation workforce. QA uses reference-based scoring with few-shot prompts; Chatbot Arena Hard follows the original pair-wise scoring setup. Main analyses report Cohen’s κ, Pearson correlation, and Kendall Tau against human judgments or Chatbot Arena ELO rankings.

Limitations

  • The paper states it investigated only three evaluator settings and a limited number of judges and panel compositions; further work is needed to test broader applicability, such as in math or reasoning evaluations, and panel selection is left to future work.
  • The authors did not run explicit latency evaluations, noting that model choice, serving platform, and other factors can impact speed.
  • Evident from the text: the PoLL results are tied to one specific three-model panel; human annotations come from a single internal workforce; and Chatbot Arena Hard initially contained generated answers only from GPT-3.5 and GPT-4, with new outputs generated for the remaining models.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2404.18796

Cites

0 citation errors · 4 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

3 of 13 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Many evaluations rely on using LLMs themselves as judges to score the quality of outputs from other LLMs.

    “many evaluations now rely on using LLMs themselves as judges to score the quality of outputs from other LLMs.”
    • Also stated in: § 1 Introduction · plain
      “More recent methods have attempted to address these issues by instead using trained or prompted models as evaluators (Sellam et al., 2020;Zheng et al., 2024;Li et al., 2024b;Kocmi and Federmann, 2023a;Shen et al., 2023;Li et al., 2023).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Sellam et al., 2020 source

      The Introduction (page not available in our copy) says NLG researchers commonly use automatic metrics and increasingly incorporate learned components. Section 3 describes a BERT-based metric that predicts quality ratings. These passages support model-based scoring of generated text, but not the more specific claim that many evaluations use LLMs as judges of other LLMs' outputs. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites Zheng et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Li et al., 2024b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Premisecitation

    Prior work has shown that model-based scoring methods often correlate better with human judgements than heuristic metrics like EM.

    “Prior work has shown that model-based scoring methods often correlate better with human judgements than heuristic metrics like EM (Bohnet et al., 2022;Zheng et al., 2024)”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Bohnet et al., 2022 source
    • cites Zheng et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Findingown data

    Using an instantiation of PoLL correlates better with human judgements compared to a single large judge (GPT-4).

    “We show that using an instantiation of PoLL correlates better with human judgements compared to a single large judge (GPT-4), while being over seven times cheaper (Sections 4.1 and 4.2).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Chatbot Arena crowdsourced annotations are ground truth for calculating correlation between evaluator models and human judgements.
    • The authors state this as an assumption: The crowd-sourced ELO rankings from Chatbot Arena are ground truth for model rankings.
  • Findingown data

    In some scenarios, GPT-4 is a relatively weak judge, exhibiting high variance with minor changes to the prompt.

    “In some scenarios, GPT-4 is a relatively weak judge, exhibiting high variance with minor changes to the prompt (Section 4.3).”
    Quote in the paperSubstantial claim
  • Findingown data

    A Panel of LLM Evaluators composed of smaller models is an effective method for evaluating LLM performance.

    “we showed how a Panel of LLM Evaluators composed of smaller models is not only an effective method for evaluating LLM performance, but also reduces intra-model bias, latency, and cost.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 13
  • May combine two claims. 12

  • Classified as probably not a substantial claim.

    Premisecitation

    Strong evaluator models generalize well across different tasks.

    “and that strong evaluator models generalize well across different tasks (Huang et al., 2024).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Huang et al., 2024 source

      Section 2.1 reports that proprietary evaluators such as GPT-3.5 and GPT-4 perform consistently well across evaluation schemes. It also finds that fine-tuned judges perform best on their training schemes and drop substantially on unfamiliar ones. The claim is therefore too broad: the paper does not say that strong evaluator models generally transfer well across tasks. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Max voting with average voting fallback yielded similar results with an identical overall ranking.

    “We also experimented with max voting with average voting fallback which yielded similar results with an identical overall ranking.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    EM unjustly penalized Command models for a tendency to write in Canadian or British English because QA dataset annotations typically format dates in American MM-DD-YYYY format.

    “We found that EM unjustly penalized Command models for a tendency to write in Canadian or British English as QA dataset annotations typically format dates in American MM-DD-YYYY format.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    These prompt changes bring GPT-4 agreement up to the level of GPT-3.5 with few-shot standard prompt, though still below Command-R, Haiku, and PoLL.

    “These changes bring the agreement level for GPT-4 up to the level of GPT-3.5 when using our few-shot standard prompt, though still below Command-R, Haiku, and PoLL.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    There is not a single 'best' judge across all settings, while PoLL performs well consistently.

    “The benefits of PoLL are bolstered by the finding that there is not a single 'best' judge across all settings, while PoLL performs well consistently.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    The 'don't overthink' prompt had a negligible positive effect on Haiku, a minor negative effect on Command R, and a severe negative impact on GPT-3.5.

    “the 'don't overthink' prompt had a negligible positive effect on Haiku, minor negative effect to Command R and a severe negative impact on GPT-3.5.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In Table 1, PoLL has the strongest correlation across various tasks, while GPT-4 is one of the weaker evaluators on this particular task setup.

    “We see that overall, PoLL has the strongest correlation across various tasks, while GPT-4 is one of the weaker evaluators on this particular task setup (See 4.3 for further analysis).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Experiments on triple annotated datasets were based on a pooled super-annotator decision based on majority voting.

    “Experiments on triple annotated datasets were based on a pooled super-annotator decision based on majority voting.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Li et al. (2023) proposed an evaluation method with multiple judges to reduce bias though only looked at large models on pair-wise evaluations.

    “In prior relevant work, Li et al. (2023) proposed an evaluation method with multiple judges to reduce bias though only looked at large models on pair-wise evaluations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Li et al., 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Running a collection of smaller models in parallel (as in PoLL) is faster than a single big model.

    “In general though, running a collection of smaller models in parallel (as in PoLL) is faster than a single big model.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Similar pooling techniques are used to reduce variance in human annotations by normalizing out natural variation and human error.

    “Similar pooling techniques are used to reduce variance in human annotations by normalizing out both natural variation in human judgements caused by their own subjective biases as well as human error (Voorhees, 1998).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Voorhees, 1998 source

  • Classified as probably not a substantial claim.

    Premisecitation

    Li et al. (2024a) determined that this evaluation correlated better with human judgements from original Chatbot Arena annotations when compared against other commonly used automated evaluation benchmarks.

    “In their analysis, they determined that this evaluation correlated better with the human judgements from the original Chatbot Arena annotations when compared against other commonly used automated evaluation benchmarks.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Li et al., 2024a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The study investigated only three evaluator settings and a limited number of judges and panel compositions.

    “In this work we investigated only three evaluator settings and a limited number of judges and panel compositions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Multiple-choice questions probe a different property than free-form generative tasks, which are oftentimes closer to the downstream use-case.

    “However, multiple-choice questions are in many ways probing a different property than that of a free-form generative task, which is oftentimes closer to the downstream use-case.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given

    QA metrics invariably lead to both false positive failures and more commonly false negatives due to an incomplete set of gold reference answers.

    “QA metrics, for example, invariably lead to both false positive failures (e.g.superfluous token overlap) and more commonly false negatives due to an incomplete set of gold reference answers (e.g.date format differencesfoot_0 , inclusion of middle initial in person's name, etc.).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Single-hop TriviaQA, Bamboogle, and multi-hop HotPotQA were singly annotated, while Natural Questions and single-hop HotPotQA were triple annotated.

    “Single-hop TriviaQA, Bamboogle, and multi-hop HotPotQA were singly annotated while Natural Questions and single-hop HotPotQA are triple annotated.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning

    The EM baseline is not LLM-dependent and therefore not dependent on prompt.

    “note that the EM baseline is not LLM-dependent and therefore not dependent on prompt”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Annotations were performed by professional annotators with diverse demographic and professional backgrounds including novelists, copywriters, copy editors, and journalists.

    “Annotations were performed by profession annotators with diverse demographic and professional backgrounds including novelists, copywriters, copy editors, and journalists.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Li et al. (2024a) mined a 'hard' subset of 500 arena prompts and created an automatic evaluation where an LLM judge (GPT-4) would score a model output against a baseline generation.

    “Recently, Li et al. (2024a) mined a 'hard' subset of 500 arena prompts and created an automatic evaluation where an LLM judge (GPT-4) would score a model output against a baseline generation (coming from a different version of GPT-4) in a head-to-head comparison.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Li et al., 2024a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

19 of 19

  • The paper relies on this without support.

    Evaluations most commonly use a single large model like GPT-4.

    “Evaluations most commonly use a single large model like GPT-4.”
    “it is most common to use the largest, most universally capable models as evaluators”

  • The authors state this as an assumption.

    Chatbot Arena crowdsourced annotations are ground truth for calculating correlation between evaluator models and human judgements.

    “Following the original work, we treat Chatbot Arena crowdsourced annotations as ground truth for calculating correlation between evaluator models and human judgements.”

  • The paper relies on this without support.

    Using the largest, most universally capable models as evaluators limits applicability and access.

    “which is both slow and costly, limiting applicability and access.”

  • The authors state this as an assumption.

    The crowd-sourced ELO rankings from Chatbot Arena are ground truth for model rankings.

    “Following Li et al. (2024a), we treat the crowd-sourced ELO rankings from Chatbot Arena as ground truth.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • This method demonstrates high agreement with expert evaluations in assessing answer quality, achieving a Pearson correlation coefficient of 0.917, compared to 0.817 for GPT-4.

    “This method demonstrates high agreement with expert evaluations in assessing answer quality, achieving a Pearson correlation coefficient of 0.917, compared to 0.817 for GPT-4 [38].”

    From Evaluating Medical Text Summaries Using Automatic Evaluation Metrics and LLM-as-a-Judge Approach: A Pilot Study (2025) · cites this paper as [38]

    Table 2 reports the stated Pearson correlations: 0.917 for PoLL and 0.817 for GPT-4. But Section 4.2 says these measure agreement between model rankings and the crowd-sourced Chatbot Arena leaderboard, not agreement with expert evaluations of individual answers. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • In scenarios where human annotations are unavailable, assembling a diverse panel of LLM-as-a-judge judges drawn from multiple model families minimizes systematic biases.

    “In scenarios where human annotations are entirely unavailable, assembling a diverse panel of LLM-as-a-judge judges [Verga et al., 2024, Badshah andSajjad, 2024, Li et al., b] drawn from multiple model families (such as GPT, Claude, and Llama) further minimizes systematic biases, ensuring fair, consistent, and robust evaluations across different models and evaluation tasks over time.”

    From Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judge (2025) · cites this paper as Verga et al., 2024

    The Introduction and Section 4.4 report that pooling judges from different model families reduces intra-model scoring bias. But Section 3.7 says the study used human reference annotations, so it does not establish the result specifically when annotations are entirely unavailable. Section 3.2 lists GPT-3.5, Claude Haiku, and Command R in the panel—not Llama—and the Conclusions limit the demonstrated consistency to the settings studied, rather than across tasks over time. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Empirical evidence shows that parallel generation with small LLMs can match or even outperform single large LLMs.

    “Empirical evidence shows that parallel generation with small LLMs can match or even outperform single large LLMs (Verga et al., 2024; Wang et al., 2025a).”

    From A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems (2025) · cites this paper as Verga et al., 2024

    The Abstract and Introduction report that a panel of smaller LLMs outperformed a single large model as judges of other models’ outputs. The claim extends that result to parallel generation by small LLMs; the paper tests evaluation, not whether those models generate equally good or better answers. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Existing aggregation methods in NLP either assign equal weight to all LLM judgments or are designed for specific tasks such as hallucination detection.

    “Existing methods in NLP either assign equal weight to all LLM judgments or are designed for specific tasks such as hallucination detection.”

    From SkillAggregation: Reference-free LLM-Dependent Aggregation (2025) · cites this paper as Verga et al., 2024

    Section 3.1 describes the paper’s own panel aggregating judges’ scores by max voting or average pooling, consistent with treating their judgments equally. Section 2.2 notes that prior multi-judge work was limited to pair-wise evaluations, but the paper does not establish that existing NLP methods generally fall into the two stated categories or identify hallucination detection as one of them. Our copy provides no page numbers for these passages. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

  • Assigning equal weight to all judges can be suboptimal because skill levels may vary across judges and tasks.

    “First, assigning equal weight to all judges, as done in Verga et al. (2024); Badshah and Sajjad (2024), can be suboptimal because skill levels may vary across judges and tasks.”

    From SkillAggregation: Reference-free LLM-Dependent Aggregation (2025) · cites this paper as Verga et al., 2024

    Section 3.1 describes average pooling for Chatbot Arena and max voting for QA, with no judge-specific weights. The Conclusions say there is no single best judge across settings. The paper does not assess whether equal weighting is suboptimal because judges’ skill levels vary, so that conclusion is not supported by this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • GPT-3.5 standard-prompt kappa differs between tables

    Minor

    Table 1 reports 0.726 for GPT-3.5 on Natural Questions, while Table 4 reports 0.729 for the same model and standard prompt.

1 of 1

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 1

  1. Karl Pearson. 1895. Vii. note on regression and inheri- tance in the case of two parents. proceedings of the royal society of London, 58(347-352):240-242.

Cited by

Sorted by citations

Related

Authors
Verga, P., Hofstatter, S., Althammer, S., Su, Y., Piktus, A., Arkhangorodsky, A., Xu, M., White, N., & Lewis, P.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2404.18796
arXiv
2404.18796
OpenAlex
W4396570275
Semantic Scholar
1a9d2a8093d2886b27e5854c1f20d873f02edfb0
Cited by
16
References
0
Citations by year
2026: 8 · 2025: 6 · 2024: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
LLM evaluation, LLM-as-a-judge, evaluation costs, LLM benchmarking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review