CoChat
Sign Up
PreprintCited by 0Artificial Intelligence ApplicationsResearch, Science, and AcademiaAI and HR Technologies

Scoring Grant Applications with Large Language Models

Open access
Thelwall, Mike, Pride, David, Osborne, Francesco, Salatino, Angelo, Knoth, Petr, Iyawa, Gloria, Yusuf, Oluwaremilekun·2026·arXiv
DOI 10.48550/arxiv.2609.25327arXiv 2609.25327
Open this paper in CoChat — free accountOpen PDF

Abstract

Purpose: Assessing grant applications is time-consuming and difficult, adding to the overall burden of academic peer review. Whilst funders are exploring whether AI can help, there is no published research into the accuracy of Large Language Models (LLMs) for scoring contemporary grants. Design/methodology/approach: This study investigates whether six open-weight LLMs (Gemma 3 1B/4B/12B/27B, DeepSeek R1 32B, Qwen 3 32B) can give useful scores for 2267 recent UK Economic and Social Research Council (ESRC), and Engineering and Physical Sciences Research Council (EPSRC) grant applications, comparing them with scores from the original reviewers and funding panel members. Findings: Although the LLM scores are individually inaccurate, when averaged and converted to ranks they correlate positively with expert average scores. The best performing LLM, Gemma 3 27B (10 iterations with varied prompts), had moderate rank correlations with average reviewer scores (mean rho=0.26). Gemma 3 27B's average correlation with individual reviewers was 0.19, which is lower than the inter-reviewer mean correlation of 0.24, suggesting that it scores are slightly weaker than individual reviewer scores. Gemma 3 27B had weak rank correlations with average panel member scores (mean rho=0.17), with lower average correlations with individual panellists (mean rho=0.14), which is substantially lower than the inter-panellist correlation (mean rho=0.38). Whilst the correlations seem too weak to replace expert review at the final panel stage, LLM scores might help with the initial reviewing state, such as by helping identify the weakest proposals for fast-track desk rejections, to replace one human reviewer, or for triangulation to check for bias.

AI snapshot

Written from the paper’s full text.

TL;DR

This study tests whether six open-weight LLMs can score 2,267 recent UK ESRC/EPSRC grant applications, comparing LLM scores with reviewer and panel scores. It finds low but positive rank correlations, with the best configuration (Gemma 3 27B, 10 varied prompts) reaching mean rho=0.263 with average reviewer scores and rho=0.165 with panel prescores, too weak to replace expert review at the final panel stage but potentially useful for triage or initial review support.

Key findings

  • Across 2,267 applications from four 2023 UKRI open calls, LLM scores were individually inaccurate: all LLMs had a higher mean and lower standard deviation than human reviewers. The best configuration, Gemma 3 27B with long and short prompts combined (10 scores), had a moderate mean rank correlation with average reviewer scores (rho=0.263); its average correlation with individual reviewers was 0.186, lower than the inter-reviewer mean correlation of 0.242.
  • For panel prescores, Gemma 3 27B had weak mean rank correlations with average panel member scores (rho=0.165) and with individual panellists (mean rho=0.144), substantially lower than the inter-panellist mean correlation of 0.381.
  • Smaller Gemma 3 variants and the two reasoning models (DeepSeek R1 32B, Qwen 3 32B) generally performed less well; correlations varied by call, and for the strongest configuration they were higher in the two EPSRC calls than in the two ESRC calls.
  • Kendall’s tau-b for Gemma 3 27B with combined prompts varied between 0.15 and 0.22, implying that a decision maker picking the better of a pair on the basis of LLM scores would match the human mean 57%–61% of the time. The paper concludes LLM scores are too weak to replace expert review at the final panel stage but might help with initial triage, replacing one human reviewer, or triangulation for bias checks.

Methods

  • Data: vision and approach sections from four 2023 UKRI responsive-mode calls (two ESRC, two EPSRC), totalling 2,267 proposals (n=524 EPSRC 1, n=282 ESRC 1, n=752 ESRC 2, n=709 EPSRC 2). Reviewer scores were integer 1–6; panellist prescores were 0–10 with one decimal. PDFs were converted to text with GROBID, and data were accessed in a secure, offline UKRI cloud Linux environment.
  • Models: Gemma 3 1B/4B/12B/27B, DeepSeek R1 32B, and Qwen 3 32B, run through Ollama. Prompts were based on UKRI assessment criteria; two system prompts (short and long) and five user-prompt wording variations gave up to 10 submissions per proposal per model, except the reasoning models used only the long variant. Default parameters, zero-shot prompting, and no fine-tuning or few-shot prompting were used.
  • Analysis: Spearman rank correlations between LLM scores and expert scores, with bootstrapped 95% confidence intervals; human inter-reviewer and inter-panellist agreement rates served as benchmarks. Kendall’s tau-b correlations were also calculated.

Limitations

  • The paper states the data cover one type of grant application from one country, a limited set of LLMs, one funder type, two research areas, a single year (2023), only the vision and approach sections, two prompt sets, one prompting strategy (zero-shot), and default LLM parameters; changing any of these could give different results. It also assumes the average human reviewer score is the correct or best available approximation, while acknowledging expert judgement is subjective and error-prone, and notes commercial LLMs can be influenced by creators’ political priorities or training data.
  • The text makes evident that only four funding calls were analysed and correlations varied across calls; final panel scores were unavailable due to UKRI system restrictions; confidence intervals were wide, limiting strong statistical conclusions; and the human agreement benchmarks used only the first three reviewers/panellists.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2609.25327

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

3 of 18 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    There is often little correlation between individual reviewer scores for grant applications.

    “reflecting the general pattern that there is often little correlation between individual reviewer scores for grant applications (e.g., Fogelholm et al. 2012;Hodgson 1995;Pier et al. 2018).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Fogelholm et al. 2012

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Hodgson 1995

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Pier et al. 2018 source
  • Findingown data

    Gemma 3 27B's average correlation with individual reviewers was 0.19.

    “Gemma 3 27B's average correlation with individual reviewers was 0.19”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.
  • Findingown data

    The best performing LLM, Gemma 3 27B (10 iterations with varied prompts), had moderate rank correlations with average reviewer scores (mean rho=0.26).

    “The best performing LLM, Gemma 3 27B (10 iterations with varied prompts), had moderate rank correlations with average reviewer scores (mean rho=0.26).”
    • Also stated in: § 3.1 LLM scores vs. initial reviewer scores · plain
      “The mean of the four correlations (one per funding call) for this configuration is moderate, at rho=0.263.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.
  • Findingown data

    Gemma 3 27B had lower average correlations with individual panellists (mean rho=0.14).

    “with lower average correlations with individual panellists (mean rho=0.14)”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.
  • Findingown data

    Gemma 3 27B's correlation with individual panellists is substantially lower than the inter-panellist correlation.

    “which is substantially lower than the inter-panellist correlation (mean rho=0.38)”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.
  • Classified as probably not a substantial claim. 11
  • May combine two claims. 18

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    LLMs can give useful reviewer suggestions for some conference submissions.

    “Although previous studies have found that LLMs can give useful reviewer suggestions for some conference submissions (Thakkar et al. 2025)”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Thakkar et al. 2025 source

  • May combine two claims.

    Premisecitation

    Commercial LLMs can be influenced by the political priorities of their creators or in the data that they are trained on, potentially allowing non-academics to influence decisions about funding research into sensitive topics.

    “A more fundamental limitation is that commercial LLMs can be influenced by the political priorities of their creators or in the data that they are trained on, potentially allowing non-academics to influence decisions about funding research into sensitive topics, such as gender identity (e.g., Ovalle et al. 2023;Voutyrakou and Voutyrakos 2026).”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Ovalle et al. 2023 source

      The Introduction (p. 1) says LLMs may propagate societal biases, and the Conclusion (p. 10) reports gender-related biases in generated text. The paper does not connect those biases to creators’ political priorities or say that non-academics can influence research-funding decisions. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites Voutyrakou and Voutyrakos 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    LLM scores are individually inaccurate but when averaged and converted to ranks they correlate positively with expert average scores.

    “Although the LLM scores are individually inaccurate, when averaged and converted to ranks they correlate positively with expert average scores.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.

  • May combine two claims.

    Findingown data

    The strongest overall configuration, Gemma 3 27B with the long and short prompts combined, had a moderate mean correlation with reviewer Scores (rho = 0.263) and a weak mean correlation with panel-member Prescores (rho = 0.165).

    “The strongest overall configuration, Gemma 3 27B with the long and short prompts combined, had a moderate mean correlation with reviewer Scores (rho = 0.263) and a weak mean correlation with panel-member Prescores (rho = 0.165).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Human agreement rates are higher for EPSRC than for ESRC.

    “Generally, the human agreement rates are higher for EPSRC than for ESRC.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.

  • May combine two claims.

    Findingown data

    For panellist prescores, Gemma 3 27B again tended to have the highest correlations, although the differences are not clear-cut and no prompt configuration performed consistently best.

    “For panellist prescores, Gemma 3 27B again tended to have the highest correlations, although the differences are not clear-cut and no prompt configuration performed consistently best.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.

  • May combine two claims.

    Findingown data

    The two reasoning models perform consistently worse than Gemma 3 27B and the smallest variant of Gemma 3 27B performs substantially worse overall.

    “the two reasoning models perform consistently worse than Gemma 3 27B and the smallest variant of Gemma 3 27B performs substantially worse overall.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.

  • May combine two claims.

    Findingown data

    When the best performing LLM is compared against individual reviewer scores rather than the mean of reviewer scores, reviewers correlate with each other individually more strongly than the LLM correlates with individual reviewers, with an average correlation of 0.186.

    “Nevertheless, if the best performing LLM is compared against the individual reviewer scores (rather than the mean of the reviewer scores) then the reviewers correlate with each other individually more strongly than the LLM correlates with the individual reviewers (Table 2), with an average correlation of 0.186.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    All LLMs had a higher mean score and a lower standard deviation than human reviewers for all calls.

    “In particular, all LLMs had a higher mean score and a lower standard deviation than human reviewers for all calls.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.

  • May combine two claims.

    Findingown data

    The LLM with the highest correlation with reviewer scores is Gemma 3 27B, and the highest correlation generally occurs when the long and short prompt scores are combined (average of 10 scores).

    “The LLM with the highest correlation with reviewer scores is Gemma 3 27B, and the highest correlation generally occurs when the long and short prompt scores are combined (i.e., the average of 10 scores, 5 from each set of prompts).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.

  • May combine two claims.

    Findingown data

    For the four funding calls, the correlations between LLM scores and mean reviewer scores were relatively low but higher than the correlations between individual reviewers.

    “The relatively low correlations between LLM scores and mean reviewer scores are nevertheless higher than the correlations between individual reviewers for the four calls”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    The data covers one type of grant application from one country and a limited set of LLMs.

    “The data covers one type of grant application from one country and a limited set of LLMs.”
    • Also stated in: § 4 Discussion · plain
      “The results are limited to one type of funding application (open calls), a single funder type (government), a limited set of LLMs, two areas (social sciences, and engineering and physical sciences), a single year (2023) and one country (the UK).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    The average score given by experts, mediated by an expert decision panel, is the closest to a correct answer that it is practical to obtain.

    “This encodes the belief that field experts' opinions are the best source of evidence about the quality of proposals and that the average score given by experts, mediated by an expert decision panel, is the closest to a correct answer that it is practical to obtain (e.g., Fogelholm et al. 2012).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Fogelholm et al. 2012

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    It is not possible through any methods to predict the most successful research to fund in advance.

    “Academic research fundamentally involves uncertainty, so it is not possible through any methods to predict the most successful research to fund in advance (Roumbanis 2021).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Roumbanis 2021 source

  • May combine two claims.

    Interpretationown data

    This configuration showed some ability to reproduce the proposal rankings implied by expert assessments, with stronger alignment to initial reviewer Scores than to panel member Prescores.

    “Thus, this configuration showed some ability to reproduce the proposal rankings implied by expert assessments, with stronger alignment to initial reviewer Scores than to panel member Prescores (on a high scoring subset).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    The greater dispersion of EPSRC prescores may also have made correlations with prescores easier to detect.

    “The greater dispersion of EPSRC Prescores may also have made correlations with Prescores easier to detect.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    The results are like those from a different year (1994), country (Sweden), field (medicine), and application type (fellowships), which gives some confidence that they are not unique to the context analysed.

    “Nevertheless, the results are like those from a different year (1994), country (Sweden), field (medicine), and application type (fellowships) (Sandström and Thelwall 2026), which gives some confidence that they are not unique to the context analysed.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sandström and Thelwall 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Open-weight LLMs scored 142 Swedish medical fellowship research proposals from 1994 and had moderate rank correlations with expert scores (mean Spearman's rho = 0.33 for Gemma 3 27B processing titles and summaries, falling to 0.24 for full texts).

    “Based on intuitive prompts submitted five times per proposal and model, there were moderate rank correlations with expert scores: a mean Spearman's rho = 0.33 for Gemma 3 27B processing proposal titles and summaries, falling to 0.24 when processing full texts.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Sandström and Thelwall 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation

    A stronger correlation was found for title/summary input for the Swedish case, suggesting that higher correlations could also be obtained for UKRI proposal summaries, although these were not available.

    “Given that a stronger correlation was found for title/summary input for the Swedish case, this suggests that higher correlations could also be obtained for UKRI proposal summaries, although these were not available.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The average human reviewer score is the correct answer, or at least the best available approximation.
    • The authors state this as an assumption: Human experts are the most reliable sources of judgement.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Changing any of these could give different results.

    “Changing any of these could give different results.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The improvement over guessing is relatively modest but perhaps comparable to existing procedures such as panel discussions, given the high rate of expert disagreement.

    “This is a relatively modest improvement on guessing (i.e., 50%), but perhaps comparable to existing procedures (e.g., panel discussions), given the high rate of expert disagreement.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    The real improvement over guessing may be higher because mean reviewer scores may also be incorrect.

    “Moreover, the real improvement over guessing may be higher given that the mean reviewer scores may also be incorrect.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    It is impossible to know how good reviewers and panel members are at selecting the best proposals to fund.

    “but it is impossible to know how good reviewers and panel members are at selecting the best proposals to fund.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    LLMs do not seem to be quite as strong as individual reviewers from a technical accuracy (correlation) standpoint, even though the differences are not large.

    “Thus, LLMs do not seem to be quite as strong as individual reviewers from a technical accuracy (correlation) standpoint, even though the differences are not large.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Human reviewer and panellist scores provide a valid benchmark for the technical accuracy of proposal evaluation.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The results are restricted to two sets of prompts, one prompting strategy (zero shot) and one set of LLM parameters (the defaults).

    “In addition, the results are restricted to two sets of prompts, one prompting strategy (zero shot) and one set of LLM parameters (the defaults).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Increasing the role of LLMs would shift the decision responsibility partly (if LLMs support reviewers) or fully (if LLMs replace reviewers) from academics to the funding councils.

    “Increasing the role of LLMs would shift the decision responsibility partly (if LLMs support reviewers) or fully (if LLMs replace reviewers) from academics to the funding councils.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Under the current UK system, UKRI designs and manages the funding approach but largely delegates the outcome to academics in the form of selected reviewers and panel members.
26 of 26

  • The authors state this as an assumption.

    The average human reviewer score is the correct answer, or at least the best available approximation.

    “the analysis assumes that the average human reviewer score is the correct answer, or at least the best available approximation”

  • The authors state this as an assumption.

    Human experts are the most reliable sources of judgement.

    “which assumed to be the most reliable sources of judgement.”

  • The authors state this as an assumption.

    Human mean scores are correct.

    “if the human mean scores are correct”

  • The paper relies on this without support.

    Human reviewer and panellist scores provide a valid benchmark for the technical accuracy of proposal evaluation.

    “Thus, LLMs do not seem to be quite as strong as individual reviewers from a technical accuracy (correlation) standpoint”

  • The paper relies on this without support.

    Under the current UK system, UKRI designs and manages the funding approach but largely delegates the outcome to academics in the form of selected reviewers and panel members.

    “Under the current UK system, although UKRI has designed the funding approach (i.e., independent reviewers, then a funding panel; the nature of the funding calls) and manage it (e.g., selecting reviewers), they largely delegate the outcome to academics in the form of the selected reviewers and panel members.”
5 of 5

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Not every LLM score has a lower standard deviation

    Minor

    The text says every LLM standard deviation is below that of human reviewers for every call. For EPSRC 1, Table 1 reports a reviewer standard deviation of 0.82 but includes LLM standard deviations of 1.59 and 0.89.

  • Swedish proposal year changes from 1994 to 1992

    Minor

    The paper describes the previously studied Swedish applications as being from 1994, but the methods section calls them 1992 applications.

  • An EPSRC 1 LLM mean is below the reviewer mean

    Minor

    The same blanket claim about higher LLM means conflicts with Table 1: its EPSRC 1 reviewer mean is 4.37, while one LLM mean is 4.36.

3 of 3

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 2

  1. Sandström, Ulf, and Mike Thelwall. 2026. "Can Large Language Models Evaluate Grant Proposal Quality? Revisiting the Wennerås and Wold Peer Review Data." Journal of Data and Information Science. DOI. · DOI · Paper page
  2. Thelwall, Mike. 2024. "Can ChatGPT Evaluate Research Quality?" Journal of Data and Information Science 9 (2): 1-21. DOI. · DOI · Paper page

Authors
Thelwall, M., Pride, D., Osborne, F., Salatino, A., Knoth, P., Iyawa, G., & Yusuf, O.
Affiliations
  • The Open University (GB)
  • University of Salford (GB)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2609.25327
arXiv
2609.25327
OpenAlex
W7214124956
Semantic Scholar
3ece7250926f3cadb418868a966d2bc90a445d85
Cited by
0
References
0
Type
posted content
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
Economic and Social Research Council
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
Keywords
large language models, peer review, research funding, academic research evaluation, rank correlation, desk rejection, bias detection, Gemma 3, DeepSeek, Qwen

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-23.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-23.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review