CoChat
Sign Up
PreprintCited by 0Information Retrieval and Search BehaviorTopic ModelingAI-based Problem Solving and Planning

LifeSearch: Benchmarking Deep Research Agents for Real-World Life and Productivity

Open access
Wang, Yibo, Jin, Senjie, Ning, Yansong, Liang, Xinbing, Luo, Haotian, Zhang, Jiazheng, He, Yuxuan, Wu, Will +1 more·2026·arXiv
DOI 10.48550/arxiv.2610.18092arXiv 2610.18092
Open this paper in CoChat — free accountOpen PDF

Abstract

Deep research (DR) agents address open-ended information needs through multi-step reasoning, long-horizon web search, and evidence synthesis. However, existing benchmarks primarily focus on academic research or synthetic tasks, leaving DR agents' capabilities in real-world life and professional productivity settings insufficiently evaluated. To bridge this gap, we introduce LifeSearch, a challenging benchmark comprising 203 Chinese and English tasks derived from real-user query seeds across 7 concrete life and productivity domains. With more than 4,500 hours of human annotation, we construct over 4,000 expert-written, fine-grained rubrics, covering both report-content criteria for coverage and accuracy and concrete criteria specifying values, units, dates, and URLs. We conduct extensive experiments on 12 advanced DR agents under a unified evaluation harness, systematically analyzing their performance patterns and failure modes. We find that even the best-performing frontier agent, GPT-5.6 Sol, achieves a score of only 63.4%, leaving substantial room for improvement on realistic life and professional DR tasks. We select the grading configuration by comparing candidate LLM judges against human rubric labels, and a human ranking study shows strong alignment between aggregate benchmark scores and human judgments (Spearman's $ρ= 0.89$, Kendall's $τ= 0.81$). LifeSearch provides a rigorous evaluation framework and foundation for advancing DR agents toward real-world life and productivity.

AI snapshot

Written from the paper’s full text.

TL;DR

LIFESEARCH is a bilingual (Chinese/English) benchmark of 203 real-user life and productivity research tasks across seven domains, annotated with 4,014 expert-written fine-grained rubrics over 4,500+ hours of human labor. Evaluating 12 frontier deep research agents under a unified harness, the best system (GPT-5.6 Sol) reaches only 63.4%, and output volume alone does not predict quality.

Key findings

  • Substantial headroom: GPT-5.6 Sol leads at 63.4%, followed by Claude Opus 5 at 56.1%; all other systems score below 50%, and no system exceeds 70% in any of the seven domains. GPT-5.6 Sol's three-run mean falls below 50% on 41 of 203 tasks (20.2%).
  • Quality transcends volume: GPT-5.6 Sol has both the highest score and largest output volume (51.5k tokens/task), but Claude Opus 5 and GLM-5.3 generate similar volumes (48.9k and 49.3k) yet differ by 6.9 percentage points; Hy4 Preview reaches 49.7% with only 33.6k tokens.
  • Evidence collection correlates with performance: mean interaction count correlates positively with overall score (Spearman's ρ = 0.76); GPT-5.6 Sol averages 86.2 iterations versus GLM-5.2's 11.0 iterations and 37.1%. Final-report length also correlates with score (ρ = 0.60).
  • Grading validity: rubric-based rankings align strongly with human rankings (Spearman's ρ = 0.89, Kendall's τ = 0.81) on a 30-task, 210-report, seven-system study, with identical ordering of the top four systems. Recurring failure modes are premature clarification, temporal scope violations, and missing required details.

Methods

  • Benchmark construction: crowd-worker query seeds filtered by LLM for search relevance/quality/implicit-event dependence, classified into seven domains, rewritten to remove operational/file dependencies, then manually screened. Experts decomposed seeds into 2–4 subtasks (Lookup, Compute, Compare, Reason), enriched them around capabilities (Search Planning, Information Synthesis, Conflict Resolution, Uncertainty Handling), and rewrote them into natural-language tasks. Result: 203 tasks (102 Chinese, 101 English) and 4,014 rubrics (mean 19.77 per task; min 11, max 34); ~40% of rubrics specify numerical values, events, or named documents.
  • Evaluation: rubric-based LLM-as-a-Judge assigns binary Pass/Fail per rubric; scores are weight-normalized within task then averaged uniformly across tasks. Judge selection compared 11 configurations of four frontier LLMs against 402 human-labeled rubrics (mean pairwise Cohen's κ = 0.715; 65 disagreements resolved by discussion), selecting Gemini 3.1 Pro with low reasoning effort and zero temperature (84.25% mean accuracy, 98.8% self-consistency).
  • Agent evaluation: 12 frontier DR agents run on all 203 tasks in a shared ReAct-style harness with web search (Serper), web fetch (Jina Reader), and code execution; three independent runs per task, reporting mean and sample standard deviation, plus iteration counts and output tokens.

Limitations

  • Stated by the paper: iteration counts include other tool actions, so cross-model comparisons "do not isolate the causal effect of additional searches"; domain-level differences are "descriptive, especially for the nine-task everyday-facts subset"; the human ranking study covers only 30 tasks and seven systems (210 reports); all models were evaluated within a few days of one another in August 2026.
  • Evident from the text: the selected judge itself has 84.25% mean accuracy against human labels, so roughly one in six rubric judgments may deviate from expert annotation; the benchmark is confined to Chinese and English; and the harness provides no user replies, which the paper notes can terminate runs when agents ask clarifying questions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2610.18092

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 5 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Hy4 Preview reaches a comparable score to GLM-5.3 (49.7% versus 49.2%) with only 33.6k tokens.

    “Hy4 Preview reaches a comparable score to GLM-5.3 (49.7% versus 49.2%) with only 33.6k tokens.”
    Quote in the paperSubstantial claim
  • Findingown data

    Hy4 Preview leads GLM-5.3 on academic and professional literature (51.4% versus 49.0%).

    “GLM-5.3 outperforms Hy4 Preview on everyday facts (51.9% versus 47.1%), while Hy4 Preview leads on academic and professional literature (51.4% versus 49.0%).”
    Quote in the paperNot a substantial claim
  • Findingown data

    No system exceeds 70% in any of the seven domains.

    “GPT-5.6 Sol and Claude Opus 5 rank first and second in all seven domains, but no system exceeds 70% in any domain.”
    • Also stated in: § 1 INTRODUCTION · plain
      “with all systems falling below 70% across domains”
    Quote in the paperSubstantial claim
  • Findingown data

    The construction of LIFESEARCH involved more than 4,500 hours of human annotation labor.

    “we spend totally more than 4,500 hours of human annotation to construct over 4,000 expert-written, fine-grained rubrics”
    • Also stated in: § 1 INTRODUCTION · plain
      “its construction involves more than 4,500 hours of human labor to produce over 4,000 expert-written, fine-grained rubrics that comprehensively assess both the coverage and the accuracy of a candidate report.”
    Quote in the paperNot a substantial claim
  • Findingown data

    GLM-5.3 outperforms Hy4 Preview on everyday facts (51.9% versus 47.1%).

    “GLM-5.3 outperforms Hy4 Preview on everyday facts (51.9% versus 47.1%), while Hy4 Preview leads on academic and professional literature (51.4% versus 49.0%).”
    Quote in the paperNot a substantial claim
  • May combine two claims. 20
  • The quote only partly states this claim. 5
  • Classified as probably not a substantial claim. 17

  • May combine two claims.

    Findingown data

    GPT-5.6 Sol achieves the highest overall score of 63.4%, followed by Claude Opus 5 at 56.1%, with all other systems scoring below 50%.

    “GPT-5.6 Sol leads with 63.4%, followed by Claude Opus 5 at 56.1%; all other systems score below 50%.”
    • Also stated in: § Abstract · plain
      “even the best-performing frontier agent, GPT-5.6 Sol, achieves a score of only 63.4%”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The annotator-assigned rubric weights correctly reflect each requirement's relative importance for fulfilling the user's request.

  • May combine two claims.

    Findingown data

    Doubao-Seed-2.1-Pro and Gemini-3.6-Flash produce 12,089 and 10,256 characters per report while scoring 43.1% and 34.7%, respectively.

    “Yet Doubao-Seed-2.1-Pro and Gemini-3.6-Flashproduce 12,089 and 10,256 characters per report while scoring 43.1% and 34.7%, respectively; Hy4 Preview reaches 49.7% with 9,959 characters.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    GPT-5.6 Sol achieves the highest overall score on LifeSearch among the 12 evaluated models, 63.44 ± 0.27.

    “GPT-5.6 Sol | 62.9 | 67.8 | 60.7 65.6 59.5 | 65.6 | 58.3 | 63.44 ± 0.27 86.2 | 51.5 |”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Findingown data

    GPT-5.6 Sol has the longest mean final-answer length per task among the 12 models, 23,679 characters.

    “GPT-5.6 Sol | 23,679”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    LIFESEARCH contains 4,014 expert-written fine-grained rubrics, averaging 19.8 rubrics per task.

    “trained expert annotators develop 4,014 fine-grained rubrics assessing both requirement coverage and factual accuracy.”
    • Also stated in: § 3.3 RUBRIC DESIGN · plain
      “The benchmark contains 4,014 rubrics, averaging 19.8 per task.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Agents sometimes default to current web data even when the query specifies an earlier cutoff, a behavior particularly noticeable in Hy4 Preview.

    “Agents sometimes default to current web data even when the query specifies an earlier cutoff, a behavior particularly noticeable in Hy4 Preview.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    A human ranking study on 30 tasks (15 per language, 210 reports from seven systems) found strong correlation between rubric-based aggregate scores and human rankings (Spearman's ρ = 0.89, Kendall's τ = 0.81), with identical ordering of the top four systems.

    “A human ranking study shows strong alignment between aggregate benchmark scores and human judgments, with Spearman's ρ = 0.89 and Kendall's τ = 0.81.”
    • Also stated in: § 4.5 HUMAN AGREEMENT · boosted
      “Within this subset, rubric-based and human rankings are strongly correlated (Spearman's ρ = 0.89, Kendall's τ = 0.81), with identical ordering of the top four systems.”
    • Also stated in: § 4.5 HUMAN AGREEMENT · plain
      “We conduct a human ranking study on 30 tasks (15 per language) to examine agreement at the report level.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Human annotators' rankings of reports by factual accuracy, completeness, and formatting quality constitute a valid ground truth for overall report quality.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    In Case 3, the response distinguishes watchers from subscribers, satisfying the metric-definition criterion, but its own disclosure dates the API values to August 21, after the requested July 20 cutoff.

    “The response distinguishes watchers from subscribers, satisfying the metric-definition criterion. However, its own disclosure dates the API values to August 21, after the requested July 20 cutoff.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    With around 20 rubrics per task, fluent summaries can omit requested entities, numerical values, dates, sources, or edge cases.

    “With around 20 rubrics per task, a fluent summary can address the broad topic while omitting requested entities, numerical values, dates, sources, or edge cases.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    In Case 4, the response supplies the requested Chan/WLS reference, but the full report omits the required comparison of Chan-WLS, Taylor iteration, and constrained least squares.

    “The response supplies the requested Chan/WLS reference. The recorded judge nevertheless finds that the full report omits the required comparison of Chan-WLS, Taylor iteration, and constrained least squares.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Claude Opus 5 and GLM-5.3 generate similar output volumes (48.9k and 49.3k tokens) yet their scores differ by 6.9 percentage points.

    “However, Claude Opus 5 and GLM-5.3 generate similar volumes (48.9k and 49.3k), yet their scores differ by 6.9 percentage points.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    GPT-5.6 Sol and Claude Opus 5 rank first and second in all seven domains.

    “GPT-5.6 Sol and Claude Opus 5 rank first and second in all seven domains, but no system exceeds 70% in any domain.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Among the seven displayed models, Hy3 is ranked 7th by human ranking but 5th by the paper's method, a change of two places.

    “Hy3 | 7 | 5 | ↓ 2”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    GPT-5.6 Sol has both the highest score and the largest output volume (51.5k tokens per task).

    “GPT-5.6 Sol has both the highest score and the largest output volume (51.5k tokens per task).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    GPT-5.6 Sol averages 86.2 iterations and scores 63.4%, whereas GLM-5.2 averages 11.0 iterations and scores 37.1%.

    “GPT-5.6 Sol averages 86.2 iterations and scores 63.4%, whereas GLM-5.2 averages 11.0 iterations and scores 37.1%.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    In Case 2, the requested crop is winter wheat, but the listed paper's title explicitly says spring wheat.

    “The requested crop is winter wheat, but the listed paper's title explicitly says spring wheat.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    In Case 1, the model response describes single-run execution and lists both start-node options but does not state that User Input and Trigger are mutually exclusive.

    “The response describes single-run execution and lists both start-node options. It does not state that User Input and Trigger are mutually exclusive.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Doubao-Seed-2.1-Pro and GPT-5.6 Sol sometimes ask follow-up questions instead of proceeding with the research task, terminating the loop before a final report is produced.

    “Doubao-Seed-2.1-Pro and GPT-5.6 Sol sometimes ask follow-up questions instead of proceeding with the research task.”
    • Also stated in: § 4.4 FAILURE ANALYSIS · plain
      “Because the autonomous harness provides no user replies, these requests can terminate the loop before a final report is produced.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Many of the rubrics are anchored to concrete numerical values, units, dates, or thresholds, which facilitates verifiable, criterion-level evaluation.

    “Crucially, many rubrics are anchored to concrete numerical values, units, dates, or thresholds, facilitating verifiable, criterion-level evaluation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Study design and conductnone given

    The rubrics are self-contained, non-redundant, and operationally precise, with target entities, factual targets, units, temporal bounds, and tolerances defined so that judgments require no additional web browsing.

    “Rubrics are self-contained, non-redundant, and operationally precise.Experts define target entities, factual targets, units, temporal bounds, and tolerances so that judgments require no additional web browsing.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Existing benchmarks provide limited coverage of real-world life and professional productivity settings, with many evaluations centered on academic research or synthetic tasks.

    “Nevertheless, as summarized in Table 1, existing benchmarks provide limited coverage of real-world life and professional productivity settings, with many evaluations remaining centered on academic research tasks or synthetic tasks.”
    • Also stated in: § Deep Research Agent Evaluation. · plain
      “However, most of these focus on academic research or synthetic tasks, leaving DR agents' capabilities in real-world life and professional productivity settings underexplored.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Du et al., 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Sharma et al., 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Wong et al., 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Three independent runs per task were conducted, reporting the mean and sample standard deviation of the run-level benchmark scores.

    “To account for variability across DR agent trajectories, we conduct three independent runs per task, and report the mean and sample standard deviation of the run-level benchmark scores.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Judge selection compared 11 judge configurations on 402 human-labeled rubrics, sorted by mean accuracy over three runs.

    “Comparison of 11 judge configurations on 402 human-labeled rubrics for judge selection, sorted by mean accuracy over three runs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Annotation quality control included an iterative cycle of annotation, inspection, and revision, and outlier screening of low-scoring and overly easy tasks.

    “We implement annotation quality control across two stages: (i) an iterative cycle of annotation, inspection, and revision to refine the rubrics; (ii) outlier screening, where low-scoring tasks are checked for flaws, while overly easy tasks are tightened.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Study design and conductown data

    All 12 DR agents were evaluated on all 203 bilingual tasks under a shared ReAct-style harness with identical tools (web search, web fetch, code execution).

    “All systems operate within a shared ReAct-style (Yao et al., 2023) harness that interleaves decisions, tool execution, and feedback from the resulting observations.Agents have access to web search, web fetch, and code execution.”
    • Also stated in: § 4.1 EXPERIMENTAL SETUP · plain
      “We evaluate 12 frontier DR agents on all 203 bilingual tasks: GPT-5.6 Sol, Claude Opus 5, Hy4 Preview, GLM-5.3,Qwen3.8-Max,DeepSeek-V4 Pro, Kimi-K3, Doubao-Seed-2.1-Pro,Claude Opus 4.8, Hy3, GLM-5.2, and Gemini-3.6-Flash.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Yao et al., 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Constructing the LifeSearch benchmark required more than 4,500 hours of human annotation.

    “Constructing the benchmark required more than 4,500 hours of human annotation.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Study design and conductown data

    Two to three domain experts independently assigned binary Pass/Fail labels to 402 rubrics, reaching mean pairwise Cohen's κ = 0.715, and all 65 initial disagreements were resolved through discussion to form the reference labels.

    “For each report, two to three domain experts independently assign binary Pass/Fail labels, reaching a substantial inter-annotator agreement (mean pairwise Cohen's κ = 0.715).”
    • Also stated in: § 3.5 JUDGE MODEL SELECTION IN DR SETTING · plain
      “All 65 initial disagreements are subsequently resolved through discussion to form the reference labels used to compare judge configurations.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Each LLM-judge configuration was run three times on identical reports.

    “Each configuration is run three times on identical reports.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    A relevant search query does not ensure that every selected paper meets the crop requirement.

    “A relevant search query does not ensure that every selected paper meets the crop requirement.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    All models listed in Table 4 were evaluated within a few days of one another in August 2026.

    “All models listed in the table are evaluated within a few days of one another in August 2026.”
    Quote in the paperNot a substantial claim
30 of 30

  • The paper relies on this without support.

    The crowd-worker-written query seeds and expert refinement yield tasks that represent real-world life and professional productivity information needs.

    “To target real-world life and professional productivity scenarios, LIFESEARCH requires both authentic task provenance and broad coverage of user needs in its construction.”

  • The paper relies on this without support.

    Human annotators' rankings of reports by factual accuracy, completeness, and formatting quality constitute a valid ground truth for overall report quality.

    “Annotators rank anonymized reports presented in randomized order according to factual accuracy, completeness, and formatting quality.”

  • The paper relies on this without support.

    The annotator-assigned rubric weights correctly reflect each requirement's relative importance for fulfilling the user's request.

    “Annotators assign each rubric a weight according to its relative importance for fulfilling the user's request.”
3 of 3

Cites

Flagged first, then by citations

2 of the 26 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Start-node options (reformatted table row). Workflow: User Input 或Trigger Chatflow: User Input(仅此一种) Source label in the model's table: Workflow & Chatflow 对比表 Case explanation (editorial). The response describes single-run execution and lists both start-node options. It does not state that User Input and Trigger are mutually exclusive. Two recorded rubric judgments R4 -Pass Rubric: 回复应定义Workflow 为单轮执行模式：接收输入→流经节点处理→返回结果，运行一次即 结束 Recorded reason: 答复在定义与定位部分明确引用官方原话，说明Workflow处理单轮任务，接收输 入、通过流程处理并返回结果，运行一次即结束，完全符合标准。 R6 -Fail Rubric: 回复应说明Workflow 的开始节点可以是User Input 或Trigger(定时/Webhook/插件事件)， 且二者互斥，不可同时使用 Recorded reason: 模型答复说明了Workflow的开始节点可以是User Input或Trigger，但未提及二者互 斥、不可同时使用。 D.2 CASE 2: WINTER-WHEAT LITERATURE SCREENING Model: Hy4 Preview Language: Chinese Domain: Academic and professional literature Task type: Literature retrieval and eligibility screening Full-report evaluation: 11/15 rubrics passed; score 57.53%. Task summary. A research group needs English-language papers in Q1 (top-quartile) journals on winter-wheat ridge/furrow or raised-bed cultivation. It requests search terms, verified bibliographic details, and journal-quartile checks against Web of Science and Journal Citation Reports (JCR). Response excerpts

Authors
Wang, Y., Jin, S., Ning, Y., Liang, X., Luo, H., Zhang, J., He, Y., Wu, W., & Clarenceai
Affiliations
Tencent (China) (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2610.18092
arXiv
2610.18092
OpenAlex
W7221836076
Semantic Scholar
2add77e7a3fa5a642cd8386cedb140eb9d2759dd
Cited by
0
References
26
Type
posted content
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
deep research agents, benchmarking, multi-step reasoning, evidence synthesis, LLM-as-a-judge, manual annotation, GPT-5.6 Sol, failure modes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-23.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-23.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review