DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivity
Open accessAbstract
We present DRACO (Deep Research Accuracy, Completeness, and Objectivity), a benchmark of complex deep research tasks. These tasks, which span 10 domains and draw on information sources from 40 countries, originate from anonymized real-world usage patterns within a large-scale deep research system. Tasks are sampled from a de-identified dataset of Perplexity Deep Research requests, then filtered and augmented to ensure that the tasks are anonymized, open-ended and complex, objectively evaluable, and representative of the broad scope of real-world deep research use cases. Outputs are graded against task-specific rubrics along four dimensions: factual accuracy (accuracy), breadth and depth of analysis (including completeness), presentation quality (including objectivity), and citation quality. DRACO is publicly available at https://hf.co/datasets/perplexity-ai/draco.
AI snapshot
Written from the paper’s full text.
TL;DR
DRACO is a 100-task cross-domain benchmark built from anonymized Perplexity Deep Research production queries, spanning 10 domains and 40 countries, with expert-designed rubrics graded by LLM-as-a-judge. Perplexity Deep Research led overall (70.5% normalized score with Opus 4.6; 72.8% pass rate), while factual accuracy and citation quality remained the weakest rubric axes across systems.
Key findings
- The benchmark contains 100 tasks across 10 domains and 40 countries, with an average of 39.3 rubric criteria per task; of 3,934 total criteria, 415 are negative. Factual accuracy dominates (20.5 criteria/task), followed by breadth/depth (8.6), presentation quality (5.6), and citation quality (4.8).
- Perplexity Deep Research ranked first overall: 70.5% normalized score and 72.8% pass rate with Opus 4.6, and 67.2%/70.9% with Opus 4.5. Other systems scored lower: Gemini Deep Research 59.0%/62.7%, OpenAI o3 52.1%/56.9%, OpenAI o4-mini 41.9%/48.0%, Claude Opus 4.6 59.8%/63.1%, and Claude Opus 4.5 46.7%/50.2%.
- By rubric axis, Perplexity achieved the highest normalized scores in Factual Accuracy (67.9%), Breadth and Depth of Analysis (73.1% with Opus 4.5), Presentation Quality (90.3%), and Citation Quality (64.6%). The largest gaps versus the second-best model were in Breadth and Depth (13.2 percentage points) and Factual Accuracy (10.1 percentage points); systems generally performed best on Presentation Quality and worst on Factual Accuracy or Citation Quality.
- Perplexity Deep Research (Opus 4.6) had the highest score and lowest average latency among deep research systems (245.3 s) but the largest average input token usage (778,711 tokens). OpenAI o3 had the highest latency (1808.1 s) and a mid-range score (52.1%); OpenAI o4-mini was most token-efficient in combined input/output usage (53,506 tokens) but scored lowest (41.9%).
Methods
- Tasks were sampled from 1,000 high-difficulty English Perplexity Deep Research queries from September–October 2025, then pre-processed to remove PII, augmented along context and scope dimensions, filtered for objectivity/tractability/difficulty, and curated to 100 tasks with manual expert review.
- Rubrics were developed with 26 domain experts across four stages, including saturation testing where about 45% of tasks were returned for revision and about 10% of rubrics were returned at final review. Each task was graded against its rubric using an open-source LLM-as-a-judge protocol with binary MET/UNMET verdicts per criterion, weighted aggregation into normalized scores, and pass rates.
- Five deep research systems plus Claude Opus 4.5 and 4.6 with web search and code execution were run on all 100 tasks. Gemini-3-Pro was the primary judge, with GPT-5.2 and Sonnet-4.5 used for robustness checks; each task was evaluated over 5 independent grading runs.
Limitations
- The paper states that DRACO evaluates single-turn interactions only, remains static despite an automatable refresh pipeline, is restricted to text-to-text evaluation, may over-specify tasks through systematic augmentation, and is currently English-only.
- The paper also states that rubric creation still relies heavily on human experts, LLM-as-a-judge absolute scores vary by judge model though rankings were stable, and system-level black-box evaluation cannot attribute performance to specific components due to harness heterogeneity.
- Evident from the text: the benchmark is derived from Perplexity production queries and the authors are affiliated with Perplexity, so the evaluation of Perplexity Deep Research on a benchmark built from its own production usage may raise conflict-of-interest concerns.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
7 of 12 cited sources checked · 3 supported · 1 flagged · accuracy not yet measured
- Premisecitation§ 2 Related Work · p. 2
Some deep research benchmarks focus on challenging but closed-ended tasks whose solutions can be checked by a deterministic algorithm against the ground truth.
“Some deep research benchmarks focus on challenging but closed-ended tasks whose solutions can be checked by a deterministic algorithm against the ground truth (e.g., Mialon et al. [2023], Phan et al. [2025], Wei et al. [2025], Krishna et al. [2024], Gupta et al. [2026]).”
Quote in the paperNot a substantial claim3 of 3 sources checked- cites Mialon et al. [2023] source
- cites Phan et al. [2025] source
The Dataset > Collection section (page unavailable) describes HLE as challenging, closed-ended questions with specified correct answers, and says exact-match answers are kept short to support automatic grading. But Evaluation > Setup (page unavailable) says the paper uses o3-mini as a judge to handle equivalent answer formats; it does not establish that all solutions can be checked by a deterministic algorithm. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites Wei et al. [2025] source
The “Easy to verify, but hard to solve” section describes BrowseComp’s challenging questions and short reference answers. But §2.3, “Grading,” says predicted answers are checked for semantic equivalence by an AI model, not a deterministic algorithm. (LLM, conf=0.90)
Experimental — accuracy not yet measured
- Findingreasoning§ 2 Related Work · p. 2
None of the listed deep research benchmarks directly draws from a widely available production deep research system.
“none directly draws from a widely available production deep research system.”
Quote in the paperNot a substantial claim - Findingnone given§ 1 Introduction · p. 2
Perplexity Deep Research consistently demonstrates the strongest performance by overall score and pass rate, across all domains and rubric categories.
“Perplexity Deep Research consistently demonstrates the strongest performance by overall score and pass rate, across all domains and rubric categories.”
Quote in the paperNot a substantial claim- Rests on an assumption: Most real-world deep research tasks require human-like judgment and open-ended analysis.
- Findingown data§ Abstract · p. 1
DRACO tasks originate from anonymized real-world usage patterns within a large-scale deep research system (Perplexity Deep Research).
“originate from anonymized real-world usage patterns within a large-scale deep research system.”
- Also stated in: § 1 Introduction · p. 1 · plain
“these tasks all originate from actual user-requested tasks”
Quote in the paperSubstantial claim - Also stated in: § 1 Introduction · p. 1 · plain
- Findingown data§ 5.2 Main Results · p. 10
OpenAI Deep Research o3 records the highest latency (1808.1 seconds).
“OpenAI Deep Research o3 records the highest latency (1808.1 seconds)”
Quote in the paperNot a substantial claim
- Findingown data§ 4.1 Rubric · p. 7
The total criteria per task ranges from 30.2 (Needle in a Haystack) to 47.6 (Finance).
“The total criteria per task ranges from 30.2 (Needle in a Haystack) to 47.6 (Finance), indicating substantial variation in evaluation granularity.”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Main Results · p. 10
Opus 4.6 yields the strongest non-Perplexity result overall, outperforming other deep research systems.
“Opus 4.6 yields the strongest non-Perplexity result overall, outperforming other deep research systems.”
Quote in the paperSubstantial claim - Findingnone given§ 6.2 Evaluation Protocol · p. 13
Relative rankings remain stable across judge models.
“While relative rankings remain stable across judge models”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Main Results · p. 12
Perplexity Deep Research with Opus 4.6 ranks first on three of the four axes.
“Perplexity Deep Research with Opus 4.6 ranks first on three of the four axes”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 10
Gemini Deep Research achieved a normalized score of 59.0%.
“followed by Gemini Deep Research (59.0%)”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 10
OpenAI o4-mini achieved a normalized score of 41.9%.
“OpenAI o4-mini (41.9%)”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 10
OpenAI Deep Research o3 and Gemini Deep Research produce substantially more output tokens (24,944 and 22,066 tokens, respectively).
“OpenAI Deep Research o3 and Gemini Deep Research produce substantially more output tokens (24,944 and 22,066 tokens, respectively)”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 12
The biggest performance gaps between Perplexity Deep Research and the second-best-performing model are in Breadth and Depth of Analysis and Factual Accuracy (13.2 and 10.1 percentage points, respectively).
“The biggest performance gaps between Perplexity Deep Research and the second-best-performing model are in Breadth and Depth of Analysis and Factual Accuracy (13.2 and 10.1 percentage points, respectively).”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Rubric · p. 7
Law and Medicine exhibit the highest proportion of negative criteria (4.7 out of 33.2 and 4.3 out of 33.7, respectively).
“Law and Medicine exhibit the highest proportion of negative criteria (4.7 out of 33.2 and 4.3 out of 33.7, respectively)”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Rubric · p. 7
Finance and Academic domains require the most comprehensive evaluation frameworks (47.6 and 41.6 criteria, respectively).
“Finance and Academic domains require the most comprehensive evaluation frameworks (47.6 and 41.6 criteria, respectively)”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Main Results · p. 10
Perplexity Deep Research with Opus 4.5 achieved a normalized score of 67.2%.
“and 67.2% (Opus 4.5)”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 10
Perplexity Deep Research substantially outperforms Opus 4.5 and 4.6 with web search and code execution.
“Perplexity Deep Research substantially outperforms Opus 4.5 and 4.6 with web search and code execution”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 11
Pass rates by domain display a similar overall pattern to normalized scores.
“Table 12 reports pass rates by domain and displays a similar overall pattern.”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Main Results · p. 10
OpenAI o3 achieved a normalized score of 52.1%.
“OpenAI o3 (52.1%)”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 10
Perplexity Deep Research with Opus 4.6 achieved the lowest average latency (245.3 seconds) among deep research systems.
“Perplexity Deep Research (Opus 4.6) attains the highest normalized score while also achieving the lowest average latency (245.3 seconds) among deep research systems”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Main Results · p. 11
Perplexity Deep Research (with Opus 4.5 or 4.6) attains the highest scores across all domains.
“Perplexity Deep Research (with Opus 4.5 or 4.6) attains the highest scores across all domains”
Quote in the paperSubstantial claim - Findingown data§ 4.1 Rubric · p. 8
Factual Accuracy dominates all domains, comprising over half of all criteria on average (20.5 out of 39.3).
“Factual Accuracy dominates all domains, comprising over half of all criteria on average (20.5 out of 39.3)”
Quote in the paperNot a substantial claim - Findingown data§ 3 Task Construction · p. 3
The sample spans 10 general and specialized domains.
“The sample spans 10 general and specialized domains (Figure 2).”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 10
Perplexity Deep Research with Opus 4.6 achieved a normalized score of 70.5%.
“Perplexity Deep Research leads with a score of 70.5% (Opus 4.6)”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 10
Perplexity Deep Research with Opus 4.6 and with Opus 4.5 consistently rank as the top two systems across repeated deep research runs.
“Perplexity Deep Research with Opus 4.6 and with Opus 4.5 consistently rank as the top two systems across repeated deep research runs.”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 10
Claude Opus 4.5 and 4.6 exhibit the lowest latency (178.4 and 192.9 seconds, respectively).
“and exhibit the lowest latency (178.4 and 192.9 seconds, respectively)”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Main Results · p. 10
Perplexity Deep Research with Opus 4.6 has the largest average input token usage (778,711 tokens).
“albeit with the largest average input token usage (778,711 tokens)”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Main Results · p. 10
OpenAI Deep Research o4-mini is the most token-efficient in terms of combined input and output usage (a total of 53,506 tokens).
“OpenAI Deep Research o4-mini is the most token-efficient in terms of combined input and output usage (a total of 53,506 tokens)”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Main Results · p. 12
Perplexity Deep Research with Opus 4.5 attains the top score for Breadth and Depth of Analysis.
“while Perplexity Deep Research with Opus 4.5 attains the top score for Breadth and Depth of Analysis”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 10
Among deep research systems, Perplexity Deep Research leads.
“Among deep research systems, Perplexity Deep Research leads”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 10
Claude Opus 4.5 and 4.6 generate the fewest output tokens (6,174 and 8,143, respectively).
“Claude Opus 4.5 and 4.6 generate the fewest output tokens (6,174 and 8,143, respectively)”
Quote in the paperNot a substantial claim - Findingnone given§ 2 Related Work · p. 2
DRACO's main contribution is a curated set of benchmark tasks that closely mirror real deep research needs and how people use deep research agents in practice.
“Our main contribution is a curated set of benchmark tasks that closely mirror real deep research needs and how people use deep research agents in practice.”
Quote in the paperSubstantial claim - Findingown data§ 5.2 Main Results · p. 11
The best-performing version of Perplexity Deep Research varies by domain.
“although the best-performing version varies by domain”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Main Results · p. 11
Law (90.2%) and Academic (82.8%) show the strongest absolute performance.
“with Law (90.2%) and Academic (82.8%) showing the strongest absolute performance”
Quote in the paperNot a substantial claim - Findingown data§ A Alternative LLM Judges · p. 16
GPT-5.2 consistently assigned lower scores than the other two judges.
“GPT-5.2 consistently assigned lower scores than the other two judges”
Quote in the paperNot a substantial claim - Findingown data§ A Alternative LLM Judges · p. 16
The relative ordering of systems was stable across all three judges.
“yet the relative ordering of systems was stable across all three”
Quote in the paperSubstantial claim - Study design and conductnone given§ 1 Introduction · p. 1
DRACO tasks are paired with task-specific, expert-grounded rubrics.
“Importantly, these tasks all originate from actual user-requested tasks and are paired with task-specific, expert-grounded rubrics.”
Quote in the paperSubstantial claim - Study design and conductnone given§ 3 Task Construction · p. 3
1,000 high-difficulty English deep research queries issued on Perplexity in September-October 2025 were randomly sampled, where difficulty is proxied by either subsequent negative user sentiment or an explicit thumbs-down rating on the model's prior response.
“We randomly sampled 1,000 high-difficulty English deep research queries issued on Perplexity in September-October 2025, where difficulty is proxied by either subsequent negative user sentiment or an explicit thumbs-down rating on the model's prior response.”
Quote in the paperSubstantial claim - Study design and conductnone given§ 6.2 Evaluation Protocol · p. 13
Rubric creation still relies heavily on human expert involvement.
“rubric creation still relies heavily on human expert involvement”
Quote in the paperSubstantial claim - Interpretationown data§ 5.2 Main Results · p. 10
The uniformly low standard deviations across systems indicate that grades are consistent across judge runs.
“The uniformly low standard deviations across systems indicate that grades are consistent across judge runs.”
Quote in the paperNot a substantial claim - Interpretationown data§ 5.2 Main Results · p. 10
Longer outputs do not necessarily achieve higher performance on the benchmark.
“yet their lower normalized scores indicate that longer outputs do not necessarily achieve higher performance on our benchmark”
Quote in the paperSubstantial claim - Study design and conductreasoning§ A Alternative LLM Judges · p. 16
Configuring the LLM judges with the specified temperature and reasoning settings ensured that differences in model outputs were primarily attributable to the system's internal architecture and reasoning capabilities rather than stochastic variations.
“This configuration ensured that differences in model outputs were primarily attributable to the system's internal architecture and reasoning capabilities rather than stochastic variations.”
Quote in the paperSubstantial claim- Rests on an assumption: Configuring LLM judges with low temperature and disabled or low reasoning reduces stochastic variation in their outputs.
- May combine two claims. 32
- Classified as probably not a substantial claim. 53
- The quote only partly states this claim. 1
May combine two claims.
Premisecitation§ 2 Related Work · p. 2DeepResearchEval, ReportBench, DeepScholar-Bench, and DRBench rely on synthetic task generation.
“DeepResearchEval [Wang et al., 2026], ReportBench [Li et al., 2025b], DeepScholar-Bench [Patel et al., 2025], and DRBench [Abaskohi et al., 2025], in contrast, rely on synthetic task generation.”
Quote in the paperNot a substantial claim2 of 3 sources checked- cites Wang et al., 2026 source
- cites Li et al., 2025b
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Patel et al., 2025 source
For DeepScholar-Bench, the paper says the opposite of the attributed claim. Section 2 (pp. 3–4) says it sources queries by scraping real arXiv papers and extracts human-written related-work sections as exemplars; it does not describe synthetic task generation. This verdict does not assess the other benchmarks cited in the sentence. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
Premisecitation§ 1 Introduction · p. 1Deep research systems integrate multi-step planning and reasoning with autonomous retrieval and evaluation of external information, enabling the system to verify claims, resolve conflicting evidence, and identify gaps in the literature.
“Unlike single-shot question answering, deep research systems integrate multi-step planning and reasoning with autonomous retrieval and evaluation of external information, enabling the system to verify claims, resolve conflicting evidence, and identify gaps in the literature [Huang et al., 2025].”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Huang et al., 2025 source
The Abstract (p. 1) describes deep research agents as combining long-horizon planning, reasoning, and multi-hop retrieval. Section 6 (p. 22) describes checking claims against independent sources and revising the retrieval strategy when evidence conflicts. The paper does not establish that these systems identify gaps in the literature; its discussion of research priorities concerns gaps in the field's own benchmarks. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
Premisecitation§ 1 Introduction · p. 1Deep research systems are increasingly relevant to knowledge-intensive domains such as academic research, medical decision support, legal analysis, and financial analysis.
“Deep research systems are increasingly relevant to knowledge-intensive domains, such as academic research [Patel et al., 2025, Zhou et al., 2025], medical decision support [Chen et al., 2025b, Wu et al., 2025], legal analysis [Li et al., 2025a], and financial analysis [Zhu et al., 2025, Bigeard et al., 2025].”
Quote in the paperSubstantial claim1 of 3 sources checked- cites Patel et al., 2025, Zhou et al., 2025 source
- cites Chen et al., 2025b, Wu et al., 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Li et al., 2025a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Findingnone given§ 2 Related Work · p. 2Expert-designed rubrics are present in the majority of the listed benchmarks; the remainder rely on automated or reference-based scoring.
“Expert-designed rubrics are present in the majority of the benchmarks; the remainder rely on automated or reference-based scoring”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ Abstract · p. 1DRACO is a benchmark of complex deep research tasks that span 10 domains and draw on information sources from 40 countries.
“We present DRACO (Deep Research Accuracy, Completeness, and Objectivity), a benchmark of complex deep research tasks. These tasks, which span 10 domains and draw on information sources from 40 countries, originate from anonymized real-world usage patterns within a large-scale deep research system.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ 4.1 Rubric · p. 6The most severe penalties are reserved for harmful medical content, with weights ranging from -50 for harmful clinical guidance to -500 for dangerous recommendations.
“The most severe penalties are reserved for harmful medical content, with weights ranging from -50 for harmful clinical guidance to -500 for dangerous recommendations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ 4.1 Rubric · p. 6In non-medical domains, penalties typically range from -10 to -25.
“In non-medical domains, penalties typically range from -10 to -25.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Findingnone given§ 6.2 Evaluation Protocol · p. 13Grading runs with different judges exhibit substantial variation in score magnitudes.
“Grading runs with different judges also exhibit substantial variation in score magnitudes.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.2 Main Results · p. 12The second place is split between Opus 4.6 (Factual Accuracy and Citation Quality) and Gemini (Breadth and Depth of Analysis and Presentation Quality).
“The second place is split between Opus 4.6 (Factual Accuracy and Citation Quality) and Gemini (Breadth and Depth of Analysis and Presentation Quality).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Rubric · p. 622% (8.6) of criteria assess the quality of analysis in terms of completeness and depth.
“22% (8.6) assess the quality of analysis in terms of completeness and depth”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Rubric · p. 612% (4.8) of criteria evaluate correct citation of primary sources.
“12% (4.8) evaluate correct citation of primary sources.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.1 Rubric · p. 8Finance and Academic tasks require the most comprehensive evaluation (47.6 and 41.6 criteria/task, respectively), driven by dense factual requirements (27.7 and 21.8, respectively), while Needle in a Haystack tasks require the fewest (30.2).
“Finance and Academic tasks require the most comprehensive evaluation (47.6 and 41.6 criteria/task, respectively), driven by dense factual requirements (27.7 and 21.8, respectively), while Needle in a Haystack tasks require the fewest (30.2)”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.2 Main Results · p. 12Across the board, agents perform best on Presentation Quality and worst on Factual Accuracy or Citation Quality.
“Across the board, agents perform best on Presentation Quality and worst on Factual Accuracy or Citation Quality.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingreasoning§ 4.1 Rubric · p. 6Approximately half of the criteria (20.5 per task) target verification of the factual accuracy of the claims.
“Approximately half of the criteria (20.5 per task) target verification of the factual accuracy of the claims, reflecting the critical importance of correctness in research tasks.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.2 Main Results · p. 11The gap between Perplexity and the second-best model is largest on Finance (21.6 percentage points), Shopping/Product Comparison (10.9), Technology (9.8), and Academic (9.3).
“The gap between Perplexity and the second-best model is largest on Finance (21.6 percentage points), Shopping/Product Comparison (10.9 percentage points), Technology (9.8 percentage points), and Academic (9.3 percentage points)”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.2 Main Results · p. 12Perplexity Deep Research (with Opus 4.5 or 4.6) demonstrates best performance in all four rubric axes, achieving the highest normalized scores in Factual Accuracy (67.9%), Breadth and Depth of Analysis (73.1%), Presentation Quality (90.3%), and Citation Quality (64.6%).
“Perplexity Deep Research (with Opus 4.5 or 4.6) demonstrates best performance in all four categories, achieving the highest normalized scores in Factual Accuracy (67.9%), Breadth and Depth of Analysis (73.1%), Presentation Quality (90.3%), and Citation Quality (64.6%).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Rubric · p. 6About 10% of rubrics were returned to Expert 1 at Stage 4.
“About 10% of rubrics were returned to Expert 1 at this stage.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 3 Task Construction · p. 3The augmentation dimensions emerged from analysis of user behavior on Perplexity Deep Research, where successful outcomes correlate with richer upfront context and well-defined analytical scope.
“These dimensions emerged from the analysis of user behavior on Perplexity Deep Research, where successful outcomes correlate with richer upfront context and well-defined analytical scope.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data§ 4.1 Rubric · p. 6Negative criteria appear in all axes but are most prevalent in Presentation Quality (32.1% of all criteria along that axis).
“The negative criteria appear in all axes but are most prevalent in Presentation Quality (32.1% of all criteria along that axis)”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Rubric · p. 6Of the 3,934 total criteria, 415 are negative.
“Of the 3,934 total criteria, 415 are negative.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Rubric · p. 7The ratio of positive to negative criteria ranges from 6.1 (Law) to 11.2 (Finance).
“The ratio of positive to negative criteria ranges from 6.1 (Law) to 11.2 (Finance)”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.2 Main Results · p. 11The gap between Perplexity and the second-best model is smallest on Law (1.6 percentage points) and Needle in a Haystack (2.2).
“the gap is the smallest on Law (1.6 percentage points) and Needle in a Haystack (2.2 percentage points)”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.1 Rubric · p. 8Breadth and Depth of Analysis and Presentation Quality remain relatively stable across domains (except for Medicine), whereas Citation Quality varies notably-highest in Academic (5.8) and lowest in Medicine (3.0).
“Breadth and Depth of Analysis and Presentation Quality remain relatively stable across domains (except for Medicine), whereas Citation Quality varies notably-highest in Academic (5.8) and lowest in Medicine (3.0).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingnone given§ 5.1 Experiment Setting · p. 9The ranking of deep research systems was stable across judge models, even though absolute score magnitudes varied.
“The ranking of deep research systems was stable across judge models, even though absolute score magnitudes varied.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Rubric · p. 6About 45% of the tasks are sent back to Expert 1 for revision at Stage 3.
“About 45% of the tasks are sent back to Expert 1 for revision at this stage.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Rubric · p. 614% (5.6) of criteria address the clarity and style of presentation, such as format, readability, and objective tone.
“14% (5.6) address the clarity and style of presentation-such as format, readability, and objective tone”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 5.2 Main Results · p. 10Pass rates exhibit an overall pattern consistent with the normalized scores.
“Table 9 reports the unweighted pass rates, which exhibit an overall pattern consistent with the normalized scores.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.2 Main Results · p. 11The second-best non-Perplexity result varies by domain: Claude Opus 4.6 ranks second in 5 domains, Gemini ranks second in 4 domains, and OpenAI o3 takes second on Academic.
“The second-best non-Perplexity result varies by domain: Claude Opus 4.6 ranks second in 5 domains (General Knowledge, UX Design, Law, Medicine, and Needle in a Haystack), Gemini ranks second in 4 domains (Finance, Shopping/Product Comparison, Technology, Personalized Assistant), while OpenAI o3 takes second on Academic.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Rubric · p. 6Each task is assessed against an average of 39.3 criteria.
“Each task is assessed against an average of 39.3 criteria.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ A Alternative LLM Judges · p. 16Perplexity Deep Research with Opus 4.6 was ranked first by every judge, followed by Perplexity Deep Research with Opus 4.5, Claude Opus 4.6, and Gemini Deep Research.
“Perplexity Deep Research with Opus 4.6 was ranked first by every judge, followed by Perplexity Deep Research with Opus 4.5, Claude Opus 4.6, and Gemini Deep Research.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ 6.2 Evaluation Protocol · p. 13Absolute scores depend on LLM judges and may not perfectly align with human expert preferences across all domains.
“absolute scores depend on LLM judges and may not perfectly align with human expert preferences across all domains”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ Abstract · p. 1Outputs are graded against task-specific rubrics along four dimensions: factual accuracy, breadth and depth of analysis, presentation quality, and citation quality.
“Outputs are graded against task-specific rubrics along four dimensions: factual accuracy (accuracy), breadth and depth of analysis (including completeness), presentation quality (including objectivity), and citation quality.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductnone given§ 1 Introduction · p. 1DRACO tasks are sampled from tens of millions of Perplexity Deep Research requests, then filtered and augmented to remove personally identifiable information (PII) and ensure both rigor and representativeness.
“Tasks are sampled from tens of millions of Perplexity Deep Research requests, then filtered and augmented to remove personally identifiable information (PII) and ensure both rigor and representativeness.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Premisenone given§ 1 Introduction · p. 1Evaluating deep research systems is challenging due to the curse of dimensionality: a comprehensive dataset must simultaneously reflect realistic use cases, span a wide range of domains, cover different regions with distinct information sources, and probe multiple underlying capabilities within each instance.
“Evaluating deep research systems is challenging due to the curse of dimensionality: a comprehensive dataset must simultaneously reflect realistic use cases, span a wide range of domains, cover different regions with distinct information sources, and probe multiple underlying capabilities within each instance.”
Quote in the paperNot a substantial claim- Rests on an assumption: Strong performance in knowledge-intensive domains requires comprehensive, in-depth, transparent, and verifiable reasoning over large, heterogeneous information corpora.
May combine two claims.
Study design and conductnone given§ Abstract · p. 1DRACO tasks were sampled from a de-identified dataset of Perplexity Deep Research requests, then filtered and augmented to ensure they are anonymized, open-ended, complex, objectively evaluable, and representative of real-world deep research use cases.
“Tasks are sampled from a de-identified dataset of Perplexity Deep Research requests, then filtered and augmented to ensure that the tasks are anonymized, open-ended and complex, objectively evaluable, and representative of the broad scope of real-world deep research use cases.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductnone given§ 3 Task Construction · p. 2Tasks are sourced from production Perplexity Deep Research queries, then systematically reformulated, augmented, and filtered to ensure they are anonymous, well-specified, bounded, demand challenging open-ended analysis, and are representative of actual user use cases.
“We source tasks from production Perplexity Deep Research queries, then systematically reformulate, augment, and filter them to ensure they are anonymous, well-specified, bounded, demand challenging open-ended analysis, and are representative of actual user use cases.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone given§ 3 Task Construction · p. 3Sampled raw queries were reformulated with an LLM to remove personally identifiable information (PII) and to reduce ambiguity.
“Sampled raw queries were reformulated with an LLM to remove personally identifiable information (PII) and to reduce ambiguity.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ 6.1 Generalization · p. 13All evaluation is currently conducted in English.
“all evaluation is currently conducted in English”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown data§ 6 Discussion · p. 12The evaluation reveals that while significant progress has been made (especially in presentation quality), substantial headroom remains (especially in factual accuracy).
“Our evaluation of frontier deep research systems reveals that while significant progress has been made (especially in presentation quality), substantial headroom remains (especially in factual accuracy).”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductown data§ 4.1 Rubric · p. 6Once Expert 2 accepted a rubric, Perplexity Deep Research was evaluated on the associated task using that rubric; if the model achieved a score above 90%, the task was returned to Expert 1 and passed through Stages 1 and 2 again.
“Once Expert 2 accepted a rubric, we evaluated Perplexity Deep Research on the associated task using that rubric; if the model achieved a score above 90% (indicating that the task was too simple or the rubric was too lenient), that task was returned to Expert 1 and passed through Stages 1 and 2 again.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductown data§ 5.2 Main Results · p. 10Normalized scores are averaged across 100 tasks, each evaluated over 5 independent LLM-as-a-judge grading runs, and standard deviations capture variability across grading runs.
“Normalized scores (%) are averaged across 100 tasks, each evaluated over 5 independent LLM-as-a-judge grading runs; standard deviations (SD) capture the variability across grading runs.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone given§ 6.1 Generalization · p. 13Systematic augmentation reduces ambiguity and improves reproducibility, but it also risks over-specifying tasks and dampening the natural variability of user queries.
“Systematic augmentation reduces ambiguity and improves reproducibility, but it also risks over-specifying tasks and dampening the natural variability of user queries.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone given§ 3 Task Construction · p. 3All queries were processed end-to-end by an automated pipeline, and no raw user queries were ever exposed to human analysts.
“All queries were processed end-to-end by an automated pipeline, and no raw user queries were ever exposed to human analysts.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown data§ 5.2 Main Results · p. 10The outperformance of Perplexity Deep Research over Opus 4.5 and 4.6 indicates the importance of agent orchestration beyond the base model.
“indicating the importance of agent orchestration beyond the base model”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 5.1 Experiment Setting · p. 9Each system was run on the full benchmark of 100 tasks.
“Each system was run on the full benchmark of 100 tasks.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ 3 Task Construction · p. 2In-house domain experts and experts recruited by The LLM Data Company verified the generated tasks.
“We work with in-house domain experts and experts recruited by The LLM Data Company to verify the generated tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown data§ 4.1 Rubric · p. 6Expert 2 reviewed the initial rubric and proposed revisions to Expert 1.
“Expert 2 reviewed the initial rubric and proposed revisions to Expert 1, which may include refining existing criteria, adding missing ones, removing incorrect or redundant items, or, in some cases, recommending that the task be dropped.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ 6.1 Generalization · p. 12The benchmark evaluates single-turn interactions only.
“The benchmark evaluates single-turn interactions only”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationreasoning§ 6.1 Generalization · p. 12The benchmark itself remains static and may not fully generalize to future deep research applications.
“the benchmark itself remains static and may not fully generalize to future deep research applications”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown data§ 4.1 Rubric · p. 5The rubrics were designed and validated with The LLM Data Company.
“We worked with The LLM Data Company to design and validate the rubrics.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductown data§ 3 Task Construction · p. 3Pre-processed queries were systematically augmented along two axes: task context (such as user persona, desired output format, and sources) and task scope (extending time horizon, adding comparative elements, and introducing geographic variation).
“Pre-processed queries were systematically augmented along two axes (Table 2): we specified task context (such as user persona, desired output format, and sources) and broadened task scope by extending the time horizon, adding comparative elements, and introducing geographic variation.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. The quote only partly states this claim.
Study design and conductown data§ 4.1 Rubric · p. 6Each task is associated with a rubric that specifies evaluation criteria along four axes: factual accuracy, breadth and depth of analysis, presentation quality, and citation quality.
“At the end of the process, each task is associated with a rubric that specifies evaluation criteria along four axes (Tables 4 and 5).”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone given§ 4.2 Grading · p. 8Responses are evaluated against the final task-specific rubrics, with scores assigned independently for each criterion.
“Responses are evaluated against the final task-specific rubrics, with scores assigned independently for each criterion.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 3 Task Construction · p. 4Augmented queries were filtered with an LLM to retain only those that are objective, tractable, and challenging.
“Augmented queries were filtered with an LLM to retain only those that are objective, tractable, and challenging.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Study design and conductreasoning§ 2 Related Work · p. 2The task construction pipeline is designed to be automatable, continuously generating fresh benchmark tasks, with human reviewers as a final safety and quality gate.
“our task construction pipeline is designed to be automatable, continuously generating fresh benchmark tasks, with human reviewers as a final safety and quality gate.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 4.2 Grading · p. 8For each criterion, the judge outputs a binary verdict (MET or UNMET), accompanied by a short justification.
“For each criterion, the judge outputs a binary verdict (MET or UNMET), accompanied by a short justification.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductnone given§ 2 Related Work · p. 2The benchmark is constructed from actual Perplexity Deep Research tasks, which are systematically reformulated to protect user privacy and augmented into challenging deep research tasks that stress current deep research agents.
“We construct the benchmark from actual Perplexity Deep Research tasks, which are systematically reformulated to protect user privacy and augmented into challenging deep research tasks that stress current deep research agents and are likely to remain difficult in the foreseeable future.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductown data§ 3 Task Construction · p. 4One hundred queries were sampled from the filtered pool based on the domain distribution to align with the underlying mix of deep research user needs on Perplexity Deep Research, and were manually reviewed by in-house domain experts to verify security and quality.
“One hundred queries were sampled from the filtered pool based on the domain distribution shown in Figure 2 to align with the underlying mix of deep research user needs on Perplexity Deep Research, and were manually reviewed by in-house domain experts to verify security and quality.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning§ 5.2 Main Results · p. 10The low output tokens and latency of Claude Opus 4.5 and 4.6 likely reflect their different configuration as non-deep research systems.
“likely reflecting their different configuration as non-deep research systems”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ 4.1 Rubric · p. 6Rubrics that passed Stages 1 through 3 underwent a final quality-assurance review by an in-house domain expert (Expert 3) together with an AI expert (Expert 4).
“Rubrics that passed Stages 1 through 3 underwent a final quality-assurance review by an in-house domain expert (Expert 3) together with an AI expert (Expert 4).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 4.1 Rubric · p. 5Twenty-six domain experts, including medical professionals, attorneys, financial analysts, software engineers, and designers, were recruited to develop rubrics for selected tasks.
“Twenty-six domain experts, including medical professionals, attorneys, financial analysts, software engineers, and designers, were recruited to develop rubrics for selected tasks.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductown data§ 5.1 Experiment Setting · p. 9Drawing on an internal human-LLM alignment study, Gemini-3-Pro was selected as the primary judge model.
“Drawing on an internal human-LLM alignment study, we selected Gemini-3-Pro as our primary judge model.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 6 Discussion · p. 12DRACO is a cross-domain benchmark derived from real-world production deep research tasks designed to bridge the gap between AI evaluations and authentic research needs.
“We introduce DRACO, a cross-domain benchmark derived from real-world production deep research tasks designed to bridge the gap between AI evaluations and authentic research needs.”
Quote in the paperSubstantial claim- Rests on an assumption: The benchmark tasks are representative of real-world production deep research tasks.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ 6.1 Generalization · p. 12The benchmark is currently restricted to text-to-text evaluation.
“Our benchmark is currently restricted to text-to-text evaluation.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning§ 6.1 Generalization · p. 12The benchmark still exhibits gaps relative to how systems are used in practice now and in the future.
“Although we source tasks from real production queries, our benchmark still exhibits gaps relative to how systems are used in practice now and in the future.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 4.2 Grading · p. 8Grading is conducted with an open-source LLM-as-a-judge protocol.
“Grading is conducted with an open-source LLM-as-a-judge protocol.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown data§ 4.1 Rubric · p. 6For each task, a domain expert drafted an initial rubric with LLM assistance, typically requiring 45-60 minutes and at least 6 interaction turns between the expert and the model per rubric.
“For each task, a domain expert (Expert 1) drafted an initial rubric with LLM assistance, typically requiring 45-60 minutes and at least 6 interaction turns between the expert and the model per rubric.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductown data§ A Alternative LLM Judges · p. 16Scoring all five deep research systems and Claude Opus 4.5 and 4.6 with three distinct LLM judges (Gemini-3-Pro, GPT-5.2, and Sonnet-4.5) was done to assess robustness.
“To assess the robustness of our evaluation methodology, we scored all five deep research systems and Claude Opus 4.5 and 4.6 with three distinct LLM judges: Gemini-3-Pro, GPT-5.2, and Sonnet-4.5.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ A Alternative LLM Judges · p. 16Gemini strongly discourages setting temperature to 0.
“while respecting that Gemini strongly discourages setting temperature to 0”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Gemini docs at https://ai.google.dev/gemini-api/docs/gemini-3#temperature
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Study design and conductnone given§ 6.3 Attribution and Decomposition · p. 13The benchmark holistically assesses end-to-end system performance.
“While our benchmark holistically assesses end-to-end system performance”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationreasoning§ 6.3 Attribution and Decomposition · p. 13It is difficult to attribute overall performance to specific parts because systems differ in their internal tools, retrieval stacks, and browsing capabilities.
“Because systems differ in their internal tools, retrieval stacks, and browsing capabilities, it is difficult to attribute overall performance to specific parts”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisecitation§ A Alternative LLM Judges · p. 16Changing the temperature (setting it below 1.0) may lead to unexpected behavior, such as looping or degraded performance, particularly in complex mathematical or reasoning tasks.
“Changing the temperature (setting it below 1.0) may lead to unexpected behavior, such as looping or degraded performance, particularly in complex mathematical or reasoning tasks; see the Gemini docs at https://ai.google.dev/gemini-api/docs/gemini-3#temperature .”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Gemini docs at https://ai.google.dev/gemini-api/docs/gemini-3#temperature
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Study design and conductreasoning§ 6.3 Attribution and Decomposition · p. 13The evaluation is system-level, treating agents as black-box products.
“We conduct system-level evaluation, treating agents as black-box products”
Quote in the paperSubstantial claim
The paper relies on this without support.
Strong performance in knowledge-intensive domains requires comprehensive, in-depth, transparent, and verifiable reasoning over large, heterogeneous information corpora.
“Strong performance in these domains requires comprehensive, in-depth, transparent, and verifiable reasoning over large, heterogeneous information corpora.”
The paper relies on this without support.
Most real-world deep research tasks require human-like judgment and open-ended analysis.
“most real-world deep research tasks require human-like judgment and open-ended analysis.”
The paper relies on this without support.
Expert-designed rubrics align more closely with human preferences compared to LLM-designed rubrics.
“Expert-designed rubrics align more closely with human preferences compared to LLM-designed rubrics”
The paper relies on this without support.
The benchmark tasks are representative of real-world production deep research tasks.
“derived from real-world production deep research tasks”
The paper relies on this without support.
The four rubric axes (Factual Accuracy, Breadth and Depth of Analysis, Presentation Quality, Citation Quality) adequately capture deep research quality.
“four rubric axes”
The paper relies on this without support.
Configuring LLM judges with low temperature and disabled or low reasoning reduces stochastic variation in their outputs.
“This configuration ensured that differences in model outputs were primarily attributable to the system's internal architecture and reasoning capabilities rather than stochastic variations.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Seed mining follows the de-identification methodology of the DRACO benchmark: requests are sampled from a de-identified pool and reworded to remove identifying details.
“First, seed mining follows the de-identification methodology of the DRACO benchmark[Zhong et al., 2026]: requests are sampled from a de-identified pool and reworded to remove identifying details.”
From WANDR: A Benchmark for Wide and Deep Research (2026) · cites this paper as Zhong et al., 2026
WANDR is the breadth-oriented counterpart to DRACO.
“WANDR is the breadth-oriented counterpart to DRACO.”
From WANDR: A Benchmark for Wide and Deep Research (2026) · cites this paper as Zhong et al., 2026
The Abstract (p. 1) describes DRACO, and Related Work (p. 2) compares it with other deep-research benchmarks, but neither identifies WANDR as its breadth-oriented counterpart. A search of our copy for “WANDR” found no passage addressing that relationship. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
System rankings are not stable across judges
MinorTable 15 reverses the ordering of OpenAI Deep Research (o3) and Claude Opus 4.5: o3 scores higher with Gemini-3-Pro (52.1 versus 46.7), but lower with GPT-5.2 (31.7 versus 31.9) and Sonnet-4.5 (49.4 versus 58.7). Thus the relative ordering is not stable across all three judges.
“We observed systematic differences in absolute score levels-GPT-5.2 consistently assigned lower scores than the other two judges-yet the relative ordering of systems was stable across all three.”
“| OpenAI Deep Research (o3) | 52.1 | 31.7 | 49.4 | | OpenAI Deep Research (o4-mini) | 41.9 | 25.3 | 41.7 | | Claude Opus 4.6 | 59.8 | 42.7 | 70.1 | | Claude Opus 4.5 | 46.7 | 31.9 | 58.7 |”
Cites
Aldrich, C. (2020) · Definitions · doi:10.32388/10gqmo · Cited by 712
Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks
Bigeard, A., Nashold, L., Krishnan, R., & Wu, S. (2025) · arXiv.org · doi:10.48550/arxiv.2508.00828 · Cited by 78 · cited 1×
“…systems are increasingly relevant to knowledge-intensive domains, such as academic research [Patel et al., 2025, Zhou et al., 2025], medical decision support [Chen et al., 2025b, Wu et al., 2025], legal analysis [Li et al., 2025a], and financial analysis [Zhu et al., 2025, Bigeard et al., 2025].”
Phan, L., Gatti, A., Han, Z., Li, N., Hu, J., Zhang, H., Zhang, C. B. C., Shaaban, M., Ling, J., Shi, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.14249 · Cited by 65
Self-citationTowards evaluating and building versatile large language models for medicine
Wu, C., Qiu, P., Liu, J., Gu, H., Li, N., Zhang, Y., Wang, Y., & Xie, W. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01390-4 · Cited by 52 · cited 1× · see citing works
“…systems are increasingly relevant to knowledge-intensive domains, such as academic research [Patel et al., 2025, Zhou et al., 2025], medical decision support [Chen et al., 2025b, Wu et al., 2025], legal analysis [Li et al., 2025a], and financial analysis [Zhu et al., 2025, Bigeard et al., 2025].”
ResearcherBench: Evaluating Deep AI Research Systems on the Frontiers of Scientific Inquiry
Xu, T.-Z., Lu, P.-R., Ye, L., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2507.16280 · Cited by 35
Ruan, J., Nair, I., Cao, S., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2506.01241 · Cited by 26 · cited 2×
“…✗ DRBench [Abaskohi et al., 2025] ✗ ✗ ✗ ✓ DeepResearchEval [Wang et al., 2026] ✗ ✗ ✓ ✗ DeepResearch Bench [Du et al., 2025] ✗ ✓ ✓ ✗ ExpertLongBench [Ruan et al., 2025] ✗ ✓ ✗ ✓ xBench [Chen et al., 2025a] ✗ ✓ ✗ ✓ ResearcherBench [Xu et al., 2025] ✗ ✓ ✗ ✓ DEER [Han et al., 2025] ✗ ✓ ✗ ✓ DR. Bench…”
A benchmark of expert-level academic questions to assess AI capabilities
Center for AI Safety, Phan, L., Gatti, A., Li, N., Khoja, A., Kim, R., Ren, R., Hausenloy, J., Zhang, O., Mazeika, M., et al. (2026) · Nature · doi:10.1038/s41586-025-09962-4 · Cited by 23 · cited 1× · see citing works
“Some deep research benchmarks focus on challenging but closed-ended tasks whose solutions can be checked by a deterministic algorithm against the ground truth (e.g., Mialon et al. [2023], Phan et al. [2025], Wei et al. [2025], Krishna et al. [2024], Gupta et al. [2026]).”
Self-citationMMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents
Huang, P., Zhong, Z.-X., Wan, Z., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2601.12346 · Cited by 22 · cited 1×
“As deep research agents begin to process and output images and videos, future benchmarks could continue to explore explicitly incorporating multimodal verification [Huang et al., 2026].”
DRBench: A Realistic Benchmark for Enterprise Deep Research
Abaskohi, A., Chen, T., Munoz-M'armol, M., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.00172 · Cited by 22 · cited 3×
“…Non-Specialized Expert Tasks Authored Domain Rubrics ReportBench [Li et al., 2025b] ✗ ✗ ✗ ✗ DeepScholar-Bench [Patel et al., 2025] ✗ ✗ ✗ ✗ DRBench [Abaskohi et al., 2025] ✗ ✗ ✗ ✓ DeepResearchEval [Wang et al., 2026] ✗ ✗ ✓ ✗ DeepResearch Bench [Du et al., 2025] ✗ ✓ ✓ ✗ ExpertLongBench [Ruan et…”
FinDeepResearch: Evaluating Deep Research Agents in Rigorous Financial Analysis
Zhu, F., Ng, X.-Y., Liu, Z., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.13936 · Cited by 14
GAIA: a benchmark for General AI Assistants
Mialon, G., Fourrier, C., Swift, C., Wolf, T., LeCun, Y., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2311.12983 · Cited by 12 · cited 1× · see citing works
“Some deep research benchmarks focus on challenging but closed-ended tasks whose solutions can be checked by a deterministic algorithm against the ground truth (e.g., Mialon et al. [2023], Phan et al. [2025], Wei et al. [2025], Krishna et al. [2024], Gupta et al. [2026]).”
Wells, T. (2012) · doi:10.5040/9781784601324.00000003 · Cited by 7
ScholarSearch: Benchmarking Scholar Searching Ability of LLMs
Zhou, J., Li, W., Liao, Y., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2506.13784 · Cited by 7
Deep Research Agents: A Systematic Examination And Roadmap
Huang, Y., Chen, Y., Zhang, H., Li, K., Zhou, H., Fang, M., Yang, L., Li, X., Shang, L., Xu, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.18096 · Cited by 3 · cited 2×
“Deep research refers to a research process in which an agentic AI system decomposes a complex query into constituent workflows, iteratively searches for diverse sources of information, and synthesizes the resulting evidence into a structured and cited report [Zhang et al., 2025].”
Self-citationBrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
Wei, J., Sun, Z., Papay, S., McKinney, S., Han, J., Fulford, I., Chung, H. W., Passos, A. T., Fedus, W., & Glaese, A. (2025) · arXiv · doi:10.48550/arxiv.2504.12516 · Cited by 2 · cited 1× · see citing works
“Some deep research benchmarks focus on challenging but closed-ended tasks whose solutions can be checked by a deterministic algorithm against the ground truth (e.g., Mialon et al. [2023], Phan et al. [2025], Wei et al. [2025], Krishna et al. [2024], Gupta et al. [2026]).”
DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
Du, M., Xu, B., Zhu, C., Wang, X., & Mao, Z. (2025) · arXiv · doi:10.48550/arxiv.2506.11763 · Cited by 2 · cited 7× · see citing works
“…2025a] ✗ ✓ ✗ ✓ ResearcherBench [Xu et al., 2025] ✗ ✓ ✗ ✓ DEER [Han et al., 2025] ✗ ✓ ✗ ✓ DR. Bench [Yao et al., 2025] ✗ ✓ ✓ ✓ LiveResearchBench [Wang et al., 2025] ✗ ✓ ✓ ✓ ResearchRubrics [Sharma et al., 2025] ✗ ✓ ✓ ✓ DeepResearch Bench II [Li et al., 2026] Stage 1: Sampling We randomly…”
InfluentialFact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation
Krishna, S., Krishna, K., Mohananey, A., Schwarcz, S., Stambler, A., Upadhyay, S., & Faruqui, M. (2024) · arXiv · doi:10.48550/arxiv.2409.12941 · Cited by 1 · cited 1× · see citing works
“Some deep research benchmarks focus on challenging but closed-ended tasks whose solutions can be checked by a deterministic algorithm against the ground truth (e.g., Mialon et al. [2023], Phan et al. [2025], Wei et al. [2025], Krishna et al. [2024], Gupta et al. [2026]).”
xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations
Chen, K., Ren, Y., Liu, Y., Hu, X., Tian, H., Xie, T., Liu, F., Zhang, H., Liu, H., Gong, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.13651 · Cited by 1 · cited 2× · see citing works
“…✗ ✗ ✓ DeepResearchEval [Wang et al., 2026] ✗ ✗ ✓ ✗ DeepResearch Bench [Du et al., 2025] ✗ ✓ ✓ ✗ ExpertLongBench [Ruan et al., 2025] ✗ ✓ ✗ ✓ xBench [Chen et al., 2025a] ✗ ✓ ✗ ✓ ResearcherBench [Xu et al., 2025] ✗ ✓ ✗ ✓ DEER [Han et al., 2025] ✗ ✓ ✗ ✓ DR. Bench [Yao et al., 2025] ✗ ✓ ✓ ✓…”
Self-citationResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents
Sharma, M., Zhang, C. B. C., Bandi, C., Wang, C., Aich, A., Nghiem, H., Rabbani, T., Htet, Y., Jang, B., Basu, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.07685 · Cited by 1 · cited 2× · see citing works
“…expert-level long-form generation [Ruan et al., 2025], and profession-aligned productivity [Chen et al., 2025a], while others span general-purpose domains that include everyday use cases [Du et al., 2025, Wang et al., 2026, Sharma et al., 2025, Yao et al., 2025, Wang et al., 2025, Li et al., 2026].”
LegalAgentBench: Evaluating LLM Agents in Legal DomainPreprint
Li, H., Chen, J., Yang, J., Ai, Q., Jia, W., Liu, Y., Lin, K., Wu, Y., Yuan, G., Hu, Y., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.17259 · Cited by 1 · cited 1× · see citing works
“…systems are increasingly relevant to knowledge-intensive domains, such as academic research [Patel et al., 2025, Zhou et al., 2025], medical decision support [Chen et al., 2025b, Wu et al., 2025], legal analysis [Li et al., 2025a], and financial analysis [Zhu et al., 2025, Bigeard et al., 2025].”
Self-citationDeep Research: A Survey of Autonomous Research AgentsPreprint
Zhang, W., Li, X., Zhang, Y., Jia, P., Wang, Y., Guo, H., Liu, Y., & Zhao, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.12752 · Cited by 1 · see citing works
ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks
Li, M., Zeng, Y., Cheng, Z., Ma, C., & Jia, K. (2025) · arXiv · doi:10.48550/arxiv.2508.15804 · Cited by 1 · cited 3× · see citing works
“Human Non-Specialized Expert Tasks Authored Domain Rubrics ReportBench [Li et al., 2025b] ✗ ✗ ✗ ✗ DeepScholar-Bench [Patel et al., 2025] ✗ ✗ ✗ ✗ DRBench [Abaskohi et al., 2025] ✗ ✗ ✗ ✓ DeepResearchEval [Wang et al., 2026] ✗ ✗ ✓ ✗ DeepResearch Bench [Du et al., 2025] ✗ ✓ ✓ ✗ ExpertLongBench [Ruan et…”
DEER: A Benchmark for Evaluating Deep Research Agents on Expert Report GenerationPreprint
Han, J., Kim, H., Lee, C., Lee, D., Park, M. H., Song, H.-S., Choi, S. J., Lee, M., & Lee, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.17776 · Cited by 1 · cited 3× · see citing works
“While many benchmarks feature human-authored tasks that are inspired by real use cases from actual searches, interviews, or workflows (e.g., Chen et al. [2025a], Xu et al. [2025], Han et al. [2025], Du et al. [2025]), none directly draws from a widely available production deep research system.”
DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents
Gupta, N., Chatterjee, R., Haas, L., Tao, C., Wang, A., Liu, C., Oiwa, H., Gribovskaya, E., Ackermann, J., Blitzer, J., et al. (2026) · arXiv · doi:10.48550/arxiv.2601.20975 · cited 1×
“Some deep research benchmarks focus on challenging but closed-ended tasks whose solutions can be checked by a deterministic algorithm against the ground truth (e.g., Mialon et al. [2023], Phan et al. [2025], Wei et al. [2025], Krishna et al. [2024], Gupta et al. [2026]).”
LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild
Wang, J., Ming, Y., Dulepet, R., Chen, Q., Xu, A., Ke, Z., Sala, F., Albarghouthi, A., Xiong, C., & Joty, S. (2025) · arXiv · doi:10.48550/arxiv.2510.14240
DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
Patel, L., Arabzadeh, N., Gupta, H., Sundar, A., Stoica, I., Zaharia, M., & Guestrin, C. (2025) · arXiv · doi:10.48550/arxiv.2508.20033 · cited 4×
“Human Non-Specialized Expert Tasks Authored Domain Rubrics ReportBench [Li et al., 2025b] ✗ ✗ ✗ ✗ DeepScholar-Bench [Patel et al., 2025] ✗ ✗ ✗ ✗ DRBench [Abaskohi et al., 2025] ✗ ✗ ✗ ✓ DeepResearchEval [Wang et al., 2026] ✗ ✗ ✓ ✗ DeepResearch Bench [Du et al., 2025] ✗ ✓ ✓ ✗ ExpertLongBench [Ruan et…”
InfluentialDeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports
Li, R., Du, M., Xu, B., Zhu, C., Wang, X., & Mao, Z. (2026) · arXiv · doi:10.48550/arxiv.2601.08536 · cited 2×
“…Bench [Yao et al., 2025] ✗ ✓ ✓ ✓ LiveResearchBench [Wang et al., 2025] ✗ ✓ ✓ ✓ ResearchRubrics [Sharma et al., 2025] ✗ ✓ ✓ ✓ DeepResearch Bench II [Li et al., 2026] Stage 1: Sampling We randomly sampled 1,000 high-difficulty English deep research queries issued on Perplexity in September–October…”
MedBrowseComp: Benchmarking Medical Deep Research and Computer Use
Chen, S., Moreira, P., Xiao, Y., Schmidgall, S., Warner, J., Aerts, H., Hartvigsen, T., Gallifant, J., & Bitterman, D. S. (2025) · arXiv · doi:10.48550/arxiv.2505.14963 · cited 1×
“…systems are increasingly relevant to knowledge-intensive domains, such as academic research [Patel et al., 2025, Zhou et al., 2025], medical decision support [Chen et al., 2025b, Wu et al., 2025], legal analysis [Li et al., 2025a], and financial analysis [Zhu et al., 2025, Bigeard et al., 2025].”
Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to ReportsPreprint
Yao, Y., Wang, Y., Zhang, Y., Lu, Y., Gu, T., Li, L., Zhao, D., Wu, K., Wang, H., Nie, P., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.02190 · cited 2×
“…[Ruan et al., 2025] ✗ ✓ ✗ ✓ xBench [Chen et al., 2025a] ✗ ✓ ✗ ✓ ResearcherBench [Xu et al., 2025] ✗ ✓ ✗ ✓ DEER [Han et al., 2025] ✗ ✓ ✗ ✓ DR. Bench [Yao et al., 2025] ✗ ✓ ✓ ✓ LiveResearchBench [Wang et al., 2025] ✗ ✓ ✓ ✓ ResearchRubrics [Sharma et al., 2025] ✗ ✓ ✓ ✓ DeepResearch Bench II [Li et…”
DeepResearchEval: An Automated Framework for Deep Research Task Construction and Agentic Evaluation
Wang, Y., Wang, L., Deng, Y., Wu, K., Xiao, Y., Yao, H., Kang, L., Ye, H., Jing, Y., & Bing, L. (2026) · arXiv · doi:10.48550/arxiv.2601.09688 · cited 3×
“DeepResearchEval [Wang et al., 2026], ReportBench [Li et al., 2025b], DeepScholar-Bench [Patel et al., 2025], and DRBench [Abaskohi et al., 2025], in contrast, rely on synthetic task generation.”
Flagged first, then by citations
27 of the 57 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Agentic Routing: The Harness-Native Data Flywheel
Liu, X.-C., Zhou, H., Zong, Y.-J., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.11399 · Cited by 6
“DRACO is particularly useful here because it scores complex deep-research outputs along factual accuracy, completeness, objectivity, presentation quality, and citation quality, rather than only exact-match task completion [40].”
Self-citationFrom Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape
Chen, H., Han, Z., Yan, Y.-K., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.08625 · Cited by 5
“DRACO (Zhong et al., 2026) and ReportLogic (Zhao et al., 2026b) adopt human-in-the-loop construction, accepting some quality risk in exchange for coverage of authentic user queries sampled from real request logs.”
Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents
Rao, D., Wong, E., & Callison-Burch, C. (2026) · arXiv.org · doi:10.48550/arxiv.2604.03173 · Cited by 5
A Survey on Rubric-Guided Reinforcement Learning for Language Models
Shan, Z., & Shao, F. (2026) · doi:10.48550/arxiv.2608.27505 · Cited by 4
“DRACO ( Zhong et al., 2026) introduces a cross-domain benchmark spanning 10 domains and 40 countries’ information sources, evaluating outputs along factual accuracy, completeness, objectivity, and citation quality.”
Luan, B.-D., Sun, R., Wang, S.-N., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.12252 · Cited by 4
“…Deng et al., 2023), and more recently to deep-research settings involving realistic tool use, report generation, scientific re-discovery, and data-centric analysis (Mialon et al., 2023; Wei et al., 2025; Du et al., 2025; Han et al., 2025; Wang et al., 2026; Zhong et al., 2026; Liu et al., 2026a,b).”
An Inspectable LLM Council for Multi-Model Research Answer Aggregation
Wu, S., Li, X., Feng, Y., Li, Y.-F., Wang, Z., & Wang, R. (2026) · doi:10.48550/arxiv.2604.02923 · Cited by 2
“DRACO makes these distinctions explicit through task-specific criteria for factual accuracy, breadth and depth, presentation, and citation quality [1].”
Vinogradova, A., Vinogradov, V., Greenwood, L., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.15019 · Cited by 2
“Bench-marks like BrowseComp [5] emphasize short browsing tasks with a single verifiable answer, while rubric-based benchmarks such as ResearchRubrics [6] and DRACO [7] evaluate grounding, reasoning, clarity, and citation quality in long-form outputs.”
WinSyn: An Automated Pipeline for Realistic Enterprise Question-Answering Evaluation
Varhade, A., Sutradhar, A., Krishnaswamy, R., et al. (2026) · doi:10.48550/arxiv.2609.12171 · Cited by 1
“We can only give a very small sample: for RAG over research papers [4], for deep research [9], and for other related tasks OfficeQA Pro [13], DRACO [19], APEX-Agents [15], PRBench [3], TheAgentCompany [16].”
Wang, B., Zhou, K., Guo, L.-F., et al. (2026) · doi:10.48550/arxiv.2608.04077 · Cited by 1
“In the same spirit, DRACO (Zhong et al. 2026) scores long-form deep-research reports along four weighted meta-dimensions (accuracy, completeness, objectivity, citation quality), evidencing that per-dimension aggregation is standard practice for long-form evaluation.”
Multi-Turn Evaluation of Deep Research Agents Under Process-Level Feedback
Sabharwal, R., Wang, H., Storkey, A., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.09748 · Cited by 1
“However, because such signals are rare in DRACO and do not help identify gaps in the research process, we exclude them from our gap inference.”
InfluentialWANDR: A Benchmark for Wide and Deep Research
Polshkov, V., Pitera, M., Yang, J., Priemko, K., Gaiduk, M., Nikolenko, A., Bykov, D., Southern, C., Yarats, D., & Ma, J. (2026) · arXiv · doi:10.48550/arxiv.2608.14747
“…one of three ways: matching gold labels for single-answer question answering; applying task-specific rubric criteria with a large language model (LLM) judge to long-form reports, as in our deep-research benchmark DRACO [Zhong et al., 2026]; or matching a gold collection in wide-search benchmarks.”
Self-citationICBCBench: An Industry Consortium Benchmark for Financial Deep Research
Li, W., Tang, Z., He, Y., Wang, C., Feng, L., Sun, X., Liu, D., Wen, Z., Wei, H., Wang, J., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.17458
“Another line of work, represented by DeepResearch Bench, ResearchRubrics, LiveResearchBench, DRACO, and related benchmarks [9, 40, 37, 31, 44, 36, 1], evaluates long-form reports through citation accuracy and checklist-based expert assessment.”
LifeSearch: Benchmarking Deep Research Agents for Real-World Life and Productivity
Wang, Y., Jin, S., Ning, Y., Liang, X., Luo, H., Zhang, J., He, Y., Wu, W., & Clarenceai (2026) · arXiv · doi:10.48550/arxiv.2610.18092
“1 I NTRODUCTION Deep Research (DR) agents solve open-ended information-seeking problems by iteratively interacting with search engines and web pages, planning multi-step retrieval, and summarizing the collected (Gou et al., 2025) ✗ ✗ ✗ ✗ Lists/tables DeepResearch Bench (Du et al., 2026) ✗ ✗ ✗ ✓ Reports WideSearch (Won…”
Influential
- Authors
- Zhong, J., Zhang, H., Southern, C., Yang, J., Wang, T., Jung, K., Zhang, S., Yarats, D., Ho, J., & Ma, J.
- Affiliations
- Harvard University (US)
- ORCID
- Wang, Thomas
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2602.11685
- OpenAlex
- W7128775057
- Semantic Scholar
- b410892c249b1e6f8802891242b66f7ad8434030
- Cited by
- 0
- References
- 57
- Type
- posted content
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- large language models, factual accuracy, completeness, objectivity, citation quality, cross-domain evaluation, presentation quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-22.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-22.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
