BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese
Open accessAbstract
As large language models (LLMs) evolve into tool-using agents, the ability to browse the web in real-time has become a critical yardstick for measuring their reasoning and retrieval competence. Existing benchmarks such as BrowseComp concentrate on English and overlook the linguistic, infrastructural, and censorship-related complexities of other major information ecosystems -- most notably Chinese. To address this gap, we introduce BrowseComp-ZH, a high-difficulty benchmark purpose-built to comprehensively evaluate LLM agents on the Chinese web. BrowseComp-ZH consists of 289 multi-hop questions spanning 11 diverse domains. Each question is reverse-engineered from a short, objective, and easily verifiable answer (e.g., a date, number, or proper noun). A two-stage quality control protocol is applied to strive for high question difficulty and answer uniqueness. We benchmark over 20 state-of-the-art language models and agentic search systems on our proposed BrowseComp-ZH. Despite their strong conversational and retrieval capabilities, most models struggle severely: a large number achieve accuracy rates below 10%, and only a handful exceed 20%. Even the best-performing system, OpenAI's DeepResearch, reaches just 42.9%. These results demonstrate the considerable difficulty of BrowseComp-ZH, where success demands not only effective retrieval strategies, but also sophisticated reasoning and information reconciliation -- capabilities that current models still struggle to master. Our dataset, construction guidelines, and benchmark results have been publicly released at https://github.com/PALIN2018/BrowseComp-ZH.
AI snapshot
Written from the paper’s full text.
TL;DR
BrowseComp-ZH is a new 289-question Chinese-web benchmark built by reverse-designing multi-hop queries from short, verifiable answers across 11 domains. It evaluates over 20 LLMs and AI search systems, finding most models below 10% accuracy and the best system, OpenAI DeepResearch, at 42.9%.
Key findings
- Most standalone LLMs perform poorly: Qwen2.5-72B-Instruct 6.6%, GPT-4o 6.2%, Claude-3.5-Sonnet 5.5%, LlaMa4 4.8%; many systems remain below 10% accuracy.
- Reasoning-capable models improve substantially: DeepSeek-R1 reaches 23.2% vs. DeepSeek-V3’s 8.7%, and Claude-3.7-Sonnet reaches 17.7% vs. Claude-3.5-Sonnet’s 5.5%.
- Agentic search systems with multi-round retrieval perform best: OpenAI DeepResearch 42.9%, Doubao (Deep Search) 26.0%, Perplexity (Research) 22.6%.
- Enabling web search can hurt: DeepSeek-R1 drops from 23.2% in direct-answer mode to 7.6% with web search enabled; calibration error also rises from 59% to 65%.
Methods
- Dataset construction used a reverse strategy: 10 expert contributors selected factual answers and crafted multi-constraint Chinese queries, tested on Baidu, Bing, and Google so answers did not appear on first-page results.
- Two-stage quality control: Stage 1 filtered 76 simple samples from 480 preliminary samples, leaving 404 high-difficulty candidates; Stage 2 used AI agents plus manual verification to eliminate 115 ambiguous samples, yielding 289 final questions across 11 domains.
- Evaluation covered over 20 open-source models, closed-source APIs, and AI search products; accuracy and calibration error were reported, with regex extraction plus GPT-4o grading for models and human extraction for AI search products.
Limitations
- Stated: the dataset is relatively small, and increasing sample size and question-type diversity would improve representativeness.
- Stated: answer uniqueness cannot be fully guaranteed, and the dynamic web means factual answers may evolve or become inconsistent over time, challenging stability and reproducibility.
- Evident from the text: evaluation depends on GPT-4o grading for model outputs and human annotators for AI search products, which may introduce grader or annotator variability; topic coverage is also uneven, e.g., Film & TV 15.6% versus Academic Papers 2.4%.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
6 of 13 cited sources checked · 4 supported · 0 flagged · accuracy not yet measured
- Study design and conductcitation§ 3.1 Dataset Construction · p. 4
Each task in BrowseComp-ZH begins with a factual answer, from which an elaborate, multi-constraint query is crafted to make direct retrieval non-trivial.
“Inspired by Wei et al. (2025), we adopt a reverse construction strategy: each task begins with a factual answer, from which an elaborate, multi-constraint query is crafted to make direct retrieval non-trivial.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Wei et al. (2025) source
- Findingreasoning§ 4.6 Analysis of AI Search Products · p. 9
Enabling search functionality for DeepSeek-R1 leads to a substantial decline in performance.
“enabling search functionality for DeepSeek-R1 leads to a substantial decline in performance”
Quote in the paperSubstantial claim - Findingown data§ 4.5 Analysis of Reasoning Ability of LLM · p. 8
Doubao (Standard) achieves 18.7% accuracy on BrowseComp-ZH.
“Doubao (Standard) (26.0% vs. 18.7%)”
Quote in the paperSubstantial claim - Findingown data§ 4.6 Analysis of AI Search Products · p. 8
Yuanbao achieves 12.2% accuracy on BrowseComp-ZH.
“Yuanbao (12.2%)”
Quote in the paperNot a substantial claim - Findingown data§ 4.7 Calibration Analysis · p. 9
The calibration error for DeepSeek-R1 increases from 59% in the direct-answer setting to 65% when search is enabled.
“the calibration error for DeepSeek-R1 increases from 59% in the direct-answer setting to 65% when search is enabled”
Quote in the paperNot a substantial claim
- Findingown data§ 4.4 Performance · p. 7
OpenAI's O1 achieved 29.1% accuracy on BrowseComp-ZH.
“OpenAI's O1 (29.1%)”
- Also stated in: § 4.4 Performance · p. 7 · plain
“O1 (29.1%)”
Quote in the paperNot a substantial claim - Also stated in: § 4.4 Performance · p. 7 · plain
- Findingown data§ 4.4 Performance · p. 7
Claude-3.5-Sonnet achieved 5.5% accuracy on BrowseComp-ZH.
“Claude-3.5-Sonnet (5.5%)”
Quote in the paperNot a substantial claim - Findingown data§ 4.5 Analysis of Reasoning Ability of LLM · p. 8
Doubao (Deep Search) achieves nearly 8% absolute improvement over Doubao (Standard).
“Doubao (Deep Search) achieves a nearly 8% absolute improvement over Doubao (Standard)”
Quote in the paperSubstantial claim - Findingown data§ 4.5 Analysis of Reasoning Ability of LLM · p. 8
DeepSeek-V3 achieves 8.7% accuracy on BrowseComp-ZH.
“DeepSeek-V3's 8.7%”
Quote in the paperSubstantial claim - Findingown data§ 4.5 Analysis of Reasoning Ability of LLM · p. 8
Doubao (Deep Search) achieves 26.0% accuracy on BrowseComp-ZH.
“Doubao (Deep Search) achieves a nearly 8% absolute improvement over Doubao (Standard) (26.0% vs. 18.7%)”
- Also stated in: § 4.6 Analysis of AI Search Products · p. 8 · plain
“Doubao (Deep Search) achieving 26.0%”
Quote in the paperSubstantial claim - Also stated in: § 4.6 Analysis of AI Search Products · p. 8 · plain
- Findingown data§ 4.4 Performance · p. 7
AI search products equipped with retrieval mechanisms outperformed other categories on BrowseComp-ZH.
“Overall, AI search products equipped with retrieval mechanisms outperformed other categories”
Quote in the paperSubstantial claim - Findingown data§ 4.7 Calibration Analysis · p. 9
Integrating search functionality results in increased calibration errors.
“integrating search functionality results in increased calibration errors”
Quote in the paperNot a substantial claim - Findingown data§ 4.5 Analysis of Reasoning Ability of LLM · p. 8
DeepSeek-R1 markedly improves upon DeepSeek-V3.
“DeepSeek-R1 achieves an accuracy of 23.2%, markedly improving upon DeepSeek-V3's 8.7%”
Quote in the paperSubstantial claim - Findingown data§ 4.4 Performance · p. 7
Google's Gemini-2.5-Pro achieved 27.3% accuracy on BrowseComp-ZH.
“Google's Gemini-2.5-Pro (27.3%)”
- Also stated in: § 4.4 Performance · p. 7 · plain
“Gemini-2.5-Pro (27.3%)”
Quote in the paperNot a substantial claim - Also stated in: § 4.4 Performance · p. 7 · plain
- Findingreasoning§ 4.5 Analysis of Reasoning Ability of LLM · p. 8
DeepSeek's Deep Search variant does not exhibit a significant performance advantage over its standard counterpart.
“DeepSeek's Deep Search variant does not exhibit a significant performance advantage over its standard counterpart.”
Quote in the paperSubstantial claim- Rests on an assumption: All versions of DeepSeek's search system perform only a single round of retrieval.
- Findingown data§ 4.6 Analysis of AI Search Products · p. 8
DeepSeek achieves 7.6% accuracy on BrowseComp-ZH.
“DeepSeek (7.6%)”
Quote in the paperSubstantial claim - Findingown data§ 4.6 Analysis of AI Search Products · p. 8
Kimi achieves 8.0% accuracy on BrowseComp-ZH.
“Kimi (8.0%)”
Quote in the paperSubstantial claim - Findingnone given§ 1 Introduction · p. 3
Naive large language models, irrespective of parameter scale or training data size, exhibit consistently poor performance on BrowseComp-ZH, with accuracies often remaining below 10%.
“Naive large language models, irrespective of parameter scale or training data size, exhibit consistently poor performance on the benchmark, with accuracies often remaining below 10%.”
Quote in the paperSubstantial claim - Findingown data§ 4.4 Performance · p. 7
DeepSeek R1, O1, and Gemini-2.5-Pro achieved accuracies exceeding 20% on BrowseComp-ZH.
“even models without browsing capabilities, such as DeepSeek R1, O1, and Gemini-2.5-Pro, which rely solely on their internal world knowledge, managed to achieve accuracies exceeding 20%”
Quote in the paperNot a substantial claim- Rests on an assumption: DeepSeek R1, O1, and Gemini-2.5-Pro are models without browsing capabilities and rely solely on internal world knowledge.
- Findingown data§ 4.5 Analysis of Reasoning Ability of LLM · p. 8
Models enhanced with reasoning capabilities consistently demonstrate substantial performance gains across comparisons.
“Across all comparisons, models enhanced with reasoning capabilities consistently demonstrate substantial performance gains.”
Quote in the paperSubstantial claim - Findingown data§ 4.4 Performance · p. 7
Closed-source APIs performed second best on BrowseComp-ZH.
“followed by closed-source APIs”
Quote in the paperNot a substantial claim - Findingown data§ 4.6 Analysis of AI Search Products · p. 8
Perplexity achieves 22.6% accuracy on BrowseComp-ZH.
“Perplexity attaining 22.6%”
Quote in the paperNot a substantial claim - Findingown data§ 4.4 Performance · p. 7
OpenAI's DeepResearch achieved the highest accuracy among all evaluated systems.
“Among all evaluated systems, OpenAI's DeepResearch achieved the highest accuracy (42.9%)”
Quote in the paperNot a substantial claim - Findingown data§ 4.4 Performance · p. 7
GPT-4o achieved 6.2% accuracy on BrowseComp-ZH.
“GPT-4o (6.2%)”
Quote in the paperNot a substantial claim - Findingown data§ 4.4 Performance · p. 7
Qwen2.5-72B-Instruct achieved 6.6% accuracy on BrowseComp-ZH.
“Qwen2.5-72B-Instruct (6.6% accuracy)”
Quote in the paperNot a substantial claim - Findingown data§ Abstract · p. 1
The best-performing system, OpenAI's DeepResearch, reaches just 42.9% accuracy on BrowseComp-ZH.
“Even the best-performing system, OpenAI's DeepResearch, reaches just 42.9%.”
Quote in the paperSubstantial claim - Findingown data§ 4.5 Analysis of Reasoning Ability of LLM · p. 8
DeepSeek-R1 achieves 23.2% accuracy on BrowseComp-ZH.
“DeepSeek-R1 achieves an accuracy of 23.2%”
Quote in the paperSubstantial claim - Findingown data§ 4.4 Performance · p. 7
Open-source models performed the worst on BrowseComp-ZH.
“while open-source models performed the worst on this challenging benchmark”
Quote in the paperNot a substantial claim - Findingown data§ 4.6 Analysis of AI Search Products · p. 9
DeepSeek-R1 accuracy drops from 23.2% in the direct-answer setting to 7.6% when web search is enabled.
“accuracy dropping from 23.2% in the direct-answer (no web access) setting to 7.6% when web search is enabled”
Quote in the paperSubstantial claim - Findingown data§ 4.4 Performance · p. 7
OpenAI's DeepResearch achieved 42.9% accuracy on BrowseComp-ZH.
“OpenAI's DeepResearch achieved the highest accuracy (42.9%)”
- Also stated in: § 4.6 Analysis of AI Search Products · p. 8 · plain
“DeepResearch reaching 42.9%”
Quote in the paperSubstantial claim - Also stated in: § 4.6 Analysis of AI Search Products · p. 8 · plain
- Findingown data§ 4.6 Analysis of AI Search Products · p. 8
Systems employing multi-round retrieval achieve significantly higher accuracy than single-retrieval systems on BrowseComp-ZH.
“Statistical analysis shows that systems employing multi-round retrieval achieve significantly higher accuracy”
Quote in the paperSubstantial claim - Findingown data§ 5 Conclusion and Discussion · p. 9
AI search products employing multi-turn retrieval outperform purely parametric models on BrowseComp-ZH.
“AI search products employing multi-turn retrieval, including DeepResearch (42.9%) and Doubao (Deep Search) (26.0%), further outperform purely parametric models”
Quote in the paperSubstantial claim - Findingown data§ 5 Conclusion and Discussion · p. 9
Most standalone LLMs achieve limited accuracy on BrowseComp-ZH.
“most standalone LLMs-such as GPT-4o, Qwen2.5-72B, and Llama-4-achieve limited accuracy”
Quote in the paperSubstantial claim - Findingown data§ 5 Conclusion and Discussion · p. 9
Models with stronger reasoning abilities demonstrate substantial improvements on BrowseComp-ZH.
“Models with stronger reasoning abilities, such as O1 (29.1%) and Gemini-2.5-Pro (27.3%), demonstrate substantial improvements”
Quote in the paperSubstantial claim - Premisenone given§ 1 Introduction · p. 2
Translated queries often collapse into keyword matches, trivializing the intended challenge.
“translated queries often collapse into keyword matches, trivializing the intended challenge”
Quote in the paperNot a substantial claim - Premisenone given§ 1 Introduction · p. 2
Structural and idiomatic differences render many translated queries unnatural or ineffective in actual search.
“structural and idiomatic differences render many translated queries unnatural or ineffective in actual search”
Quote in the paperNot a substantial claim - Premisereasoning§ 1 Introduction · p. 2
Directly translating English browsing benchmarks into Chinese does not yield a meaningful evaluation framework.
“Crucially, directly translating English browsing benchmarks into Chinese does not yield a meaningful evaluation framework .”
Quote in the paperSubstantial claim - Interpretationown data§ 4.4 Performance · p. 7
The BrowseComp-ZH benchmark is effective in differentiating models across a wide range of performance levels.
“These results demonstrate the effectiveness of our benchmark in differentiating models across a wide range of performance levels.”
Quote in the paperSubstantial claim - Study design and conductnone given§ Abstract · p. 1
Each question in BrowseComp-ZH is reverse-engineered from a short, objective, and easily verifiable answer (e.g., a date, number, or proper noun).
“Each question is reverse-engineered from a short, objective, and easily verifiable answer (e.g., a date, number, or proper noun).”
Quote in the paperSubstantial claim - Interpretationown data§ 4.6 Analysis of AI Search Products · p. 9
Integrating retrieval capabilities without robust post-retrieval reasoning and alignment strategies may, in some cases, hinder rather than enhance model performance.
“integrating retrieval capabilities without robust post-retrieval reasoning and alignment strategies may, in some cases, hinder rather than enhance model performance”
Quote in the paperSubstantial claim - Premisereasoning§ 1 Introduction · p. 2
There remains a surprising lack of direct evaluation of browsing capabilities themselves-i.e., the ability of LLMs to effectively retrieve, filter, and reason over information from the web.
“While an increasing number of studies Li et al. (2023a); Fernández-Pichel et al. (2024); Fan et al. (2024); Vu et al. (2023); Lai et al. (2025); He et al. (2024); Lai et al. (2024); Xiong et al. (2024) demonstrate that web browsing greatly improves LLM performance on downstream tasks, there remains a surprising lack of direct evaluation of browsing capabilities themselves-i.e., the ability of LLMs”
Quote in the paperSubstantial claim - Interpretationown data§ 4.4 Performance · p. 7
The high accuracies of DeepSeek R1, O1, and Gemini-2.5-Pro demonstrate the strength of high-capacity language models.
“demonstrating the strength of high-capacity language models”
Quote in the paperSubstantial claim - Interpretationown data§ 4.5 Analysis of Reasoning Ability of LLM · p. 8
Enhanced reasoning facilitates more accurate and iterative retrieval for complex queries.
“highlighting the benefits of enhanced reasoning in facilitating more accurate and iterative retrieval for complex queries”
Quote in the paperSubstantial claim - Interpretationown data§ 4.4 Performance · p. 7
The BrowseComp-ZH dataset is challenging.
“highlighting the challenging nature of the dataset we proposed”
Quote in the paperSubstantial claim - Interpretationown data§ 4.6 Analysis of AI Search Products · p. 9
Reconciling retrieved evidence with internal representations remains non-trivial for large language models.
“effectively reconciling retrieved evidence with internal representations remains non-trivial”
Quote in the paperSubstantial claim - Interpretationown data§ 4.4 Performance · p. 7
Qwen2.5-72B-Instruct, GPT-4o, and Claude-3.5-Sonnet exhibited limited performance, consistently achieving low accuracy scores across tasks.
“several systems, such as Qwen2.5-72B-Instruct (6.6% accuracy), GPT-4o (6.2%), and Claude-3.5-Sonnet (5.5%), exhibited limited performance, consistently achieving low accuracy scores across tasks”
Quote in the paperSubstantial claim - Interpretationreasoning§ 4.5 Analysis of Reasoning Ability of LLM · p. 8
The single-round retrieval of DeepSeek's search system limits the extent to which improved reasoning capabilities can be leveraged.
“limiting the extent to which improved reasoning capabilities can be leveraged”
Quote in the paperSubstantial claim- Rests on an assumption: All versions of DeepSeek's search system perform only a single round of retrieval.
- Interpretationown data§ 5 Conclusion and Discussion · p. 9
The limited accuracy of standalone LLMs highlights the difficulty of the benchmark.
“highlighting the difficulty of the benchmark”
Quote in the paperSubstantial claim - Interpretationown data§ 5 Conclusion and Discussion · p. 9
Test-time scaling through iterative retrieval is effective.
“illustrating the effectiveness of test-time scaling through iterative retrieval”
Quote in the paperSubstantial claim - Interpretationown data§ 5 Conclusion and Discussion · p. 9
Reasoning plays a critical role for complex question answering.
“underscoring the critical role of reasoning for complex question answering”
Quote in the paperSubstantial claim - Study design and conductreasoning§ 5 Conclusion and Discussion · p. 9
Answer uniqueness in BrowseComp-ZH cannot be fully guaranteed despite rigorous validation.
“although we apply rigorous validation to ensure answer uniqueness, it cannot be fully guaranteed”
Quote in the paperNot a substantial claim
- May combine two claims. 34
- Classified as probably not a substantial claim. 27
May combine two claims.
Premisenone given§ 1 Introduction · p. 2Wei et al. (2025) primarily operates within the English-language web, missing the linguistic, structural, and cultural complexities inherent to other language environments.
“Nonetheless, Wei et al. (2025) primarily operates within the English-language web, missing the linguistic, structural, and cultural complexities inherent to other language environments.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Wei et al. (2025) source
The Introduction and Related work and discussion describe BrowseComp as a benchmark for finding hard-to-find information on the web, but do not specify that it primarily uses the English-language web. Searches for English, non-English, multilingual, and cultural or linguistic limitations found no passage assessing its coverage of other language environments. (LLM, conf=0.73)
Experimental — accuracy not yet measured
May combine two claims.
Premisenone given§ 2 Related Work · p. 4Liu et al. (2023) and Li et al. (2023b) focus on domain-specific medical QA, but do not evaluate open-ended retrieval or browsing strategies.
“Liu et al. (2023) and Li et al. (2023b) focus on domain-specific medical QA, but do not evaluate open-ended retrieval or browsing strategies.”
Quote in the paperSubstantial claim1 of 2 sources checked- cites Liu et al. (2023) source
- cites Li et al. (2023b)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Premisecitation§ 2 Related Work · p. 4Wei et al. (2025) remains confined to the English web and lacks generalizability to non-English environments with fragmented platforms and diverse linguistic structures.
“While it offers finer-grained evaluation for web browsing agents, it remains confined to the English web and lacks generalizability to non-English environments with fragmented platforms and diverse linguistic structures.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Wei et al. (2025) source
The Introduction and Section 5 describe BrowseComp as a benchmark for agents searching the web, but do not state that it is confined to the English web. Searches for English-only coverage, multilingual evaluation, and non-English generalizability found no passage addressing fragmented platforms or diverse linguistic structures. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
Premisecitation§ 1 Introduction · p. 2Wei et al. (2025) introduced a benchmark of reverse-designed, evidence-grounded queries that challenge English-language agents to search and reason over difficult-to-access information.
“To address this gap, Wei et al. (2025) introduced a benchmark of reverse-designed, evidence-grounded queries that challenge English-language agents to search and reason over difficult-to-access information.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Wei et al. (2025) source
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitation§ 4.2 Setting · p. 7The grading prompts used in BrowseComp-ZH are adapted from those used in the original BrowseComp benchmark.
“The grading prompts are adapted from those used in the original BrowseComp benchmark Wei et al. (2025).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Wei et al. (2025) source
May combine two claims.
Findingown data§ 1 Introduction · p. 3Not all systems benefit from retrieval integration; enabling web search for DeepSeek-R1 results in a substantial decline in performance, with accuracy dropping from 23.2% in the direct-answer (no web access) setting to 7.6% when web search is enabled.
“While well-designed retrieval pipelines can significantly improve performance, not all systems benefit from retrieval integration. For instance, enabling web search for DeepSeekR1 results in a substantial decline in performance, with accuracy dropping from 23.2% in the direct-answer (no web access) setting to 7.6% when web search is enabled.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ Abstract · p. 1BrowseComp-ZH consists of 289 multi-hop questions spanning 11 diverse domains.
“BrowseComp-ZH consists of 289 multi-hop questions spanning 11 diverse domains.”
- Also stated in: § 1 Introduction · p. 3 · plain
“The final BrowseComp-ZH dataset consists of 289 complex questions, each with multiple constraints and unique answers, spanning 11 diverse domains, including film, technology, law, and medicine.”
Quote in the paperSubstantial claim- Also stated in: § 1 Introduction · p. 3 · plain
May combine two claims.
Findingreasoning§ 3.2 Quality Control · p. 5This process eliminates 115 ambiguous samples, resulting in a final benchmark of 289 validated questions that maintains high levels of difficulty and answer verifiability.
“This process eliminates 115 ambiguous samples, resulting in a final benchmark of 289 validated questions that maintains high levels of difficulty and answer verifiability.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 3.3 Data Statistics · p. 5The most represented categories in BrowseComp-ZH include Film & TV (15.6%), Art (13.8%), and Geography (12.8%); the dataset also features Music (11.1%), History (10.0%), and Medicine (9.0%); topics like Policy & Law (3.5%) and Academic Papers (2.4%) have fewer samples.
“As illustrated, the most represented categories include Film & TV (15.6%), Art (13.8%), and Geography (12.8%), reflecting the diverse interests of annotators and a broad coverage of Chinese web content. The dataset also features Music (11.1%), History (10.0%), and Medicine (9.0%). Conversely, topics like Policy & Law (3.5%) and Academic Papers (2.4%) have fewer samples”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 1 Introduction · p. 3Models endowed with inherent reasoning capabilities exhibit consistent improvements in accuracy on BrowseComp-ZH; for example, DeepSeek-R1 surpasses DeepSeek-V3 by 14.5%, while Claude-3.7-Sonnet outperforms Claude-3.5 by 12.2%.
“Models endowed with inherent reasoning capabilities exhibit consistent improvements in accuracy. For example, DeepSeek-R1 surpasses DeepSeek-V3 by 14.5%, while Claude-3.7Sonnet outperforms Claude-3.5 by 12.2%.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ Abstract · p. 1Most models struggle severely on BrowseComp-ZH: a large number achieve accuracy rates below 10%, and only a handful exceed 20%.
“Despite their strong conversational and retrieval capabilities, most models struggle severely: a large number achieve accuracy rates below 10%, and only a handful exceed 20%.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 3.2 Quality Control · p. 5In this stage, annotators identified 76 simple samples, and after filtering them out, 404 high-difficulty candidates remained.
“In this stage, annotators identified 76 simple samples, and after filtering them out, we are left with 404 high-difficulty candidates.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ 3.1 Dataset Construction · p. 5This process yields 480 preliminary samples.
“This process yields 480 preliminary samples, which are subsequently reviewed via the quality control procedure detailed in Section 3.2.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 1 Introduction · p. 3Agentic systems augmented with external retrieval mechanisms tend to achieve comparatively higher scores on BrowseComp-ZH; OpenAI's DeepResearch and Doubao (Deep Search) achieve accuracies of 42.9% and 26.0%, respectively.
“Agentic systems augmented with external retrieval mechanisms tend to achieve comparatively higher scores. Notably, OpenAI's DeepResearch and Doubao (Deep Search) demonstrate that well-orchestrated retrieval and reasoning pipelines can substantially enhance performance, achieving accuracies of 42.9% and 26.0%, respectively.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 3.3 Data Statistics · p. 6Answer length in BrowseComp-ZH generally falls within the range of 5 to 10 characters.
“In Fig. 3 (b), the answer length distribution shows that answers are generally short, typically falling within the range of 5 to 10 characters.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingnone given§ 5 Conclusion and Discussion · p. 9The question-answer pairs in BrowseComp-ZH require multi-hop retrieval, information filtering, and logical reasoning to derive concise, factual answers.
“we construct challenging question-answer pairs that require multi-hop retrieval, information filtering, and logical reasoning to derive concise, factual answers”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Findingnone given§ 5 Conclusion and Discussion · p. 9The current BrowseComp-ZH dataset is relatively small.
“the current dataset is relatively small”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 5 Conclusion and Discussion · p. 9BrowseComp-ZH contains 289 high-quality samples across 11 diverse topics.
“We construct 289 high-quality samples across 11 diverse topics”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Premisenone given§ 1 Introduction · p. 2Canonical information pathways in English (e.g., Wikipedia, IMDb) lack equivalents or exhibit vastly different structures in the Chinese web.
“canonical information pathways in English (e.g., Wikipedia, IMDb) lack equivalents or exhibit vastly different structures in the Chinese web”
Quote in the paperNot a substantial claimMay combine two claims.
Premisereasoning§ 1 Introduction · p. 2The Chinese web poses unique challenges for retrieval and reasoning, including information scattered across heterogeneous platforms, inconsistent naming conventions, unreliable indexing of deep or domain-specific content, and linguistic properties that break standard keyword-based retrieval paths.
“Moreover, the Chinese web presents its own set of retrieval obstacles-fragmented and platform-specific content, inconsistent naming conventions, and linguistic traits such as ellipsis, cultural references, and implicit reasoning that undermine linear search strategies.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.
Interpretationreasoning§ 3.3 Data Statistics · p. 6These length distributions reflect the high information density of the dataset, emphasizing both complexity in the questions and simplicity in the answers.
“These length distributions reflect the high information density of the dataset, emphasizing both complexity in the questions and simplicity in the answers.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ 1 Introduction · p. 3These findings suggest that BrowseComp-ZH not only challenges models' browsing and reasoning capabilities but also exposes their limitations in processing, evaluating, and aligning retrieved information with internal representations.
“These findings suggest that BrowseComp-ZH not only challenges models' browsing and reasoning capabilities but also exposes their limitations in processing, evaluating, and aligning retrieved information with internal representations.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone given§ 3.1 Dataset Construction · p. 4In Stage 2, each question combines temporal, spatial, categorical, or descriptive conditions to ensure answer uniqueness; at least two dimensions are typically combined.
“Each question combines temporal, spatial, categorical, or descriptive conditions to ensure answer uniqueness; not all conditions are required, but at least two dimensions are typically combined”
Quote in the paperSubstantial claimMay combine two claims.
Premisecitation§ 2 Related Work · p. 4Lyu et al. (2025) evaluates RAG systems under the CRUD paradigm, yet primarily emphasizes generation quality over retrieval path validation.
“Lyu et al. (2025) evaluates RAG systems under the CRUD (Create, Read, Update, Delete) paradigm, yet primarily emphasizes generation quality over retrieval path validation.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Lyu et al. (2025)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S4 (based).
Premisenone given§ 1 Introduction · p. 2BrowseComp-ZH is the first benchmark specifically designed to evaluate web-enabled LLMs in the Chinese information environment.
“To fill this gap, we introduce BrowseComp-ZH , the first benchmark specifically designed to evaluate web-enabled LLMs in the Chinese information environment.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown data§ Abstract · p. 1The results demonstrate the considerable difficulty of BrowseComp-ZH, where success demands not only effective retrieval strategies, but also sophisticated reasoning and information reconciliation capabilities that current models still struggle to master.
“These results demonstrate the considerable difficulty of BrowseComp-ZH, where success demands not only effective retrieval strategies, but also sophisticated reasoning and information reconciliation-capabilities that current models still struggle to master.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 3.1 Dataset Construction · p. 4To ensure high-quality annotation, 10 expert contributors (with Master's or PhD degrees) who have extensive experience in both LLM usage and web search were recruited.
“To ensure high-quality annotation, we recruited 10 expert contributors (with Master's or PhD degrees) who have extensive experience in both LLM usage and web search.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone given§ 3.1 Dataset Construction · p. 4In Stage 1, selected facts must be objective statements that can be independently verified through reliable sources, without the need for interpretation or inference.
“The selected facts must be objective statements that can be independently verified through reliable sources, without the need for interpretation or inference”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductnone given§ 1 Introduction · p. 2Every candidate query in BrowseComp-ZH is verified on Baidu, Bing, and Google, and is accepted only if the answer is not surfaced in the first-page results of any engine.
“Every candidate query is verified on three widely-used search engines: Baidu 1 , Bing 2 , and Google 3 , and is accepted only if the answer is not surfaced in the first-page results of any engine.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone given§ 3.1 Dataset Construction · p. 4Annotators test each query on Baidu, Bing, and Google, using three distinct keyword combinations per search engine; if the correct answer appears on the first page of any search engine, the query is revised or further constrained.
“Annotators test each query on Baidu, Bing, and Google, using three distinct keyword combinations per search engine. If the correct answer appears on the first page of any search engine, the query will be revised or further constrained”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 1 Introduction · p. 2Stage 1 of BrowseComp-ZH quality control screens for keyword retrievability.
“Stage 1 screens for keyword retrievability as above.”
Quote in the paperSubstantial claimMay combine two claims.
Premisenone given§ 2 Related Work · p. 4Hu et al. (2024) evaluates Chinese web search agents but lacks rigorous control over task difficulty and answer accessibility.
“Hu et al. (2024) evaluates Chinese web search agents but lacks rigorous control over task difficulty and answer accessibility.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Hu et al. (2024)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim. May combine two claims.
Interpretationreasoning§ 1 Introduction · p. 3These questions require multi-step reasoning and are difficult to answer directly through search engines, making them ideal for evaluating the ability of Chinese LLM agents to conduct multi-hop retrieval, perform factual reasoning, and integrate online information.
“These questions require multi-step reasoning and are difficult to answer directly through search engines, making them ideal for evaluating the ability of Chinese LLM agents to conduct multihop retrieval, perform factual reasoning, and integrate online information.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductnone given§ 1 Introduction · p. 2All queries, evidence chains, and browsing steps in BrowseComp-ZH are authored in Chinese and fully localized.
“All queries, evidence chains, and browsing steps are authored in Chinese and fully localized.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ 2 Related Work · p. 4BrowseComp-ZH establishes a high-difficulty benchmark for Chinese web retrieval, supporting systematic evaluation of agents' ability to navigate fragmented, unstructured, and linguistically diverse Chinese information sources.
“It establishes a high-difficulty benchmark for Chinese web retrieval, supporting systematic evaluation of agents' ability to navigate fragmented, unstructured, and linguistically diverse Chinese information sources.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ 4.6 Analysis of AI Search Products · p. 8Tasks in BrowseComp-ZH often involve multi-faceted queries, making it challenging to obtain accurate answers through a single retrieval operation.
“This trend aligns with the nature of tasks in BrowseComp-ZH, which often involve multi-faceted queries, making it challenging to obtain accurate answers through a single retrieval operation.”
Quote in the paperSubstantial claimMay combine two claims.
Premisecitation§ 2 Related Work · p. 4Xu et al. (2024) focuses on dynamic, time-sensitive QA tasks but does not emphasize multi-hop retrieval.
“Xu et al. (2024) focuses on dynamic, time-sensitive QA tasks but does not emphasize multi-hop retrieval.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Xu et al. (2024)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Study design and conductown data§ 1 Introduction · p. 2BrowseComp-ZH employs a human-in-the-loop approach where multiple top-performing AI agents first perform reasoning and provide answers based on the designed questions, and these results are then manually verified by annotators to check if any additional answers satisfy the constraints of the question.
“We employ a human-in-the-loop approach, where multiple top-performing AI agents first perform reasoning and provide answers based on the designed questions. These results are then manually verified by annotators to check if any additional answers satisfy the constraints of the question.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoning§ 3.1 Dataset Construction · p. 5All questions are tested using GPT-4o and DeepSeek, both operating in web-enabled search mode; if both models consistently retrieve the correct answer with minimal effort, the query is revised to increase its complexity.
“To further discourage shortcut-based resolution, all questions are tested using GPT-4o and DeepSeek, both operating in web-enabled search mode. If both models consistently retrieve the correct answer with minimal effort, the query will be revised to increase its complexity by methods including introducing implicit constraints or obfuscating key lexical signals.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone given§ 3.1 Dataset Construction · p. 4In Stage 1, selected facts must be concrete and specific enough to exclude overly generic or widely known common-sense facts.
“they must be concrete and specific enough to exclude overly generic or widely known common-sense facts.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ Abstract · p. 1A two-stage quality control protocol is applied to strive for high question difficulty and answer uniqueness.
“A two-stage quality control protocol is applied to strive for high question difficulty and answer uniqueness.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductreasoning§ 2 Related Work · p. 4BrowseComp-ZH introduces three key innovations: reverse-designed tasks in native Chinese to avoid translation artifacts; rigorous multi-step validation to ensure high retrieval difficulty and answer verifiability; and broad model coverage for evaluating both open-source and proprietary agents.
“To address these limitations in Chinese retrieval benchmarking, BrowseComp-ZH introduces three key innovations: (1) reverse-designed tasks in native Chinese to avoid translation artifacts; (2) rigorous multi-step validation to ensure high retrieval difficulty and answer verifiability; and (3) broad model coverage for evaluating both open-source and proprietary agents.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 3.1 Dataset Construction · p. 5Each sample includes at least one authoritative source URL that validates the logical connection between the query constraints and the target answer.
“Each sample includes at least one authoritative source URL that validates the logical connection between the query constraints and the target answer.”
Quote in the paperSubstantial claimMay combine two claims.
Premisereasoning§ 2 Related Work · p. 4Many answers in existing English benchmarks can be retrieved via simple keyword searches, so these benchmarks often fail to evaluate an agent's ability to plan complex search trajectories and synthesize information across documents.
“However, since many answers can be retrieved via simple keyword searches, these benchmarks often fail to evaluate an agent's ability to plan complex search trajectories and synthesize information across documents.”
Quote in the paperNot a substantial claimMay combine two claims.
Premisereasoning§ 1 Introduction · p. 2Existing English-centric benchmarks struggle to generalize, leaving a critical gap in assessing real-world browsing capabilities in non-English environments.
“As a result, existing English-centric benchmarks struggle to generalize, leaving a critical gap in assessing real-world browsing capabilities in non-English environments.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 1 Introduction · p. 2Stage 2 of BrowseComp-ZH quality control focuses on ensuring the uniqueness of the answers.
“Stage 2 focuses on ensuring the uniqueness of the answers.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone given§ 5 Conclusion and Discussion · p. 9BrowseComp-ZH is the first benchmark specifically designed to evaluate the web browsing and reasoning capabilities of large language models in the Chinese information environment.
“This study introduces BrowseComp-ZH, the first benchmark specifically designed to evaluate the web browsing and reasoning capabilities of large language models (LLMs) in the Chinese information environment.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 4.7 Calibration Analysis · p. 9The trend of increased calibration errors with search is consistent with observations reported in BrowseComp.
“This trend is consistent with the observations reported in BrowseComp.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites BrowseComp
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Study design and conductcitation§ A.1 Instructions for model prediction · p. 13In the evaluation, the authors followed the BrowseComp instructions for model predictions.
“In the evaluation, we followed the BrowseComp instructions for model predictions.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites BrowseComp
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Interpretationnone given§ 5 Conclusion and Discussion · p. 9The Chinese web environment has fragmented information and inconsistent indexing that complicate single-shot search strategies.
“These results reflect the inherent challenges of the Chinese web environment, where fragmented information and inconsistent indexing complicate single-shot search strategies.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ 5 Conclusion and Discussion · p. 9The two-stage quality control pipeline includes a three-engine keyword validation process and human-in-the-loop verification.
“we implement a rigorous two-stage quality control pipeline that includes a three-engine keyword validation process and human-in-the-loop verification”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ 5 Conclusion and Discussion · p. 9Increasing the sample size and diversity of question types would improve the representativeness of BrowseComp-ZH.
“increasing both the sample size and the diversity of question types would improve its representativeness”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductreasoning§ 5 Conclusion and Discussion · p. 9The quality control pipeline ensures that answers are both difficult to retrieve and unambiguous.
“ensuring that answers are both difficult to retrieve and unambiguous”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductcitation§ A.2 Instruction for grading · p. 13The authors adopt the same grading prompt as used in BrowseComp and employ GPT-4o for the grading process.
“We adopt the same grading prompt as used in BrowseComp and employ GPT-4o for the grading process.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites BrowseComp
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
The paper relies on this without support.
All versions of DeepSeek's search system perform only a single round of retrieval.
“all versions of DeepSeek's search system perform only a single round of retrieval”
The paper relies on this without support.
The dynamic nature of the web means that factual answers may evolve or become inconsistent over time.
“the dynamic nature of the web means that factual answers may evolve or become inconsistent over time”
The paper relies on this without support.
DeepSeek R1, O1, and Gemini-2.5-Pro are models without browsing capabilities and rely solely on internal world knowledge.
“even models without browsing capabilities, such as DeepSeek R1, O1, and Gemini-2.5-Pro, which rely solely on their internal world knowledge”
The paper relies on this without support.
The tasks in BrowseComp-ZH require both extensive world knowledge and strong reasoning abilities.
“the tasks in our proposed BrowseComp-ZH benchmark require both extensive world knowledge and strong reasoning abilities”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing benchmarks for deep search agents, such as BrowseComp, rely on black-box web search APIs to retrieve supporting documents in real time.
“These benchmarks typically evaluate agents using blackbox web search APIs to retrieve supporting documents in real time (Zhou et al., 2025; Chen et al., 2025).”
From BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents (2026) · cites this paper as Zhou et al., 2025
The Abstract (p. 1) frames real-time web browsing as the capability being evaluated, and §4.6 (p. 8) describes evaluated search systems performing retrieval. But §4.2 (p. 7) says the authors evaluated AI search products through human GUI interactions, not black-box web search APIs; the paper does not establish that benchmarks typically use such APIs to retrieve supporting documents. (LLM, conf=0.85)
Experimental — accuracy not yet measuredCurrent benchmarks typically focus on isolated capabilities such as web browsing and information retrieval, or generative abilities disconnected from real-time information acquisition.
“Current benchmarks typically focus on assessing isolated capabilities-such as web browsing and information retrieval [28, 38, 39], or generative abilities disconnected from real-time information acquisition [23, 1, 31].”
From DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents (2025) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Grok3 accuracy is incompatible with 289 questions
MinorAn accuracy of 12.9% cannot result from an integer number of correct answers out of 289 questions. The nearest counts, 37 and 38, yield 12.8% and 13.1% when rounded to one decimal place.
“| Grok3 (Research) | AI Search Product | - | Y | 12.9% | 39 | xAI |”
“resulting in a final benchmark of 289 validated questions”
Yuanbao accuracy is incompatible with 289 questions
MinorAn accuracy of 12.2% cannot result from an integer number of correct answers out of 289 questions. The nearest counts, 35 and 36, round to 12.1% and 12.5%.
“| Yuanbao (Hunyuan Model) | AI Search Product | - | Y | 12.2% | 56 | Tencent |”
“resulting in a final benchmark of 289 validated questions”
Cites
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 1× · see citing works
“In parallel, retrieval-augmented generation (RAG) frameworks Guu et al. (2020); Lewis et al. (2020) have been widely adopted in QA Wiratunga et al. (2024), summarization Edge et al. (2024), and fact verification Martin et al. (2024), serving as a mechanism for injecting external knowledge into LLMs.”
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · cited 1× · see citing works
“To assess retrieval capabilities, a variety of widely used English benchmarks have been proposed, including TriviaQA Joshi et al. (2017), HotpotQA Yang et al. (2018), FEVER Thorne et al. (2018), KILT Petroni et al. (2020), and GAIA Mialon et al. (2023).”
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · cited 1× · see citing works
“To assess retrieval capabilities, a variety of widely used English benchmarks have been proposed, including TriviaQA Joshi et al. (2017), HotpotQA Yang et al. (2018), FEVER Thorne et al. (2018), KILT Petroni et al. (2020), and GAIA Mialon et al. (2023).”
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D. S., & Zettlemoyer, L. (2017) · arXiv · doi:10.48550/arxiv.1705.03551 · Cited by 1,650
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343 · cited 1× · see citing works
“To assess retrieval capabilities, a variety of widely used English benchmarks have been proposed, including TriviaQA Joshi et al. (2017), HotpotQA Yang et al. (2018), FEVER Thorne et al. (2018), KILT Petroni et al. (2020), and GAIA Mialon et al. (2023).”
Toolformer: Language Models Can Teach Themselves to Use Tools
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2302.04761 · Cited by 1,150 · cited 1× · see citing works
“Representative works such as WebGPT Nakano et al. (2021), Toolformer Schick et al. (2023), and ReAct Yao et al. (2023) explore how LLMs leverage search engines, tool usage, and reasoning strategies to tackle complex question-answering tasks.”
Toolformer: language models can teach themselves to use tools
Schick, T., Dwivedi-Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2997 · Cited by 1,150 · see citing works
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · cited 1× · see citing works
“As large language models (LLMs) evolve from static knowledge repositories to dynamic agents capable of using external tools, tasks that involve web browsing have emerged as a critical lens through which to evaluate their real-world reasoning and information-seeking capabilities Xi et al. (2025).”
KILT: a Benchmark for Knowledge Intensive Language Tasks
Petroni, F., Piktus, A., Fan, A., Lewis, P., Yazdani, M., Cao, N. D., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., et al. (2020) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2009.02252 · Cited by 940
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671470 · Cited by 827 · see citing works
ReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608 · cited 1× · see citing works
“Representative works such as WebGPT Nakano et al. (2021), Toolformer Schick et al. (2023), and ReAct Yao et al. (2023) explore how LLMs leverage search engines, tool usage, and reasoning strategies to tackle complex question-answering tasks.”
REALM: Retrieval-Augmented Language Model Pre-Training
Guu, K., Lee, K., Tung, Z., Pasupat, P., & Chang, M.-W. (2020) · arXiv · doi:10.48550/arxiv.2002.08909 · Cited by 574 · cited 1× · see citing works
“In parallel, retrieval-augmented generation (RAG) frameworks Guu et al. (2020); Lewis et al. (2020) have been widely adopted in QA Wiratunga et al. (2024), summarization Edge et al. (2024), and fact verification Martin et al. (2024), serving as a mechanism for injecting external knowledge into LLMs.”
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.12948 · Cited by 538 · see citing works
Self-citationKILT: a Benchmark for Knowledge Intensive Language Tasks
Petroni, F., Piktus, A., Fan, A., Lewis, P., Yazdani, M., De Cao, N., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.200 · Cited by 357 · cited 1× · see citing works
“To assess retrieval capabilities, a variety of widely used English benchmarks have been proposed, including TriviaQA Joshi et al. (2017), HotpotQA Yang et al. (2018), FEVER Thorne et al. (2018), KILT Petroni et al. (2020), and GAIA Mialon et al. (2023).”
DeepSeek-AI, Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.19437 · Cited by 294 · see citing works
Self-citationWebGPT: Browser-assisted question-answering with human feedback
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.09332 · Cited by 272 · cited 1× · see citing works
“Representative works such as WebGPT Nakano et al. (2021), Toolformer Schick et al. (2023), and ReAct Yao et al. (2023) explore how LLMs leverage search engines, tool usage, and reasoning strategies to tackle complex question-answering tasks.”
Wiratunga, N., Abeyratne, R., Jayawardena, L., Martin, K., Massie, S., Nkisi-Orji, I., Weerasinghe, R., Liret, A., & Fleisch, B. (2024) · International Conference on Case-Based Reasoning · doi:10.48550/arxiv.2404.04302 · Cited by 168
From local to global: a graph rag approach to query-focused summarization
Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., & Larson, J. (2024) · arXiv · doi:10.48550/arxiv.2404.16130 · Cited by 159 · cited 1× · see citing works
“In parallel, retrieval-augmented generation (RAG) frameworks Guu et al. (2020); Lewis et al. (2020) have been widely adopted in QA Wiratunga et al. (2024), summarization Edge et al. (2024), and fact verification Martin et al. (2024), serving as a mechanism for injecting external knowledge into LLMs.”
Lyu, Y., li, Z., Niu, S., Xiong, F., Tang, B., Wang, W., Wu, H., Liu, H., Xu, T., & Chen, E. (2024) · ACM Transactions on Information Systems · doi:10.1145/3701228 · Cited by 144 · see citing works
Wiratunga, N., Abeyratne, R., Jayawardena, L., Martin, K., Massie, S., Nkisi-Orji, I., Weerasinghe, R., Liret, A., & Fleish, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-63646-2_29 · Cited by 136 · see citing works
Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.15115 · Cited by 128 · see citing works
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., & Manning, C. D. (2018) · arXiv · doi:10.48550/arxiv.1809.09600 · Cited by 97 · see citing works
When Search Engine Services Meet Large Language Models: Visions and Challenges
Xiong, H., Bian, J., Li, Y., Li, X., Du, M., Wang, S., Yin, D., & Helal, S. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3451185 · Cited by 82 · cited 1× · see citing works
“…Li et al. (2023a); Fernández-Pichel et al. (2024); Fan et al. (2024); Vu et al. (2023); Lai et al. (2025); He et al. (2024); Lai et al. (2024); Xiong et al. (2024) demonstrate that web browsing greatly improves LLM performance on downstream tasks, there remains a surprising lack of direct…”
AutoWebGLM: A Large Language Model-based Web Navigating Agent
Lai, H., Liu, X., Iong, I. L., Yao, S., Chen, Y., Shen, P., Yu, H., Zhang, H., Zhang, X., Dong, Y., et al. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671620 · Cited by 69 · cited 1× · see citing works
“…number of studies Li et al. (2023a); Fernández-Pichel et al. (2024); Fan et al. (2024); Vu et al. (2023); Lai et al. (2025); He et al. (2024); Lai et al. (2024); Xiong et al. (2024) demonstrate that web browsing greatly improves LLM performance on downstream tasks, there remains a surprising…”
Benchmarking Large Language Models on CMExam - A Comprehensive Chinese Medical Exam Dataset
Liu, J., Zhou, P., Hua, Y., Chong, D., Tian, Z., Liu, A., Wang, H., You, C., Guo, Z., Zhu, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.03030 · Cited by 36 · see citing works
Self-citationLet LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmark
Xu, Z., Li, Y.-H., Ding, R.-X., et al. (2024) · International Conference on Computational Linguistics · doi:10.48550/arxiv.2402.19248 · Cited by 30
GAIA: a benchmark for General AI Assistants
Mialon, G., Fourrier, C., Swift, C., Wolf, T., LeCun, Y., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2311.12983 · Cited by 12 · see citing works
Huatuo-26M, a Large-scale Chinese Medical QA DatasetPreprint
Li, J., Wang, X., Wu, X., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.01526 · Cited by 12 · see citing works
The Web Can Be Your Oyster for Improving Large Language ModelsPreprint
Li, J., Tang, T., Zhao, W., Wang, J., Nie, J.-Y., & Wen, J.-R. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2305.10998 · Cited by 10 · see citing works
Fernández-Pichel, M., Pichel, J. C., & Losada, D. (2024) · arXiv.org · doi:10.48550/arxiv.2407.12468 · Cited by 8
FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation
Vu, T., Iyyer, M., Wang, X., Constant, N., Wei, J., Wei, J., Tar, C., Sung, Y.-H., Zhou, D., Le, Q., et al. (2023) · arXiv · Cited by 7 · cited 1× · see citing works
“While an increasing number of studies Li et al. (2023a); Fernández-Pichel et al. (2024); Fan et al. (2024); Vu et al. (2023); Lai et al. (2025); He et al. (2024); Lai et al. (2024); Xiong et al. (2024) demonstrate that web browsing greatly improves LLM performance on downstream tasks, there remains…”
Semantic Verification in Large Language Model-based Retrieval Augmented Generation
Martin, A., Witschel, H. F., Mandl, M. M., & Stockhecke, M. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v3i1.31199 · Cited by 7 · cited 1× · see citing works
“In parallel, retrieval-augmented generation (RAG) frameworks Guu et al. (2020); Lewis et al. (2020) have been widely adopted in QA Wiratunga et al. (2024), summarization Edge et al. (2024), and fact verification Martin et al. (2024), serving as a mechanism for injecting external knowledge into LLMs.”
Benchmarking Large Language Models on CMExam - A comprehensive Chinese Medical Exam Dataset
Liu, J., Zhou, P., Hua, Y., et al. (2023) · neural information processing systems · doi:10.52202/075280-2283 · Cited by 6 · see citing works
Self-citationWebGLM: Towards an Efficient and Reliable Web-Enhanced Question-Answering System
Lai, H., Liu, X., Yu, H., Xu, Y., Iong, I. L., Yao, S., Zeng, A., Du, Z., Dong, Y., & Tang, J. (2025) · ACM Transactions on Information Systems · doi:10.1145/3729421 · Cited by 5 · see citing works
Level-Navi Agent: A Framework and benchmark for Chinese Web Search Agents
Hu, C., Xie, S., Wang, B., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2502.15690 · Cited by 4
BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
Wei, J., Sun, Z., Papay, S., McKinney, S., Han, J., Fulford, I., Chung, H. W., Passos, A. T., Fedus, W., & Glaese, A. (2025) · arXiv · doi:10.48550/arxiv.2504.12516 · Cited by 2 · cited 3× · see citing works
“Nonetheless, Wei et al. (2025) primarily operates within the English-language web, missing the linguistic, structural, and cultural complexities inherent to other language environments.”
InfluentialZero-Indexing Internet Search Augmented Generation for Large Language ModelsPreprint
He, G., Dai, Z., Zhu, J., Zhao, B., Hu, Q., Li, C., Peng, Y. X., Wang, C., & Yuan, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.19478 · Cited by 1 · see citing works
Zhiheng, X., Wenxiang, C., Xin, G., Wei, H., Yiwen, D., Boyang, H., Ming, Z., Junzhe, W., Senjie, J., Enyu, Z., et al. (2026) · SCIENTIA SINICA Informationis · doi:10.1360/ssi-2025-0532 · Cited by 1
FEVER: a large-scale dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.05355
Flagged first, then by citations
Cited by
Sorted by citations
WebSailor: Navigating Super-human Reasoning for Web AgentPreprint
Li, K., Zhang, Z.-W., Yin, H.-F., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2507.02592 · Cited by 232 · see citing works
UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement LearningPreprint
Wang, H., Zou, H., Song, H., Feng, J., Fang, J., Lu, J., Liu, L., Luo, Q., Liang, S., Huang, S., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.02544
“For browser use , Online-Mind2Web [77] contains 300 realistic tasks across 136 websites, while BrowseComp-en [73] and BrowseComp-zh [88] provide high-difficulty multi-hop questions.”
Self-citationWebDancer: Towards Autonomous Information Seeking Agency
Wu, J.-L., Li, B., Fang, R.-N., et al. (2025) · Neural Information Processing Systems · doi:10.48550/arxiv.2505.22648 · Cited by 204
ReSum: Unlocking Long-Horizon Search Intelligence via Context SummarizationPreprint
Wu, X., Li, K., Zhao, Y., Zhang, L., Ou, L., Yin, H., Zhang, Z., Yu, X., Zhang, D., Jiang, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.13313
“The key modification of ReSum-RL is that ReSum naturally segments long trajectories into 1 https://huggingface.co/Qwen/Qwen3-30B-A3B-Thinking-2507 multiple episodes when summarization occurs.”
InfluentialWebShaper: Agentically Data Synthesizing via Information-Seeking Formalization
Tao, Z.-W., Wu, J.-L., Yin, W., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2507.15061 · Cited by 124
“Concurrently, several recent benchmarks, such as GAIA (Mialon et al., 2023), BrowseComp (Wei et al., 2025), and BrowseComp-zh (Zhou et al., 2025), provide only test sets, which restricts their applicability for training agents.”
AgentFold: Long-Horizon Web Agents with Proactive Context ManagementPreprint
Ye, R., Zhang, Z., Li, K., Yin, H., Tao, Z., Zhao, Y., Su, L., Zhang, L., Qiao, Z., Wang, X., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.24699
“We consider 3 information-seeking benchmarks including BrowseComp (Wei et al., 2025), BrowseComp-ZH (Zhou et al., 2025a), and WideSearch-en (the most detailed metric: Item-F1) (Wong et al., 2025); and 1 general benchmark: GAIA (text-only subset) (Mialon et al., 2023).”
Miro Team, Bai, S., Bing, L., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2511.11793 · Cited by 83
“A similar pattern is observed on the BrowseComp-ZH [26] benchmark, with a 6.1-point improvement over GLM-4.”
InfluentialDeep Research: A Survey of Autonomous Research AgentsPreprint
Zhang, W., Li, X., Zhang, Y., Jia, P., Wang, Y., Guo, H., Liu, Y., & Zhao, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.12752 · Cited by 1 · see citing works
WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon AgentsPreprint
Qiao, Z., Chen, G., Chen, X., Yu, D., Yin, W., Wang, X., Zhang, Z., Li, B., Yin, H., Li, K., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.13309
“• BrowseComp-zh (Zhou et al., 2025) -”
Seed1.8 Model Card: Towards Generalized Real-World Agency
Seed, B. (2026) · arXiv.org · doi:10.48550/arxiv.2603.20633 · Cited by 67
“8 on GAIA [44], BrowseComp-en [77], BrowseComp-zh [95], MM-BrowseComp [35], WideSearch [78], HLE [51], SWE-Bench Verified [47], Multi-SWE-Bench [87], AInstein-SWE-Bench, Terminal Bench 2.0 [64], U-Artifacts (internal), DeepConsult [62], DeepResearchBench [18], DiscoX [91], BFCL-v4 [50], and τ 2…”
InfluentialLi, K., Zhang, Z., Yin, H., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.13305
“This improvement successfully translates to our validation benchmarks, where performance on both BrowseComp-EN and BrowseComp-ZH shows a corresponding, albeit oscillating, upward trajectory.”
InfluentialTowards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs
Li, Y., Zhang, W., Yang, Y., Huang, W.-C., Wu, Y., Luo, J., Bei, Y., Zou, H. P., Luo, X., Zhao, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.09477 · Cited by 2
REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agents
Chu, Z., Wang, X., Hong, J., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.14234 · Cited by 55
“Our evaluation suite includes: Humanity’s Last Exam [26], BrowseComp [40], BrowseComp-ZH [47], GAIA [24].”
From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning AgentsPreprint
Zhang, W., Li, Y., Bei, Y., Luo, J., Wan, G., Yang, L., Xie, C., Yang, Y., Huang, W., Miao, C., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.18959
“In comparison, the OpenAI Deep Research agent achieves significantly higher scores—51.5% on BrowseComp, 42.9% on BrowseComp-ZH, and 26.6% on HLE—demonstrating the effectiveness of reasoning-integrated search in advancing the frontier of intelligent information-seeking systems.”
InfluentialSelf-citationScaling Agents via Continual Pre-trainingPreprint
Su, L., Zhang, Z., Li, G., Chen, Z., Wang, C., Song, M., Wang, X., Li, K., Wu, J., Chen, X., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.13310
“Open-source deep research agents have witnessed remarkable growth, achieving impressive results on deep research benchmarks including BrowseComp-en (Wei et al., 2025), BrowseComp-zh (Zhou et al., 2025b) Xbench-DeepSearch (Xbench-Team, 2025), and GAIA (Mialon et al., 2023).”
InfluentialMM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents
Li, S.-L., Bu, X.-Y., Wang, W., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2508.13186 · Cited by 45
“However, BrowseComp and its derivative works [3, 56] focus solely on textual information and overlook the need for multimodal understanding.”
Self-citationOpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis
Li, Z., Jiang, D., Ma, X.-G., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2603.20278 · Cited by 38
OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Data
Du, Y., Ye, R., Tang, S., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2603.15594 · Cited by 38
“Notably, on BrowseComp-ZH, OpenSeeker surpasses RedResearcher by 21.6% (48.4% vs 26.8%).”
InfluentialWang, W.-X., Xu, X.-X., An, W., Dai, F., Gao, W., He, Y.-C., Huang, J., Ji, Q., Jin, H., Li, X., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2512.24873 · Cited by 34
Self-citationMiroThinker-1.7 & H1: Towards Heavy-Duty Research Agents via VerificationPreprint
Бай, М., Bing, L., Lei, L., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2603.15726 · Cited by 31 · see citing works
“As shown in Figure 1, our flagship system MiroThinker-H1 achieves 88.2 and 84.4 on BrowseComp [8] and BrowseComp-ZH [9], respectively, outperforming best open-source and commercial research agents.”
InfluentialSelf-citationFinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning
Hu, L., Jiao, J., Liu, J., Ren, Y., Wen, Z., Zhang, K., Zhang, X., Gao, X., He, T., Hu, F., et al. (2025) · arXiv · Cited by 2
“…proposed for evaluating browsing capabilities, while they fall short along two axes: ( i ) General-purpose browsing benchmarks, like BrowseComp [23], BrowseComp-ZH [27], and BrowseComp-Plus [7], are intentionally domain-agnostic and center on lookup-oriented tasks with short, verifiable targets.”
MedBrowseComp: Benchmarking Medical Deep Research and Computer Use
Chen, S., Moreira, P., Xiao, Y., Schmidgall, S., Warner, J., Aerts, H., Hartvigsen, T., Gallifant, J., & Bitterman, D. S. (2025) · arXiv · doi:10.48550/arxiv.2505.14963
“BrowseComp and its offshoot BrowseComp-ZH push AI agents to perform complex web searches for difficult-to-find facts [28, 33].”
Lin, M.-H., Wu, Z., Xu, Z.-C., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.16724 · Cited by 29
“These benchmarks jointly evaluate the agent’s ability to (i) retrieve relevant information and (ii) synthesize evidence into correct, verifiable answers.”
LongCat-Flash-Thinking-2601 Technical ReportPreprint
Meituan LongCat Team, Gui, A., Li, B., Tao, B., Zhou, B., Chen, B., Zhang, C., Zhang, C., Gao, C., Zhang, C., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.16725
“We evaluate agentic search capability on BrowseComp [Wei et al., 2025] and BrowseComp-ZH [Zhou et al., 2025], and report results under both settings with and without context management.”
Self-citationTongyi DeepResearch Technical Report
Li, T., Zhang, B., Zhang, D.-C., et al. (2025) · doi:10.48550/arxiv.2510.24701 · Cited by 26
DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking
Lan, T., Zhu, B., Jia, Q., Ren, J., Li, H., Wang, L., Xu, Z., Luo, W., & Zhang, K. (2025) · arXiv · doi:10.48550/arxiv.2510.20168
“Existing deep search benchmarks such as GAIA [Mialon et al., 2023], BrowseComp [Wei et al., 2025] and BrowseComp-zh [Zhou et al., 2025b] require agents to employ multi-hop web browsing and deep reasoning to identify target answers.”
Self-citationSu, S., Xing, S., Dong, X., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.22808 · Cited by 20
“MiroFlow achieves state-of-the-art results on all major benchmarks, including GAIA [34], BrowseComp-EN/ZH [67, 77], HLE [44], xBench-DS [9], and Fu-tureX [75], outperforming both commercial/closed-source and open-source agent frameworks.”
The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents
Xuan, W., Zeng, Q., Qi, H.-L., et al. (2026) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2601.07264 · Cited by 19
Zhang, J.-J., Lv, X., Feng, L., et al. (2026) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2601.06021 · Cited by 18
“Considering the relatively small dataset size of BrowseComp-ZH, xbench-DeepSearch, and GAIA, we repeat their evaluation 3 times and report the average accuracy.”
Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric Design
Zhu, B., Jia, Q., Lan, T., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2603.28376 · Cited by 17
Self-citationLearning Query-Specific Rubrics from Human Preferences for DeepResearch Report GenerationPreprint
Lv, C., Zhou, J., Zhao, W.-T., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.03619 · Cited by 17 · see citing works
“Benchmarks such as GAIA (Mialon et al., 2023; Russell et al., 2025), BrowseComp (Wei et al., 2025; Zhou et al., 2025), and HLE (Phan et al., 2025) provide verifiable targets, enabling agent training via Reinforcement Learning with Verifiable Rewards (RLVR) (Jin et al., 2025; Liu et al., 2025).”
Self-citationRethinking Agentic Search with Pi-Serini: Is Lexical Retrieval Sufficient?
Hsu, T.-H., Yang, J.-H., & Lin, J. (2026) · arXiv.org · doi:10.48550/arxiv.2605.10848 · Cited by 16
“…their evaluation has also shifted toward more realistic research settings: Bench-marks such as BrowseComp (Wei et al., 2025) and BrowseComp-ZH (Zhou et al., 2025) evaluate open-web browsing ability, while BrowseComp-Plus provides a fixed-corpus setting for controlled comparison of search…”
BrowseMaster: Towards Scalable Web Browsing via Tool-Augmented Programmatic Agent Pair
Pang, X., Tang, S., Ye, R., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2508.09129 · Cited by 16
“Evaluation employs xVerify-9B (Chen et al., 2025a) for BrowseComp, BrowseComp-zh, and xBench-DeepResearch, GPT-4o (OpenAI, 2024a) for WebWalkerQA and GAIA following Wu et al. (2025b).”
InfluentialParallelMuse: Agentic Parallel Thinking for Deep Information Seeking
Li, B.-X., Zhang, D.-C., Wu, J.-L., Yin, W.-B., Tao, Z.-W., Zhao, Y., Zhang, L.-W., Shen, H., Fang, R.-N., Xie, P.-J., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.24698 · Cited by 14
“These benchmarks jointly assess both deep search and reasoning capabilities, with BrowseComp and BrowseComp-zh placing greater emphasis on deep search, HLE focusing more on reasoning, and GAIA providing a balanced evaluation across both dimensions.”
InfluentialRE-TRAC: REcursive TRAjectory Compression for Deep Search Agents
Zhu, J., Zhang, G.-R., Ma, X., Xu, L., Zhang, M., Yang, R., Wang, S., Qiu, K., Wu, Z.-R., Dai, Q., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.02486 · Cited by 13
“Our RE-TRAC-30B-A3B model consistently achieves 8% ∼ 10% improvement on BrowseComp, BrowseComp-ZH, GAIA and XBench, compared with its base model Tongyi-DeepResearch, which is also the previous strongest baseline with 30B parameters.”
InfluentialPushing Test-Time Scaling Limits of Deep Search with Asymmetric Verification
Zeng, W., He, K., Kuang, C.-Q., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.06135 · Cited by 13
“On BrowseComp-zh, GLM-4.”
InfluentialReinforcement Learning Foundations for Deep Research Systems: A SurveyPreprint
Li, W. J., Chen, Z., Lin, J., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2509.06733 · Cited by 13 · see citing works
Self-citationLongSeeker: Elastic Context Orchestration for Long-Horizon Search Agents
Lu, Y.-J., Ye, R., Du, Y., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2605.05191 · Cited by 12
“We train LongSeeker on 10k synthesized trajectories and achieve 61.5% on BrowseComp and 62.5% on BrowseComp-ZH, outperforming strong long-horizon search baselines.”
InfluentialLiang, J., Han, J., Li, W., Wang, X., Zhang, Z., Jiang, Z., Liao, Y., Li, T., Huang, Y., Shen, H., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.17091 · Cited by 1 · see citing works
“(2) BrowseComp-ZH [27]: A benchmark specifically designed to measure multi-hop web search and chain-based reasoning within the Chinese web ecosystem.”
OffSeeker: Online Reinforcement Learning Is Not All You Need for Deep Research Agents
Zhou, Y.-H., Zheng, K., Chen, Q.-G., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2601.18467 · Cited by 12
Self-citationInteractComp: Evaluating Search Agents With Ambiguous Queries
Deng, M., Huang, L., Fan, Y., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.24668 · Cited by 12
“Interaction benchmarks like IN3 (Qian et al., 2024) and Tau-Bench (Yao et al., 2024) focus on general conversation but lack grounding in verifiable search tasks.”
InfoDeepSeek: Benchmarking Agentic Information Seeking for Retrieval-Augmented GenerationPreprint
Xi, Y., Lin, J., Zhu, M., Xiao, Y., Ou, Z., Liu, J., Wan, T., Chen, B., Liu, W., Wang, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.15872 · Cited by 9 · see citing works
“Most benchmarks include assessing generation quality, i.e. , answer accuracy [51, 58, 2, 54, 48, 43, 64].”
Revisiting Text Ranking in Deep Research
Meng, C., Ou, L., MacAvaney, S., & Dalton, D. J. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808557 · Cited by 4 · see citing works
“Motivation .”
Repurposing Synthetic Data for Fine-grained Search Agent Supervision
Zhao, Y., Li, K., Wu, X.-X., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.24694 · Cited by 11
“Notably, Web-30B-E-GRPO achieves the best performance among open-source agents on BrowseComp (12.9) and BrowseComp-ZH (26.4), even surpassing advanced models like Claude-4-Sonnet on BrowseComp, and narrows the gap with others.”
InfluentialRhinoInsight: Improving Deep Research through Control Mechanisms for Model Behavior and Context
Lei, Y., Si, S., Wang, W., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2511.18743 · Cited by 10
“• BrowseComp-ZH (Zhou et al., 2025) is the first high-difficulty benchmark for evaluating real-world web-browsing and reasoning capabilities of LLMs within the Chinese information ecosystem.”
SAM: State-Adaptive Memory for Long-Horizon Reasoning AgentPreprint
Hu, Y., Qian, H., Wang, S., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.24468
“…agent benchmarks that stress complementary aspects of memory-intensive reasoning: BrowseComp [34] (long-range web browsing), BrowseComp-ZH [46] (cross-lingual multi-hop search), WideSearch [35] (broad exploration over large search spaces), and HLE [24] (knowledge-intensive scientific…”
AgentSwing: Adaptive Parallel Context Management Routing for Long-Horizon Web Agents
Feng, Z., Su, L., Zhang, Z., Wang, X., Zhang, X., Wang, X., Fang, R., Zhang, Q., Li, B., Cai, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2603.27490
“We evaluate AgentSwing on three challenging deep information-seeking benchmarks: BrowseComp (Wei et al., 2025), BrowseComp-ZH (Zhou et al., 2025), and Humanity’s Last Exam (HLE) (Phan et al., 2025).”
InfluentialVeriWeb: Verifiable Long-Chain Web Benchmark for Agentic Information-Seeking
Liu, S., Liu, M., Zhou, H., Cui, Z., Zhou, Y., Zhou, Y., Gao, J., Zhou, H., Yang, Y., Fan, W., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.04026
SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering AgentsPreprint
Zheng, P., Shang, Z., Jiang, S., Tian, W., Zhu, D., Ma, Z., Yuan, D., & Zhang, Q. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.08149
“Recent agentic benchmarks focus on assessments on web search [12, 34, 41], productivity workflows [29, 30, 42], cybersecurity [19, 33, 43], and software engineering [10, 14, 16, 24, 36].”
FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents
Deng, J.-L., Chen, Y.-M., Xiang, X.-Q., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.12087 · Cited by 8
“This appendix provides the graph-expansion algorithm, fact and fuzzing examples, and refinement cases accompanying Section 3.”
Masking Stale Observations Helps Search Agents - Until It Doesn't: A Regime Map and Its Mechanism
Zhang, H., Xu, Q.-X., Li, Z.-F., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.00408 · Cited by 8
“4 on BrowseComp-ZH), while the strongest backbone DS-V4-Flash-Max is saturated everywhere ( +0 .”
InfluentialOpenSeeker-v2: Pushing the Limits of Search Agents with Informative and High-Difficulty Trajectories
Du, Y., Ye, R., Tang, S., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2605.04036 · Cited by 8
“Though trained with only 10.6k samples, OpenSeeker-v2 achieves a new SOTA across four representative benchmarks: 46.0% on BrowseComp, 58.1% on BrowseComp-ZH, 34.6% on Humanity’s Last Exam, and 78.0% on xbench, significantly outperforms Tongyi DeepResearch and RedSearcher that are extensively trained via CPT, SFT, and…”
InfluentialTowards Efficient Agents: A Co-Design of Inference Architecture and SystemPreprint
Lin, W., Zhen, H., Yang, S., Wang, X. Y., Liu, R., Chen, H., Zhang, W., Zhou, C., Li, Y., Chen, C., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.18337
“• Tasks: Experiments were conducted on the BrowseComp-zh (Chinese version) [38] benchmark, which involves multi-step tasks requiring web searches.”
EcomBench: Towards Holistic Evaluation of Foundation Agents in E-commerce
Min, R., Qiao, Z.-L., Xu, Z., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2512.08868 · Cited by 7
“At its core, assessing the foundational capabilities of these advanced LLM agents has become essential, where a wide range of benchmarks have been proposed (Mialon et al., 2023; Wei et al., 2025; Zhou et al., 2025; Xbench Team, 2025; Yao et al., 2022; Wong et al., 2025; Yao et al., 2024).”
InfoAgent: Advancing Autonomous Information-Seeking Agents
Zhang, G.-R., Zhu, J., Yang, R., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2509.25189 · Cited by 7
STAMP: Provenance-Guided Credit Assignment for Deep Search Agents
Xu, K., Xu, H., Chen, X.-R., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.11172 · Cited by 6
“On Qwen3-30B-A3B-Thinking-2507 (Team, 2025), we evaluate STAMP in a live-web deep-search setting on BrowseComp (Wei et al., 2025), BrowseComp-ZH (Zhou et al., 2025), and xbench-DS (Xbench-Team, 2025).”
InfluentialS1-DeepResearch: Beyond Search, Toward Real-World Long-Horizon Research AgentsPreprint
Dong, Y. H., Xiao, X., Dong, L., Jin, X., Li, Z., Zhang, H., 王杜云, & Xu, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.15367
PBSD: Privileged Bayesian Self-Distillation for Long-Horizon Credit Assignment
Tian, Y., Wang, R., Wen, X., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.09348 · Cited by 6
“BrowseComp-ZH [34] extends this setting to the Chinese web, introducing multi-hop questions that require retrieval and reasoning over Chinese information sources.”
Argus: Evidence Assembly for Scalable Deep Research Agents
Zhang, Z., Su, L.-C., Chen, Z., Lin, X., Xu, H.-T., Du, S., Yang, K.-Y., An, B., Bing, L., & Wang, X. (2026) · arXiv.org · doi:10.48550/arxiv.2605.16217 · Cited by 6
“BrowseComp [33] and its Chinese counterpart BrowseComp-ZH [34] probe multi-step web browsing on adversarially constructed factual questions that resist single-hop search. xbench DeepSearch-2510 [35] targets deep search and tool use through professionally annotated Chinese tasks with dynamic updates.”
DLLM Agent: See Farther, Run Faster
Zhen, H., Lin, W.-Z., Liu, R.-X., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.07451 · Cited by 6
“We adopt the public B ROWSE C OMP - ZH benchmark [35], which requires multi-turn web browsing, iterative retrieval, and reasoning over dynamically collected evidence to answer user queries.”
InfluentialDeep Search with Hierarchical Meta-Cognitive Monitoring Inspired by Cognitive Neuroscience
Sun, Z.-X., Wang, Q., Yu, W.-J., et al. (2026) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3805712.3809549 · Cited by 6 · see citing works
Nested Browser-Use Learning for Agentic Information Seeking
Li, B., Wu, J.-L., Yin, W., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2512.23647 · Cited by 6
“…agents (Team, 2025c; x.ai, 2025; AI, 2025; Team, 2025a; Gemini, 2025), powered by large language models (LLMs) have achieved strong performance on a variety of challenging wide and deep search tasks (Mialon et al., 2023; Wei et al., 2025; Zhou et al., 2025; Xbench-Team, 2025; Wong et al., 2025).”
BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents
Ou, L., Li, K., Yin, H.-F., Zhang, L.-W., Zhang, Z.-W., Wu, X.-X., Ye, R., Qiao, Z.-L., Jiang, Y., Xie, P.-J., et al. (2025) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2510.23458 · Cited by 7
“1 models, evaluating them on two challenging information-seeking benchmarks: BrowseComp (Wei et al., 2025), which tests the ability to locate hard-to-find information on the web, and its Chinese counterpart, BrowseComp-zh (Zhou et al., 2025a).”
ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capability
Liu, W.-T., Bi, J., Zhou, W.-Q., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.14145 · Cited by 5
“…into the reasoning loop, LLM-based agents have achieved strong performance across a wide spectrum of downstream tasks, spanning text-based question answering [7, 8, 9, 10], visual question answering [11, 12, 13], long-form text report generation [14, 15], and domain-grounded tool use [16, 17].”
Mach-Mind-4-Flash Technical ReportPreprint
Foundation Model Team (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.09375
“We assess deep-search ability with BrowseComp / BrowseComp-zh [80; 81] and X-Bench [82] (persistent multi-constraint web browsing).”
LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know?
Fan, H., Wang, X., Chu, Z., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2605.28721 · Cited by 5
“We conduct the pilot study on four challenging agentic benchmarks: BrowseComp [12], BrowseComp-ZH [24], HLE [25], and GAIA [26].”
DR-Venus: Towards Frontier Edge-Scale Deep Research Agents with Only 10K Open Data
Venus Team, Dai, S.-H., Deng, Y., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2604.19859 · Cited by 5
InfoSeeker: A Scalable Hierarchical Parallel Agent Framework for Web Information SeekingPreprint
Lee, K. Y., Huang, Y., He, Z., Zhou, H., Luo, W., Shao, K., Fang, M., & Wang, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.02971
Nanbeige4.1-3B: A Small General Model that Reasons, Aligns, and ActsPreprint
Yang, C., Peng, G., Zhu, J., Le, R., Feng, R., Zhang, T., Xu, X., Song, Y., Jia, Y., Wen, Y., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.13367
“We evaluate our model on a range of long-horizon benchmarks, including GAIA [9], BrowseComp [16], BrowseComp-ZH [23], Humanity’s Last Exam (HLE) [12], SEAL-0 [11], xBench-DeepSearch-2505 [3], and xBench-DeepSearch-2510 [3].”
Table-as-Search: Formulate Long-Horizon Agentic Information Seeking as Table Completion
Lan, T., Henry, F., Zhu, B., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.06724 · Cited by 5
“To rigorously evaluate TaS across distinct long-horizon agentic infoseeking, we employ three categories of benchmarks: (1) Deep Search : We utilize GAIA (text-only) (Mialon et al., 2023) and BrowseComp-ZH (Zhou et al., 2025) to assess multi-step reasoning and precise filtering capabilities.”
InfluentialSelf-citationThink in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoning
Wang, H.-N., Du, C., Kawaguchi, K., & Pang, T.-Y. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2512.02874 · Cited by 5
PhysLab: A Benchmark Dataset for Multi-Granularity Visual Parsing of Physics Experiments
Zou, M., Zeng, Q., Miao, Y., Liu, S., Wang, Z., Liu, H., & Zhou, W. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3758221
“To ensure annotation quality, we implemented multi-round cross-checking and sample-based review procedures [64].”
ActiveMem: Dynamic Latent Memory Trees for Long-Horizon AgentsPreprint
Wu, S.-L., Wang, J., Du, Z., & Gan, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33244
“Regarding evaluation, we consider three information-seeking benchmarks—BrowseComp (Wei et al. 2025), BrowseComp-ZH (Zhou et al. 2025a), and WideSearch-en (Wong et al. 2025)—alongside the text-only subset of the general agent benchmark GAIA (Mialon et al. 2024).”
LongCat Sparse Attention: Taming the Lightning via Streaming-aware Hierarchical Cross-Layer Indexing
Zan, W., Zhang, J., Tan, J.-C., et al. (2026) · doi:10.48550/arxiv.2608.01662 · Cited by 4
“…[Jimenez et al., 2023], and TerminalBench [Merrill et al., 2026]), agentic tool use ( τ 2 -Bench [Barres et al., 2025], VitaBench [He et al., 2025], and MCP-Atlas [Bandi et al., 2026]), and agentic search (BrowseComp [Wei et al., 2025, Zhou et al., 2025] and RWSearch [Zhu et al., 2026]).”
Zhang, C., Dong, K.-C., Li, D., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.07773 · Cited by 4
Self-citationInteraction as Intelligence: Deep Research With Human-AI Partnership
Ye, L., Cai, X., Wang, X., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2507.15759 · Cited by 4
“…our hypothesis that experts with higher cognitive capabilities demonstrate enhanced collaboration with AI in transparent dialogue environments, we measured our system performance through the accuracy of browsecomp-ZH (Zhou et al., 2025), a benchmark assessing agents’ web browsing capabilities.”
BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent
Chen, Z., Ma, X., Zhuang, S., Nie, P., Zou, K., Liu, A., Green, J., Patel, K., Meng, R., Su, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.06600 · Cited by 3 · see citing works
“Recent extensions of BrowseComp concepts, such as ZH-BrowseComp [4] and MedBrowseComp [5], further expand to multilingual queries and domain-specific challenges.”
Self-citationContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL
Pan, Z.-S., Pei, Q., Lu, J.-R., et al. (2026) · doi:10.48550/arxiv.2608.28476 · Cited by 3
“The evaluation on deep search tasks includes GAIA (the text-only subset with 103 examples) [38], BrowseComp [3], BrowseComp-ZH [39], and xBench-DeepSearch [40].”
ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit Assignment
Lu, Y.-J., Ye, R., Wang, J.-J., et al. (2026) · doi:10.48550/arxiv.2608.05102 · Cited by 3
SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents
Liang, Z.-X., Liu, Y.-F., Huang, Y.-D., et al. (2026) · doi:10.48550/arxiv.2608.05212 · Cited by 3
“Queries are drawn from five deep-search benchmarks: BrowseComp (Wei et al. 2025), BrowseComp-ZH (Zhou et al. 2025), DeepSearchQA (Gupta et al. 2026), Seal-0 (Pham et al. 2026), and xBench-DeepSearch (Chen et al. 2025).”
SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Task
Mei, L., Yu, X.-H., Chen, C., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.24850 · Cited by 3
“A similar pattern is observed on BrowseComp-zh: SearchArt-27B has a mean of 18.5 turns compared with 17.2 for Qwen3.”
InfluentialAgentCompass: A Unified Evaluation Infrastructure for Agent CapabilitiesPreprint
Chen, K., Ding, Z., Ge, J., Jiang, S., Li, M., Li, Q., Li, Z., Li, Z., Liang, T., Liu, S., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.13705
Self-citationAgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
Xi, Z., Yang, D., Liu, J., Huang, J., Guo, H., Huang, B., Chen, T., Zhang, Q., 卢中杭, Liu, C., et al. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.2058
Self-citationLu, Y.-H., Zeng, Q.-C., Qi, H.-L., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.15345 · Cited by 3
“Recent browsing-agent benchmarks beyond English, such as BrowseComp-ZH (Zhou et al., 2025), broaden the linguistic scope of agent evaluation but remain primarily monolingual: ques-a tions, evidence, and answers all stay within the same language.”
SearchSwarm: Towards Delegation Intelligence in Agentic LLMs for Long-Horizon Deep Research
Ning, P., Chen, Q., Tao, K., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.09730 · Cited by 3
K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts
Lee, N., Yoon, D., Son, G., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.02404 · Cited by 3
“Following BrowseComp and BrowseComp-ZH (Wei et al., 2025; Zhou et al., 2025), we report expected calibration error over five equal-width confidence bins, computed as the weighted average gap between mean confidence and empirical accuracy.”
IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents
Feng, Y., Huang, Q., Xie, X.-Y., et al. (2026) · doi:10.48550/arxiv.2601.06676 · Cited by 3
“…Alzubi et al. 2025; Li et al. 2025) and end-to-end agents trained with reinforcement learning (Jin et al. 2025; Zheng et al. 2025 (Wei et al. 2025; Zhou et al. 2025), long-form writing (Bai et al. 2025; Wu et al. 2025), citation-grounded reports (Shao et al. 2024), and long-horizon research (Du…”
Zhou, Y., Lei, Y., Si, S., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2512.14244 · Cited by 3
“As shown in Table 6, on the noise-intensive BrowseComp-ZH , our method delivers dramatic gains.”
MARINE: Theoretical Optimization and Design for Multi-Agent Recursive IN-context Enhancement
Zhang, H., Lu, J., Du, Y., Gao, Y.-Q., Huang, L., Wang, B., Tan, F., & Zou, P. (2025) · arXiv.org · doi:10.48550/arxiv.2512.07898 · Cited by 3
DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
Du, M., Xu, B., Zhu, C., Wang, X., & Mao, Z. (2025) · arXiv · doi:10.48550/arxiv.2506.11763 · Cited by 2 · see citing works
“These include evaluations for agents in scientific domains [2, 12, 17, 3], creative writing [31, 1, 23], code generation and software engineering [10, 40, 22, 9, 32], and in their roles as human assistants, often enhanced by capabilities such as web Browse and tool-use [28, 38, 33, 6, 26].”
BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents
Chen, Z., Ma, X., Zhuang, S., Nie, P., Zou, K., Sharifymoghaddam, S., Liu, A., Green, J., Patel, K., Meng, R., et al. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.1023 · Cited by 2 · see citing works
“These benchmarks typically evaluate agents using black-box web search APIs to retrieve supporting documents in real time (Zhou et al., 2025; Chen et al., 2025).”
Self-citationLast Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents
Li, Z., Yu, C.-Y., Zang, L., et al. (2026) · doi:10.48550/arxiv.2608.29685 · Cited by 2
“5-122B-A10B, averaged across two benchmarks (BrowseComp and BrowseComp-zh) are shown in Figure 5(a).”
ACE: Pluggable Adaptive Context Elasticizer across AgentsPreprint
Liao, N., Long, Z., Wang, X., Yang, X., Wang, Y., Zhuang, Z., Cai, X., Weng, R., & Yan, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.31564
GAIA-v2-LILT: Multilingual Adaptation of Agent Benchmark beyond Translation
Kim, Y., Uhlig, K., & Wuebker, J. (2026) · Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026) · doi:10.48550/arxiv.2604.24929 · Cited by 2
JADE: Expert-Grounded Dynamic Evaluation for Open-Ended Professional Tasks
Lin, L., Liu, J.-Y., Yang, T.-Y., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.06486 · Cited by 2
“By integrating multi-step reasoning with tool-use capabilities, these agents are increasingly deployed in complex, long-horizon professional workflows (Luo et al., 2025; Jimenez et al., 2024).”
Table-as-Search: Agentic Information Seeking is Table Completion
Lan, T., Henry, F., Zhu, B., et al. (2026) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2026.findings-acl.743 · Cited by 2
“To rigorously evaluate TaS across distinct long-horizon agentic infoseeking, we employ three categories of benchmarks: (1) Deep Search : We utilize GAIA (text-only) (Mialon et al., 2023) and BrowseComp-ZH (Zhou et al., 2025) to assess multi-step reasoning and precise filtering capabilities.”
InfluentialSelf-citationReportBench: Evaluating Deep Research Agents via Academic Survey Tasks
Li, M., Zeng, Y., Cheng, Z., Ma, C., & Jia, K. (2025) · arXiv · doi:10.48550/arxiv.2508.15804 · Cited by 1 · see citing works
FinHopper: A Progressive Multi-Hop Benchmark for Complex Financial Agents with Step-wise Evaluation
Wu, Y., Lin, M.-P., Ying, Q., Dan, L., Zhao, H.-X., Liu, Q.-R., Shen, Y.-T., Wu, S.-H., Luo, W.-H., Du, X.-K., et al. (2026) · doi:10.48550/arxiv.2610.26036 · Cited by 1
Avoiding What Your Teacher Already Knows: Entity Familiarity for Deep-Search Distillation
Liu, Y.-F., Liang, Z.-X., Tan, W.-T., et al. (2026) · doi:10.48550/arxiv.2610.16251 · Cited by 1
“Table 7 summarizes results on BrowseComp (BC), BrowseComp-ZH (BC-ZH) [43], SEAL-0 [23], the text-only subset of Humanity’s Last Exam with tools [24], and xbench-DeepSearch-2510 [35].”
Iris: Climbing to the Search Frontier
Liu, Z.-Y., Liu, H., Wang, Z.-C., et al. (2026) · doi:10.48550/arxiv.2609.04304 · Cited by 1
“We evaluate our models on four challenging benchmarks: BrowseComp [23], BrowseComp-ZH [31], DeepSearchQA [8], and HLE [16].”
Self-citation
- Authors
- Zhou, P., Leon, B., Ying, X., Zhang, C., Shao, Y., Ye, Q., Chong, D., Jin, Z., Xie, C., Cao, M., Gu, Y., Hong, S., Ren, J., Chen, J., Liu, C., & Hua, Y.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2504.19314
- OpenAlex
- W6929371760
- Semantic Scholar
- 34c20505098fd48ff19e4b5c4901febdd5d27181
- Cited by
- 0
- References
- 34
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- large language models, web browsing, benchmarking, multi-hop question answering, LLM agents, agentic search, information retrieval, information reconciliation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-22.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-22.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
