The DOI matches its DataCite record; 38 of the 38 references it cites were found in a registry or an index; 6 of the 13 sources its claims cite were checked for support.
BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese
Open accessAbstract
As large language models (LLMs) evolve into tool-using agents, the ability to browse the web in real-time has become a critical yardstick for measuring their reasoning and retrieval competence. Existing benchmarks such as BrowseComp concentrate on English and overlook the linguistic, infrastructural, and censorship-related complexities of other major information ecosystems -- most notably Chinese. To address this gap, we introduce BrowseComp-ZH, a high-difficulty benchmark purpose-built to comprehensively evaluate LLM agents on the Chinese web. BrowseComp-ZH consists of 289 multi-hop questions spanning 11 diverse domains. Each question is reverse-engineered from a short, objective, and easily verifiable answer (e.g., a date, number, or proper noun). A two-stage quality control protocol is applied to strive for high question difficulty and answer uniqueness. We benchmark over 20 state-of-the-art language models and agentic search systems on our proposed BrowseComp-ZH. Despite their strong conversational and retrieval capabilities, most models struggle severely: a large number achieve accuracy rates below 10%, and only a handful exceed 20%. Even the best-performing system, OpenAI's DeepResearch, reaches just 42.9%. These results demonstrate the considerable difficulty of BrowseComp-ZH, where success demands not only effective retrieval strategies, but also sophisticated reasoning and information reconciliation -- capabilities that current models still struggle to master. Our dataset, construction guidelines, and benchmark results have been publicly released at https://github.com/PALIN2018/BrowseComp-ZH.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
BrowseComp-ZH is a new 289-question Chinese-web benchmark built by reverse-designing multi-hop queries from short, verifiable answers across 11 domains. It evaluates over 20 LLMs and AI search systems, finding most models below 10% accuracy and the best system, OpenAI DeepResearch, at 42.9%.
Key findings
- Most standalone LLMs perform poorly: Qwen2.5-72B-Instruct 6.6%, GPT-4o 6.2%, Claude-3.5-Sonnet 5.5%, LlaMa4 4.8%; many systems remain below 10% accuracy.
- Reasoning-capable models improve substantially: DeepSeek-R1 reaches 23.2% vs. DeepSeek-V3’s 8.7%, and Claude-3.7-Sonnet reaches 17.7% vs. Claude-3.5-Sonnet’s 5.5%.
- Agentic search systems with multi-round retrieval perform best: OpenAI DeepResearch 42.9%, Doubao (Deep Search) 26.0%, Perplexity (Research) 22.6%.
- Enabling web search can hurt: DeepSeek-R1 drops from 23.2% in direct-answer mode to 7.6% with web search enabled; calibration error also rises from 59% to 65%.
Methods
- Dataset construction used a reverse strategy: 10 expert contributors selected factual answers and crafted multi-constraint Chinese queries, tested on Baidu, Bing, and Google so answers did not appear on first-page results.
- Two-stage quality control: Stage 1 filtered 76 simple samples from 480 preliminary samples, leaving 404 high-difficulty candidates; Stage 2 used AI agents plus manual verification to eliminate 115 ambiguous samples, yielding 289 final questions across 11 domains.
- Evaluation covered over 20 open-source models, closed-source APIs, and AI search products; accuracy and calibration error were reported, with regex extraction plus GPT-4o grading for models and human extraction for AI search products.
Limitations
- Stated: the dataset is relatively small, and increasing sample size and question-type diversity would improve representativeness.
- Stated: answer uniqueness cannot be fully guaranteed, and the dynamic web means factual answers may evolve or become inconsistent over time, challenging stability and reproducibility.
- Evident from the text: evaluation depends on GPT-4o grading for model outputs and human annotators for AI search products, which may introduce grader or annotator variability; topic coverage is also uneven, e.g., Film & TV 15.6% versus Academic Papers 2.4%.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
39 Own result · 20 Own interpretation · 18 Own method · 16 No basis given · 6 Sources not found · 3 Supported · 2 Couldn’t confirm · 1 Partly supported · accuracy not yet measured
Every source the paper cites for this claim was checked and supports it.
In the paper
“Inspired by Wei et al. (2025), we adopt a reverse construction strategy: each task begins with a factual answer, from which an elaborate, multi-constraint query is crafted to make direct retrieval non-trivial.”
What the sources say
- Supportedcites Wei et al. (2025) source
In “Easy to verify, but hard to solve” (page not specified), BrowseComp says trainers were encouraged to start with a fact and construct an inverted question whose answer would be hard to find. It describes building the question from characteristics of a seed, supporting the cited inspiration for the reverse-construction strategy. The claim that BrowseComp-ZH adopts that strategy is the citing authors’ own description, not a claim about the source. (LLM, conf=0.93)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“enabling search functionality for DeepSeek-R1 leads to a substantial decline in performance”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Doubao (Standard) (26.0% vs. 18.7%)”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Yuanbao (12.2%)”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the calibration error for DeepSeek-R1 increases from 59% in the direct-answer setting to 65% when search is enabled”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“OpenAI's O1 (29.1%)”
- Also stated in: § 4.4 Performance · p. 7 · plain
“O1 (29.1%)”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4.4 Performance · p. 7 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Claude-3.5-Sonnet (5.5%)”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Doubao (Deep Search) achieves a nearly 8% absolute improvement over Doubao (Standard)”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“DeepSeek-V3's 8.7%”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Doubao (Deep Search) achieves a nearly 8% absolute improvement over Doubao (Standard) (26.0% vs. 18.7%)”
- Also stated in: § 4.6 Analysis of AI Search Products · p. 8 · plain
“Doubao (Deep Search) achieving 26.0%”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4.6 Analysis of AI Search Products · p. 8 · plain
- It may state two separate claims as one. 34
- Probably not a substantial claim. 27
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Nonetheless, Wei et al. (2025) primarily operates within the English-language web, missing the linguistic, structural, and cultural complexities inherent to other language environments.”
What the sources say
- Could not verifycites Wei et al. (2025) sourceExperimental — accuracy not yet measured
The Introduction and Related work and discussion describe BrowseComp as a benchmark for finding hard-to-find information on the web, but do not specify that it primarily uses the English-language web. Searches for English, non-English, multilingual, and cultural or linguistic limitations found no passage assessing its coverage of other language environments. (LLM, conf=0.73)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Liu et al. (2023) and Li et al. (2023b) focus on domain-specific medical QA, but do not evaluate open-ended retrieval or browsing strategies.”
What the sources say
- Supportedcites Liu et al. (2023) source
The Experimental Setting evaluates answers to medical multiple-choice questions and open-ended solution explanations, not retrieval or browsing strategies. The Conclusion and Discussions identifies CMExam as a medical question-answering benchmark. This verdict concerns Liu et al. only; it does not assess the separate claim about Li et al. (LLM, conf=0.88)
- Not matchedcites Li et al. (2023b)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“While it offers finer-grained evaluation for web browsing agents, it remains confined to the English web and lacks generalizability to non-English environments with fragmented platforms and diverse linguistic structures.”
What the sources say
- Could not verifycites Wei et al. (2025) sourceExperimental — accuracy not yet measured
The Introduction and Section 5 describe BrowseComp as a benchmark for agents searching the web, but do not state that it is confined to the English web. Searches for English-only coverage, multilingual evaluation, and non-English generalizability found no passage addressing fragmented platforms or diverse linguistic structures. (LLM, conf=0.74)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“To address this gap, Wei et al. (2025) introduced a benchmark of reverse-designed, evidence-grounded queries that challenge English-language agents to search and reason over difficult-to-access information.”
What the sources say
- Supportedcites Wei et al. (2025) source
The Introduction (page unavailable) introduces BrowseComp as a benchmark requiring agents to browse many websites. “Easy to verify, but hard to solve” describes constructing inverted questions from known facts, while “Data collection and verification” says the questions were supported by evidence. The Related work section describes its focus as hard-to-find information on the web. (LLM, conf=0.89)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The grading prompts are adapted from those used in the original BrowseComp benchmark Wei et al. (2025).”
What the sources say
- Supportedcites Wei et al. (2025) source
Section 2.3 Grading says the full BrowseComp grading prompt appears in Appendix B. The Additional information section reproduces that prompt, identifying the original prompt to which the citing sentence refers. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While well-designed retrieval pipelines can significantly improve performance, not all systems benefit from retrieval integration. For instance, enabling web search for DeepSeekR1 results in a substantial decline in performance, with accuracy dropping from 23.2% in the direct-answer (no web access) setting to 7.6% when web search is enabled.”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“BrowseComp-ZH consists of 289 multi-hop questions spanning 11 diverse domains.”
- Also stated in: § 1 Introduction · p. 3 · plain
“The final BrowseComp-ZH dataset consists of 289 complex questions, each with multiple constraints and unique answers, spanning 11 diverse domains, including film, technology, law, and medicine.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 1 Introduction · p. 3 · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This process eliminates 115 ambiguous samples, resulting in a final benchmark of 289 validated questions that maintains high levels of difficulty and answer verifiability.”
plaindirectn = 115- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As illustrated, the most represented categories include Film & TV (15.6%), Art (13.8%), and Geography (12.8%), reflecting the diverse interests of annotators and a broad coverage of Chinese web content. The dataset also features Music (11.1%), History (10.0%), and Medicine (9.0%). Conversely, topics like Policy & Law (3.5%) and Academic Papers (2.4%) have fewer samples”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Models endowed with inherent reasoning capabilities exhibit consistent improvements in accuracy. For example, DeepSeek-R1 surpasses DeepSeek-V3 by 14.5%, while Claude-3.7Sonnet outperforms Claude-3.5 by 12.2%.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Despite their strong conversational and retrieval capabilities, most models struggle severely: a large number achieve accuracy rates below 10%, and only a handful exceed 20%.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this stage, annotators identified 76 simple samples, and after filtering them out, we are left with 404 high-difficulty candidates.”
plainn = 76- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This process yields 480 preliminary samples, which are subsequently reviewed via the quality control procedure detailed in Section 3.2.”
plaindirectn = 480- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Agentic systems augmented with external retrieval mechanisms tend to achieve comparatively higher scores. Notably, OpenAI's DeepResearch and Doubao (Deep Search) demonstrate that well-orchestrated retrieval and reasoning pipelines can substantially enhance performance, achieving accuracies of 42.9% and 26.0%, respectively.”
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Fig. 3 (b), the answer length distribution shows that answers are generally short, typically falling within the range of 5 to 10 characters.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“we construct challenging question-answer pairs that require multi-hop retrieval, information filtering, and logical reasoning to derive concise, factual answers”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“the current dataset is relatively small”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We construct 289 high-quality samples across 11 diverse topics”
plainn = 289- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“canonical information pathways in English (e.g., Wikipedia, IMDb) lack equivalents or exhibit vastly different structures in the Chinese web”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Moreover, the Chinese web presents its own set of retrieval obstacles-fragmented and platform-specific content, inconsistent naming conventions, and linguistic traits such as ellipsis, cultural references, and implicit reasoning that undermine linear search strategies.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These length distributions reflect the high information density of the dataset, emphasizing both complexity in the questions and simplicity in the answers.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These findings suggest that BrowseComp-ZH not only challenges models' browsing and reasoning capabilities but also exposes their limitations in processing, evaluating, and aligning retrieved information with internal representations.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Each question combines temporal, spatial, categorical, or descriptive conditions to ensure answer uniqueness; not all conditions are required, but at least two dimensions are typically combined”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“Lyu et al. (2025) evaluates RAG systems under the CRUD (Create, Read, Update, Delete) paradigm, yet primarily emphasizes generation quality over retrieval path validation.”
What the sources say
- Not matchedcites Lyu et al. (2025)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To fill this gap, we introduce BrowseComp-ZH , the first benchmark specifically designed to evaluate web-enabled LLMs in the Chinese information environment.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results demonstrate the considerable difficulty of BrowseComp-ZH, where success demands not only effective retrieval strategies, but also sophisticated reasoning and information reconciliation-capabilities that current models still struggle to master.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To ensure high-quality annotation, we recruited 10 expert contributors (with Master's or PhD degrees) who have extensive experience in both LLM usage and web search.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The selected facts must be objective statements that can be independently verified through reliable sources, without the need for interpretation or inference”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Every candidate query is verified on three widely-used search engines: Baidu 1 , Bing 2 , and Google 3 , and is accepted only if the answer is not surfaced in the first-page results of any engine.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Annotators test each query on Baidu, Bing, and Google, using three distinct keyword combinations per search engine. If the correct answer appears on the first page of any search engine, the query will be revised or further constrained”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Stage 1 screens for keyword retrievability as above.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Hu et al. (2024) evaluates Chinese web search agents but lacks rigorous control over task difficulty and answer accessibility.”
What the sources say
- Not matchedcites Hu et al. (2024)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These questions require multi-step reasoning and are difficult to answer directly through search engines, making them ideal for evaluating the ability of Chinese LLM agents to conduct multihop retrieval, perform factual reasoning, and integrate online information.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All queries, evidence chains, and browsing steps are authored in Chinese and fully localized.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It establishes a high-difficulty benchmark for Chinese web retrieval, supporting systematic evaluation of agents' ability to navigate fragmented, unstructured, and linguistically diverse Chinese information sources.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This trend aligns with the nature of tasks in BrowseComp-ZH, which often involve multi-faceted queries, making it challenging to obtain accurate answers through a single retrieval operation.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Xu et al. (2024) focuses on dynamic, time-sensitive QA tasks but does not emphasize multi-hop retrieval.”
What the sources say
- Not matchedcites Xu et al. (2024)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We employ a human-in-the-loop approach, where multiple top-performing AI agents first perform reasoning and provide answers based on the designed questions. These results are then manually verified by annotators to check if any additional answers satisfy the constraints of the question.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To further discourage shortcut-based resolution, all questions are tested using GPT-4o and DeepSeek, both operating in web-enabled search mode. If both models consistently retrieve the correct answer with minimal effort, the query will be revised to increase its complexity by methods including introducing implicit constraints or obfuscating key lexical signals.”
plaindirectn = 480- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“they must be concrete and specific enough to exclude overly generic or widely known common-sense facts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“A two-stage quality control protocol is applied to strive for high question difficulty and answer uniqueness.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To address these limitations in Chinese retrieval benchmarking, BrowseComp-ZH introduces three key innovations: (1) reverse-designed tasks in native Chinese to avoid translation artifacts; (2) rigorous multi-step validation to ensure high retrieval difficulty and answer verifiability; and (3) broad model coverage for evaluating both open-source and proprietary agents.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Each sample includes at least one authoritative source URL that validates the logical connection between the query constraints and the target answer.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, since many answers can be retrieved via simple keyword searches, these benchmarks often fail to evaluate an agent's ability to plan complex search trajectories and synthesize information across documents.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As a result, existing English-centric benchmarks struggle to generalize, leaving a critical gap in assessing real-world browsing capabilities in non-English environments.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Stage 2 focuses on ensuring the uniqueness of the answers.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This study introduces BrowseComp-ZH, the first benchmark specifically designed to evaluate the web browsing and reasoning capabilities of large language models (LLMs) in the Chinese information environment.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“This trend is consistent with the observations reported in BrowseComp.”
What the sources say
- Not matchedcites BrowseComp
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“In the evaluation, we followed the BrowseComp instructions for model predictions.”
What the sources say
- Not matchedcites BrowseComp
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results reflect the inherent challenges of the Chinese web environment, where fragmented information and inconsistent indexing complicate single-shot search strategies.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“we implement a rigorous two-stage quality control pipeline that includes a three-engine keyword validation process and human-in-the-loop verification”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“increasing both the sample size and the diversity of question types would improve its representativeness”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“ensuring that answers are both difficult to retrieve and unambiguous”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“We adopt the same grading prompt as used in BrowseComp and employ GPT-4o for the grading process.”
What the sources say
- Not matchedcites BrowseComp
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
The paper relies on this without support.
All versions of DeepSeek's search system perform only a single round of retrieval.
“all versions of DeepSeek's search system perform only a single round of retrieval”
The paper relies on this without support.
The dynamic nature of the web means that factual answers may evolve or become inconsistent over time.
“the dynamic nature of the web means that factual answers may evolve or become inconsistent over time”
The paper relies on this without support.
DeepSeek R1, O1, and Gemini-2.5-Pro are models without browsing capabilities and rely solely on internal world knowledge.
“even models without browsing capabilities, such as DeepSeek R1, O1, and Gemini-2.5-Pro, which rely solely on their internal world knowledge”
The paper relies on this without support.
The tasks in BrowseComp-ZH require both extensive world knowledge and strong reasoning abilities.
“the tasks in our proposed BrowseComp-ZH benchmark require both extensive world knowledge and strong reasoning abilities”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These benchmarks typically evaluate agents using blackbox web search APIs to retrieve supporting documents in real time (Zhou et al., 2025; Chen et al., 2025).”
From BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents (2026) · cites this paper as Zhou et al., 2025
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Current benchmarks typically focus on assessing isolated capabilities-such as web browsing and information retrieval [28, 38, 39], or generative abilities disconnected from real-time information acquisition [23, 1, 31].”
From DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents (2025) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.4 Performance · p. 8
“| Grok3 (Research) | AI Search Product | - | Y | 12.9% | 39 | xAI |”
§ 3.2 Quality Control · p. 5
“resulting in a final benchmark of 289 validated questions”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.4 Performance · p. 8
“| Yuanbao (Hunyuan Model) | AI Search Product | - | Y | 12.2% | 56 | Tencent |”
§ 3.2 Quality Control · p. 5
“resulting in a final benchmark of 289 validated questions”
Citations
Cites
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 1×
“In parallel, retrieval-augmented generation (RAG) frameworks Guu et al. (2020); Lewis et al. (2020) have been widely adopted in QA Wiratunga et al. (2024), summarization Edge et al. (2024), and fact verification Martin et al. (2024), serving as a mechanism for injecting external knowledge into LLMs.”
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · cited 1×
“To assess retrieval capabilities, a variety of widely used English benchmarks have been proposed, including TriviaQA Joshi et al. (2017), HotpotQA Yang et al. (2018), FEVER Thorne et al. (2018), KILT Petroni et al. (2020), and GAIA Mialon et al. (2023).”
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · cited 1×
“To assess retrieval capabilities, a variety of widely used English benchmarks have been proposed, including TriviaQA Joshi et al. (2017), HotpotQA Yang et al. (2018), FEVER Thorne et al. (2018), KILT Petroni et al. (2020), and GAIA Mialon et al. (2023).”
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D. S., & Zettlemoyer, L. (2017) · arXiv · doi:10.48550/arxiv.1705.03551 · Cited by 1,650
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343 · cited 1×
“To assess retrieval capabilities, a variety of widely used English benchmarks have been proposed, including TriviaQA Joshi et al. (2017), HotpotQA Yang et al. (2018), FEVER Thorne et al. (2018), KILT Petroni et al. (2020), and GAIA Mialon et al. (2023).”
Toolformer: Language Models Can Teach Themselves to Use Tools
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2302.04761 · Cited by 1,150 · cited 1×
“Representative works such as WebGPT Nakano et al. (2021), Toolformer Schick et al. (2023), and ReAct Yao et al. (2023) explore how LLMs leverage search engines, tool usage, and reasoning strategies to tackle complex question-answering tasks.”
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · cited 1×
“As large language models (LLMs) evolve from static knowledge repositories to dynamic agents capable of using external tools, tasks that involve web browsing have emerged as a critical lens through which to evaluate their real-world reasoning and information-seeking capabilities Xi et al. (2025).”
KILT: a Benchmark for Knowledge Intensive Language Tasks
Petroni, F., Piktus, A., Fan, A., Lewis, P., Yazdani, M., Cao, N. D., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., et al. (2020) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2009.02252 · Cited by 940
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671470 · Cited by 827
ReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608 · cited 1×
“Representative works such as WebGPT Nakano et al. (2021), Toolformer Schick et al. (2023), and ReAct Yao et al. (2023) explore how LLMs leverage search engines, tool usage, and reasoning strategies to tackle complex question-answering tasks.”
Cited by
Sorted by citations
WebSailor: Navigating Super-human Reasoning for Web AgentPreprint
Li, K., Zhang, Z., Yin, H., Zhang, L., Ou, L., Wu, J., Yin, W., Li, B., Tao, Z., Wang, X., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.02592 · Cited by 319
UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement LearningPreprint
Wang, H., Zou, H., Song, H., Feng, J., Fang, J., Lu, J., Liu, L., Luo, Q., Liang, S., Huang, S., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.02544 · Cited by 209
“For browser use , Online-Mind2Web [77] contains 300 realistic tasks across 136 websites, while BrowseComp-en [73] and BrowseComp-zh [88] provide high-difficulty multi-hop questions.”
Self-citationWebDancer: Towards Autonomous Information Seeking Agency
Wu, J.-L., Li, B., Fang, R.-N., et al. (2025) · Neural Information Processing Systems · doi:10.48550/arxiv.2505.22648 · Cited by 204
ReSum: Unlocking Long-Horizon Search Intelligence via Context SummarizationPreprint
Wu, X., Li, K., Zhao, Y., Zhang, L., Ou, L., Yin, H., Zhang, Z., Yu, X., Zhang, D., Jiang, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.13313 · Cited by 166
“The key modification of ReSum-RL is that ReSum naturally segments long trajectories into 1 https://huggingface.co/Qwen/Qwen3-30B-A3B-Thinking-2507 multiple episodes when summarization occurs.”
InfluentialWebShaper: Agentically Data Synthesizing via Information-Seeking Formalization
Tao, Z.-W., Wu, J.-L., Yin, W., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2507.15061 · Cited by 124
“Concurrently, several recent benchmarks, such as GAIA (Mialon et al., 2023), BrowseComp (Wei et al., 2025), and BrowseComp-zh (Zhou et al., 2025), provide only test sets, which restricts their applicability for training agents.”
AgentFold: Long-Horizon Web Agents with Proactive Context ManagementPreprint
Ye, R., Zhang, Z., Li, K., Yin, H., Tao, Z., Zhao, Y., Su, L., Zhang, L., Qiao, Z., Wang, X., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.24699 · Cited by 126
“We consider 3 information-seeking benchmarks including BrowseComp (Wei et al., 2025), BrowseComp-ZH (Zhou et al., 2025a), and WideSearch-en (the most detailed metric: Item-F1) (Wong et al., 2025); and 1 general benchmark: GAIA (text-only subset) (Mialon et al., 2023).”
MiroMind Team, Bai, S., Bing, L., Chen, C., Chen, G., Chen, Y., Chen, Z., Chen, Z., Dai, Y., Dong, X., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.11793 · Cited by 97
“A similar pattern is observed on the BrowseComp-ZH [26] benchmark, with a 6.1-point improvement over GLM-4.”
InfluentialDeep Research: A Survey of Autonomous Research AgentsPreprint
Zhang, W., Li, X., Zhang, Y., Jia, P., Wang, Y., Guo, H., Liu, Y., & Zhao, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.12752 · Cited by 88
WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon AgentsPreprint
Qiao, Z., Chen, G., Chen, X., Yu, D., Yin, W., Wang, X., Zhang, Z., Li, B., Yin, H., Li, K., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.13309 · Cited by 82
“• BrowseComp-zh (Zhou et al., 2025) -”
Seed1.8 Model Card: Towards Generalized Real-World AgencyPreprint
Seed, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.20633 · Cited by 59
“8 on GAIA [44], BrowseComp-en [77], BrowseComp-zh [95], MM-BrowseComp [35], WideSearch [78], HLE [51], SWE-Bench Verified [47], Multi-SWE-Bench [87], AInstein-SWE-Bench, Terminal Bench 2.0 [64], U-Artifacts (internal), DeepConsult [62], DeepResearchBench [18], DiscoX [91], BFCL-v4 [50], and τ 2…”
InfluentialLi, K., Zhang, Z., Yin, H., Ye, R., Zhao, Y., Zhang, L., Ou, L., Zhang, D., Wu, X., Wu, J., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.13305 · Cited by 60
“This improvement successfully translates to our validation benchmarks, where performance on both BrowseComp-EN and BrowseComp-ZH shows a corresponding, albeit oscillating, upward trajectory.”
InfluentialTowards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs
Li, Y., Zhang, W., Yang, Y., Huang, W.-C., Wu, Y., Luo, J., Bei, Y., Zou, H. P., Luo, X., Zhao, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.09477 · Cited by 2
REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search AgentsPreprint
Chu, Z., Wang, X. Y., Hong, J., Fan, H., Huang, Y., Yang, Y., Xu, G., Zhao, C., Xiang, C., Hu, S., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.14234 · Cited by 60
“Our evaluation suite includes: Humanity’s Last Exam [26], BrowseComp [40], BrowseComp-ZH [47], GAIA [24].”
From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning AgentsPreprint
Zhang, W., Li, Y., Bei, Y., Luo, J., Wan, G., Yang, L., Xie, C., Yang, Y., Huang, W., Miao, C., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.18959 · Cited by 64
“In comparison, the OpenAI Deep Research agent achieves significantly higher scores—51.5% on BrowseComp, 42.9% on BrowseComp-ZH, and 26.6% on HLE—demonstrating the effectiveness of reasoning-integrated search in advancing the frontier of intelligent information-seeking systems.”
InfluentialSelf-citationScaling Agents via Continual Pre-trainingPreprint
Su, L., Zhang, Z., Li, G., Chen, Z., Wang, C., Song, M., Wang, X., Li, K., Wu, J., Chen, X., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.13310 · Cited by 39
“Open-source deep research agents have witnessed remarkable growth, achieving impressive results on deep research benchmarks including BrowseComp-en (Wei et al., 2025), BrowseComp-zh (Zhou et al., 2025b) Xbench-DeepSearch (Xbench-Team, 2025), and GAIA (Mialon et al., 2023).”
InfluentialMM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents
Li, S.-L., Bu, X.-Y., Wang, W., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2508.13186 · Cited by 45
“However, BrowseComp and its derivative works [3, 56] focus solely on textual information and overlook the need for multimodal understanding.”
Self-citationOpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis
Li, Z., Jiang, D., Ma, X.-G., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2603.20278 · Cited by 38
OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Data
Du, Y., Ye, R., Tang, S., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2603.15594 · Cited by 38
“Notably, on BrowseComp-ZH, OpenSeeker surpasses RedResearcher by 21.6% (48.4% vs 26.8%).”
InfluentialWang, W., Xu, X., An, W., Dai, F., Gao, W., He, Y., Huang, J., Ji, Q., Jin, H., Li, X., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.24873 · Cited by 32
Self-citationMiroThinker-1.7 & H1: Towards Heavy-Duty Research Agents via VerificationPreprint
M. Team, Bai, S. K., Bing, L., Lei, L., Li, R., Li, X. W., Lin, X. S., Min, E., Su, L., Wang, B. L., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.15726 · Cited by 37
“As shown in Figure 1, our flagship system MiroThinker-H1 achieves 88.2 and 84.4 on BrowseComp [8] and BrowseComp-ZH [9], respectively, outperforming best open-source and commercial research agents.”
InfluentialSelf-citationFinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning
Hu, L., Jiao, J., Liu, J., Ren, Y., Wen, Z., Zhang, K., Zhang, X., Gao, X., He, T., Hu, F., et al. (2025) · arXiv · Cited by 2
“…proposed for evaluating browsing capabilities, while they fall short along two axes: ( i ) General-purpose browsing benchmarks, like BrowseComp [23], BrowseComp-ZH [27], and BrowseComp-Plus [7], are intentionally domain-agnostic and center on lookup-oriented tasks with short, verifiable targets.”
MedBrowseComp: Benchmarking Medical Deep Research and Computer Use
Chen, S., Moreira, P., Xiao, Y., Schmidgall, S., Warner, J., Aerts, H., Hartvigsen, T., Gallifant, J., & Bitterman, D. S. (2025) · arXiv · doi:10.48550/arxiv.2505.14963
“BrowseComp and its offshoot BrowseComp-ZH push AI agents to perform complex web searches for difficult-to-find facts [28, 33].”
Lin, M., Wu, Z., Xu, Z., Liu, H., Tang, X., He, Q., Aggarwal, C. C., Liu, H., Zhang, X., & Wang, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.16724 · Cited by 35
“These benchmarks jointly evaluate the agent’s ability to (i) retrieve relevant information and (ii) synthesize evidence into correct, verifiable answers.”
LongCat-Flash-Thinking-2601 Technical ReportPreprint
Meituan LongCat Team, Gui, A., Li, B., Tao, B., Zhou, B., Chen, B., Zhang, C., Zhang, C., Gao, C., Zhang, C., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.16725
“We evaluate agentic search capability on BrowseComp [Wei et al., 2025] and BrowseComp-ZH [Zhou et al., 2025], and report results under both settings with and without context management.”
Self-citationTongyi DeepResearch Technical ReportPreprint
Tongyi DeepResearch Team, Li, B., Zhang, B., Zhang, D., Huang, F., Li, G., Chen, G., Yin, H., Wu, J., Zhou, J., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.24701 · Cited by 295
DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking
Lan, T., Zhu, B., Jia, Q., Ren, J., Li, H., Wang, L., Xu, Z., Luo, W., & Zhang, K. (2025) · arXiv · doi:10.48550/arxiv.2510.20168
“Existing deep search benchmarks such as GAIA [Mialon et al., 2023], BrowseComp [Wei et al., 2025] and BrowseComp-zh [Zhou et al., 2025b] require agents to employ multi-hop web browsing and deep reasoning to identify target answers.”
Self-citationSu, S., Xing, S., Dong, X., Zhong, M., Wang, B., Zhu, X., Chen, Y., Wang, W., Deng, Y., Zhu, P., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.22808 · Cited by 22
“MiroFlow achieves state-of-the-art results on all major benchmarks, including GAIA [34], BrowseComp-EN/ZH [67, 77], HLE [44], xBench-DS [9], and Fu-tureX [75], outperforming both commercial/closed-source and open-source agent frameworks.”
The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents
Xuan, W., Zeng, Q., Qi, H.-L., et al. (2026) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2601.07264 · Cited by 19
Zhang, J.-J., Lv, X., Feng, L., et al. (2026) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2601.06021 · Cited by 18
“Considering the relatively small dataset size of BrowseComp-ZH, xbench-DeepSearch, and GAIA, we repeat their evaluation 3 times and report the average accuracy.”
Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric DesignPreprint
Zhu, B., Jia, Q., Lan, T., Ren, J., Gu, F., Jiang, F., Wang, L., Xu, Z., & Luo, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.28376 · Cited by 17
Self-citationLearning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generation
Lv, C., Zhou, J., Zhao, W., Xu, J., Dou, S., Huang, Z., Tian, M., Wang, X., Liu, Y., Zhou, P., et al. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.48550/arxiv.2602.03619 · Cited by 20
“Benchmarks such as GAIA (Mialon et al., 2023; Russell et al., 2025), BrowseComp (Wei et al., 2025; Zhou et al., 2025), and HLE (Phan et al., 2025) provide verifiable targets, enabling agent training via Reinforcement Learning with Verifiable Rewards (RLVR) (Jin et al., 2025; Liu et al., 2025).”
Self-citationRethinking Agentic Search with Pi-Serini: Is Lexical Retrieval Sufficient?
Hsu, T.-H., Yang, J.-H., & Lin, J. (2026) · arXiv.org · doi:10.48550/arxiv.2605.10848 · Cited by 16
“…their evaluation has also shifted toward more realistic research settings: Bench-marks such as BrowseComp (Wei et al., 2025) and BrowseComp-ZH (Zhou et al., 2025) evaluate open-web browsing ability, while BrowseComp-Plus provides a fixed-corpus setting for controlled comparison of search…”
BrowseMaster: Towards Scalable Web Browsing via Tool-Augmented Programmatic Agent PairPreprint
Pang, X., Tang, S., Ye, R., Du, Y., Du, Y., & Chen, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.09129 · Cited by 17
“Evaluation employs xVerify-9B (Chen et al., 2025a) for BrowseComp, BrowseComp-zh, and xBench-DeepResearch, GPT-4o (OpenAI, 2024a) for WebWalkerQA and GAIA following Wu et al. (2025b).”
InfluentialParallelMuse: Agentic Parallel Thinking for Deep Information SeekingPreprint
Li, B., Zhang, D., Wu, J., Yin, W., Tao, Z., Zhao, Y., Zhang, L., Shen, H., Fang, R., Xie, P., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.24698 · Cited by 15
“These benchmarks jointly assess both deep search and reasoning capabilities, with BrowseComp and BrowseComp-zh placing greater emphasis on deep search, HLE focusing more on reasoning, and GAIA providing a balanced evaluation across both dimensions.”
InfluentialRE-TRAC: REcursive TRAjectory Compression for Deep Search Agents
Zhu, J., Zhang, G.-R., Ma, X., Xu, L., Zhang, M., Yang, R., Wang, S., Qiu, K., Wu, Z.-R., Dai, Q., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.02486 · Cited by 13
“Our RE-TRAC-30B-A3B model consistently achieves 8% ∼ 10% improvement on BrowseComp, BrowseComp-ZH, GAIA and XBench, compared with its base model Tongyi-DeepResearch, which is also the previous strongest baseline with 30B parameters.”
InfluentialPushing Test-Time Scaling Limits of Deep Search with Asymmetric VerificationPreprint
Zeng, W., He, K., Kuang, C., Li, X., & He, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.06135 · Cited by 9
“On BrowseComp-zh, GLM-4.”
InfluentialReinforcement Learning Foundations for Deep Research Systems: A SurveyPreprint
Li, W., Chen, Z., Lin, J., Cao, H., Han, W., Liang, S., Zhang, Z., Dong, K., Li, D., Zhang, C., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.06733 · Cited by 14
Self-citationLongSeeker: Elastic Context Orchestration for Long-Horizon Search AgentsPreprint
Lu, Y., Ye, R., Du, Y., Wang, J., Liu, S., & Chen, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.05191 · Cited by 10
“We train LongSeeker on 10k synthesized trajectories and achieve 61.5% on BrowseComp and 62.5% on BrowseComp-ZH, outperforming strong long-horizon search baselines.”
InfluentialLiang, J., Han, J., Li, W., Wang, X., Zhang, Z., Jiang, Z., Liao, Y., Li, T., Huang, Y., Shen, H., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.17091 · Cited by 12
“(2) BrowseComp-ZH [27]: A benchmark specifically designed to measure multi-hop web search and chain-based reasoning within the Chinese web ecosystem.”
OffSeeker: Online Reinforcement Learning Is Not All You Need for Deep Research Agents
Zhou, Y.-H., Zheng, K., Chen, Q.-G., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2601.18467 · Cited by 12
Self-citationInteractComp: Evaluating Search Agents With Ambiguous QueriesPreprint
Deng, M., Huang, L., Fan, Y., Kong, F., Zhang, J., Ren, F., Bai, J., Yang, F., Miao, D., Yu, Z., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.24668 · Cited by 14
“Interaction benchmarks like IN3 (Qian et al., 2024) and Tau-Bench (Yao et al., 2024) focus on general conversation but lack grounding in verifiable search tasks.”
InfoDeepSeek: Benchmarking Agentic Information Seeking for Retrieval-Augmented GenerationPreprint
Xi, Y., Lin, J., Zhu, M., Xiao, Y., Ou, Z., Liu, J., Wan, T., Chen, B., Liu, W., Wang, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.15872 · Cited by 34
“Most benchmarks include assessing generation quality, i.e. , answer accuracy [51, 58, 2, 54, 48, 43, 64].”
Revisiting Text Ranking in Deep Research
Meng, C., Ou, L., MacAvaney, S., & Dalton, D. J. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808557 · Cited by 4
“Motivation .”
Repurposing Synthetic Data for Fine-grained Search Agent SupervisionPreprint
Zhao, Y., Li, K., Wu, X., Zhang, L., Zhang, D., Li, B., Song, M., Chen, Z., Wang, C., Wang, X., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.24694 · Cited by 11
“Notably, Web-30B-E-GRPO achieves the best performance among open-source agents on BrowseComp (12.9) and BrowseComp-ZH (26.4), even surpassing advanced models like Claude-4-Sonnet on BrowseComp, and narrows the gap with others.”
InfluentialRhinoInsight: Improving Deep Research through Control Mechanisms for Model Behavior and ContextPreprint
Lei, Y., Si, S., Wang, W., Wu, Y., Chen, G., Qi, F., & Sun, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.18743 · Cited by 11
“• BrowseComp-ZH (Zhou et al., 2025) is the first high-difficulty benchmark for evaluating real-world web-browsing and reasoning capabilities of LLMs within the Chinese information ecosystem.”
SAM: State-Adaptive Memory for Long-Horizon Reasoning AgentPreprint
Hu, Y., Qian, H., Wang, S., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.24468
“…agent benchmarks that stress complementary aspects of memory-intensive reasoning: BrowseComp [34] (long-range web browsing), BrowseComp-ZH [46] (cross-lingual multi-hop search), WideSearch [35] (broad exploration over large search spaces), and HLE [24] (knowledge-intensive scientific…”
AgentSwing: Adaptive Parallel Context Management Routing for Long-Horizon Web Agents
Feng, Z., Su, L., Zhang, Z., Wang, X., Zhang, X., Wang, X., Fang, R., Zhang, Q., Li, B., Cai, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2603.27490
“We evaluate AgentSwing on three challenging deep information-seeking benchmarks: BrowseComp (Wei et al., 2025), BrowseComp-ZH (Zhou et al., 2025), and Humanity’s Last Exam (HLE) (Phan et al., 2025).”
InfluentialVeriWeb: Verifiable Long-Chain Web Benchmark for Agentic Information-Seeking
Liu, S., Liu, M., Zhou, H., Cui, Z., Zhou, Y., Zhou, Y., Gao, J., Zhou, H., Yang, Y., Fan, W., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.04026
SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering AgentsPreprint
Zheng, P., Shang, Z., Jiang, S., Tian, W., Zhu, D., Ma, Z., Yuan, D., & Zhang, Q. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.08149
“Recent agentic benchmarks focus on assessments on web search [12, 34, 41], productivity workflows [29, 30, 42], cybersecurity [19, 33, 43], and software engineering [10, 14, 16, 24, 36].”
FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents
Deng, J.-L., Chen, Y.-M., Xiang, X.-Q., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.12087 · Cited by 8
“This appendix provides the graph-expansion algorithm, fact and fuzzing examples, and refinement cases accompanying Section 3.”
Masking Stale Observations Helps Search Agents - Until It Doesn't: A Regime Map and Its Mechanism
Zhang, H., Xu, Q.-X., Li, Z.-F., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.00408 · Cited by 8
“4 on BrowseComp-ZH), while the strongest backbone DS-V4-Flash-Max is saturated everywhere ( +0 .”
InfluentialOpenSeeker-v2: Pushing the Limits of Search Agents with Informative and High-Difficulty TrajectoriesPreprint
Du, Y., Ye, R., Tang, S., Huang, K., Zhu, X., Cai, Y., & Chen, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.04036 · Cited by 9
“Though trained with only 10.6k samples, OpenSeeker-v2 achieves a new SOTA across four representative benchmarks: 46.0% on BrowseComp, 58.1% on BrowseComp-ZH, 34.6% on Humanity’s Last Exam, and 78.0% on xbench, significantly outperforms Tongyi DeepResearch and RedSearcher that are extensively trained via CPT, SFT, and…”
InfluentialTowards Efficient Agents: A Co-Design of Inference Architecture and SystemPreprint
Lin, W., Zhen, H., Yang, S., Wang, X. Y., Liu, R., Chen, H., Zhang, W., Zhou, C., Li, Y., Chen, C., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.18337 · Cited by 9
“• Tasks: Experiments were conducted on the BrowseComp-zh (Chinese version) [38] benchmark, which involves multi-step tasks requiring web searches.”
EcomBench: Towards Holistic Evaluation of Foundation Agents in E-commercePreprint
Min, R., Qiao, Z., Xu, Z., Zhai, J., Gao, W., Chen, X., Sun, H., Zhang, Z., Wang, X., Zhou, H., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.08868 · Cited by 11
“At its core, assessing the foundational capabilities of these advanced LLM agents has become essential, where a wide range of benchmarks have been proposed (Mialon et al., 2023; Wei et al., 2025; Zhou et al., 2025; Xbench Team, 2025; Yao et al., 2022; Wong et al., 2025; Yao et al., 2024).”
InfoAgent: Advancing Autonomous Information-Seeking AgentsPreprint
Zhang, G., Zhu, J., Yang, R., Qiu, K., Zhang, M., Wu, Z.-R., Dai, Q., Liu, B., Luo, C., Yang, Z., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.25189 · Cited by 13
STAMP: Provenance-Guided Credit Assignment for Deep Search Agents
Xu, K., Xu, H., Chen, X.-R., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.11172 · Cited by 6
“On Qwen3-30B-A3B-Thinking-2507 (Team, 2025), we evaluate STAMP in a live-web deep-search setting on BrowseComp (Wei et al., 2025), BrowseComp-ZH (Zhou et al., 2025), and xbench-DS (Xbench-Team, 2025).”
InfluentialS1-DeepResearch: Beyond Search, Toward Real-World Long-Horizon Research AgentsPreprint
Dong, Y. H., Xiao, X., Dong, L., Jin, X., Li, Z., Zhang, H., 王杜云, & Xu, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.15367 · Cited by 5
PBSD: Privileged Bayesian Self-Distillation for Long-Horizon Credit AssignmentPreprint
Tian, Y., Wang, R., Wen, X., Li, J., Sun, S., (30203), L. S., Bian, J., & Zhao, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.09348 · Cited by 7
“BrowseComp-ZH [34] extends this setting to the Chinese web, introducing multi-hop questions that require retrieval and reasoning over Chinese information sources.”
Argus: Evidence Assembly for Scalable Deep Research AgentsPreprint
Zhang, Z., Su, L., Chen, Z., Lin, X., Xu, H., Du, S. S., Yang, K., An, B., Bing, L., & Wang, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.16217 · Cited by 5
“BrowseComp [33] and its Chinese counterpart BrowseComp-ZH [34] probe multi-step web browsing on adversarially constructed factual questions that resist single-hop search. xbench DeepSearch-2510 [35] targets deep search and tool use through professionally annotated Chinese tasks with dynamic updates.”
DLLM Agent: See Farther, Run Faster
Zhen, H., Lin, W.-Z., Liu, R.-X., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.07451 · Cited by 6
“We adopt the public B ROWSE C OMP - ZH benchmark [35], which requires multi-turn web browsing, iterative retrieval, and reasoning over dynamically collected evidence to answer user queries.”
InfluentialDeep Search with Hierarchical Meta-Cognitive Monitoring Inspired by Cognitive Neuroscience
Sun, Z.-X., Wang, Q., Yu, W.-J., et al. (2026) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3805712.3809549 · Cited by 6
Nested Browser-Use Learning for Agentic Information Seeking
Li, B., Wu, J.-L., Yin, W., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2512.23647 · Cited by 6
“…agents (Team, 2025c; x.ai, 2025; AI, 2025; Team, 2025a; Gemini, 2025), powered by large language models (LLMs) have achieved strong performance on a variety of challenging wide and deep search tasks (Mialon et al., 2023; Wei et al., 2025; Zhou et al., 2025; Xbench-Team, 2025; Wong et al., 2025).”
BrowseConf: Confidence-Guided Test-Time Scaling for Web AgentsPreprint
Ou, L., Li, K., Yin, H., Zhang, L., Zhang, Z., Wu, X., Ye, R., Qiao, Z., Xie, P., Zhou, J., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.23458 · Cited by 7
“1 models, evaluating them on two challenging information-seeking benchmarks: BrowseComp (Wei et al., 2025), which tests the ability to locate hard-to-find information on the web, and its Chinese counterpart, BrowseComp-zh (Zhou et al., 2025a).”
ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use CapabilityPreprint
Liu, W., Bi, J., Zhou, W., Feng, J., Ma, Y., Han, A., & Lu, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.14145 · Cited by 9
“…into the reasoning loop, LLM-based agents have achieved strong performance across a wide spectrum of downstream tasks, spanning text-based question answering [7, 8, 9, 10], visual question answering [11, 12, 13], long-form text report generation [14, 15], and domain-grounded tool use [16, 17].”
Mach-Mind-4-Flash Technical ReportPreprint
Foundation Model Team (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.09375 · Cited by 6
“We assess deep-search ability with BrowseComp / BrowseComp-zh [80; 81] and X-Bench [82] (persistent multi-constraint web browsing).”
LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know?Preprint
Fan, H., Wang, X., Chu, Z., Wang, Q., Wang, Z., Liu, M., Qin, B., & XingYu (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.28721 · Cited by 7
“We conduct the pilot study on four challenging agentic benchmarks: BrowseComp [12], BrowseComp-ZH [24], HLE [25], and GAIA [26].”
DR-Venus: Towards Frontier Edge-Scale Deep Research Agents with Only 10K Open Data
Venus Team, Dai, S.-H., Deng, Y., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2604.19859 · Cited by 5
InfoSeeker: A Scalable Hierarchical Parallel Agent Framework for Web Information SeekingPreprint
Lee, K. Y., Huang, Y., He, Z., Zhou, H., Luo, W., Shao, K., Fang, M., & Wang, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.02971
Nanbeige4.1-3B: A Small General Model that Reasons, Aligns, and ActsPreprint
Yang, C., Peng, G., Zhu, J., Le, R., Feng, R., Zhang, T., Xu, X., Song, Y., Jia, Y., Wen, Y., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.13367
“We evaluate our model on a range of long-horizon benchmarks, including GAIA [9], BrowseComp [16], BrowseComp-ZH [23], Humanity’s Last Exam (HLE) [12], SEAL-0 [11], xBench-DeepSearch-2505 [3], and xBench-DeepSearch-2510 [3].”
Table-as-Search: Formulate Long-Horizon Agentic Information Seeking as Table CompletionPreprint
Lan, T., Henry, F., Zhu, B., Jia, Q., Ren, J., Pu, Q., Li, H., Wang, L., Xu, Z., & Luo, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.06724 · Cited by 3
“To rigorously evaluate TaS across distinct long-horizon agentic infoseeking, we employ three categories of benchmarks: (1) Deep Search : We utilize GAIA (text-only) (Mialon et al., 2023) and BrowseComp-ZH (Zhou et al., 2025) to assess multi-step reasoning and precise filtering capabilities.”
InfluentialSelf-citationThink in Parallel, Answer as One: Logit Averaging for Open-Ended ReasoningPreprint
Wang, H., Du, C., Kawaguchi, K., & Pang, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.02874 · Cited by 4
PhysLab: A Benchmark Dataset for Multi-Granularity Visual Parsing of Physics Experiments
Zou, M., Zeng, Q., Miao, Y., Liu, S., Wang, Z., Liu, H., & Zhou, W. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3758221
“To ensure annotation quality, we implemented multi-round cross-checking and sample-based review procedures [64].”
ActiveMem: Dynamic Latent Memory Trees for Long-Horizon AgentsPreprint
Wu, S.-L., Wang, J., Du, Z., & Gan, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33244 · Cited by 2
“Regarding evaluation, we consider three information-seeking benchmarks—BrowseComp (Wei et al. 2025), BrowseComp-ZH (Zhou et al. 2025a), and WideSearch-en (Wong et al. 2025)—alongside the text-only subset of the general agent benchmark GAIA (Mialon et al. 2024).”
LongCat Sparse Attention: Taming the Lightning via Streaming-aware Hierarchical Cross-Layer IndexingPreprint
Zan, W., Zhang, J., Tan, J., Liu, H., Wang, C., Li, X., Ma, D., Wu, G., Lu, Y., Li, F., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.01662 · Cited by 7
“…[Jimenez et al., 2023], and TerminalBench [Merrill et al., 2026]), agentic tool use ( τ 2 -Bench [Barres et al., 2025], VitaBench [He et al., 2025], and MCP-Atlas [Bandi et al., 2026]), and agentic search (BrowseComp [Wei et al., 2025, Zhou et al., 2025] and RWSearch [Zhu et al., 2026]).”
Zhang, C., Dong, K., Li, D., Li, W., Yang, Q., Han, W., & Liu, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.07773 · Cited by 2
Self-citationInteraction as Intelligence: Deep Research With Human-AI Partnership
Ye, L., Cai, X., Wang, X., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2507.15759 · Cited by 4
“…our hypothesis that experts with higher cognitive capabilities demonstrate enhanced collaboration with AI in transparent dialogue environments, we measured our system performance through the accuracy of browsecomp-ZH (Zhou et al., 2025), a benchmark assessing agents’ web browsing capabilities.”
BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent
Chen, Z., Ma, X., Zhuang, S., Nie, P., Zou, K., Liu, A., Green, J., Patel, K., Meng, R., Su, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.06600 · Cited by 3
“Recent extensions of BrowseComp concepts, such as ZH-BrowseComp [4] and MedBrowseComp [5], further expand to multilingual queries and domain-specific challenges.”
Self-citationContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL
Pan, Z.-S., Pei, Q., Lu, J.-R., et al. (2026) · doi:10.48550/arxiv.2608.28476 · Cited by 3
“The evaluation on deep search tasks includes GAIA (the text-only subset with 103 examples) [38], BrowseComp [3], BrowseComp-ZH [39], and xBench-DeepSearch [40].”
ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit AssignmentPreprint
Lu, Y., Ye, R., Wang, J., Du, Y., Jin, T., Liu, S., & Chen, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.05102 · Cited by 4
SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search AgentsPreprint
Liang, Z., Liu, Y., Huang, Y., Zhao, H., Huang, B., Wang, J., Duan, N., & Cao, Q. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.05212 · Cited by 3
“Queries are drawn from five deep-search benchmarks: BrowseComp (Wei et al. 2025), BrowseComp-ZH (Zhou et al. 2025), DeepSearchQA (Gupta et al. 2026), Seal-0 (Pham et al. 2026), and xBench-DeepSearch (Chen et al. 2025).”
SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified TaskPreprint
Mei, L., Yu, X., Chen, C., Liu, L., Chen, X., Ma, J., Feng, C., Huang, L., Mo, S., Kang, S., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.24850 · Cited by 6
“A similar pattern is observed on BrowseComp-zh: SearchArt-27B has a mean of 18.5 turns compared with 17.2 for Qwen3.”
InfluentialAgentCompass: A Unified Evaluation Infrastructure for Agent CapabilitiesPreprint
Chen, K., Ding, Z., Ge, J., Jiang, S., Li, M., Li, Q., Li, Z., Li, Z., Liang, T., Liu, S., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.13705
Self-citationAgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
Xi, Z., Yang, D., Liu, J., Huang, J., Guo, H., Huang, B., Chen, T., Zhang, Q., 卢中杭, Liu, C., et al. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.2058
Self-citationLu, Y.-H., Zeng, Q.-C., Qi, H.-L., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.15345 · Cited by 3
“Recent browsing-agent benchmarks beyond English, such as BrowseComp-ZH (Zhou et al., 2025), broaden the linguistic scope of agent evaluation but remain primarily monolingual: ques-a tions, evidence, and answers all stay within the same language.”
SearchSwarm: Towards Delegation Intelligence in Agentic LLMs for Long-Horizon Deep ResearchPreprint
Lan, X., Chen, Q., Tao, K., Tang, X., Wang, T., Cao, Q., Kong, X., Wen, Z., Zhang, Z., & Zhou, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.09730 · Cited by 4
K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts
Lee, N., Yoon, D., Son, G., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.02404 · Cited by 3
“Following BrowseComp and BrowseComp-ZH (Wei et al., 2025; Zhou et al., 2025), we report expected calibration error over five equal-width confidence bins, computed as the weighted average gap between mean confidence and empirical accuracy.”
IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents
Feng, Y., Huang, Q., Xie, X.-Y., et al. (2026) · doi:10.48550/arxiv.2601.06676 · Cited by 3
“…Alzubi et al. 2025; Li et al. 2025) and end-to-end agents trained with reinforcement learning (Jin et al. 2025; Zheng et al. 2025 (Wei et al. 2025; Zhou et al. 2025), long-form writing (Bai et al. 2025; Wu et al. 2025), citation-grounded reports (Shao et al. 2024), and long-horizon research (Du…”
Zhou, Y., Yu, L., Si, S., Sun, Q., Wang, W., Wu, Y., Wen, H., Chen, G., Qi, F., & Sun, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.14244 · Cited by 6
“As shown in Table 6, on the noise-intensive BrowseComp-ZH , our method delivers dramatic gains.”
MARINE: Theoretical Optimization and Design for Multi-Agent Recursive IN-context EnhancementPreprint
Zhang, H., Lu, J., Du, Y., Gao, Y., Huang, L., Wang, B., Tan, F., & Zou, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.07898 · Cited by 3
DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
Du, M., Xu, B., Zhu, C., Wang, X., & Mao, Z. (2025) · arXiv · doi:10.48550/arxiv.2506.11763 · Cited by 2
“These include evaluations for agents in scientific domains [2, 12, 17, 3], creative writing [31, 1, 23], code generation and software engineering [10, 40, 22, 9, 32], and in their roles as human assistants, often enhanced by capabilities such as web Browse and tool-use [28, 38, 33, 6, 26].”
BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents
Chen, Z., Ma, X., Zhuang, S., Nie, P., Zou, K., Sharifymoghaddam, S., Liu, A., Green, J., Patel, K., Meng, R., et al. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.1023 · Cited by 2
“These benchmarks typically evaluate agents using black-box web search APIs to retrieve supporting documents in real time (Zhou et al., 2025; Chen et al., 2025).”
Self-citationLast Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents
Li, Z., Yu, C.-Y., Zang, L., et al. (2026) · doi:10.48550/arxiv.2608.29685 · Cited by 2
“5-122B-A10B, averaged across two benchmarks (BrowseComp and BrowseComp-zh) are shown in Figure 5(a).”
ACE: Pluggable Adaptive Context Elasticizer across AgentsPreprint
Liao, N., Long, Z., Wang, X., Yang, X., Wang, Y., Zhuang, Z., Cai, X., Weng, R., & Yan, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.31564 · Cited by 3
GAIA-v2-LILT: Multilingual Adaptation of Agent Benchmark beyond Translation
Kim, Y., Uhlig, K., & Wuebker, J. (2026) · Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026) · doi:10.48550/arxiv.2604.24929 · Cited by 2
JADE: Expert-Grounded Dynamic Evaluation for Open-Ended Professional Tasks
Lin, L., Liu, J.-Y., Yang, T.-Y., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.06486 · Cited by 2
“By integrating multi-step reasoning with tool-use capabilities, these agents are increasingly deployed in complex, long-horizon professional workflows (Luo et al., 2025; Jimenez et al., 2024).”
Table-as-Search: Agentic Information Seeking is Table Completion
Lan, T., Henry, F., Zhu, B., Jia, Q., Ren, J., Pu, Q., Li, H., Wang, L., Xu, Z., & Luo, W. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.743 · Cited by 1
“To rigorously evaluate TaS across distinct long-horizon agentic infoseeking, we employ three categories of benchmarks: (1) Deep Search : We utilize GAIA (text-only) (Mialon et al., 2023) and BrowseComp-ZH (Zhou et al., 2025) to assess multi-step reasoning and precise filtering capabilities.”
InfluentialSelf-citationReportBench: Evaluating Deep Research Agents via Academic Survey Tasks
Li, M., Zeng, Y., Cheng, Z., Ma, C., & Jia, K. (2025) · arXiv · doi:10.48550/arxiv.2508.15804 · Cited by 1
FinHopper: A Progressive Multi-Hop Benchmark for Complex Financial Agents with Step-wise Evaluation
Wu, Y., Lin, M.-P., Ying, Q., Dan, L., Zhao, H.-X., Liu, Q.-R., Shen, Y.-T., Wu, S.-H., Luo, W.-H., Du, X.-K., et al. (2026) · doi:10.48550/arxiv.2610.26036 · Cited by 1
Avoiding What Your Teacher Already Knows: Entity Familiarity for Deep-Search DistillationPreprint
Liu, Y., Liang, Z., Tan, W., Wang, Y., Wang, J., Duan, N., & Cao, Q. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.16251 · Cited by 1
“Table 7 summarizes results on BrowseComp (BC), BrowseComp-ZH (BC-ZH) [43], SEAL-0 [23], the text-only subset of Humanity’s Last Exam with tools [24], and xbench-DeepSearch-2510 [35].”
Iris: Climbing to the Search FrontierPreprint
Liu, Z., Liu, H., Wang, Z., Qin, Y., Liang, J., Chu, X., Chen, S., Gu, Y., Luo, Z., Hu, Y., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.04304 · Cited by 1
“We evaluate our models on four challenging benchmarks: BrowseComp [23], BrowseComp-ZH [31], DeepSearchQA [8], and HLE [16].”
Self-citation
Metadata
- Authors
- Zhou, P., Leon, B., Ying, X., Zhang, C., Shao, Y., Ye, Q., Chong, D., Jin, Z., Xie, C., Cao, M., Gu, Y., Hong, S., Ren, J., Chen, J., Liu, C., & Hua, Y.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2504.19314
- OpenAlex
- W6929371760
- Semantic Scholar
- 34c20505098fd48ff19e4b5c4901febdd5d27181
- Cited by
- 0
- References
- 34
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- large language models, web browsing, benchmarking, multi-hop question answering, LLM agents, agentic search, information retrieval, information reconciliation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-22, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-22 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
