Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index; none of the 16 sources its claims cite is checked for support yet.

A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation

Open access

Abstract

We introduce ChronoQA, a benchmark dataset for Chinese question answering focused on evaluating temporal reasoning in Retrieval-Augmented Generation (RAG) systems. Built from over 300,000 news articles published between 2019 and 2024, ChronoQA contains 5,176 questions covering absolute, aggregate, and relative temporal types, with both explicit and implicit time expressions. The dataset features both single- and multi-document scenarios, reflecting real-world requirements for temporal alignment and logical consistency. By providing structured evaluation across a wide range of temporal tasks, ChronoQA offers a dynamic, reliable, and scalable resource for benchmarking RAG systems in evolving knowledge environments.

Overview

CoChat Summary

TL;DR

ChronoQA is a Chinese benchmark of 5,176 temporal-sensitive QA pairs built from over 300,000 news articles published between 2019 and 2024, designed to evaluate temporal reasoning in retrieval-augmented generation. It covers absolute, aggregate, and relative temporal types, explicit and implicit time expressions, and single- and multi-document scenarios.

Key findings

  • ChronoQA contains 5,176 QA pairs: absolute 2,529, aggregate 1,911, relative 736; explicit time expressions 2,000, implicit 3,176; single-document 3,261, multi-document 1,915 (37%); answer types entity 2,556, time 864, numerical 507, judgement 1,045, other 204.
  • Quality validation manually reviewed ~6,000 examples, finding over 95% met quality standards with Cohen’s Kappa = 0.85; all identified error pairs were removed.
  • Retrieval baselines at K = 10 showed Query Decomposition best overall: Recall 68.2, MAP 57.6, NDCG 78.9; on multiple-document questions: Recall 53.9, MAP 53.1, NDCG 83.2. Native RAG at K = 10 scored overall Recall 61.8, MAP 53.5, NDCG 71.8.
  • Error analysis of 100 cases found relative/implicit time expressions caused 69% of failures versus 31% for absolute/explicit; day-level granularity caused 62% of errors, month 27%, year 11%; retrieval failure accounted for 72% of incorrect answers, while 7% failed even with perfect retrieval.

Methods

  • Source articles came from public news sources (e.g., Sina News) covering January 1, 2019, to August 30, 2024, averaging approximately 171.8 articles per day and roughly 350k textual units; after cleaning and deduplication, gpt-4o-mini extracted 294,696 “intensive temporal paragraphs.”
  • Single temporal QA pairs were generated with gpt-4o from those paragraphs using structured prompts, yielding over 10,000 standalone pairs after automated filtering; multi-document questions were composed via semantic similarity (bge-large-zh-1.5 embeddings, cosine similarity) and temporal proximity, using parallel and series reasoning circuits with o1-mini.
  • Verification combined rule-based filtering, gpt-4o evaluation of fluency/factual accuracy/temporal relevance, and manual review; retrieval experiments compared Native RAG, Temporal Filter, Query Rewrite, and Query Decomposition at K = 5 and K = 10 using Recall, MAP, and NDCG.

Limitations

  • The paper does not state a dedicated limitations section.
  • Evident from the text: the dataset is Chinese-language and news-based, with source coverage bounded to January 1, 2019–August 30, 2024, so temporal and topical coverage is tied to that corpus despite the claimed scalable/updatable framework.
  • Evident from the text: manual validation covered ~6,000 examples rather than the full final set, error analysis examined only 100 cases, direct LLM performance is reported only in Fig. 8 without numeric values in the text, and retrieval baselines are limited to four methods at K = 5 and K = 10.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41597-025-06098-y

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Overview of construction process of the ChronoQA dataset.
Fig. 1 Overview of construction process of the ChronoQA dataset.CC BY-NC-ND · PubMed Central
Fig. 2 Yearly distribution of collected articles from 2019 to 2024.
Fig. 2 Yearly distribution of collected articles from 2019 to 2024.CC BY-NC-ND · PubMed Central
Fig. 3 The prompt template guiding the LLM for single temporal QA generation.
Fig. 3 The prompt template guiding the LLM for single temporal QA generation.CC BY-NC-ND · PubMed Central
Fig. 4 The composition process for multi-document reasoning circuits. (a) In a Parallel Circuit, two independent QA pairs from different documents are synthesized into a single query that requires aggregating both pieces of information. (b) In a Series Circuit, the answer from one QA pair (e.g., a date) serves as a necessary input to resolve the second question, forming a dependency chain.
Fig. 4 The composition process for multi-document reasoning circuits. (a) In a Parallel Circuit, two independent QA pairs from different documents are synthesized into a single query that requires aggregating both pieces of information. (b) In a Series Circuit, the answer from one QA pair (e.g., a date) serves as a necessary input to resolve the second question, forming a dependency chain.CC BY-NC-ND · PubMed Central
Fig. 5 Representative examples from ChronoQA.
Fig. 5 Representative examples from ChronoQA.CC BY-NC-ND · PubMed Central
Fig. 6 Distribution of question lengths (number of characters) in ChronoQA.
Fig. 6 Distribution of question lengths (number of characters) in ChronoQA.CC BY-NC-ND · PubMed Central
Fig. 7 Thematic distribution of source news articles in ChronoQA. The chart displays the proportion of articles across seven major categories, confirming the dataset’s topical diversity.
Fig. 7 Thematic distribution of source news articles in ChronoQA. The chart displays the proportion of articles across seven major categories, confirming the dataset’s topical diversity.CC BY-NC-ND · PubMed Central
Fig. 8 Direct LLM Performance on ChronoQA.
Fig. 8 Direct LLM Performance on ChronoQA.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

0 Valid · 1 Not valid · 8 Not checked · 53 Own results and interpretations

27 more claims

Create a free account to see all 62 claims and to verify the 15 references still open.

See all claims — free account
  • It may state two separate claims as one. 23
  • Probably not a substantial claim. 14
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Citations

Cites

Show all 25 references

Research works printed but not linked: 2

  1. Abdallah, A., Piryani, B., Wallat, J., Anand, A. & Jatowt, A. Extending dense passage retrieval with temporal information. Preprint at https://arxiv.org/abs/2502.21024 (2025). · arXiv
  2. Nguyen, T. et al. MS MARCO: A human generated machine reading comprehension dataset. In Proc. Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches, co-located with NIPS 2016, vol. 1773 (2016).

Cited by

See all 20

Metadata

Authors
Chen, Z., Min, E., Zhao, X., Li, Y., Jia, X., Liao, J., Li, J., Wang, S., Hu, B., & Yin, D.
Affiliations
  • National University of Defense Technology (CN)
  • Baidu (China) (CN)
  • Harbin Institute of Technology (CN)
Corresponding author
Zhao, Xiang
Venue
Scientific Data
Published
2025-11-21
Volume
12
Issue
1
Pages
1855-1855
ISSN
2052-4463 (ISSN-L)
OpenAlex
W4416466501
Cited by
20
References
25
Citations by year
2026: 18 · 2025: 2
Type
data paper
Language
en
Open access
Licence
Locations
  • Scientific Data (publishedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
National Natural Science Foundation of China (72301284, U23A20296, 62272469)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
temporal reasoning, retrieval-augmented generation, question answering dataset, Chinese question answering, news articles, temporal expressions, temporal alignment, logical consistency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation. https://app.cochat.ai/papers/10.1038/s41597-025-06098-y
The paper
Chen, Z., Min, E., Zhao, X., Li, Y., Jia, X., Liao, J., Li, J., Wang, S., Hu, B., & Yin, D. (2025). A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation. Scientific Data. https://doi.org/10.1038/s41597-025-06098-y

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.