The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index; none of the 16 sources its claims cite is checked for support yet.
A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation
Open accessAbstract
We introduce ChronoQA, a benchmark dataset for Chinese question answering focused on evaluating temporal reasoning in Retrieval-Augmented Generation (RAG) systems. Built from over 300,000 news articles published between 2019 and 2024, ChronoQA contains 5,176 questions covering absolute, aggregate, and relative temporal types, with both explicit and implicit time expressions. The dataset features both single- and multi-document scenarios, reflecting real-world requirements for temporal alignment and logical consistency. By providing structured evaluation across a wide range of temporal tasks, ChronoQA offers a dynamic, reliable, and scalable resource for benchmarking RAG systems in evolving knowledge environments.
Overview
CoChat Summary
TL;DR
ChronoQA is a Chinese benchmark of 5,176 temporal-sensitive QA pairs built from over 300,000 news articles published between 2019 and 2024, designed to evaluate temporal reasoning in retrieval-augmented generation. It covers absolute, aggregate, and relative temporal types, explicit and implicit time expressions, and single- and multi-document scenarios.
Key findings
- ChronoQA contains 5,176 QA pairs: absolute 2,529, aggregate 1,911, relative 736; explicit time expressions 2,000, implicit 3,176; single-document 3,261, multi-document 1,915 (37%); answer types entity 2,556, time 864, numerical 507, judgement 1,045, other 204.
- Quality validation manually reviewed ~6,000 examples, finding over 95% met quality standards with Cohen’s Kappa = 0.85; all identified error pairs were removed.
- Retrieval baselines at K = 10 showed Query Decomposition best overall: Recall 68.2, MAP 57.6, NDCG 78.9; on multiple-document questions: Recall 53.9, MAP 53.1, NDCG 83.2. Native RAG at K = 10 scored overall Recall 61.8, MAP 53.5, NDCG 71.8.
- Error analysis of 100 cases found relative/implicit time expressions caused 69% of failures versus 31% for absolute/explicit; day-level granularity caused 62% of errors, month 27%, year 11%; retrieval failure accounted for 72% of incorrect answers, while 7% failed even with perfect retrieval.
Methods
- Source articles came from public news sources (e.g., Sina News) covering January 1, 2019, to August 30, 2024, averaging approximately 171.8 articles per day and roughly 350k textual units; after cleaning and deduplication, gpt-4o-mini extracted 294,696 “intensive temporal paragraphs.”
- Single temporal QA pairs were generated with gpt-4o from those paragraphs using structured prompts, yielding over 10,000 standalone pairs after automated filtering; multi-document questions were composed via semantic similarity (bge-large-zh-1.5 embeddings, cosine similarity) and temporal proximity, using parallel and series reasoning circuits with o1-mini.
- Verification combined rule-based filtering, gpt-4o evaluation of fluency/factual accuracy/temporal relevance, and manual review; retrieval experiments compared Native RAG, Temporal Filter, Query Rewrite, and Query Decomposition at K = 5 and K = 10 using Recall, MAP, and NDCG.
Limitations
- The paper does not state a dedicated limitations section.
- Evident from the text: the dataset is Chinese-language and news-based, with source coverage bounded to January 1, 2019–August 30, 2024, so temporal and topical coverage is tied to that corpus despite the claimed scalable/updatable framework.
- Evident from the text: manual validation covered ~6,000 examples rather than the full final set, error analysis examined only 100 cases, direct LLM performance is reported only in Fig. 8 without numeric values in the text, and retrieval baselines are limited to four methods at K = 5 and K = 10.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
0 Valid · 1 Not valid · 8 Not checked · 53 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“covering absolute, aggregate, and relative temporal types”
§ A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“with both explicit and implicit time expressions”
§ A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Built from over 300,000 news articles published between 2019 and 2024”
§ A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We introduce ChronoQA, a benchmark dataset for Chinese question answering focused on evaluating temporal reasoning in Retrieval-Augmented Generation (RAG) systems.”
§ A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChronoQA contains 5,176 questions”
§ A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
27 more claims
Create a free account to see all 62 claims and to verify the 15 references still open.
- It may state two separate claims as one. 23
- Probably not a substantial claim. 14
- The quote only partly states this claim. 2
Citations
Cites
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · cited 1×
“More advanced RAG benchmarks like HotpotQA [16] introduce multi-hop retrieval, requiring models to synthesize information across multiple sources.”
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · cited 1×
“Early QA datasets, such as Natural Questions (NQ)[17], TriviaQA [14], and MS MARCO [15], primarily assess open-domain retrieval, relying on web documents and knowledge graphs.”
Large language models generate functional protein sequences across diverse families
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., et al. (2023) · Nature Biotechnology · doi:10.1038/s41587-022-01618-2 · Cited by 1,078
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang, Z., & Tang, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.26 · Cited by 955
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876
C-Pack: Packed Resources For General Chinese Embeddings
Xiao, S., Liu, Z., Zhang, P., Muennighoff, N., Lian, D., & Nie, J.-Y. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657878 · Cited by 523
C-Pack: Packed Resources For General Chinese Embeddings
Xiao, S., Liu, Z., Zhang, P., Muennighoff, N., Lian, D., & Nie, J.-Y. (2023) · arXiv · doi:10.48550/arxiv.2309.07597 · Cited by 514
Benchmarking Retrieval-Augmented Generation for Medicine
Xiong, G., Jin, Q., Lu, Z., & Zhang, A. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.372 · Cited by 310
Research works printed but not linked: 2
- Abdallah, A., Piryani, B., Wallat, J., Anand, A. & Jatowt, A. Extending dense passage retrieval with temporal information. Preprint at https://arxiv.org/abs/2502.21024 (2025). · arXiv
- Nguyen, T. et al. MS MARCO: A human generated machine reading comprehension dataset. In Proc. Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches, co-located with NIPS 2016, vol. 1773 (2016).
Cited by
TEMPO: Realistic Multi-Domain Benchmark for Temporal Reasoning-Intensive Retrieval
Abdallah, A., Ali, M., Abdul-Mageed, M., & Jatowt, A. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.291 · Cited by 11
Nam, Y., Kim, J. H., Song, H., Yoon, S., Lee, J.-G., & Lee, B. S. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.398 · Cited by 4
Rank4Gen: RAG-Preference-Aligned Document Set Selection and Ranking
Fan, Y., Chu, Y., Xia, Z., Chen, X., Liu, J., Liang, H., Ma, J., He, B., Sun, Y., Zhai, J., et al. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.942 · Cited by 3
Kim, S., Wang, J., Xie, X., & Whang, S. E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.02045 · Cited by 3
Advancing Temporal Sensitive Question Answering through Progressive Multi-Step Reflection
Chen, Z., Min, E., Zhao, X. H., Li, Y., Jia, X., Liao, J., Wang, S., Hu, B., & Yin, D. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761292 · Cited by 1
Self-citation
Metadata
- Authors
- Chen, Z., Min, E., Zhao, X., Li, Y., Jia, X., Liao, J., Li, J., Wang, S., Hu, B., & Yin, D.
- Affiliations
- National University of Defense Technology (CN)
- Baidu (China) (CN)
- Harbin Institute of Technology (CN)
- Corresponding author
- Zhao, Xiang
- ORCID
- Venue
- Scientific Data
- Published
- 2025-11-21
- Volume
- 12
- Issue
- 1
- Pages
- 1855-1855
- ISSN
- 2052-4463 (ISSN-L)
- PMC
- PMC12638886
- OpenAlex
- W4416466501
- Cited by
- 20
- References
- 25
- Citations by year
- 2026: 18 · 2025: 2
- Type
- data paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Scientific Data (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (72301284, U23A20296, 62272469)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- temporal reasoning, retrieval-augmented generation, question answering dataset, Chinese question answering, news articles, temporal expressions, temporal alignment, logical consistency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation. https://app.cochat.ai/papers/10.1038/s41597-025-06098-y
- The paper
- Chen, Z., Min, E., Zhao, X., Li, Y., Jia, X., Liao, J., Li, J., Wang, S., Hu, B., & Yin, D. (2025). A Question Answering Dataset for Temporal-Sensitive Retrieval-Augmented Generation. Scientific Data. https://doi.org/10.1038/s41597-025-06098-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.