CoChat
Sign Up
Cited by 35Topic ModelingAdvanced Graph Neural NetworksMultimodal Machine Learning Applications

The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index; 1 of the 16 sources its claims cite were checked for support.

TimeR4 : Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering

Open access
Qian, Xinying, Zhang, Ying, Zhao, Yu, Zhou, Baohang, Sui, Xuhui, Zhang, Li, Song, Kehui·2024·Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
DOI 10.18653/v1/2024.emnlp-main.394
Open this paper in CoChat — free accountOpen PDF

Abstract

Temporal Knowledge Graph Question Answering (TKGQA) aims to answer temporal questions using knowledge in Temporal Knowledge Graphs (TKGs).Previous works employ pre-trained TKG embeddings or graph neural networks to incorporate the knowledge of TKGs.However, these methods fail to fully understand the complex semantic information of time constraints.In contrast, Large Language Models (LLMs) have shown exceptional performance in knowledge graph reasoning, unifying both semantic understanding and structural reasoning.To further enhance LLMs' temporal reasoning ability, this paper aims to integrate temporal knowledge from TKGs into LLMs through a Time-aware Retrieve-Rewrite-Retrieve-Rerank framework, which we named TimeR 4 .Specifically, to reduce temporal hallucination in LLMs, we propose a retrieve-rewrite module to rewrite questions using background knowledge stored in the TKGs, thereby acquiring explicit time constraints.Then, we implement a retrievererank module aimed at retrieving semantically and temporally relevant facts from the TKGs and reranking according to the temporal constraints.To achieve this, we fine-tune a retriever using the contrastive time-aware learning framework.Our approach achieves great improvements, with relative gains of 47.8% and 22.5% on two datasets, underscoring its effectiveness in boosting the temporal reasoning abilities of LLMs.Our code is available at https://github.com/qianxinying/TimeR4 .

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

TimeR4 is a time-aware retrieve-rewrite-retrieve-rerank framework that integrates temporal knowledge graphs into LLMs for temporal knowledge graph question answering. It reports relative Hits@1 gains of 47.8% and 22.5% on two TKGQA datasets.

Key findings

  • TimeR4 achieves the best performance across all experimental settings on MULTITQ and TimeQuestions, with reported relative improvements of 47.8% and 22.5% on Hits@1.
  • Ablations show each module matters: removing fact retrieval decreases Hits@1 by 13.2%, while removing rewriting, time-aware retrieval, or reranking also causes significant declines.
  • The optimal number of retrieved facts is 15; performance dips slightly at 20 facts despite higher answer coverage, suggesting extra facts introduce noise.
  • Fine-tuning LLaMA2 nearly doubled its results, and adding TimeR4 improves base LLMs substantially (e.g., LLaMA2 overall from 18.5 to 39.1 and ChatGPT from 10.2 to 41.4 in Table 6).

Methods

  • TimeR4 uses four modules: fact retrieval from a Fact Knowledge Store, question rewriting with an LLM to make implicit temporal constraints explicit, time-aware retrieval from a Temporal Knowledge Store fine-tuned with contrastive time-aware learning using three negative types, and reranking with a time-filtering function before fine-tuning an LLM for reasoning.
  • Experiments use MULTITQ (from ICEWS05-15, 500K QA pairs, multi-granularity) and TimeQuestions (16K temporal questions, four categories, year granularity). LLaMA2-Chat-7B is fine-tuned for 2 epochs on 2 NVIDIA A6000 GPUs, using only 20% of MULTITQ training data; SentenceBERT is fine-tuned for 10 epochs, rewriting uses gpt-3.5-turbo-0125, µ is 0.4, and 15 facts are retrieved.
  • Evaluation compares Hits@1 against pre-trained LM, embedding-based, TKGQA, and LLM baselines, plus ablations, retrieved-fact-number analysis, retrieval coverage analysis, and case studies.

Limitations

  • Stated: complex temporal fact retrieval can still be improved, and controlling LLM answer format without fine-tuning is difficult; standardizing answer formats or developing a more reasonable evaluation method is left for future work.
  • Evident from the text: evaluation can penalize semantically correct but non-standard outputs, as when an LLM generates “May” instead of the ground-truth “2012-05”; ChatGPT can also outperform TimeR4 on multiple-answer questions by returning lists of possible answers.
  • Evident from the text: TimeQuestions is much smaller and only includes year-level granularity, which may limit how strongly the method’s advantages appear on that dataset.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.18653/v1/2024.emnlp-main.394

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

1 of 16 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured

Verify all 2 references — free account
  • Findingown data

    Replacing the fact retriever module decreases overall performance by 13.2% on hits@1.

    “It can be observed that after replacing the fact retriever module, the overall performances decrease by 13.2% on hits@1, which indicates that the fact retriever module achieves accurate fact recognition.”
    Quote in the paperSubstantial claim
  • Findingown data

    Without fine-tuning, LLMs tend to randomly generate a list of irrelevant answers.

    “It can be observed in the results that without fine-tuned, LLMs tend to randomly generate a list of irrelevant answers, but in comparison, Chat-GPT performs better than LLaMA2.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Removing the rewriting strategy causes a significant decrease in performance.

    “The results showed a significant decrease, indicating that the rewriting strategy effectively aids LLMs in mitigating the hallucination of implicit temporal questions.”
    Quote in the paperSubstantial claim
  • Findingown data

    LLaMA2 is more prone to generating irrelevant responses without fine-tuning.

    “LLaMA2 is more prone to generating irrelevant responses without fine-tuning.”
    Quote in the paperNot a substantial claim
  • Findingreasoning

    PLMs (BERT, ALBERT) and LLMs (LLaMA2, ChatGPT) exhibit the lowest performance on the TKGQA task.

    “We find that PLMs (BERT.ALBERT) and LLMs (LLaMA2, ChatGPT) exhibit the lowest performance on the TKGQA task.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one. 21
  • The quote only partly states this claim. 1
  • Probably not a substantial claim. 12

  • It may state two separate claims as one.

    Premisecitation

    LLMs have demonstrated exceptional performance in knowledge graph reasoning and can unify semantic understanding and graph reasoning.

    “Large language models (LLMs) have demonstrated exceptional performance in knowledge graph reasoning (Sun et al., 2024;Luo et al., 2024) and can unify semantic understanding and graph reasoning (Huang and Chang, 2023;Wei et al., 2023).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Sun et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Luo et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Huang and Chang, 2023 source

      Huang and Chang (2023) is cited for the claim that LLMs can unify semantic understanding and graph reasoning, not for the preceding claim about exceptional performance in knowledge graph reasoning. The passages in §4.1 discuss symbolic and relational reasoning benchmarks, but do not address that unification. Searches for the claim’s wording and for knowledge-graph reasoning found no passage supporting or contradicting it. (LLM, conf=0.72)

      Experimental — accuracy not yet measured

  • The quote only partly states this claim.

    Findingown data

    TimeR4 achieves the best performance in all experimental settings on MultiTQ and TimeQuestions.

    “TimeR 4 achieves the best performance in all experimental settings, indicating its superiority on the TKGQA task.”
    • Also stated in: § 5.2 Main Results · plain
      “For the MultiTQ dataset, TimeR 4 achieves stateof-the-art performance across all question types.”
    • Also stated in: § 5.2 Main Results · plain
      “As for TimeQuestions, TimeR 4 also achieves the best results across all question types.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.

    Findingnone given

    After fine-tuning, the output of TimeR4 adheres more closely to standard conventions.

    “Furthermore, after fine-tuning, the output of TimeR 4 adheres more closely to standard conventions.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingnone given

    MULTITQ is larger and covers a wider time range than TimeQuestions.

    “MULTITQ is larger and covers a wider time range, making it harder to retrieve relevant facts.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

    Findingnone given

    Controlling the answer format during the generation process of LLMs without fine-tuning is difficult.

    “Additionally, controlling the answer format during the generation process of LLMs without finetuning is difficult, as discussed in Section 5.5.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    ChatGPT performs better than LLaMA2 in the case study.

    “It can be observed in the results that without fine-tuned, LLMs tend to randomly generate a list of irrelevant answers, but in comparison, Chat-GPT performs better than LLaMA2.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    There is a slight performance dip at n = 20 despite larger answer coverage with 20 retrieved facts.

    “Interestingly, there is a slight performance dip at n = 20, despite the larger answer coverage obtained with 20 retrieved facts.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    Across various question types, the Time-Aware Retriever answers most questions correctly that the Fact Retriever does, while also handling a significantly larger set of complex questions.

    “Across various question types, our Time-Aware Retriever answers most questions correctly that the Fact Retriever does, while also handling a significantly larger set of complex questions (second row).”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    TimeR4 achieves significantly higher accuracy and only generates the relevant answers.

    “TimeR 4 , on the other hand, achieves significantly higher accuracy and only generates the relevant answers.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Interpretationown data

    After fine-tuning, the Time-Aware Retriever can retrieve more temporal information, while the Fact Retriever struggles to capture temporal information in the facts, leading to low answer coverage in the retrieved results.

    “This indicates that after fine-tuning, the Time-Aware Retriever can retrieve more temporal information, while the Fact Retriever struggles to capture temporal information in the facts, leading to low answer coverage in the retrieved results.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Premisecitation

    Previous retrieval methods focus solely on semantic matching, so retrieved knowledge neglects time constraints and is ineffective for reasoning.

    “However, these retrieval methods focus solely on semantic matching, thus the retrieved knowledge neglects the time constraints of the question, rendering it ineffective for reasoning.”
    • Also stated in: § 2.1 KG-enhanced LLMs · plain
      “However, they only consider semantic similarity and neglect temporal constraints.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Li et al., 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Interpretationreasoning

    ChatGPT performs better than TimeR4 in multiple questions mainly because ChatGPT tends to provide all possible answers.

    “This is mainly because ChatGPT tends to provide all possible answers.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Interpretationreasoning

    The retrieve-rewrite module rewrites questions using background knowledge stored in TKGs, thereby acquiring explicit time constraints and reducing temporal hallucination in LLMs.

    “to reduce temporal hallucination in LLMs, we propose a retrieve-rewrite module to rewrite questions using background knowledge stored in the TKGs, thereby acquiring explicit time constraints.”
    • Also stated in: § 1 Introduction · plain
      “we employ the retrieve-rewrite strategy.We perform fact retrieval from the Facts Knowledge Store (FKS) to obtain relevant facts for implicit questions.Subsequently, we rewrite these questions by replacing temporal facts with specific timestamps, ensuring that all questions contain explicit temporal information.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    MULTITQ is the largest known TKGQA dataset constructed from ICEWS05-15, with 500K unique question-answer pairs.

    “MULTITQ is the largest known TKGQA dataset constructed from the ICEWS05-15 (García-Durán et al., 2018b), which has 500K unique questionanswer pairs.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Chen et al., 2023b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites García-Durán et al., 2018b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper may give no reason why it holds.

    Interpretationreasoning

    The low performance of PLMs and LLMs on TKGQA might be due to lack of necessary temporal knowledge, leading to errors in reasoning.

    “This might be due to the lack of necessary temporal knowledge, thus leading to errors in reasoning.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Interpretationown data

    Fewer facts do not provide sufficient temporal knowledge, leading to poor performance.

    “On the other hand, fewer facts do not provide sufficient temporal knowledge, also leading to poor performance.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    TimeQuestions has 16K temporal questions, divided into four categories, and only includes a time granularity of years.

    “TimeQuestions is another challenging dataset, which has 16K temporal questions and is divided into four categories.However, TimeQuestions only includes a time granularity of years and is much smaller in size.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jia et al., 2021

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. It may state two separate claims as one.

    Premisenone given

    Agent-based methods are inefficient for complex reasoning tasks due to reliance on multiple LLM-calls, and greedy decision-making is susceptible to error propagation.

    “However, they prove inefficient for complex reasoning tasks due to their reliance on multiple LLM-calls.Furthermore, the greedy decision-making process is susceptible to error propagation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Interpretationreasoning

    The higher performance of ChatGPT and LLaMA2 on TimeQuestions might be because TimeQuestions are built on the Wikidata knowledge graph and most LLMs are pretrained on the Wikidata knowledge graph corpus, so they store relevant information.

    “This might be because Timequestions are built on the Wikidata (Vrandečić and Krötzsch, 2014) knowledge graph, and most LLMs are pretrained on the Wikidata knowledge graph corpus.Therefore, they store some relevant information, enabling them to answer such questions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Vrandečić and Krötzsch, 2014

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • Rests on an assumption: Most LLMs are pretrained on the Wikidata knowledge graph corpus.

  • It may state two separate claims as one.

    Interpretationown data

    The proposed time-aware retriever method is inherently time-aware and performs better than the non-time-aware fact retriever.

    “The results show a significant decline, indicating that our proposed time-aware retriever method is inherently time-aware and performs better than the non-time-aware fact retriever.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. It may state two separate claims as one.

    Interpretationnone given

    The retrieve-rerank module retrieves semantically and temporally relevant facts from TKGs and reranks according to temporal constraints.

    “we implement a retrievererank module aimed at retrieving semantically and temporally relevant facts from the TKGs and reranking according to the temporal constraints.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Premisecitation

    Some TKGs, such as ICEWS, lack entity-linking tools, resulting in poor performance with entity-linking-based methods.

    “However, some TKGs, such as ICEWS (García-Durán et al., 2018a), lack entity-linking tools, resulting in poor performance with such methods (Chen et al., 2023b).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites García-Durán et al., 2018a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Chen et al., 2023b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    ARI integrates LLMs through a knowledge adaptability framework and abstract methodological guidance, but cannot be applied to smaller-scale LLMs.

    “ARI (Chen et al., 2023a) integrates LLMs through a knowledge adaptability framework and abstract methodological guidance.However, it cannot be applied to smaller-scale LLMs.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Chen et al., 2023a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Interpretationown data

    Fine-tuning can effectively regulate the output of LLMs and help LLMs learn patterns of temporal reasoning, thereby enhancing the results.

    “It can be found that the results of LLaMA2 with fine-tuning almost doubled, indicating that fine-tuning can effectively regulate the output of LLMs and help LLMs learn patterns of temporal reasoning, thereby enhancing the results.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Interpretationreasoning

    In the TimeQuestions dataset, relevant facts can often be retrieved by the fact retrieval module, which is why the proposed strategy is less prominent on this dataset.

    “Conversely, in the TimeQuestions dataset, relevant facts can often be retrieved by the fact retrieval module, which is why our strategy is less prominent on this dataset.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim.

    Interpretationreasoning

    The time-filtering function filters out irrelevant facts and focuses more on time-related ones.

    “To further refine the retrieval process, we design a time-filtering function to filter out irrelevant facts and focus more on time-related ones.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. It may state two separate claims as one.

    Interpretationreasoning

    TimeR4 is a Time-aware Retrieve-Rewrite-Retrieve-Rerank framework that integrates temporal knowledge from TKGs into LLMs.

    “this paper aims to integrate temporal knowledge from TKGs into LLMs through a Time-aware Retrieve-Rewrite-Retrieve-Rerank framework, which we named TimeR 4 .”
    • Also stated in: § 1 Introduction · plain
      “we propose TimeR 4 , a Time-aware Retrieve-Rewrite-Retrieve-Rerank framework for the TKGQA task.”
    • Also stated in: § 1 Introduction · plain
      “We integrate LLMs with TKGs and propose TimeR 4 , a Time-aware Retrieve-Rewrite-Retrieve-Rerank framework, which effectively overcomes the limitations in handling temporal knowledge in LLMs.”
    Quote in the paperSubstantial claim
27 of 27

  • The paper relies on this without support.

    The TKGQA task is more challenging than the KGQA task due to the added temporal dimension of reasoning.

    “The TKGQA task is more challenging than the KGQA task due to the added temporal dimension of reasoning.”

  • The paper relies on this without support.

    Most LLMs are pretrained on the Wikidata knowledge graph corpus.

    “most LLMs are pretrained on the Wikidata knowledge graph corpus”
2 of 2

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • LLaMA2 TimeQuestions overall score differs between tables

    Major

    Table 3 reports 27.1 for LLaMA2 on TimeQuestions, while Table 6 reports 28.9 for the unmodified LLaMA2 model on that dataset.

  • MULTITQ development total is overstated

    Minor

    The six development-set category counts sum to 57,979, not the reported 587,979.

    Reported 587979 · recomputed 57979

2 of 2

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 12

  1. Alberto García-Durán, Sebastijan Dumančić, and Math- ias Niepert. 2018b. Learning sequence encoders for temporal knowledge graph completion. arXiv preprint arXiv:1809.03202. · arXiv
  2. Zhen Jia, Soumajit Pramanik, Rishiraj Saha Roy, and Gerhard Weikum. 2021. Complex temporal question answering on knowledge graphs. In Proceedings of the 30th ACM international conference on informa- tion & knowledge management, pages 792-802.
  3. Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Struct- gpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645. · arXiv
  4. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535-547. · DOI · Paper page
  5. Yonghao Liu, Mengyu Li Di Liang, Fausto Giunchiglia, Ximing Li, Sirui Wang, Wei Wu, Lan Huang, Xi- aoyue Feng, and Renchu Guan. 2023. Local and global: temporal question answering via information fusion. In Proceedings of the Thirty-Second Inter- national Joint Conference on Artificial Intelligence, pages 5141-5149.
  6. Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2024. Reasoning on graphs: Faithful and interpretable large language model reasoning. Preprint, arXiv:2310.01061. · arXiv
  7. Costas Mavromatis, Prasanna Lakkur Subramanyam, Vassilis N Ioannidis, Adesoji Adeshina, Phillip R Howard, Tetiana Grinberg, Nagib Hakim, and George Karypis. 2022. Tempoqr: temporal question reason- ing over knowledge graphs. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 5825-5833.
  8. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. CoRR, abs/1908.10084. · arXiv
  9. Apoorv Saxena, Soumen Chakrabarti, and Partha Taluk- dar. 2021. Question answering over temporal knowl- edge graphs. arXiv preprint arXiv:2106.01515. · arXiv
  10. Apoorv Saxena, Aditay Tripathi, and Partha Talukdar. 2020. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In Proceedings of the 58th Annual Meeting of the As- sociation for Computational Linguistics, pages 4498- 4507, Online. Association for Computational Lin- guistics. · DOI
  11. Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Heung-Yeung Shum, and Jian Guo. 2023. Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph. arXiv preprint arXiv:2307.07697. · arXiv
  12. Denny Vrandečić and Markus Krötzsch. 2014. Wiki- data: a free collaborative knowledgebase. Commun. ACM, 57(10):78-85. · DOI

Cited by

Sorted by citations

Related

Metadata

Authors
Qian, X., Zhang, Y., Zhao, Y., Zhou, B., Sui, X., Zhang, L., & Song, K.
Affiliations
  • Nankai University (CN)
  • Tiangong University (CN)
ORCID
Venue
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Publisher
Association for Computational Linguistics
Published
2024
Pages
6942-6952
DOI
10.18653/v1/2024.emnlp-main.394
OpenAlex
W4404782541
Semantic Scholar
4cf85f9436bb8b5b1c68715f44d6b67254413ef8
Cited by
35
References
23
Citations by year
2026: 28 · 2025: 7
FWCI
5.53
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
Funders
  • National Natural Science Foundation of China (62302243, 62272250)
  • Nankai University
  • Natural Science Foundation of Tianjin City (23JCYBJC01230, 22JCQNJC01580, 22JCJQJC00150)
  • Fundamental Research Funds for the Central Universities
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
temporal knowledge graphs, large language models, temporal reasoning, retrieval-augmented generation, question rewriting, contrastive learning, knowledge graph question answering

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review