CoChat
Sign Up
Cited by 16Topic ModelingAdvanced Graph Neural NetworksMultimodal Machine Learning Applications

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index; none of the 8 sources its claims cite is checked for support yet.

TempoQR: Temporal Question Reasoning over Knowledge Graphs

Open access
Mavromatis, Costas, Subramanyam, Prasanna Lakkur, Ioannidis, Vassilis N., Adeshina, Adesoji, Howard, Phillip R, Grinberg, Tetiana, Hakim, Nagib, Karypis, George·2022·Proceedings of the AAAI Conference on Artificial Intelligence
DOI 10.1609/aaai.v36i5.20526
Open this paper in CoChat — free account

Abstract

Knowledge Graph Question Answering (KGQA) involves retrieving facts from a Knowledge Graph (KG) using natural language queries. A KG is a curated set of facts consisting of entities linked by relations. Certain facts include also temporal information forming a Temporal KG (TKG). Although many natural questions involve explicit or implicit time constraints, question answering (QA) over TKGs has been a relatively unexplored area. Existing solutions are mainly designed for simple temporal questions that can be answered directly by a single TKG fact. This paper puts forth a comprehensive embedding-based framework for answering complex questions over TKGs. Our method termed temporal question reasoning (TempoQR) exploits TKG embeddings to ground the question to the specific entities and time scope it refers to. It does so by augmenting the question embeddings with context, entity and time-aware information by employing three specialized modules. The first computes a textual representation of a given question, the second combines it with the entity embeddings for entities involved in the question, and the third generates question-specific time embeddings. Finally, a transformer-based encoder learns to fuse the generated temporal information with the question representation, which is used for answer predictions. Extensive experiments show that TempoQR improves accuracy by 25--45 percentage points on complex temporal questions over state-of-the-art approaches and it generalizes better to unseen question types.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

TempoQR is an embedding-based framework for answering complex temporal questions over temporal knowledge graphs by combining context-, entity-, and time-aware question representations. It reports large gains over prior methods on CronQuestions, especially for complex temporal questions.

Key findings

  • On CronQuestions, TempoQR-Hard reaches overall Hits@1 0.918 and complex-question Hits@1 0.864, compared with CronKGQA at 0.647 overall and 0.392 complex; TempoQR-Soft reaches 0.799 overall and 0.655 complex.
  • The paper reports that TempoQR improves accuracy by 25–45 percentage points on complex temporal questions over state-of-the-art approaches.
  • Hard supervision helps “first/last” questions: TempoQR-Hard scores 0.853 vs TempoQR-Soft 0.570, a stated 27% absolute improvement; for “before/after” the improvement is only 3%, and TempoQR-Soft is 8% worse on “time join” questions.
  • On unseen complex question types, TempoQR-Soft performs best, but Hits@1 remains below 20% for all methods.

Methods

  • TempoQR uses pre-trained TKG embeddings from TComplEx and a BERT-based language model, with three modules: context-aware question representation, entity-aware grounding to TKG entities, and time-aware representation via either hard retrieval from TKG facts or soft inference in the embedding space; a transformer encoder fuses these for answer prediction.
  • Experiments use CronQuestions over a Wikidata TKG with 125k entities, 203 relations, 1.7k timestamps, and 328k facts; CronQuestions has 410k QA pairs, with 350k for training and 30k each for validation and testing, plus 261 additional unseen complex QA pairs used only for inference.
  • Configuration: TComplEx embeddings dimension D=512; LM and TKG embeddings frozen during QA; fusion encoder has l=6 transformer layers with 8 heads; Adam with learning rate 0.0002; trained for up to 20 epochs; evaluated with Hits@1/Hits@10 and corrupted-TKG robustness settings.

Limitations

  • Stated: TempoQR-Hard depends on TKG quality during QA and is greatly affected by corrupted TKGs; TempoQR-Soft sometimes cannot infer accurate temporal information from the embedding space.
  • Stated: Neither hard nor soft supervision handles unseen question types effectively, with Hits@1 below 20% for all methods, motivating future work on generalization.
  • Evident from the text: evaluation is limited to CronQuestions/Wikidata and a small set of 261 additional unseen QA pairs; no other datasets or human evaluation are reported.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1609/aaai.v36i5.20526

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 of 8 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Random embeddings perform equally with TComplEx embeddings.

    “This is confirmed since RandEmb have the perform equally with TComplEx embeddings.”
    Quote in the paperSubstantial claim
  • Findingown data

    The largest performance drop under a corrupted TKG is observed for TempoQR-Hard for 'first/last' questions.

    “The largest performance drop is observed for TempoQR-Hard for 'first/last' questions, which indicates that it cannot generalize well to such questions under a corrupted TKG.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Hard-supervision over soft-supervision improves 'before/after' questions by only 3%.

    “On the other hand, this does not equally benefit 'before/after' questions; the improvement of hard-supervision over soft-supervision is only 3%.”
    Quote in the paperNot a substantial claim
  • Findingown data

    For 'before/after' questions, TempoQR-Hard performs worse than TempoQR-Soft even when facts are corrupted by probability p = 0.2.

    “For 'before/after' questions, it performs worse than TempoQR-Soft even when the facts are corrupted by a probability p = 0.2.”
    Quote in the paperSubstantial claim
  • Findingown data

    The time embeddings themselves do not play an important role.

    “Table 6 indicates is that the time embeddings themselves do not play an important role.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one. 20
  • The quote only partly states this claim. 1
  • Probably not a substantial claim. 2

  • It may state two separate claims as one.

    Findingown data

    TempoQR-Hard is the only method that depends on the quality of the TKG during QA and is greatly affected by a corrupted TKG.

    “TempoQR-Hard is the only method that depends on the quality of the TKG during QA and is greatly affected by a corrupted TKG.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    Depending on the information that is corrupted, some methods benefit over others, e.g., TempoQR-Hard compared to EntityQR for p = 0.33 at 'time join' questions.

    “Finally, depending on the information that is corrupted, some methods benefit over others, i.e., compare TempoQR-Hard to EntityQR for p = 0.33 at 'time join' questions.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    For 'first/last' and 'time join' questions, soft supervision introduces noise and decreases performance compared to TempoQR-Hard by 1.9 and 6.7 points, respectively.

    “For 'first/last' and 'time join' question, the soft supervision introduces noise and actually decreases the performance compared to TempoQR-Hard by 1.9 and 6.7 points, respectively.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    Both soft and hard supervision improve CronKGQA performance by 2.5% and 6% over the original CronKGQA.

    “First, we see that both supervision approaches have a positive effect on CronKGQA, where the performance is improved by 2.5% and 6% over the original method (CronKGQA-Soft and CronKGQA-Hard, respectively, compared to CronKGQA).”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    Neither TempoQR-Soft nor TempoQR-Hard can tackle unseen questions effectively, with Hits@1 below 20% for all methods.

    “In general, neither of these methods seems to be able to tackle unseen questions effectively, i.e., Hits@1 is below 20% for all the methods.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    CronKGQA is the least affected by a corrupted TKG, but its performance is still much lower.

    “CornKGQA is the least affected by a corrupted TKG, but, still, its performance is much lower.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    Methods that rely on TKG embeddings (TempoQR-Hard, TempoQR-Soft) are greatly affected when a corrupted TKG is given for both TKG embedding and QA, performing similar to each other and to EntityQR when corruption probability p is large, e.g., p = 0.5.

    “We can see that methods that rely on TKG embeddings (TempoQR-Hard, TempoQR-Soft) are greatly affected, since they perform similar to each other as well to EntityQR when the corruption probability p is large, e.g., p = 0.5.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    BERT, RoBERTa, EmbedKGQA and EaE cannot answer correctly more than 35% (Hits@1) and 76% (Hits@10) of simple questions.

    “The other methods (BERT, RoBERTa, EmbedKGQA and EaE) cannot answer correctly more the 35% (Hits@1) and 76% (Hits@10).”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    BERT, RoBERTa, EmbedKGQA and EaE have 35%-65% and 11%-40% worse overall accuracy for Hits@1 and Hits@10, respectively, compared to TempoQR, EntityQR and CronKGQA.

    “Similarly, BERT, RoBERTa, Embed-KGQA and EaE have 35%-65% and 11%-40% worse overall accuracy for Hits@1 and Hits@10, respectively, compared to TempoQR, EntityQR and CronKGQA.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    Adding temporal information (time-aware step) improves complex question answering over EntityQR; TempoQR-Soft improves 9% Hits@1 and TempoQR-Hard improves more than 30% Hits@1.

    “Furthermore, comparing Tem-poQR to EntityQR, we see the benefit of adding temporal information to the question (time-aware step).The absolute improvement of TempoQR-Soft over EntityQR is 9% at Hits@1 for complex questions, while the respective improvement of TempoQR-Hard is more than 30%.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    TempoQR-Hard is more robust for 'first/last' and 'time join' questions, where it better handles the noncorrupted timestamps of the facts.

    “Finally, TempoQR-Hard is more robust for 'first/last' and 'time join' questions, where it better handles the noncorrupted timestamps of the facts.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    Using the start and end timestamps is a key decision that provides the best results, with TComplEx-Sampled having the lowest performance.

    “As we can see, using the start and end timestamps is a key decision that provides the best results (TComplEx-Sampled has the lowest performance).”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingreasoning

    TempoQR vastly outperforms TComplEx on complex temporal questions because TComplEx cannot handle complex questions.

    “The first row of the Table shows that TempoQR vastly outperforms TComplEx since the latter cannot handle complex questions.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    TempoQR improves accuracy by 25-45 percentage points at Hits@1 on complex temporal questions over state-of-the-art approaches.

    “Extensive experiments show that TempoQR improves accuracy by 25-45 percentage points on complex temporal questions over state-of-the-art approaches”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Interpretationreasoning

    Positional embeddings perform best for 'before/after' questions because the timestamp ordering is preserved in the embedding space and allows for better 'before/after' reasoning.

    “This happens because the timestamp ordering is preserved in the embedding space and allows for better 'before/after' reasoning.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Interpretationreasoning

    Random embeddings perform best for 'first/last' questions because the timestamps are well-separated in the embedding space.

    “They also perform the best for 'first/last' questions, since the timestamps are well-separated in the embedding space.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. The paper may give no reason why it holds.

    Interpretationnone given

    By summing the time embeddings, all the information is encoded.

    “Whereas, by summing the time embeddings, we ensure that all the information is encoded.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Interpretationreasoning

    TempoQR-Soft generalizes better than TempoQR-Hard on unseen questions because it learns to handle temporal constraints in the embedding space, whereas TempoQR-Hard simply uses the TKG to answer such questions.

    “Since it learns to handle temporal constraints in the embedding space, it generalizes better than TempoQR-Hard which simply uses the TKG to answer such questions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    TComplEx employs additional regularizers to improve the quality of the learned embeddings, such as enforcing close timestamps to have similar embeddings.

    “TComplEx employs additional regularizers to improve the quality of the learned embeddings such as enforcing close timestamps to have similar embeddings (closeness of time).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lacroix, Obozinski, and Usunier 2020 source

  • It may state two separate claims as one.

    Premisecitation

    TComplEx is a TKG embedding scoring function (decoder) and can only answer questions that involve a single fact, e.g., (s, r, ?, τ), with a provided TKG relation r.

    “TComplEx is a TKG embedding scoring function (decoder) and can only answer questions that involve a single fact, e.g., (s, r, ?, τ ), with a provided TKG relation r.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [14]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Interpretationreasoning

    TempoQR-Hard provides accurate temporal information by retrieving it from the TKG, while TempoQR-Soft sometimes cannot infer as accurate information from the embedding space.

    “This confirms that TempoQR-Hard provides accurate temporal information by retrieving it from the TKG, while TempoQR-Soft sometimes cannot infer as accurate information from the embedding space.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Interpretationreasoning

    TempoQR handles questions that involve multiple facts and does not rely on a specific relation r.

    “TempoQR handles questions that involve multiple facts and does not rely on a specific relation r.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Premisecitation

    CronKGQA achieves a 65% performance improvement over non-TKG embedding methods for simple questions, but only a 15% improvement for complex questions.

    “This is confirmed by the experiments presented in (Saxena, Chakrabarti, and Talukdar 2021), where CronKGQA achieves a 65% performance improvement over non-TKG embedding methods for simple questions, but only a 15% improvement for complex questions as the ones here.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Saxena, Chakrabarti, and Talukdar 2021

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

23 of 23

  • The paper relies on this without support.

    TKG embeddings are designed for simple questions and do not by themselves handle complex temporal questions.

    “Although TKG embeddings are designed for simple questions, our method overcomes this shortcoming by incorporating additional temporal information in the question representation q to better handle constraints.”

  • The paper relies on this without support.

    The TKG facts involving the annotated entities of a question provide the correct time scope for answering the question.

    “We utilize the annotated entities of the question to retrieve the relative time scope from the underlying TKG.For the example question “Which movie won the Best Picture after The Godfather?”, the entities 'Best Picture' and 'The Godfather' appear together in a TKG fact with timestamp 1972.Hence, the time embedding of 1972 can be utilized to further enhance the question representation.”

  • The paper relies on this without support.

    The soft-supervision can recover missing temporal facts by operating in the embedding space.

    “The second approach infers question-specific temporal information based on the question's representation.We term this approach soft-supervised, since it may recover missing temporal facts by operating in the embedding space.”
3 of 3

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • EntityQR-Hard does not drop below EntityQR overall

    Minor

    The text says both supervised EntityQR variants perform worse than EntityQR, but Table 3 reports a slightly higher overall Hits@1 for EntityQR-Hard (0.748) than for EntityQR (0.745).

1 of 1

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. Miller, A.; Fisch, A.; Dodge, J.; Karimi, A.-H.; Bordes, A.; and Weston, J. 2016. Key-Value Memory Networks for Di- rectly Reading Documents. In Proceedings of the 2016 Con- ference on Empirical Methods in Natural Language Pro- cessing, 1400-1409. Austin, Texas: Association for Com- putational Linguistics.
  2. Qiu, Y.; Wang, Y.; Jin, X.; and Zhang, K. 2020. Stepwise Reasoning for Multi-Relation Question Answering over Knowledge Graph with Weak Supervision. In Proceedings of the 13th ACM International Conference on Web Search and Data Mining, WSDM '20. New York, NY, USA: Asso- ciation for Computing Machinery. ISBN 9781450368223.
  3. Saxena, A.; Tripathi, A.; and Talukdar, P. 2020. Improving Multi-hop Question Answering over Knowledge Graphs us- ing Knowledge Base Embeddings. In ACL.
  4. Souza Costa, T.; Gottschalk, S.; and Demidova, E. 2020. Event-QA: A dataset for event-centric question answering over knowledge graphs. In Proceedings of the 29th ACM In- ternational Conference on Information & Knowledge Man- agement, 3157-3164.

Cited by

Sorted by citations

Related

Metadata

Authors
Mavromatis, C., Subramanyam, P. L., Ioannidis, V. N., Adeshina, A., Howard, P. R., Grinberg, T., Hakim, N., & Karypis, G.
Affiliations
  • University of Minnesota (US)
  • University of Massachusetts Amherst (US)
  • Amazon (United States) (US)
  • Intel (United States) (US)
ORCID
Venue
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2022-06-28
Volume
36
Issue
5
Pages
5825-5833
ISSN
  • 2159-5399 (ISSN-L)
  • 2374-3468
DOI
10.1609/aaai.v36i5.20526
OpenAlex
W4226406360
Cited by
16
References
31
Citations by year
2026: 3 · 2025: 4 · 2024: 2 · 2023: 4 · 2022: 3
FWCI
1.39
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
temporal question answering, temporal knowledge graphs, knowledge graph question answering, question embedding, entity embeddings

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review