Relation-Aware Question Answering for Heterogeneous Knowledge Graphs
Open accessAbstract
Multi-hop Knowledge Base Question Answering(KBQA) aims to find the answer entity in a knowledge graph (KG), which requires multiple steps of reasoning.Existing retrievalbased approaches solve this task by concentrating on the specific relation at different hops and predicting the intermediate entity within the reasoning path.During the reasoning process of these methods, the representation of relations are fixed but the initial relation representation may not be optimal.We claim they fail to utilize information from head-tail entities and the semantic connection between relations to enhance the current relation representation, which undermines the ability to capture information of relations in KGs.To address this issue, we construct a dual relation graph where each node denotes a relation in the original KG (primal entity graph) and edges are constructed between relations sharing same head or tail entities.Then we iteratively do primal entity graph reasoning, dual relation graph information propagation, and interaction between these two graphs.In this way, the interaction between entity and relation is enhanced, and we derive better entity and relation representations.Experiments on two public datasets, WebQSP and CWQ, show that our approach achieves a significant performance gain over the prior state-of-theart.Our code is available on https://github. com/yanmenxue/RAH-KBQA.
AI snapshot
Written from the paper’s full text.
TL;DR
The paper proposes a relation-aware iterative framework for multi-hop KBQA that jointly reasons over a primal entity graph and a dual relation graph, using head-tail entity information and relation co-occurrence to improve relation representations. It reports new state-of-the-art results on WebQSP and CWQ among retrieval-based methods.
Key findings
- On WebQSP, the method achieves 77.2 Hits@1 and 72.6 F1, which the paper reports as a gain of 0.8 Hits@1 and 1.7 F1 over prior state-of-the-art REAREV.
- On CWQ, it achieves 54.4 Hits@1 and 49.3 F1; the paper reports gains of 1.5 Hits@1 and 5.3 F1 on CWQ over existing methods.
- Ablations on WebQSP show removing dual relation graph propagation drops performance by 0.4 Hits@1 and 1.5 F1; removing dual-primal interaction drops it by 0.1 Hits@1 and 0.6 F1; replacing attention with relation co-occurrence drops F1 by 0.9.
- The TransE-score evaluation shows the method improves relation/entity representation consistency over NSM across graph-size and connectivity groups, with overall scores of 5.56 for the proposed method versus 0.91 for NSM.
Methods
- The approach is an IR-based retriever-reasoner for multi-hop KBQA without logical-form supervision. It builds a question-specific primal entity subgraph and a dual relation graph where each node is a relation and edges connect relations sharing the same head or tail entity.
- It iteratively runs four modules: SentenceBERT-based question instruction generation; primal entity graph reasoning with attention; dual relation graph information propagation via GAT; and dual-primal graph interaction that updates relations from head-tail entities using the TransE assumption and updates entities from adjacent relations with separate head/tail projectors. Decoding uses an MLP with focal loss and instruction adaptation.
- Experiments use WebQSP (4,327 questions over Freebase, 1–2 hops, average 1,430 entities per subgraph) and CWQ (34,689 questions, up to 4 hops, average 1,306 entities per subgraph). Training uses Adam with batch size 8, learning rate 7e-4, reasoning steps 3, and hidden size 128; evaluation metrics are Hits@1 and F1.
Limitations
- Stated: the authors do not investigate the effect of pretrained knowledge graph embeddings, which may contain more structural and semantic information from large-scale KGs; they leave incorporating pretrained KG embeddings into the framework for future work.
- Evident from the text: evaluation is limited to two public datasets, WebQSP and CWQ, and the main comparisons are against IR-based methods; the paper notes that SP-based methods with ground-truth executable queries can outperform when such supervision is available.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 2 cited sources checked · 0 supported · 0 flagged
- Findingown data
The semantic consistency of our relation representation is better with a larger size and higher connectivity of relation graph.
“Moreover, the semantic consistency of our relation representation is better with a larger size and higher connectivity of relation graph.”
Quote in the paperSubstantial claim - Findingown data
Removing the dual relation graph propagation module causes performance drops by 0.4% score on Hits@1 and 1.5% score on F1.
“By Table 9, we can see the performance drops by 0.4% score on Hits@1 and 1.5% score on F1.”
Quote in the paperSubstantial claim - Findingown data
The SOTA SP-method DecAF drops by 6 and 18 Hits@1 score on WebQSP and CWQ with answer as the only supervision.
“The SOTA SP-method DecAF, drops by 6 and 18 Hits@1 score on WebQSP and CWQ with answer as the only supervision.”
Quote in the paperSubstantial claim - Findingown data
Replacing the attention mechanism in dual graph propagation with co-occurrence statistics of relations causes performance drops by 0.9% score on F1.
“In this ablation, we employ co-occurrence of relations (Wu et al., 2019) to replace attention mechanism in Eq. 8.”
- Also stated in: § Does interaction between dual relation graph and primal entity graph matter? · plain
“By table 9, we can see the performance drops by 0.9% score on F1.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Wu et al., 2019
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Also stated in: § Does interaction between dual relation graph and primal entity graph matter? · plain
- Findingown data
In the answer-only supervision setting, our method outperforms DecAF by 3 and 4 Hits@1 score on WebQSP and CWQ.
“In the setting, our method outperforms DecAF by 3 and 4 Hits@1 score on WebQSP and CWQ.”
Quote in the paperSubstantial claim
- Findingown data
Our approach derives better improvement over NSM on cases with larger size and high connectivity of dual relation graph and primal entity graph.
“From Table 3-6, overall, our approach derives better improvement over NSM on cases with larger size and high connectivity of dual relation graph and primal entity graph.”
Quote in the paperSubstantial claim - Findingown data
Removing the interaction between dual graph and primal graph causes performance drops by 0.1% score on Hits@1 and 0.6% score on F1.
“By table 9, we can see the performance drops by 0.1% score on Hits@1 and 0.6% score on F1.”
Quote in the paperSubstantial claim - Findingown data
Our approach outperforms the previous Average F1 score on 4 groups of cases in WebQSP classified by relation number.
“Our approach outperforms the previous Average F1 score on 4 groups of cases in WebQSP classified by relation number.”
Quote in the paperSubstantial claim - Findingown data
Our approach outperforms all existing methods on two benchmark datasets.
“We conduct experiments on two benchmark datasets in this field and our approach outperforms all the existing methods.”
- Also stated in: § Limitations · plain
“Our approach outperforms all the existing methods on two datasets, becoming the new state-of-the-art of IR-based methods.”
Quote in the paperSubstantial claim - Also stated in: § Limitations · plain
- Findingown data
If losing the ground-truth executable queries supervision signal, the performance of SP-based methods drops by a large degree.
“If losing the ground-truth executable queries supervision signal, the performance of SP-based methods will drop by a large degree.”
Quote in the paperSubstantial claim - Findingown data
SP-based methods utilize ground-truth executable queries as stronger supervision, which significantly improves KBQA performance with IR-based methods.
“SP-based methods utilize ground-truth executable queries as the stronger supervision, which will significantly improves the KBQA performance with IRbased methods.”
Quote in the paperSubstantial claim - Findingown data
Our approach significantly improves TransE-score over NSM in cases with different sizes and connectivity of relation graph.
“By Table 8, our approach significantly improves the TransE-score over NSM in cases with different sizes and connectivity of relation graph.”
Quote in the paperSubstantial claim - Interpretationown data
Our significant improvement on the more complex dataset CWQ shows our ability to address complex relation information.
“Our significant improvement on the more complex dataset CWQ shows our ability to address complex relation information.”
Quote in the paperSubstantial claim - Interpretationreasoning
Our approach incorporates information from head-tail entities and semantic connection between relations to derive better relation and entity representations.
“Our approach incorporates information from head-tail entities and semantic connection between relations to derive better relation and entity representations.”
Quote in the paperSubstantial claim - Interpretationown data
Our dual graph propagation module provides relevant semantic information from neighborhood relations which share the same head or tail entity to enhance the current relation.
“It shows our dual graph propagation module provides relevant semantic information from neighborhood relations which share the same head or tail entity to enhance the current relation.”
Quote in the paperSubstantial claim - Interpretationreasoning
One crucial point for the model to retrieve the answer in the WebQSP example is to understand the meaning of relations 'Contained_by' and 'Contains'.
“So one crucial point for the model to retrieve the answer is to understand the meaning of relation”
Quote in the paperSubstantial claim - Interpretationown data
Our representations of relations and entities are much more corresponding to the semantic relationship between them.
“It demonstrates our representations of relations and entities are much more corresponding to the semantic relationship between them.”
Quote in the paperSubstantial claim - Interpretationown data
Introducing attention mechanism to weigh the co-relation between relations is more flexible and efficient than freezing the weights as the co-occurrence statistics.
“It shows introducing attention mechanism to weigh the co-relation between relations is more flexible and efficient than freezing the weights as the co-occurrence statistics.”
Quote in the paperSubstantial claim - Interpretationown data
Our iteratively primal graph reasoning, dual graph information propagation, and dual-primal graph interaction framework is robustly efficient.
“It demonstrates the robust efficiency of our iteratively primal graph reasoning, dual graph information propagation, and dual-primal graph interaction framework.”
Quote in the paperNot a substantial claim - Interpretationown data
The weights of connection between relations should update as the reasoning process proceeds.
“The weights of connection between relations should update as the reasoning process proceeds.”
Quote in the paperSubstantial claim - Interpretationreasoning
The semantic information of entity type narrows down the meaning scope of relations 'Contained_by' and 'Contains' into locative relations.
“The semantic information of entity type narrows down the meaning scope of relation”
Quote in the paperSubstantial claim - Interpretationown data
Our dual graph propagation and dual-primal graph interaction module are efficient for complex and multi-relational KGs.
“It illustrates the efficiency of our dual graph propagation and dual-primal graph interaction module for complex and multi-relational KGs.”
Quote in the paperSubstantial claim - Interpretationnone given
Our model is efficient in handling complex questions over KBs with heterogeneous graphs.
“We showed that our model is efficient in handling complex questions over KBs with heterogeneous graphs.”
Quote in the paperSubstantial claim - Premiseown data
The relations 'Contained_by' and 'Contains' have head-tail entities of type 'location' such as 'Niger River', 'Guinea', and 'Africa'.
“On one hand, the relation”
Quote in the paperSubstantial claim - Interpretationown data
Our performance in F1 shows our approach can significantly improve the prediction of the whole answer set.
“Our performance in F1 shows our approach can significantly improve the prediction of the whole answer set.”
Quote in the paperSubstantial claim - Interpretationreasoning
With larger size and higher connectivity, our dual-primal interaction module exploits more abundant information from head-tail entities into relation representation.
“and the dual-primal interaction module exploits more abundant information from head-tail entities into relation representation.”
Quote in the paperSubstantial claim - Interpretationreasoning
With larger size and higher connectivity, our dual graph propagation module incorporates more neighbor relation information to enhance the representation of the current relation.
“With the larger size and higher connectivity, our dual graph propagation module incorporates more neighbor relation information to enhance the representation of current relation”
Quote in the paperSubstantial claim - Interpretationnone given
We are the first to incorporate semantic information of head-tail entities and co-relation among relations to enhance relation representation for KBQA.
“We are the first to incorporate semantic information of head-tail entities and co-relation among relations to enhance the relation representation for KBQA.”
Quote in the paperNot a substantial claim - Interpretationown data
Our interaction between primal and dual graphs incorporates semantic information of head-tail entities which is crucial to represent the corresponding relation.
“It shows our interaction between primal and dual graphs incorporates semantic information of head-tail entities which is crucial to represent the corresponding relation.”
Quote in the paperSubstantial claim - Interpretationreasoning
The massive intersection between head-tail entity sets of relations 'Contained_by' and 'Contains' implies a high semantic connection between these two relations.
“On the other hand, the massive intersection between head-tail entity sets of relation”
Quote in the paperSubstantial claim
- May combine two claims. 10
- The quote only partly states this claim. 3
- Classified as probably not a substantial claim. 3
May combine two claims.
Findingown dataThe baseline NSM fails to address the specific relation information and mistakes relation 'Award_nomination' and 'Award_nominee' for the meaning of acting, predicting the wrong answer 'Alan'.
“The baseline NSM fails to address the specific relation information and mistakes relation”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOur approach achieves a significant performance gain over prior state-of-the-art on WebQSP and CWQ, specifically 0.8% Hits@1 and 1.7% F1 in WebQSP and 1.5% Hits@1 and 5.3% F1 in CWQ.
“Experiments on two public datasets, WebQSP and CWQ, show that our approach achieves a significant performance gain over the prior state-of-theart.”
- Also stated in: § 1 Introduction · plain
“We do experiments on two benchmark datasets in the field of KBQA and our approach outperforms the existing methods by 0.8% Hits@1 score and 1.7% F1 score in WebQSP, 1.5% Hits@1 score and 5.3% F1 score in CWQ (Talmor and Berant, 2018), becoming the new state-of-the-art.”
- Also stated in: § 1 Introduction · plain
“Our approach outperforms existing methods in two benchmark datasets in this field, becoming the new state-of-the-art of retrievalreasoning methods.”
Quote in the paperSubstantial claim- Also stated in: § 1 Introduction · plain
The quote only partly states this claim.
Findingown dataOur model gains stable improvement over the baseline with diverse reasoning steps from 2 to 4 on CWQ.
“Table 10 shows we gain stable improvement over the baseline with diverse reasoning steps.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataIt is effective to introduce the dual relation graph and iteratively do primal graph reasoning, dual graph information propagation, and dual-primal graph interaction.
“It is effective to introduce the dual relation graph and iteratively do primal graph reasoning, dual graph information propagation, and dual-primal graph interaction.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenWe do not investigate the effect of pretrained knowledge graph embedding which contains more structural and semantic information from a large scale of knowledge graphs.
“However, we do not investigate the effect of pretrained knowledge graph embedding which contains more structural and semantic information from a large scale of knowledge graphs.”
Quote in the paperNot a substantial claimMay combine two claims.
Premisenone givenExisting methods fail to utilize information from head-tail entities and the semantic connection between relations to enhance the current relation representation, undermining their ability to capture relation information in KGs.
“We claim they fail to utilize information from head-tail entities and the semantic connection between relations to enhance the current relation representation, which undermines the ability to capture information of relations in KGs.”
- Also stated in: § 1 Introduction · plain
“We claim the semantic connection between relations which is co-relation and the semantic information from head and tail (head-tail) entities to build relation representation is neglected.”
Quote in the paperNot a substantial claim- Also stated in: § 1 Introduction · plain
May combine two claims.
Interpretationown dataIncorporating head-tail entities information and semantic co-relation between relations enhances the construction of relation representation, which stimulates the entity representation conversely.
“Incorporating head-tail entities information and the semantic co-relation between relations enhance the construction of relation representation, which stimulates the entity representation conversely.”
Quote in the paperSubstantial claimMay combine two claims.
Premisenone givenExisting retrieval-based KBQA methods fix relation representations during reasoning, and the initial relation representation may not be optimal.
“During the reasoning process of these methods, the representation of relations are fixed but the initial relation representation may not be optimal.”
- Also stated in: § 1 Introduction · hedged
“During the reasoning process of existing methods, the representation of relations are fixed but the initial relation representation may not be optimal.”
Quote in the paperNot a substantial claim- Also stated in: § 1 Introduction · hedged
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
InterpretationreasoningOur iterative unified reasoning framework on dual relation graph and primal entity graph can fuse the representation of relations and entities in a deeper way.
“This framework could fuse the representation of relations and entities in a deeper way.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: The representation learning for one relation can enhance the other.
- The authors state this as an assumption: Relations sharing the same head or tail entity should have semantic communication.
- The authors state this as an assumption: The subtraction between head-tail entity representation composes semantic information of the relation.
May combine two claims.
InterpretationreasoningEntity types of head-tail entities for relation 'Film_starring' are film and cast, and for relation 'Performance_actor' are cast and person; this helps our model address the meaning of these 2 relations as acting by dual-primal graph interaction.
“Similarly, the information that entity types of head-tail entities for relation”
Quote in the paperSubstantial claimMay combine two claims.
PremisereasoningIn the WebQSP example, answering the question requires the model to hop from the topic entity 'Niger River' by the reasoning path 'Contained_by', and the candidate entity 'Niger' must connect to 'Sahel' with relation 'Contains' to satisfy the constraint 'of the Sahel'.
“In Figure 1, to answer the question, the model has to hop from the topic entity”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningReasoning methods are important in real scenes where ground-truth logical forms are hard to obtain.
“So the reasoning methods are important in real scenes where the ground-truth logical forms are hard to obtain.”
Quote in the paperNot a substantial claim- Rests on an assumption: Ground-truth executable query forms bring much more cost and difficulty to annotate in practical settings.
The quote only partly states this claim.
Interpretationnone givenOur dual graph propagation module assigns important attention among these relations in the reasoning path like constraint relation 'character', incorporating neighborhood co-relation to enhance the learning of their representations.
“Moreover, our dual graph propagation module assigns important attention among these relations in the reasoning path like constraint relation”
Quote in the paperSubstantial claimMay combine two claims. The quote only partly states this claim.
InterpretationreasoningEntity types of head-tail entities for relation 'Award_nomination' are film and award, and for relation 'Award_nominee' are award and person; their meanings can be inferred as nominating instead of acting by our dual-primal graph interaction module.
“However, considering that the type of head-tail entity for relation”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationHe et al. (2021) adopt a teacher-student network in multi-hop KBQA, but it ignores information from head-tail entities and semantic connection with neighbor relations to build the representation of current relation.
“He et al. (2021) adopt the teacher-student network in multihop KBQA.However, it ignores the information from head-tail entities and semantic connection with neighbor relations to build the representation of current relation.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites He et al. (2021)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
The authors state this as an assumption.
The representation learning for one relation can enhance the other.
“and we assume the representation learning for one of relations can enhance the other.”
The authors state this as an assumption.
Relations sharing the same head or tail entity should have semantic communication.
“If two relations in primal graph share same head or tail entity, we believe they should have semantic communication and we connect them in the dual graph.”
The authors state this as an assumption.
The subtraction between head-tail entity representation composes semantic information of the relation.
“Based on TransE method, we believe the subtraction between head-tail entity representation composes semantic information of the relation.”
The paper relies on this without support.
The semantic roles of head entity and tail entity in one relation are different.
“Considering the semantic roles of head entity and tail entity in one relation are different, we adopt two separate projectors to handle the relations which hold the entity e as head entity or tail entity respectively:”
The paper relies on this without support.
SP-based methods need executable logical forms as supervision, which are hard to obtain in real scenes.
“SP-based methods need executable logical forms as supervision, which are hard to obtain in real scenes.”
The paper relies on this without support.
Ground-truth executable query forms bring much more cost and difficulty to annotate in practical settings.
“The ground-truth executable query forms bring much more cost and difficulty to annotate in practical.”
The authors state this as an assumption.
By the assumption of TransE, the representation of a relation should be a translation operating on the embedding of its head entity and tail entity, satisfying e_head + r ≈ e_tail.
“By the assumption of TransE, the representation of relation should be translation operating on the embedding of its head entity and tail entity, which satisfies e head + r ≈ e tail .”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
WebQSP split counts exceed the stated total
MinorThe listed WebQSP train, dev, and test counts sum to 4,737 questions, not the stated 4,327.
Reported 4327 · recomputed 4737
“Datasets train dev test entities WebQSP 2,848 250 1,639 1,429.8CWQ 27,639 3,519 3,531 1,305.8WebQuestionsSP(WebQSP) (Yih et al., 2015) includes 4,327 natural language questions”
Cites
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · see citing works
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1710.10903 · Cited by 12,107
Graph Attention NetworksPreprint
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 12,099 · see citing works
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858826 · Cited by 9,556 · see citing works
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020) · AI Open · doi:10.1016/j.aiopen.2021.01.001 · Cited by 6,880 · see citing works
Self-citation(2013) · Cited by 5,726 · see citing works
Freebase: a collaboratively created graph database for structuring human knowledge
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376746 · Cited by 4,852 · see citing works
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · see citing works
Semantic Parsing on Freebase from Question-Answer Pairs
Berant, J., Chou, A., Frostig, R., & Liang, P. (2013) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1160 · Cited by 2,318 · see citing works
Building Watson: An Overview of the DeepQA Project
Ferrucci, D. A., Brown, E. W., Chu‐Carroll, J., Fan, J., Gondek, D. C., Kalyanpur, A. A., Lally, A., Murdock, J. W., Nyberg, E., Prager, J. M., et al. (2010) · AI Magazine · doi:10.1609/aimag.v31i3.2303 · Cited by 1,523 · see citing works
Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base
Yih, W.-T., Chang, M.-W., He, X., & Gao, J. (2015) · doi:10.3115/v1/p15-1128 · Cited by 769 · see citing works
(2020) · doi:10.18653/v1/2020.acl-main.412 · Cited by 542 · see citing works
The Web as a Knowledge-Base for Answering Complex Questions
Talmor, A., & Berant, J. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1059 · Cited by 520 · see citing works
Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text
Sun, H., Dhingra, B., Zaheer, M., Mazaitis, K., Salakhutdinov, R., & Cohen, W. W. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1455 · Cited by 510 · see citing works
Variational Reasoning for Question Answering With Knowledge Graph
Zhang, Y., Dai, H., Kozareva, Z., Smola, A. J., & Song, L. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12057 · Cited by 490 · see citing works
Question Answering over Freebase with Multi-Column Convolutional Neural Networks
Dong, L. M., Wei, F., Zhou, M., & Xu, K. (2015) · doi:10.3115/v1/p15-1026 · Cited by 451 · see citing works
PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text
Sun, H., Bedrax-Weiss, T., & Cohen, W. W. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1242 · Cited by 397 · see citing works
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
Chen, L., Berant, J., Le, Q. V., Forbus, K. D., & Lao, N. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1003 · Cited by 387 · see citing works
Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs
Wu, Y., Liu, X., Feng, Y., Wang, Z., Yan, R., & Zhao, D. (2019) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · Cited by 314 · see citing works
Self-citationRelation-Aware Entity Alignment for Heterogeneous Knowledge Graphs
Wu, Y., Liu, X., Feng, Y., Wang, Z., Yan, R., & Zhao, D. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/733 · Cited by 241 · see citing works
Self-citationImproving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signals
He, G., Lan, Y., Jiang, J., Zhao, W. X., & Wen, J.-R. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441753 · Cited by 228 · see citing works
Stepwise Reasoning for Multi-Relation Question Answering over Knowledge Graph with Weak Supervision
Qiu, Y., Wang, Y., Jin, X., et al. (2020) · doi:10.1145/3336191.3371812 · Cited by 164 · see citing works
RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering
Ye, X., Yavuz, S., Hashimoto, K., Zhou, Y., & Xiong, C. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.417 · Cited by 135 · see citing works
Shi, J., Cao, S., Hou, L., Li, J., & Zhang, H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.341 · Cited by 119 · see citing works
An Interpretable Reasoning Network for Multi-Relation Question AnsweringPreprint
Zhou, M., Huang, M., & Zhu, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.04726 · Cited by 109 · see citing works
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · arXiv · doi:10.48550/arxiv.1908.10084 · Cited by 86 · see citing works
Dialog-to-action: conversational question answering over a large-scale knowledge base
Guo, D., Tang, D., Duan, N., Zhou, M., & Yin, J. P. (2018) · neural information processing systems · Cited by 81 · see citing works
Complex Program Induction for Querying Knowledge Bases in the Absence of Gold Programs
Saha, A., Ansari, G. A., Laddha, A., Sankaranarayanan, K., & Chakrabarti, S. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00262 · Cited by 53 · see citing works
Scalable Neural Methods for Reasoning With a Symbolic Knowledge BasePreprint
Cohen, W. W., Sun, H., Hofer, R. A., & Siegler, M. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06115 · Cited by 47 · see citing works
Program Transfer for Answering Complex Questions over Knowledge Bases
Cao, S., Shi, J., Yao, Z., Lv, X., Yu, J., Hou, L., Li, J., Liu, Z., & Xiao, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.559 · Cited by 39 · see citing works
ReaRev: Adaptive Reasoning for Question Answering over Knowledge Graphs
Mavromatis, C., & Karypis, G. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.181 · Cited by 34 · see citing works
Simple Question Answering with Subgraph Ranking and Joint-Scoring
Zhao, W., Chung, T., Goyal, A., et al. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1029 · Cited by 31 · see citing works
Expanding End-to-End Question Answering on Differentiable Knowledge Graphs with Intersection
Sen, P., Oliya, A., & Saffari, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.694 · Cited by 24 · see citing works
DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge BasesPreprint
Yu, D., Zhang, S., Ng, P., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.00063 · Cited by 24 · see citing works
Modelling Long-distance Node Relations for KBQA with Global Dynamic Graph
Wang, X., Zhao, S., Han, J., Cheng, B. H., Yang, H., Ao, J., & Li, Z. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.231 · Cited by 13 · see citing works
SQALER: Scaling Question Answering by Decoupling Multi-Hop and Logical ReasoningPreprint
Atzeni, M., Bogojeska, J., & Loukas, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14266 · Cited by 7 · see citing works
Program Transfer for Answering Complex Questions over Knowledge BasesPreprint
Cao, S., Shi, J., Yao, Z., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05743
Jiang, J., Zhou, K., Zhao, W. X., & Wen, J.-R. (2022) · arXiv · doi:10.48550/arxiv.2212.00959
ReaRev: Adaptive Reasoning for Question Answering over Knowledge Graphs
Mavromatis, C., & Karypis, G. (2022) · arXiv · doi:10.48550/arxiv.2210.13650
Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text
Sun, H., Dhingra, B., Zaheer, M., et al. (2018) · arXiv · doi:10.48550/arxiv.1809.00782
The Web as a Knowledge-base for Answering Complex Questions
Talmor, A., & Berant, J. (2018) · arXiv · doi:10.48550/arxiv.1803.06643
RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering
Ye, X., Yavuz, S., Hashimoto, K., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.08678
Flagged first, then by citations
Cited by
Sorted by citations
Advancing rule learning in knowledge graphs with structure-aware graph transformer
Xu, K., Chen, M., Feng, Y., & Dong, Z. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103976 · Cited by 10 · see citing works
LSECG: LLM-based semantic enhancement and context-guided GNNs in multi-hop KGQA
Cheng, K., Zuo, Z., Liao, Y., & Tohti, T. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-026-03216-z
- Authors
- Du, H., Huang, Q., Li, C., Zhang, C., Li, Y., & Zhao, D.
- Affiliations
- Peking University (CN)
- Ant Group (China) (CN)
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Published
- 2023
- Pages
- 13582-13592
- OpenAlex
- W4389518732
- Semantic Scholar
- 8703e9b1a40fa71790f487233881abeaf9a29f7e
- Cited by
- 2
- References
- 33
- Citations by year
- 2026: 1 · 2024: 1
- FWCI
- 0.36
- Type
- conference paper
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- cc-by
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- knowledge base question answering, multi-hop reasoning, heterogeneous knowledge graphs, relational representation, information dissemination, entity representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-23.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-23.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
