CoChat
Sign Up
PreprintCited by 51Advanced Graph Neural NetworksTopic ModelingSemantic Web and Ontologies

Faithful Embeddings for Knowledge Base Queries

Open access
Sun, Haitian, Arnold, Andrew O., Bedrax-Weiss, Tania, Pereira, Fernando, Cohen, William W.·2020·arXiv
DOI 10.48550/arxiv.2004.03658arXiv 2004.03658
Open this paper in CoChat — free accountOpen PDF

Abstract

The deductive closure of an ideal knowledge base (KB) contains exactly the logical queries that the KB can answer. However, in practice KBs are both incomplete and over-specified, failing to answer some queries that have real-world answers. \emph{Query embedding} (QE) techniques have been recently proposed where KB entities and KB queries are represented jointly in an embedding space, supporting relaxation and generalization in KB inference. However, experiments in this paper show that QE systems may disagree with deductive reasoning on answers that do not require generalization or relaxation. We address this problem with a novel QE method that is more faithful to deductive reasoning, and show that this leads to better performance on complex queries to incomplete KBs. Finally we show that inserting this new QE module into a neural question-answering system leads to substantial improvements over the state-of-the-art.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2004.03658

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing KBQA systems do not have a modular reasoner to perform logical reasoning using information available in the KB.

    “However, existing KBQA systems (Diefenbach et al. 2020;Vakulenko et al. 2019;Sun et al. 2020) do not have a modular reasoner to perform logical reasoning using the information available in the knowledge base.”

    From A Semantic Parsing and Reasoning-Based Approach to Knowledge Base Question Answering (2021) · cites this paper as Sun et al. 2020

    The Introduction (p. 2) calls EmQL a module in a KBQA system. Section 4.2 (p. 7) explicitly describes embedding EmQL as a reasoning component and says an earlier KBQA model also had a reasoner, ReifKB. The source therefore contradicts the claim that existing KBQA systems lack a modular reasoner. (LLM, conf=0.90)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Existing KBQA approaches face challenges including complex question understanding, scarcity of end-to-end training data, and the need for reasoning.

    “Knowledge Base Question Answering (KBQA) is a task where existing techniques have faced significant challenges, such as the need for complex question understanding, reasoning, and large training datasets.”

    From A Semantic Parsing and Reasoning-Based Approach to Knowledge Base Question Answering (2021) · cites this paper as Sun et al. 2020

    Section 4.2 (p. 7) describes KBQA as taking natural-language questions and using a reasoning component; §4.1 (p. 7) discusses difficulties with complex queries. The paper reports training-data sizes and a benefit from pre-training (§4.2, p. 9), but does not characterize scarce end-to-end data or a need for large training datasets as a challenge for existing KBQA techniques. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 27

  1. [1][1] B. H. Bloom. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM , 13(7):422-426, 1970. -
  2. [2][2] A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, and O. Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in neural information processing systems , pages 2787-2795, 2013. -
  3. [3][3] W. W. Cohen, M. Siegler, and R. A. Hofer. Neural query language: A knowledge base query language for tensorflow. CoRR , abs/1905.06209, 2019. URL http://arxiv.org/abs/1905. 06209 . - · arXiv
  4. [4][4] W. W. Cohen, H. Sun, R. A. Hofer, and M. Siegler. Scalable neural methods for reasoning with a symbolic knowledge base. arXiv preprint arXiv:2002.06115 , 2020. Appeared in ICLR-2020. - · arXiv
  5. [5][5] G. Cormode and S. Muthukrishnan. An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms , 55(1):58-75, 2005. -
  6. [6][6] A. Daniely, N. Lazic, Y. Singer, and K. Talwar. Sketching and neural networks. arXiv preprint arXiv:1604.05753 , 2016. - · arXiv
  7. [7][7] R. Das, A. Neelakantan, D. Belanger, and A. McCallum. Chains of reasoning over entities, relations, and text using recurrent neural networks. arXiv preprint arXiv:1607.01426 , 2016. - · arXiv
  8. [8][8] T. Demeester, T. Rocktäschel, and S. Riedel. Lifted rule injection for relation embeddings. arXiv preprint arXiv:1606.08359 , 2016. - · arXiv
  9. [10][10] K. Guu, J. Miller, and P. Liang. Traversing knowledge graphs in vector space. arXiv preprint arXiv:1506.01094 , 2015. - · arXiv
  10. [11][11] W. Hamilton, P. Bajaj, M. Zitnik, D. Jurafsky, and J. Leskovec. Embedding logical queries on knowledge graphs. In Advances in Neural Information Processing Systems , pages 2026-2037, 2018. -
  11. [12][12] Y. Lin, Z. Liu, H. Luan, M. Sun, S. Rao, and S. Liu. Modeling relation paths for representation learning of knowledge bases. arXiv preprint arXiv:1506.00379 , 2015. - · arXiv
  12. [13][13] H. Liu, Y. Wu, and Y. Yang. Analogical inference for multi-relational embeddings. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 , pages 2168-2178. JMLR. org, 2017. -
  13. [14][14] A. Miller, A. Fisch, J. Dodge, A.-H. Karimi, A. Bordes, and J. Weston. Key-value memory networks for directly reading documents. EMNLP , 2016. -
  14. [15][15] A. H. Miller, A. Fisch, J. Dodge, A. Karimi, A. Bordes, and J. Weston. Key-value memory networks for directly reading documents. CoRR , abs/1606.03126, 2016. URL http://arxiv. org/abs/1606.03126 . - · arXiv
  15. [16][16] S. Mussmann and S. Ermon. Learning and inference via maximum inner product search. In International Conference on Machine Learning , pages 2587-2596, 2016. -
  16. [17][17] P. Rastogi, A. Poliak, and B. Van Durme. Training relation embeddings under logical constraints. In KG4IR@ SIGIR , pages 25-31, 2017. -
  17. [19][19] A. Saxena, A. Tripathi, and P. Talukdar. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. ACL , 2020. -
  18. [20][20] H. Sun, B. Dhingra, M. Zaheer, K. Mazaitis, R. Salakhutdinov, and W. W. Cohen. Open domain question answering using early fusion of knowledge bases and text. EMNLP , 2018. -
  19. [21][21] H. Sun, T. Bedrax-Weiss, and W. W. Cohen. Pullnet: Open domain question answering with iterative retrieval on knowledge bases and text. arXiv preprint arXiv:1904.09537 , 2019. - · arXiv
  20. [22][22] K. Svozil. Quantum logic . Springer Science & Business Media, 1998. -
  21. [24][24] L. Vilnis and A. McCallum. Word representations via gaussian embedding. arXiv preprint arXiv:1412.6623 , 2014. - · arXiv
  22. [25][25] L. Vilnis, X. Li, S. Murty, and A. McCallum. Probabilistic embedding of knowledge graphs with box lattice measures. arXiv preprint arXiv:1805.06627 , 2018. - · arXiv
  23. [26][26] M. Wang, R. Wang, J. Liu, Y. Chen, L. Zhang, and G. Qi. Towards empty answers in sparql: Approximating querying with rdf embedding. In D. Vrandeˇ ci´ c, K. Bontcheva, M. C. SuárezFigueroa, V. Presutti, I. Celino, M. Sabou, L.-A. Kaffee, and E. Simperl, editors, The Semantic Web - ISWC 2018 , pages 513-529, Cham, 2018. Springer International Publishing. ISBN 978-3-030-00671-6. -
  24. [27][27] Q. Wang, Z. Mao, B. Wang, and L. Guo. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering , 29(12):2724-2743, 2017. -
  25. [28][28] W.-t. Yih, M.-W. Chang, X. He, and J. Gao. Semantic parsing via staged query graph generation: Question answering with knowledge base. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages 1321-1331, Beijing, China, July 2015. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/ P15-1128 . -
  26. [29][29] M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola. Deep sets. In Advances in neural information processing systems , pages 3391-3401, 2017. -
  27. [30][30] Y. Zhang, H. Dai, Z. Kozareva, A. J. Smola, and L. Song. Variational reasoning for question answering with knowledge graph. In AAAI , 2018.

Cited by

Sorted by citations

Authors
Sun, H., Arnold, A. O., Bedrax-Weiss, T., Pereira, F., & Cohen, W. W.
Affiliations
Google (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2004.03658
arXiv
2004.03658
OpenAlex
W3041520157
Cited by
51
References
3
Citations by year
2025: 4 · 2024: 6 · 2023: 11 · 2022: 9 · 2021: 18 · 2020: 3
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
Keywords
query embedding, knowledge base queries, deductive closure, incomplete knowledge bases, deductive reasoning, query relaxation, query generalization, knowledge base inference

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review