CoChat
Sign Up
PreprintCited by 74Advanced Graph Neural NetworksTopic ModelingSemantic Web and Ontologies

BoxE: A Box Embedding Model for Knowledge Base Completion

Open access
Abboud, Ralph, Ceylan, İsmail İlkan, Lukasiewicz, Thomas, Salvatori, Tommaso·2020·arXiv
DOI 10.48550/arxiv.2007.06267arXiv 2007.06267
Open this paper in CoChat — free accountOpen PDF

Abstract

Knowledge base completion (KBC) aims to automatically infer missing facts by exploiting information already present in a knowledge base (KB). A promising approach for KBC is to embed knowledge into latent spaces and make predictions from learned embeddings. However, existing embedding models are subject to at least one of the following limitations: (1) theoretical inexpressivity, (2) lack of support for prominent inference patterns (e.g., hierarchies), (3) lack of support for KBC over higher-arity relations, and (4) lack of support for incorporating logical rules. Here, we propose a spatio-translational embedding model, called BoxE, that simultaneously addresses all these limitations. BoxE embeds entities as points, and relations as a set of hyper-rectangles (or boxes), which spatially characterize basic logical properties. This seemingly simple abstraction yields a fully expressive model offering a natural encoding for many desired logical properties. BoxE can both capture and inject rules from rich classes of rule languages, going well beyond individual inference patterns. By design, BoxE naturally applies to higher-arity KBs. We conduct a detailed experimental analysis, and show that BoxE achieves state-of-the-art performance, both on benchmark knowledge graphs and on more general KBs, and we empirically show the power of integrating logical rules.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2007.06267

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Query2Box and BoxE make use of boxes in the loss function of their models, but entities themselves are represented as vectors.

    “Query2Box (Ren et al., 2020) and BoxE (Abboud et al., 2020) make use of boxes in the loss function of their models, however entities themselves are represented as vectors, and thus these models do not benefit from the probabilistic semantics of box embeddings, which we rely on heavily for modeling UKGs.”

    From Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021) · cites this paper as Abboud et al., 2020

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 45

  1. [1][1] Ivana Balazevic, Carl Allen, and Timothy M. Hospedales. TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the Ninth International Joint Conference on Natural Language Processing, EMNLP-IJCNLP , 2019. -
  2. [2][2] Kurt D. Bollacker, Robert P. Cook, and Patrick Tufts. Freebase: A shared database of structured general human knowledge. In Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, AAAI , 2007. -
  3. [3][3] Antoine Bordes, Sumit Chopra, and Jason Weston. Question answering with subgraph embeddings. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP , 2014. -
  4. [4][4] Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Proceedings of the Twenty-Sixth Annual Conference on Advances in Neural Information Processing Systems (NIPS) , 2013. -
  5. [5][5] L. Sunil Chandran, Mathew C. Francis, and Naveen Sivadasan. Geometric representation of graphs in low dimension using axis parallel boxes. Algorithmica , 56(2):129-140, 2008. -
  6. [6][6] Kai-Wei Chang, Wen-tau Yih, Bishan Yang, and Christopher Meek. Typed tensor decomposition of knowledge bases for relation extraction. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP , 2014. -
  7. [7][7] Thomas Demeester, Tim Rocktäschel, and Sebastian Riedel. Lifted rule injection for relation embeddings. In Proceedings of the 2016 Annual Conference on Empirical Methods in Natural Language Processing, EMNLP , 2016. -
  8. [8][8] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2D knowledge graph embeddings. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, AAAI , 2018. -
  9. [9][9] Boyang Ding, Quan Wang, Bin Wang, and Li Guo. Improving knowledge graph embedding using simple constraints. In Proceedings of the Fifth-Sixth Annual Meeting of the Association for Computational Linguistics, ACL , 2018. -
  10. [10][10] Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. Knowledge Vault: A web-scale approach to probabilistic knowledge fusion. In Proceedings of the Twentieth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD , 2014. -
  11. [11][11] Bahare Fatemi, Perouz Taslakian, David Vázquez, and David Poole. Knowledge hypergraphs: Extending knowledge graphs beyond binary relations. CoRR , abs/1906.00137, 2019. - · arXiv
  12. [12][12] Jun Feng, Minlie Huang, Mingdong Wang, Mantong Zhou, Yu Hao, and Xiaoyan Zhu. Knowledge graph embedding by flexible translation. In Proceedings of the Fifteenth International Conference on the Principles of Knowledge Representation and Reasoning, KR , 2016. -
  13. [13][13] Shu Guo, Quan Wang, Lihong Wang, Bin Wang, and Li Guo. Jointly embedding knowledge graphs and logical rules. In Proceedings of the 2016 Annual Conference on Empirical Methods in Natural Language Processing, EMNLP , 2016. -
  14. [14][14] Víctor Gutiérrez-Basulto and Steven Schockaert. From knowledge graph embedding to ontology embedding? an analysis of the compatibility between vector space representations and rules. In Proceedings of the Sixteenth International Conference on the Principles of Knowledge Representation and Reasoning, KR , 2018. -
  15. [15][15] Frank L Hitchcock. The expression of a tensor or a polyadic as a sum of products. Journal of Mathematics and Physics , 6(1-4):164-189, 1927. -
  16. [17][17] Guoliang Ji, Kang Liu, Shizhu He, and Jun Zhao. Knowledge graph completion with adaptive sparse transfer matrix. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI , 2016. -
  17. [18][18] Seyed Mehran Kazemi and David Poole. Simple embedding for link prediction in knowledge graphs. In Proceedings of the Thirty-First Annual Conference on Advances in Neural Information Processing Systems, NeurIPS , 2018. -
  18. [19][19] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the Third International Conference on Learning Representations, ICLR , 2015. -
  19. [20][20] Timothée Lacroix, Nicolas Usunier, and Guillaume Obozinski. Canonical tensor decomposition for knowledge base completion. In Proceedings of the Thirty-Fifth International Conference on Machine Learning, ICML , 2018. -
  20. [21][21] Yann LeCun. Self-supervised learning. In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI, Invited Talk , 2020. -
  21. [22][22] Xiang Li, Luke Vilnis, Dongxu Zhang, Michael Boratko, and Andrew McCallum. Smoothing the geometry of probabilistic box embeddings. In Proceedings of the Seventh International Conference on Learning Representations, ICLR , 2019. -
  22. [23][23] Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, AAAI , 2015. -
  23. [24][24] Yu Liu, Quanming Yao, and Yong Li. Generalizing tensor decomposition for n-ary relational knowledge bases. In Proceedings of the The Web Conference, WWW '20 , 2020. -
  24. [25][25] Xin Lv, Lei Hou, Juanzi Li, and Zhiyuan Liu. Differentiating concepts and instances for knowledge graph embedding. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP , 2018. -
  25. [26][26] Farzaneh Mahdisoltani, Joanna Biega, and Fabian M. Suchanek. YAGO3: A knowledge base from multilingual wikipedias. In Proceedings of the Seventh Biennial Conference on Innovative Data Systems Research, CIDR , 2015. -
  26. [27][27] Pasquale Minervini, Thomas Demeester, Tim Rocktäschel, and Sebastian Riedel. Adversarial sets for regularising neural link predictors. In Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, UAI , 2017. -
  27. [28][28] Tom M. Mitchell, William W. Cohen, Estevam R. Hruschka Jr., Partha P. Talukdar, Bo Yang, Justin Betteridge, Andrew Carlson, Bhavana Dalvi Mishra, Matt Gardner, Bryan Kisiel, Jayant Krishnamurthy, Ni Lao, Kathryn Mazaitis, Thahir Mohamed, Ndapandula Nakashole, Emmanouil A. Platanios, Alan Ritter, Mehdi Samadi, Burr Settles, Richard C. Wang, Derry Wijaya, Abhinav Gupta, Xinlei Chen, Abulhair Saparov, Malcolm Greaves, and Joel Welling. Never-ending learning. Communications of the ACM , 61(5):103-115, 2018. -
  28. [29][29] Deepak Nathani, Jatin Chauhan, Charu Sharma, and Manohar Kaul. Learning attention-based embeddings for relation prediction in knowledge graphs. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL , 2019. -
  29. [30][30] Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. A three-way model for collective learning on multi-relational data. In Proceedings of the Twenty-Eighth International Conference on Machine Learning, ICML , 2011. -
  30. [31][31] Fred S. Roberts. On the Boxicity and Cubicity of a graph. In Recent Progress in Combinatorics, Academic Press , 1968. -
  31. [32][32] Tim Rocktäschel, Sameer Singh, and Sebastian Riedel. Injecting logical background knowledge into embeddings for relation extraction. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL , 2015. -
  32. [33][33] Daniel Ruffinelli, Samuel Broscheit, and Rainer Gemulla. You CAN teach an old dog new tricks! On training knowledge graph embeddings. In Proceedings of the Eighth International Conference on Learning Representations, ICLR , 2020. -
  33. [34][34] Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Y. Ng. Reasoning with neural tensor networks for knowledge base completion. In Proceedings of the Twenty-Seventh Annual Conference on Neural Information Processing Systems, NIPS , 2013. -
  34. [35][35] Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Y. Ng. Reasoning with neural tensor networks for knowledge base completion. In Proceedings of the Twenty-Sixth Annual Conference on Advances in Neural Information Processing Systems , NIPS , 2013. -
  35. [36][36] Sandeep Subramanian and Soumen Chakrabarti. New embedded representations and evaluation protocols for inferring transitive relations. In Proceedings of the Fourty-First International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR , 2018. -
  36. [40][40] Ledyard R Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika , 31(3):279-311, 1966. -
  37. [41][41] Luke Vilnis, Xiang Li, Shikhar Murty, and Andrew McCallum. Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures. In Proceedings of the Fifty-Sixth Annual Meeting of the Association for Computational Linguistics (ACL) , 2018. -
  38. [42][42] Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In Proceedings of the Twenty-Seventh ACM International Conference on Information and Knowledge Management, CIKM , 2018. -
  39. [43][43] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, AAAI , 2014. -
  40. [44][44] Jianfeng Wen, Jianxin Li, Yongyi Mao, Shini Chen, and Richong Zhang. On the Representation and Embedding of Knowledge Bases beyond Binary Relations. In Proceedings of the TwentyFifth International Joint Conference on Artificial Intelligence, IJCAI , 2016. -
  41. [45][45] Robert West, Evgeniy Gabrilovich, Kevin Murphy, Shaohua Sun, Rahul Gupta, and Dekang Lin. Knowledge base completion via search-based question answering. In Proceedings of the Twenty-Third International World Wide Web Conference, WWW , 2014. -
  42. [46][46] Ruobing Xie, Zhiyuan Liu, and Maosong Sun. Representation learning of knowledge graphs with hierarchical types. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI , 2016. -
  43. [47][47] Chenyan Xiong, Russell Power, and Jamie Callan. Explicit semantic ranking for academic search via knowledge graph embedding. In Proceedings of the Twenty-Sixth International Conference on World Wide Web, WWW , 2017. -
  44. [48][48] Bishan Yang and Tom M. Mitchell. Leveraging knowledge bases in LSTMs for improving machine reading. In Proceedings of the Fifty-Fifth Annual Meeting of the Association for Computational Linguistics, ACL , 2017. -
  45. [49][49] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the Third International Conference on Learning Representations, ICLR , 2015.

Cited by

Sorted by citations

Authors
Abboud, R., Ceylan, İ. İ., Lukasiewicz, T., & Salvatori, T.
Affiliations
University of Oxford (GB)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
Volume
33
Pages
9649-9661
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2007.06267
arXiv
2007.06267
OpenAlex
W3040868708
Cited by
74
References
3
Citations by year
2025: 10 · 2024: 32 · 2023: 16 · 2022: 10 · 2021: 6
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • neural information processing systems
Funders
  • DeepMind
  • University of Oxford (EP/N510129/1)
  • AXA Research Fund
  • Engineering and Physical Sciences Research Council (N510129, EP/N510129/, EP/N510129/1, EP/R013667/1, M025268, EP/M025268/1, EP/M025268/)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
Keywords
knowledge base completion, box embeddings, hyperrectangles, knowledge graph embedding, inference patterns

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review