Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures
Open accessAbstract
Embedding methods which enforce a partial order or lattice structure over the concept space, such as Order Embeddings (OE) (Vendrov et al., 2016), are a natural way to model transitive relational data (e.g.entailment graphs).However, OE learns a deterministic knowledge base, limiting expressiveness of queries and the ability to use uncertainty for both prediction and learning (e.g.learning from expectations).Probabilistic extensions of OE (Lai and Hockenmaier, 2017) have provided the ability to somewhat calibrate these denotational probabilities while retaining the consistency and inductive bias of ordered models, but lack the ability to model the negative correlations found in real-world knowledge.In this work we show that a broad class of models that assign probability measures to OE can never capture negative correlation, which motivates our construction of a novel box lattice and accompanying probability measure to capture anticorrelation and even disjoint concepts, while still providing the benefits of probabilistic modeling, such as the ability to perform rich joint and conditional queries over arbitrary sets of concepts, and both learning from and predicting calibrated uncertainty.We show improvements over previous approaches in modeling the Flickr and WordNet entailment graphs, and investigate the power of the model. * Equal contribution.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many real-world knowledge graphs contain hierarchical relations.
“Many real-world knowledge graphs contain hierarchical relations [10, 11, 26].”
From Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones (2021) · cites this paper as [11]
Recent works represent random variables as more complex geometric objects, such as cones and axis-aligned hyperrectangles (boxes), and use volume as the probability measure.
“recent works (Lai and Hockenmaier, 2017;Vilnis et al., 2018;Li et al., 2019;Dasgupta et al., 2020) represent random variables as more complex geometric objects, such as cones and axis-aligned hyperrectangles (boxes), and use volume as the probability measure.”
From Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021) · cites this paper as Vilnis et al., 2018
Box volumes can be interpreted as unnormalized probabilities.
“Vilnis et al. (2018) note that this allows one to interpret box volumes as unnormalized probabilities.”
From Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021) · cites this paper as Vilnis et al., 2018
Box embeddings capture probabilistic semantics based on box volumes, and are capable of compactly representing conditional probability distributions.
“Box embeddings not only demonstrate improved performance on modeling hierarchies, such embeddings also capture probabilistic semantics based on box volumes, and are capable of compactly representing conditional probability distributions.”
From Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021) · cites this paper as Vilnis et al., 2018
Box embeddings are naturally good at capturing transitive relations.
“box embeddings are naturally good at capturing transitive relations, as shown in the recent study (Vilnis et al., 2018).”
From Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021) · cites this paper as Vilnis et al., 2018
Probabilistic box embeddings represent elements using axis-aligned hyperrectangles (boxes).
“Introduced by Vilnis et al. (2018), probabilistic box embeddings represent elements using axis-aligned hyperrectangles (or boxes).”
From Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021) · cites this paper as Vilnis et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · see citing works
Flagged first, then by citations
Research works printed but not linked: 18
- Ben Athiwaratkun and Andrew Gordon Wilson. 2018. On modeling hierarchical data via probabilistic or- der embeddings. In International Conference on Learning Representations.
- Mohit Bansal, David Burkett, Gerard De Melo, and Dan Klein. 2014. Structured learning for taxon- omy induction with belief propagation. In Proceed- ings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), volume 1, pages 1041-1051.
- Antoine Bordes, Nicolas Usunier, Alberto Garcia- Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi- relational data. In Advances in neural information processing systems, pages 2787-2795.
- Michael Collins, Sanjoy Dasgupta, and Robert E Schapire. 2002. A generalization of principal com- ponents analysis to the exponential family. In Ad- vances in neural information processing systems, pages 617-624.
- Katrin Erk. 2009. Representing words as regions in vector space. In Proceedings of the Thirteenth Con- ference on Computational Natural Language Learn- ing, CoNLL '09, pages 57-65, Stroudsburg, PA, USA. Association for Computational Linguistics.
- Salar Fattahi and Somayeh Sojoudi. 2017. Graphi- cal lasso and thresholding: Equivalence and closed- form solutions. arXiv preprint arXiv:1708.09479. · arXiv
- Jerome Friedman, Trevor Hastie, and Robert Tibshi- rani. 2008. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3):432- 441.
- Xavier Glorot and Yoshua Bengio. 2010. Understand- ing the difficulty of training deep feedforward neu- ral networks. In Proceedings of the Thirteenth In- ternational Conference on Artificial Intelligence and Statistics, pages 249-256.
- Shizhu He, Kang Liu, Guoliang Ji, and Jun Zhao. 2015. Learning to represent knowledge graphs with gaus- sian embedding. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, CIKM '15, pages 623- 632, New York, NY, USA. ACM. · DOI
- David Heckerman, Dan Geiger, and David M Chicker- ing. 1995. Learning bayesian networks: The com- bination of knowledge and statistical data. Machine learning, 20(3):197-243.
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735-1780.
- Alice Lai and Julia Hockenmaier. 2017. Learning to predict denotational probabilities for modeling en- tailment. In EACL.
- Xiang Li, Luke Vilnis, and Andrew McCallum. 2017. Improved representation learning for predicting commonsense ontologies. NIPS Workshop on Struc- tured Prediction.
- Maximillian Nickel and Douwe Kiela. 2017. Poincaré embeddings for learning hierarchical representa- tions. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett, editors, Advances in Neural Information Pro- cessing Systems 30, pages 6338-6347.
- Daniel M Roy and Yee W Teh. 2009. The mondrian process. In Advances in neural information process- ing systems, pages 1377-1384.
- Ivan Vendrov, Ryan Kiros, Sanja Fidler, and Raquel Urtasun. 2016. Order-embeddings of images and language. In ICLR.
- Luke Vilnis and Andrew McCallum. 2015. Word rep- resentations via gaussian embedding. In ICLR.
- Adriaan C. Zaanen. 1997. Introduction to Operator Theory in Riesz Spaces. Springer Berlin Heidelberg.
Cited by
Sorted by citations
Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings
Ren, H., Hu, W., & Leskovec, J. (2020) · arXiv · doi:10.48550/arxiv.2002.05969 · Cited by 135 · see citing works
Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning
Chen, X., Boratko, M., Chen, M., Dasgupta, S. S., Li, X. L., & McCallum, A. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.68 · Cited by 47 · see citing works
“In order to encode probability distributions in the embedding space, recent works (Lai and Hocken-maier, 2017; Vilnis et al., 2018; Li et al., 2019; Dasgupta et al., 2020) represent random variables as more complex geometric objects, such as cones and axis-aligned hyperrectangles ( boxes ), and use…”
BackgroundMethodResultInfluentialSelf-citationModeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones
Bai, Y., Ying, R., Ren, H., & Leskovec, J. (2021) · arXiv · doi:10.48550/arxiv.2110.14923 · Cited by 31 · see citing works
“Many real-world knowledge graphs contain hierarchical relations [10, 11, 26].”
BackgroundBox-To-Box Transformations for Modeling Joint Hierarchies
Dasgupta, S. S., Li, X. L., Boratko, M., Zhang, D., & McCallum, A. (2021) · Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021) · doi:10.18653/v1/2021.repl4nlp-1.28 · Cited by 2 · see citing works
Self-citation
- Authors
- Vilnis, L., Li, X., Murty, S., & McCallum, A.
- Affiliations
- University of Massachusetts Amherst (US)
- Chinese Academy of Sciences (CN)
- Indian Institute of Technology Indore (IN)
- Venue
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2018
- Pages
- 263-272
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2798621714
- Cited by
- 111
- References
- 1
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 17 · 2023: 14 · 2022: 13 · 2021: 20 · 2020: 16 · 2019: 12 · 2018: 3
- FWCI
- 5.20
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1514053)
- Chan Zuckerberg Initiative
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
- Keywords
- order embeddings, lattice structures, partially ordered sets, negative correlation, anticorrelation, uncertainty calibration, WordNet, Flickr
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
