PreprintCited by 115Advanced Graph Neural NetworksTopological and Geometric Data AnalysisNeural Networks and Applications

The DOI matches its DataCite record; 37 of the 37 references it cites were found in a registry or an index; 15 of the 29 sources its claims cite were checked for support.

Hyperbolic Entailment Cones for Learning Hierarchical Embeddings

Open access

Abstract

Learning graph representations via low-dimensional embeddings that preserve relevant network properties is an important class of problems in machine learning. We here present a novel method to embed directed acyclic graphs. Following prior work, we first advocate for using hyperbolic spaces which provably model tree-like structures better than Euclidean geometry. Second, we view hierarchical relations as partial orders defined using a family of nested geodesically convex cones. We prove that these entailment cones admit an optimal shape with a closed form expression both in the Euclidean and hyperbolic spaces, and they canonically define the embedding learning process. Experiments show significant improvements of our method over strong recent baselines both in terms of representational capacity and generalization.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

The paper introduces geodesically convex entailment cones for embedding directed acyclic graphs and hierarchical relations, with closed-form optimal cone shapes in Euclidean and hyperbolic space. On WordNet hypernym link prediction, hyperbolic entailment cones outperform strong baselines, especially as more transitive-closure edges are available during training.

Key findings

  • The optimal entailment cones satisfying axial symmetry, rotation invariance, continuity, and transitivity admit closed-form expressions in both Euclidean and hyperbolic geometry; the authors also derive a closed-form exponential map in the Poincaré ball.
  • On WordNet link prediction, hyperbolic cones achieve test F1 of 80.1%, 86.0%, and 92.8% at embedding dimension 5 for 10%, 25%, and 50% of non-basic transitive-closure edges in training, and 85.9%, 91.0%, and 94.4% at dimension 10 for the same settings.
  • Hyperbolic cones are better than all baselines in all settings except 0% training edges, where order embeddings are better; once a small percentage of transitive-closure edges is available, improvements are sometimes more than 8% F1.
  • Hyperbolic cones show the largest growth when transitive-closure edges are added at train time.

Methods

  • The model represents an entailment relation \((u,v)\) as \(v\) belonging to the entailment cone \(S_{\psi(u)}^u\), trained with a max-margin loss on angles using margin \(\gamma=0.01\), 10 negative edges per positive edge, batch size 10, learning rate 1e-4, and projection outside an \(\epsilon=0.1\) ball.
  • Data: the full transitive closure of the WordNet noun hierarchy with the root removed, containing 82,114 nodes and 661,127 edges; basic transitive-reduction edges are always in training, while non-basic edges are split into validation (5%), test (5%), and training fractions of 0%, 10%, 25%, or 50%.
  • Baselines are Simple Euclidean embeddings, Poincaré embeddings, and Order embeddings; evaluation uses precision, recall, and F1, with thresholds tuned on validation, and negative validation/test pairs generated by corrupting each positive edge with five \(u'\) and five \(v'\) non-edges.

Limitations

  • The paper states that future work should test whether entailment cones can embed more complex data such as sentences or images; the current experiments are limited to WordNet noun hierarchy link prediction.
  • The paper reports that order embeddings are better in the 0% training-edge setting, and that initialization is very important because the loss is highly non-convex; hyperbolic cones are initialized from Poincaré embeddings pretrained for 100 epochs and rescaled by 0.7.
  • The paper states that full Riemannian optimization did not give significant empirical improvements over the retraction approximation, so the faster retraction was used; evident from the text, evaluation is restricted to low-dimensional settings (5 and 10 dimensions) and one WordNet-based DAG task.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.1804.01882

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Convex cones in a complete Riemannian manifold.
Figure 1 Convex cones in a complete Riemannian manifold.From the PDF
Figure 2 Poincar´ e angular cones satisfying Eq. 26 for K = 0.1. Left: examples of cones for points with Euclidean norm varying from 0.1 to 0.9. Right: transitivity for various points on the border of their parent cones.
Figure 2 Poincar´ e angular cones satisfying Eq. 26 for K = 0.1. Left: examples of cones for points with Euclidean norm varying from 0.1 to 0.9. Right: transitivity for various points on the border of their parent cones.From the PDF
Figure 3 Two dimensional embeddings of two datasets: a toy uniform tree of depth 7 and branching factor 3, with root removed (left); the mammal subtree of WordNet with 4230 relations, 1165 nodes and top 2 nodes removed (right). (Nickel & Kiela, 2017) (each left side) has most of the nodes and edges collapsed on the space border, while our hyperbolic cones (each right side) nicely reveal the data structure.
Figure 3 Two dimensional embeddings of two datasets: a toy uniform tree of depth 7 and branching factor 3, with root removed (left); the mammal subtree of WordNet with 4230 relations, 1165 nodes and top 2 nodes removed (right). (Nickel & Kiela, 2017) (each left side) has most of the nodes and edges collapsed on the space border, while our hyperbolic cones (each right side) nicely reveal the data structure.From the PDF
From Cor. 1.2 we know that the points O,x,x ′ , y, z are coplanar. We denote this plane by P . Furthermore, the metric of the Poincar´ e ball is conformal with the Euclidean metric. Given these two facts, we derive that
From Cor. 1.2 we know that the points O,x,x ′ , y, z are coplanar. We denote this plane by P . Furthermore, the metric of the Poincar´ e ball is conformal with the Euclidean metric. Given these two facts, we derive thatFrom the PDF

Verification

Claims the paper makes

15 No basis given · 11 Own interpretation · 8 Supported · 7 Own result · 3 Couldn’t confirm · 3 No full text of the sources · 2 Partly supported · 2 Sources not found · 2 Not checked yet · accuracy not yet measured

Show all 26 claims

  • It may state two separate claims as one. 15
  • Probably not a substantial claim. 16
  • The quote only partly states this claim. 3
27 of 27
  • The authors state this as an assumption.

    The origin should entail the entire space D^n, i.e. S_0 = D^n.

    “we first note that the origin should 'entail' the entire space D n , i.e. S 0 = D n .”
  • The authors state this as an assumption.

    Cones should have an aperture as large as possible to maximize model capacity.

    “Since we are interested in cones with an aperture as large as possible (to maximize model capacity)”
  • The authors state this as an assumption.

    The exponential map should be injective.

    “Note that, in the above definition, we desire that the exponential map be injective.”
  • The authors state this as an assumption.

    Hierarchical relations can be modeled as partial orders defined using a family of nested geodesically convex cones.

    “we view hierarchical relations as partial orders defined using a family of nested geodesically convex cones.”
  • The authors state this as an assumption.

    The optimal entailment cones should satisfy axial symmetry, rotation invariance, continuous aperture, and transitivity.

    “we want the definition of cones to exhibit the following four intuitive properties detailed below.”
  • The paper relies on this without support.

    The basic edges (transitive reduction) are hard to predict.

    “These edges are hard to predict, so we will always include them in the training set.”
  • The paper relies on this without support.

    The tree root carries little information and only has trivial edges to predict.

    “We remove the tree root since it carries little information and only has trivial edges to predict.”
  • The paper relies on this without support.

    Hyperbolic geometry is powerful for embedding hierarchical structures.

    “Hyperbolic geometry has proven to be powerful for embedding hierarchical structures.”
  • The paper relies on this without support.

    The WordNet noun hierarchy exhibits a latent hierarchical structure.

    “We evaluate the representational and generalization power of hyperbolic entailment cones and of other baselines using data that exhibits a latent hierarchical structure.”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

  • Glove: Global Vectors for Word Representation

    Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 3×

    “A large line of research focuses on embedding discrete data such as graphs (Grover & Leskovec, 2016; Goyal & Ferrara, 2017) or linguistic instances (Mikolov et al., 2013; Pennington et al., 2014; Kiros et al., 2015) into continuous spaces that exhibit certain desirable geometric properties.”
    Background
  • Distributed Representations of Words and Phrases and their Compositionality

    Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 3×

    “A large line of research focuses on embedding discrete data such as graphs (Grover & Leskovec, 2016; Goyal & Ferrara, 2017) or linguistic instances (Mikolov et al., 2013; Pennington et al., 2014; Kiros et al., 2015) into continuous spaces that exhibit certain desirable geometric properties.”
    Background
  • node2vec: Scalable Feature Learning for Networks

    Grover, A., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939754 · Cited by 10,926 · cited 1×

    “A large line of research focuses on embedding discrete data such as graphs (Grover & Leskovec, 2016; Goyal & Ferrara, 2017) or linguistic instances (Mikolov et al., 2013; Pennington et al., 2014; Kiros et al., 2015) into continuous spaces that exhibit certain desirable geometric properties.”
    Background
  • Untitled work

    (2013) · Cited by 5,726

  • Introduction to WordNet: An On-line Lexical Database *

    Miller, G. A., Beckwith, R., Fellbaum, C., Gross, D., & Miller, K. J. (1990) · International Journal of Lexicography · doi:10.1093/ijl/3.4.235 · Cited by 4,197 · cited 2×

    “, 2015) and use the full transitive closure of the WordNet noun hierarchy (Miller et al., 1990).”
    BackgroundMethod
  • Geometric Deep Learning: Going beyond Euclidean data

    Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661 · cited 2×

    “, 2001), Social Networks), one has to deal with data whose latent anatomy is best defined by nonEuclidean spaces such as Riemannian manifolds (Bronstein et al., 2017).”
    Background
  • A Comprehensive Introduction to Differential Geometry.

    Kramer, A. D., & Spivak, M. (1973) · American Mathematical Monthly · doi:10.2307/2319112 · Cited by 3,525 · cited 1×

    “For an in-depth introduction, we refer the reader to (Spivak, 1979) and (Hopper & Andrews, 2010).”
    Method
  • Latent Space Approaches to Social Network Analysis

    Hoff, P. D., Raftery, A. E., & Handcock, M. S. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502388618906 · Cited by 1,917 · cited 2×

    “…for various tasks and applications, such as link prediction in knowledge bases (Nickel et al., 2011; Bor-des et al., 2013) or in social networks (Hoff et al., 2002), text disambiguation (Ganea & Hofmann, 2017), word hyper-nymy (Shwartz et al., 2016), textual entailment (Rockt¨aschel et al.,…”
    Background
  • Hyperbolic Groups

    Gromov, M. (1987) · Mathematical Sciences Research Institute Publications · doi:10.1007/978-1-4613-9586-7_3 · Cited by 1,817

  • A Three-Way Model for Collective Learning on Multi-Relational Data

    Nickel, M., Tresp, V., & Kriegel, H. (2011) · Cited by 1,806

Show all 31 references

Research works printed but not linked: 1

  1. Bl¨ asius, T., Friedrich, T., Krohmer, A., and Laue, S. Efficient embedding of scale-free graphs in the hyperbolic plane. In LIPIcs-Leibniz International Proceedings in Informatics , volume 57. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2016.

Cited by

Sorted by citations

See all 115

Metadata

Authors
Ganea, O.-E., Bécigneul, G., & Hofmann, T.
Affiliations
ETH Zurich (CH)
Venue
arXiv
Publisher
arXiv
Published
2018
Volume
4
Pages
1646-1655
ISSN
2331-8422 (ISSN-L)
OpenAlex
W2796314760
Cited by
115
References
31
Citations by year
2025: 7 · 2024: 18 · 2023: 15 · 2022: 15 · 2021: 22 · 2020: 19 · 2019: 16 · 2018: 3
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • International Conference on Machine Learning
Funders
  • National Science Foundation
  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (167176)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Topological and Geometric Data Analysis
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords
hyperbolic embeddings, directed acyclic graphs, hierarchical embedding, partially ordered sets, graph representation learning, tree-like structures, hyperbolic geometry

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-23, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-23 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review