CoChat
Sign Up
PreprintCited by 668Advanced Graph Neural NetworksTopic ModelingNeural Networks and Applications

Poincaré Embeddings for Learning Hierarchical Representations

Open access
Nickel, Maximilian, Kiela, Douwe·2017·arXiv
DOI 10.48550/arxiv.1705.08039arXiv 1705.08039
Open this paper in CoChat — free accountOpen PDF

Abstract

Representation learning has become an invaluable approach for learning from symbolic data such as text and graphs. However, while complex symbolic datasets often exhibit a latent hierarchical structure, state-of-the-art methods typically learn embeddings in Euclidean vector spaces, which do not account for this property. For this purpose, we introduce a new approach for learning hierarchical representations of symbolic data by embedding them into hyperbolic space -- or more precisely into an n-dimensional Poincaré ball. Due to the underlying hyperbolic geometry, this allows us to learn parsimonious representations of symbolic data by simultaneously capturing hierarchy and similarity. We introduce an efficient algorithm to learn the embeddings based on Riemannian optimization and show experimentally that Poincaré embeddings outperform Euclidean embeddings significantly on data with latent hierarchies, both in terms of representation capacity and in terms of generalization ability.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper introduces Poincaré embeddings, which embed symbolic data into an n-dimensional Poincaré ball via Riemannian optimization to capture both hierarchy and similarity. Experiments on WordNet taxonomies, collaboration networks, and lexical entailment show that Poincaré embeddings outperform Euclidean embeddings, especially in low dimensions.

Key findings

  • On the transitive closure of the WordNet noun hierarchy (82,115 nouns and 743,241 hypernymy relations), Poincaré embeddings achieved mean rank 4.9 and MAP 0.823 at dimensionality 5 in reconstruction, and mean rank 5.7 and MAP 0.825 at dimensionality 5 in link prediction; Euclidean embeddings at dimensionality 5 had mean rank 3542.3 and MAP 0.024 in reconstruction, and mean rank 3311.1 and MAP 0.024 in link prediction.
  • On collaboration networks, Poincaré embeddings outperformed Euclidean embeddings especially in low dimensions: for ASTROPH at dimensionality 10, reconstruction MAP was 0.703 vs. 0.376 and link prediction MAP was 0.671 vs. 0.508; for CONDMAT at dimensionality 10, reconstruction MAP was 0.799 vs. 0.356 and link prediction MAP was 0.539 vs. 0.308; for GRQC at dimensionality 10, reconstruction MAP was 0.990 vs. 0.522 and link prediction MAP was 0.660 vs. 0.438; for HEPPH at dimensionality 10, reconstruction MAP was 0.811 vs. 0.434 and link prediction MAP was 0.683 vs. 0.642.
  • On the noun part of HYPERLEX (2,163 rated noun pairs), Poincaré embeddings achieved Spearman’s ρ = 0.512, compared with Euclidean ρ = 0.389 and the listed baselines FR 0.283, SLQS-Sim 0.229, WN-Basic 0.240, WN-WuP 0.214, WN-LCh 0.214, and Vis-ID 0.253. The same embeddings achieved accuracy 0.86 on WBLESS.
  • A two-dimensional Poincaré embedding trained on the WordNet mammals subtree achieved mean rank 1.26 and MAP 0.927.

Methods

  • The approach embeds symbols into the Poincaré ball model of hyperbolic space and optimizes a problem-specific loss via stochastic Riemannian optimization, using the Riemannian gradient, retraction \(R_\theta(v)=\theta+v\), projection to keep embeddings within the ball with \(\epsilon=10^{-5}\), random initialization from \(U(-0.001,0.001)\), and a burn-in phase with reduced learning rate \(\eta/c\), \(c=10\), for 10 epochs.
  • Data include the transitive closure of the WordNet noun hierarchy (82,115 nouns, 743,241 hypernymy relations); four collaboration networks ASTROPH (N=18,772; E=198,110), CONDMAT (N=23,133; E=93,497), GRQC (N=5,242; E=14,496), and HEPPH (N=12,008; E=118,521); HYPERLEX noun pairs (2,163); and WBLESS.
  • For taxonomy embedding, the loss is a soft ranking loss with 10 negative examples per positive example, evaluated by mean rank and MAP. For networks, edge probability is modeled via a Fermi-Dirac distribution with cross-entropy loss and negative sampling. For lexical entailment, a score function with penalty \(\alpha(\|v\|-\|u\|)\), \(\alpha=10^3\), is evaluated by Spearman’s rank correlation.

Limitations

  • The paper states that its main focus was to evaluate general properties of hyperbolic geometry for embedding symbolic data, and that future work includes expanding applications, for instance to multi-relational data, and deriving models tailored to specific applications such as word embeddings. It also states that natural gradient based optimization already produces good embeddings and scales to large datasets, but expects that a full Riemannian optimization approach can further increase quality and lead to faster convergence.
  • Evident from the text: the empirical evaluation is limited to taxonomy embedding, network link prediction, and lexical entailment; no multi-relational or broader symbolic-data experiments are reported.
  • Evident from the text: the translational score function has more information for asymmetric is-a relations and is “not fully unsupervised,” while the Poincaré and Euclidean distances are symmetric; the Poincaré distance captures hierarchy through norm but does not directly model ordered input pairs unless an additional asymmetric score is used.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.1705.08039

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

8 of 20 cited sources checked · 4 supported · 1 flagged · accuracy not yet measured

  • Findingcitation
    § 4.3 Lexical Entailment · p. 8

    The same embeddings achieved a state-of-the-art accuracy of 0.86 on WBLESS, which evaluates non-graded lexical entailment.

    “The same embeddings also achieved a state-of-the-art accuracy of 0 . 86 on WBLESS [33, 14], which evaluates non-graded lexical entailment.”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites [33] [33] Julie Weeds, Daoud Clarke, Jeremy Reffin, David Weir, and Bill Keller. Learning to distinguish hypernyms and co-hyponyms. In Proceedings of the 25th International Conference on Computational Linguistics COLING , pages 2249-2259. Dublin City University and Association for Computational Linguistics, 2014. -

      Section 3.3.1 describes a BLESS dataset for lexical entailment, and Table 3 reports accuracy on that dataset. Our copy does not report a WBLESS evaluation or a state-of-the-art accuracy of 0.86; the reported hyponym BLESS accuracies reach 0.75. Searches for “WBLESS,” “0.86,” and “86% accuracy” found no supporting passage. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [14] [14] Douwe Kiela, Laura Rimell, Ivan Vulic, and Stephen Clark. Exploiting image generality for lexical entailment detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL 2015) , pages 119-124. ACL, 2015. - source

      Table 2 (Results) reports WBLESS accuracy of 0.75 for image dispersion and 0.74 for centroid distance, not 0.86. The Results say dispersion matches the best-performing supervised method; the paper does support describing WBLESS as a non-graded entailment-detection evaluation. (LLM, conf=0.90)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
  • Premisecitation
    § Abstract · p. 1

    Many complex symbolic datasets exhibit a latent hierarchical structure.

    “complex symbolic datasets often exhibit a latent hierarchical structure”
    • Also stated in: § 1 Introduction · p. 1 · plain
      “large datasets whose objects can be organized according to a latent hierarchy - a property that is inherent in many complex datasets”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [25] [25] Erzsébet Ravasz and Albert-László Barabási. Hierarchical organization in complex networks. Physical Review E , 67(2):026112, 2003. - source

      The paper reports hierarchical organization in several complex networks, including a language network linking synonymous words (Section II, p. 3). This supports the possibility of latent hierarchy in symbolic data, but it does not establish that complex symbolic datasets in general often exhibit it. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [1] [1] Aaron B Adcock, Blair D Sullivan, and Michael W Mahoney. Tree-like structure in large social and information networks. In Data Mining (ICDM), 2013 IEEE 13th International Conference on , pages 1-10. IEEE, 2013. - source
  • Findingown data
    § 4.3 Lexical Entailment · p. 8

    The ranking based on Poincaré embeddings clearly outperforms all state-of-the-art methods evaluated in [32] on HYPERLEX.

    “It can be seen that the ranking based on Poincaré embeddings clearly outperforms all state-of-the-art methods evaluated in [32].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [32] [32] Ivan Vuli´ c, Daniela Gerz, Douwe Kiela, Felix Hill, and Anna Korhonen. Hyperlex: A large-scale evaluation of graded lexical entailment. arXiv preprint arXiv:1608.02117 , 2016. - source
  • Premisecitation
    § 4.2 Network Embeddings · p. 7

    Edges in complex networks can often be explained via latent hierarchies over their nodes.

    “Since edges in complex networks can often be explained via latent hierarchies over their nodes [8], we are interested in the benefits of Poincaré embeddings both in terms representation size and generalization performance.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [8] [8] Aaron Clauset, Cristopher Moore, and Mark EJ Newman. Hierarchical structure and the prediction of missing links in networks. Nature , 453(7191):98-101, 2008. - source
  • Findingown data
    § 4.1 Embedding Taxonomies · p. 6

    Poincaré embeddings are very successful in embedding large taxonomies, both with regard to representation capacity and generalization performance.

    “It can be seen that Poincaré embeddings are very successful in the embedding of large taxonomies - both with regard to their representation capacity and their generalization performance.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The transitive closure of the WORDNET noun hierarchy exhibits a clear latent hierarchical structure.
  • May combine two claims. 23
  • Classified as probably not a substantial claim. 12

  • May combine two claims.

    Premisereasoning
    § 3.1 Optimization · p. 4

    The Poincaré ball has a Riemannian manifold structure, so Equation (2) can be optimized via stochastic Riemannian optimization methods such as RSGD or RSVRG.

    “Since the Poincaré Ball has a Riemannian manifold structure, we can optimize Equation (2) via stochastic Riemannian optimization methods such as RSGD [5] or RSVRG [34].”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [5] [5] Silvere Bonnabel. Stochastic gradient descent on riemannian manifolds. IEEE Trans. Automat. Contr. , 58(9):2217-2229, 2013. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [34] [34] Hongyi Zhang, Sashank J. Reddi, and Suvrit Sra. Riemannian SVRG: fast stochastic optimization on riemannian manifolds. In Advances in Neural Information Processing Systems 29 , pages 4592-4600, 2016. - source
    • The authors state this as an assumption: We are given a problem-specific loss function L(Θ) that encourages semantically similar objects to be close in the embedding space according to their Poincaré distance.

  • May combine two claims.

    Interpretationreasoning
    § 3.1 Optimization · p. 5

    The algorithm is straightforward to parallelize via Hogwild because updates are sparse and collisions are very unlikely on large-scale data.

    “Moreover, the algorithm is straightforward to parallelize via methods such as Hogwild [26], as the updates are sparse (only a small number of embeddings are modified in an update) and collisions are very unlikely on large-scale data.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [26] [26] Benjamin Recht, Christopher Ré, Stephen J. Wright, and Feng Niu. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in Neural Information Processing Systems 24 , pages 693-701, 2011. - source
    • The authors state this as an assumption: We are given a problem-specific loss function L(Θ) that encourages semantically similar objects to be close in the embedding space according to their Poincaré distance.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction · p. 1

    The existence of power-law distributions in datasets can often be traced back to hierarchical structures.

    “the existence of power-law distributions in datasets can often be traced back to hierarchical structures [25]”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [25] [25] Erzsébet Ravasz and Albert-László Barabási. Hierarchical organization in complex networks. Physical Review E , 67(2):026112, 2003. - source

      The paper’s opening section (p. 1) says that the scale-free properties of several real networks are a consequence of hierarchical organization, and its hierarchical model has a power-law degree distribution (Section I, p. 2). The claim extends that finding from network degree distributions to power-law distributions in datasets generally, which the paper does not establish. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data
    § Abstract · p. 1

    Poincaré embeddings outperform Euclidean embeddings significantly on data with latent hierarchies, both in representation capacity and generalization ability.

    “show experimentally that Poincaré embeddings outperform Euclidean embeddings significantly on data with latent hierarchies, both in terms of representation capacity and in terms of generalization ability”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: There exists a latent hierarchy in which the symbols can be organized.

  • May combine two claims.

    Findingnone given
    § 5 Discussion and Future Work · p. 8

    Natural gradient based optimization already produces very good embeddings and scales to large datasets.

    “Furthermore, we have shown that natural gradient based optimization already produces very good embeddings and scales to large datasets.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § 3.2 Training Details · p. 5

    Initializing embeddings randomly from U(-0.001,0.001) causes embeddings to be initialized close to the origin of B^d.

    “First, we initialize all embeddings randomly from the uniform distribution U ( -0 . 001 , 0 . 001) . This causes embeddings to be initialized close to the origin of B d .”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 2

    Poincaré embeddings are successful in predicting links in collaboration networks and outperform Euclidean embeddings, especially in low dimensions.

    “On collaboration networks, we also show that Poincaré embeddings are successful in predicting links in graphs where they outperform Euclidean embeddings, especially in low dimensions.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: There exists a latent hierarchy in which the symbols can be organized.

  • May combine two claims.

    Findingown data
    § 4.2 Network Embeddings · p. 7

    Poincaré embeddings perform very well on the four social network datasets and, especially in the low-dimensional regime, outperform Euclidean embeddings.

    “It can be seen that Poincaré embeddings perform again very well on these datasets and - especially in the low-dimensional regime - outperform Euclidean embeddings.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The four social networks ASTROPH, CONDMAT, GRQC, and HEPPH have latent hierarchies over their nodes.

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 2

    Poincaré embeddings provide high quality embeddings of large taxonomies, both with and without missing data.

    “Experimentally, we show that our approach can provide high quality embeddings of large taxonomies - both with and without missing data.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: There exists a latent hierarchy in which the symbols can be organized.

  • May combine two claims.

    Premisereasoning
    § 3 Poincaré Embeddings · p. 4

    The Poincaré distance is symmetric and the hierarchical organization of the space is solely determined by the distance of points to the origin.

    “Furthermore, please note that Equation (1) is symmetric and that the hierarchical organization of the space is solely determined by the distance of points to the origin.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: There exists a latent hierarchy in which the symbols can be organized.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 1 Introduction · p. 2

    The Poincaré ball model is well-suited for gradient-based optimization.

    “We base our approach on a particular model of hyperbolic space, i.e., the Poincaré ball model, as it is well-suited for gradient-based optimization.”
    • Also stated in: § 3 Poincaré Embeddings · p. 3 · plain
      “we will base our approach on the Poincaré ball model, as it is well-suited for gradient-based optimization”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: There exists a latent hierarchy in which the symbols can be organized.

  • May combine two claims.

    Interpretationreasoning
    § 3 Poincaré Embeddings · p. 4

    The Poincaré distance changes smoothly with respect to the location of points, and this locality property is key for finding continuous embeddings of hierarchies.

    “It can be seen from Equation (1), that the distance within the Poincaré ball changes smoothly with respect to the location of u and v . This locality property of the Poincaré distance is key for finding continuous embeddings of hierarchies.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: There exists a latent hierarchy in which the symbols can be organized.

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 1

    Non-linear embeddings can mitigate the dimensionality problem, but complex graph patterns can still require computationally infeasible embedding dimensionality.

    “Although non-linear embeddings can mitigate this problem [7], complex graph patterns can still require a computationally infeasible embedding dimensionality.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7] [7] Guillaume Bouchard, Sameer Singh, and Theo Trouillon. On approximate reasoning capabilities of low-rank vector spaces. AAAI Spring Syposium on Knowledge Representation and Reasoning (KRR): Integrating Symbolic and Neural Approaches , 2015. -

  • May combine two claims.

    Premisereasoning
    § 2 Embeddings and Hyperbolic Geometry · p. 2

    Hyperbolic disc area and circle length grow exponentially with their radius.

    “both disc area and circle length grow exponentially with r”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 2

    Hyperbolic space can be thought of as a continuous version of trees and is naturally equipped to model hierarchical structures.

    “Informally, hyperbolic space can be thought of as a continuous version of trees and as such it is naturally equipped to model hierarchical structures.”
    • Also stated in: § 2 Embeddings and Hyperbolic Geometry · p. 2 · plain
      “Intuitively, hyperbolic spaces can be thought of as continuous versions of trees or vice versa, trees can be thought of as “discrete hyperbolic spaces” [16].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [16] [16] Dmitri Krioukov, Fragkiskos Papadopoulos, Maksim Kitsak, Amin Vahdat, and Marián Boguná. Hyperbolic geometry of complex networks. Physical Review E , 82(3):036106, 2010. - source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.1 Optimization · p. 5

    Using retraction Rθ(v)=θ+v in combination with the Riemannian gradient corresponds to the natural gradient method.

    “As retraction operation we use R θ ( v ) = θ + v . In combination with the Riemannian gradient, this corresponds then to the well-known natural gradient method [2].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [2] [2] Shun-ichi Amari. Natural gradient works efficiently in learning. Neural Computation , 10(2): 251-276, 1998. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § Abstract · p. 1

    Poincaré embeddings can learn parsimonious representations that simultaneously capture hierarchy and similarity.

    “Due to the underlying hyperbolic geometry, this allows us to learn parsimonious representations of symbolic data by simultaneously capturing hierarchy and similarity.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: There exists a latent hierarchy in which the symbols can be organized.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning
    § 2 Embeddings and Hyperbolic Geometry · p. 2

    Increasing the dimensionality of the embedding can lead to computational problems in runtime and memory complexity as well as to overfitting.

    “As the number of parameters increases, this can lead to computational problems in terms of runtime and memory complexity as well as to overfitting.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisereasoning
    § 3.1 Optimization · p. 4

    The Poincaré ball is a conformal model of hyperbolic space, so angles between adjacent vectors are identical to their angles in Euclidean space.

    “Since the Poincaré ball is a conformal model of hyperbolic space, the angles between adjacent vectors are identical to their angles in the Euclidean space.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § Abstract · p. 1

    State-of-the-art embedding methods typically learn embeddings in Euclidean vector spaces, which do not account for latent hierarchical structure.

    “state-of-the-art methods typically learn embeddings in Euclidean vector spaces, which do not account for this property”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given
    § 3 Poincaré Embeddings · p. 3

    The hyperboloid model is constrained to points where <x,x>=-1, while the distance function of the Beltrami-Klein model requires computing the location of ideal points on the boundary of the unit ball.

    “For instance, the hyperboloid model is constrained to points where 〈 x , x 〉 = -1 , while the distance function of the Beltrami-Klein model requires to compute the location of ideal points on the boundary ∂ B of the unit ball.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.3 Lexical Entailment · p. 7

    Poincaré embeddings allow making graded assertions about hierarchical relationships because hierarchies are represented in a continuous space.

    “An interesting aspect of Poincaré embeddings is that they allow us to make graded assertions about hierarchical relationships as hierarchies are represented in a continuous space.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisereasoning
    § 4 Evaluation · p. 5

    For the Poincaré distance and the Euclidean distance, randomly permuting the order of (u,v) yields the identical embedding, while this is not the case for the translational score function.

    “For the Poincaré distance and the Euclidean distance we could randomly permute the order of ( u, v ) and obtain the identical embedding, while this is not the case for the translational score function.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4 Evaluation · p. 5

    The translational score function was proposed by Bordes et al. for modeling large-scale graph-structured data.

    “For asymmetric data, we also include the score function d ( u , v ) = ‖ u -v + r ‖ 2 , as proposed by Bordes et al. [6] for modeling large-scale graph-structured data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] [6] Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26 , pages 2787-2795, 2013. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data
    § 5 Discussion and Future Work · p. 8

    Poincaré embeddings enable very parsimonious representations, allowing high-quality embeddings of large-scale taxonomies.

    “First, Poincaré embeddings enable very parsimonious representations whats allows us to learn high-quality embeddings of large-scale taxonomies.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The transitive closure of the WORDNET noun hierarchy exhibits a clear latent hierarchical structure.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given
    § 4.3 Lexical Entailment · p. 8

    Methods in Table 3 that are prefixed with WN also use WORDNET as a basis and therefore are most comparable.

    “Methods in Table 3 that are prefixed with WN also use WORDNET as a basis and therefore are most comparable.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.2 Network Embeddings · p. 7

    In the Fermi-Dirac model, r corresponds to the radius around each point u such that points within this radius are likely to have an edge with u.

    “Here, r corresponds to the radius around each point u such that points within this radius are likely to have an edge with u .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16] [16] Dmitri Krioukov, Fragkiskos Papadopoulos, Maksim Kitsak, Amin Vahdat, and Marián Boguná. Hyperbolic geometry of complex networks. Physical Review E , 82(3):036106, 2010. - source

  • May combine two claims.

    Interpretationreasoning
    § 4.1 Embedding Taxonomies · p. 7

    The robustness of Poincaré embeddings in link prediction is attributable to the structural bias of Poincaré embeddings, which could lead to reduced overfitting on data with a clear latent hierarchy.

    “We attribute this result to the structural bias of Poincaré embeddings, what could lead to reduced overfitting on this kind of data with a clear latent hierarchy.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The transitive closure of the WORDNET noun hierarchy exhibits a clear latent hierarchical structure.

  • May combine two claims.

    Interpretationreasoning
    § 4 Evaluation · p. 5

    The translational score function is not fully unsupervised and is only applicable where hierarchical information is available.

    “As such, it is not fully unsupervised and only applicable where this hierarchical information is available.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 4.2 Network Embeddings · p. 7

    In the Fermi-Dirac model, t specifies the steepness of the logistic function and influences both average clustering and the degree distribution.

    “The parameter t specifies the steepness of the logistic function and influences both average clustering as well as the degree distribution [16].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16] [16] Dmitri Krioukov, Fragkiskos Papadopoulos, Maksim Kitsak, Amin Vahdat, and Marián Boguná. Hyperbolic geometry of complex networks. Physical Review E , 82(3):036106, 2010. - source
30 of 30

  • The authors state this as an assumption.

    There exists a latent hierarchy in which the symbols can be organized.

    “We assume that there exists a latent hierarchy in which the symbols can be organized.”

  • The paper relies on this without support.

    In many datasets such as text corpora, multiple latent hierarchies can co-exist.

    “First, in many datasets such as text corpora, multiple latent hierarchies can co-exist, which can not always be modeled in two dimensions.”

  • The authors state this as an assumption.

    We are given a problem-specific loss function L(Θ) that encourages semantically similar objects to be close in the embedding space according to their Poincaré distance.

    “We assume we are given a problem-specific loss function L (Θ) which encourages semantically similar objects to be close in the embedding space according to their Poincaré distance.”

  • The authors state this as an assumption.

    The gradient of the loss function L is known.

    “the Euclidean gradient ∇ E = ∂ L ( θ ) ∂d ( θ , x ) ∂d ( θ , x ) ∂ θ depends on the gradient of L , which we assume is known”

  • The paper relies on this without support.

    The transitive closure of the WORDNET noun hierarchy exhibits a clear latent hierarchical structure.

    “In the first set of experiments, we are interested in evaluating the ability of Poincaré embeddings to embed data that exhibits a clear latent hierarchical structure. For this purpose, we conduct experiments on the transitive closure of the WORDNET noun hierarchy [18] in two settings:”

  • The paper relies on this without support.

    The four social networks ASTROPH, CONDMAT, GRQC, and HEPPH have latent hierarchies over their nodes.

    “We performed our experiments on four commonly used social networks, i.e, ASTROPH, CONDMAT, GRQC, and HEPPH. These networks represent scientific collaborations such that there exists an undirected edge between two persons if they co-authored a paper.”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The approach of Nickel & Kiela (2017) suffers from its loss function causing most points to collapse on the border of the Poincaré ball.

    “However, the approach of (Nickel & Kiela, 2017) suffers from a few drawbacks: first, their loss function causes most points to collapse on the border of the Poincar´ e ball, as exemplified in Figure 3.”

    From Hyperbolic Entailment Cones for Learning Hierarchical Embeddings (2018) · cites this paper as Nickel & Kiela, 2017

    Section 3 (p. 4) says leaf nodes can be placed close to the Poincaré ball’s boundary to represent hierarchy. It does not say that most points collapse there, or that the loss function causes such a collapse. Section 3.2 (p. 5) instead describes a training step intended to avoid moving too far toward the boundary. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Previous work (Nickel & Kiela, 2017) optimizing word embeddings in the Poincaré ball used the retraction map R_x(v):=x+v as a first order approximation of exp_x(v).

    “Previous work (Nickel &amp; Kiela, 2017) optimizing word embeddings in the Poincar´ e ball used the retraction map R x ( v ) := x + v as a first order approximation of exp x ( v ) .”

    From Hyperbolic Entailment Cones for Learning Hierarchical Embeddings (2018) · cites this paper as Nickel &amp; Kiela, 2017

    Section 3.1 (p. 5) explicitly says the authors use the retraction Rθ(v) = θ + v for optimization in the Poincaré ball. The paper does not describe it in terms of an exponential-map approximation, and its Discussion (p. 8) treats models tailored to word embeddings as future work rather than the application optimized here. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Hyperbolic embeddings can naturally capture hierarchical structures.

    “Hyperbolic embeddings can naturally capture hierarchical structures.”

    From Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones (2021) · cites this paper as [10]

  • Many real-world knowledge graphs contain hierarchical relations.

    “Many real-world knowledge graphs contain hierarchical relations [10, 11, 26].”

    From Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones (2021) · cites this paper as [10]

  • Poincaré embeddings represent entities in hyperbolic space, leveraging the inductive bias of negative curvature to fit hierarchies.

    “Poincaré embeddings (Nickel and Kiela, 2017) represent entities in hyperbolic space, leveraging the inductive bias of negative curvature to fit hierarchies.”

    From Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021) · cites this paper as Nickel and Kiela, 2017

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Euclidean disk area has an extra factor of two

    Minor

    The stated circle length, 2πr, implies an enclosed disk area of πr2, not 2πr2. The two formulas are inconsistent.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. [2][2] Shun-ichi Amari. Natural gradient works efficiently in learning. Neural Computation , 10(2): 251-276, 1998. -
  2. [10][10] Mikhael Gromov. Hyperbolic groups. In Essays in group theory , pages 75-263. Springer, 1987. -
  3. [35][35] George Kingsley Zipf. Human Behaviour and the Principle of Least Effort: an Introduction to Human Ecology . Addison-Wesley, 1949.

Cited by

Sorted by citations

Related

Authors
Nickel, M., & Kiela, D.
ORCID
Nickel, Maximilian
Venue
arXiv
Publisher
arXiv
Published
2017
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1705.08039
arXiv
1705.08039
OpenAlex
W2618798060
Semantic Scholar
1590bd1bca945fc6ff50b8cdf2da14ea2061c79a
Cited by
667
References
28
Citations by year
2026: 3 · 2025: 46 · 2024: 90 · 2023: 55 · 2022: 35 · 2021: 151 · 2020: 146 · 2019: 100 · 2018: 37 · 2017: 4
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • CERN Document Server (European Organization for Nuclear Research) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords
Poincaré embeddings, hyperbolic space, hierarchical representation, symbolic data, Riemannian optimization, hyperbolic geometry, embedding learning, generalization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-23.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-23.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review