PreprintCited by 668Advanced Graph Neural NetworksTopic ModelingNeural Networks and Applications

The DOI matches its DataCite record; 33 of the 33 references it cites were found in a registry or an index; 8 of the 20 sources its claims cite were checked for support.

Poincaré Embeddings for Learning Hierarchical Representations

Open access

Abstract

Representation learning has become an invaluable approach for learning from symbolic data such as text and graphs. However, while complex symbolic datasets often exhibit a latent hierarchical structure, state-of-the-art methods typically learn embeddings in Euclidean vector spaces, which do not account for this property. For this purpose, we introduce a new approach for learning hierarchical representations of symbolic data by embedding them into hyperbolic space -- or more precisely into an n-dimensional Poincaré ball. Due to the underlying hyperbolic geometry, this allows us to learn parsimonious representations of symbolic data by simultaneously capturing hierarchy and similarity. We introduce an efficient algorithm to learn the embeddings based on Riemannian optimization and show experimentally that Poincaré embeddings outperform Euclidean embeddings significantly on data with latent hierarchies, both in terms of representation capacity and in terms of generalization ability.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

The paper introduces Poincaré embeddings, which embed symbolic data into an n-dimensional Poincaré ball via Riemannian optimization to capture both hierarchy and similarity. Experiments on WordNet taxonomies, collaboration networks, and lexical entailment show that Poincaré embeddings outperform Euclidean embeddings, especially in low dimensions.

Key findings

  • On the transitive closure of the WordNet noun hierarchy (82,115 nouns and 743,241 hypernymy relations), Poincaré embeddings achieved mean rank 4.9 and MAP 0.823 at dimensionality 5 in reconstruction, and mean rank 5.7 and MAP 0.825 at dimensionality 5 in link prediction; Euclidean embeddings at dimensionality 5 had mean rank 3542.3 and MAP 0.024 in reconstruction, and mean rank 3311.1 and MAP 0.024 in link prediction.
  • On collaboration networks, Poincaré embeddings outperformed Euclidean embeddings especially in low dimensions: for ASTROPH at dimensionality 10, reconstruction MAP was 0.703 vs. 0.376 and link prediction MAP was 0.671 vs. 0.508; for CONDMAT at dimensionality 10, reconstruction MAP was 0.799 vs. 0.356 and link prediction MAP was 0.539 vs. 0.308; for GRQC at dimensionality 10, reconstruction MAP was 0.990 vs. 0.522 and link prediction MAP was 0.660 vs. 0.438; for HEPPH at dimensionality 10, reconstruction MAP was 0.811 vs. 0.434 and link prediction MAP was 0.683 vs. 0.642.
  • On the noun part of HYPERLEX (2,163 rated noun pairs), Poincaré embeddings achieved Spearman’s ρ = 0.512, compared with Euclidean ρ = 0.389 and the listed baselines FR 0.283, SLQS-Sim 0.229, WN-Basic 0.240, WN-WuP 0.214, WN-LCh 0.214, and Vis-ID 0.253. The same embeddings achieved accuracy 0.86 on WBLESS.
  • A two-dimensional Poincaré embedding trained on the WordNet mammals subtree achieved mean rank 1.26 and MAP 0.927.

Methods

  • The approach embeds symbols into the Poincaré ball model of hyperbolic space and optimizes a problem-specific loss via stochastic Riemannian optimization, using the Riemannian gradient, retraction \(R_\theta(v)=\theta+v\), projection to keep embeddings within the ball with \(\epsilon=10^{-5}\), random initialization from \(U(-0.001,0.001)\), and a burn-in phase with reduced learning rate \(\eta/c\), \(c=10\), for 10 epochs.
  • Data include the transitive closure of the WordNet noun hierarchy (82,115 nouns, 743,241 hypernymy relations); four collaboration networks ASTROPH (N=18,772; E=198,110), CONDMAT (N=23,133; E=93,497), GRQC (N=5,242; E=14,496), and HEPPH (N=12,008; E=118,521); HYPERLEX noun pairs (2,163); and WBLESS.
  • For taxonomy embedding, the loss is a soft ranking loss with 10 negative examples per positive example, evaluated by mean rank and MAP. For networks, edge probability is modeled via a Fermi-Dirac distribution with cross-entropy loss and negative sampling. For lexical entailment, a score function with penalty \(\alpha(\|v\|-\|u\|)\), \(\alpha=10^3\), is evaluated by Spearman’s rank correlation.

Limitations

  • The paper states that its main focus was to evaluate general properties of hyperbolic geometry for embedding symbolic data, and that future work includes expanding applications, for instance to multi-relational data, and deriving models tailored to specific applications such as word embeddings. It also states that natural gradient based optimization already produces good embeddings and scales to large datasets, but expects that a full Riemannian optimization approach can further increase quality and lead to faster convergence.
  • Evident from the text: the empirical evaluation is limited to taxonomy embedding, network link prediction, and lexical entailment; no multi-relational or broader symbolic-data experiments are reported.
  • Evident from the text: the translational score function has more information for asymmetric is-a relations and is “not fully unsupervised,” while the Poincaré and Euclidean distances are symmetric; the Poincaré distance captures hierarchy through norm but does not directly model ordered input pairs unless an additional asymmetric score is used.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.1705.08039

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

8 of 20 cited sources checked · 4 supported · 1 flagged · accuracy not yet measured

Show all 30 claims

  • It may state two separate claims as one. 23
  • Probably not a substantial claim. 12
30 of 30
  • The authors state this as an assumption.

    There exists a latent hierarchy in which the symbols can be organized.

    “We assume that there exists a latent hierarchy in which the symbols can be organized.”
  • The paper relies on this without support.

    In many datasets such as text corpora, multiple latent hierarchies can co-exist.

    “First, in many datasets such as text corpora, multiple latent hierarchies can co-exist, which can not always be modeled in two dimensions.”
  • The authors state this as an assumption.

    We are given a problem-specific loss function L(Θ) that encourages semantically similar objects to be close in the embedding space according to their Poincaré distance.

    “We assume we are given a problem-specific loss function L (Θ) which encourages semantically similar objects to be close in the embedding space according to their Poincaré distance.”
  • The authors state this as an assumption.

    The gradient of the loss function L is known.

    “the Euclidean gradient ∇ E = ∂ L ( θ ) ∂d ( θ , x ) ∂d ( θ , x ) ∂ θ depends on the gradient of L , which we assume is known”
  • The paper relies on this without support.

    The transitive closure of the WORDNET noun hierarchy exhibits a clear latent hierarchical structure.

    “In the first set of experiments, we are interested in evaluating the ability of Poincaré embeddings to embed data that exhibits a clear latent hierarchical structure. For this purpose, we conduct experiments on the transitive closure of the WORDNET noun hierarchy [18] in two settings:”
  • The paper relies on this without support.

    The four social networks ASTROPH, CONDMAT, GRQC, and HEPPH have latent hierarchies over their nodes.

    “We performed our experiments on four commonly used social networks, i.e, ASTROPH, CONDMAT, GRQC, and HEPPH. These networks represent scientific collaborations such that there exists an undirected edge between two persons if they co-authored a paper.”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Euclidean disk area has an extra factor of two

    Minor

    The stated circle length, 2πr, implies an enclosed disk area of πr2, not 2πr2. The two formulas are inconsistent.

1 of 1

Citations

Cites

Flagged first, then by citations

Show all 28 references

Research works printed but not linked: 3

  1. [2][2] Shun-ichi Amari. Natural gradient works efficiently in learning. Neural Computation , 10(2): 251-276, 1998. -
  2. [10][10] Mikhael Gromov. Hyperbolic groups. In Essays in group theory , pages 75-263. Springer, 1987. -
  3. [35][35] George Kingsley Zipf. Human Behaviour and the Principle of Least Effort: an Introduction to Human Ecology . Addison-Wesley, 1949.

Cited by

Sorted by citations

Items 101–200 of 668

Metadata

Authors
Nickel, M., & Kiela, D.
Venue
arXiv
Publisher
arXiv
Published
2017
ISSN
2331-8422 (ISSN-L)
OpenAlex
W2618798060
Cited by
667
References
28
Citations by year
2026: 3 · 2025: 46 · 2024: 90 · 2023: 55 · 2022: 35 · 2021: 151 · 2020: 146 · 2019: 100 · 2018: 37 · 2017: 4
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion)
  • CERN Document Server (European Organization for Nuclear Research) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords
Poincaré embeddings, hyperbolic space, hierarchical representation, symbolic data, Riemannian optimization, hyperbolic geometry, embedding learning, generalization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-23, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-23 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review