RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

On FB15k-237, the key of link prediction is to model and infer the symmetry/antisymmetry and composition patterns.

“FB15k-237 (Toutanova & Chen, 2015) is a subset of FB15k, where inverse relations are deleted. Therefore, the key of link prediction on FB15k-237 boils down to model and infer the symmetry/antisymmetry and composition patterns.”

• cites Toutanova & Chen, 2015 sourcePartly supported

  Section 4.3 describes FB15KSelected as a subset of FB15K that retains one relation from each detected inverse or near-duplicate set, leaving 237 relations. Section 4.2 says the resulting task requires non-trivial inference after trivial cases involving symmetry or inverse relations are addressed. Neither section says that link prediction on this dataset boils down to modeling symmetry, antisymmetry, and composition patterns; composition and antisymmetry are not established as the key patterns. Page numbers are unavailable in our copy. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

On FB15k, the key of link prediction is to model and infer the symmetry/antisymmetry and inversion patterns.

“Toutanova & Chen (2015) showed that almost 81% of the test triplets ( x , r , y ) can be inferred via a directly linked triplet ( x , r ′ , y ) or ( y , r ′ , x ) . Therefore, the key of link prediction on FB15k is to model and infer the symmetry/antisymmetry and inversion patterns.”

• cites Toutanova & Chen (2015) sourcePartly supported

  Section 4.2 reports that almost 81% of FB15K test triples have an entity pair directly linked in the training graph, and connects easy inferences to relation symmetry or inverse relations. The paper does not identify antisymmetry as a pattern behind that figure, nor establish that modeling these patterns is the key to FB15K link prediction overall. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

RotatE outperforms all state-of-the-art models on FB15k and WN18.

“Table 4 summarizes our results on FB15k and WN18. We can see that RotatE outperforms all the state-of-the-art models.”

pRotatE and RotatE perform similarly on FB15k and WN18.

“The performance of pRotatE and RotatE are similar on these two datasets.”

The difference between RotatE and pRotatE is much larger on FB15k-237 and WN18RR than on FB15k and WN18.

“The difference between RotatE and pRotatE is much larger on FB15k-237 and WN18RR, where there are a lot of composition patterns.”

RotatE significantly outperforms existing state-of-the-art models for link prediction on multiple benchmark knowledge graphs.

“Experimental results on multiple benchmark knowledge graphs show that the proposed RotatE model is not only scalable, but also able to infer and model various relation patterns and significantly outperform existing state-of-the-art models for link prediction.”

• Also stated in: § 1 INTRODUCTION · p. 2 · boosted

  “Experimental results show that the RotatE model significantly outperforms existing state-of-the-art approaches.”

The improvement of RotatE over state-of-the-art models is much more significant on FB15k-237 and WN18RR.

“Table 5 summarizes our results on FB15k-237 and WN18RR, where the improvement is much more significant.”

RotatE outperforms state-of-the-art models on the Countries benchmark.

“In addition, RotatE also outperforms state-of-the-art models on Countries (Bouchard et al., 2015), a benchmark explicitly designed for composition pattern inference and modeling.”

• cites Bouchard et al., 2015Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

RotatE outperforms state-of-the-art models on YAGO3-10.

“Table 11 shows that the RotatE model also outperforms state-of-the-art models on YAGO3-10.”

DistMult achieves good performance on FB15K.

“Surprisingly, DistMult achieves good performance on this dataset although it cannot model the antisymmetry and inversion patterns.”

The re-implemented TransE outperforms all state-of-the-art models on FB15k-237.

“in addition, our re-implementation of TransE also outperforms all the state-of-the-art models on FB15k-237.”

Negative sampling loss is only effective on RotatE, not on TransE, on FB15k-237.

“while negative sampling loss is only effective on RotatE;”

DistMult performance significantly decreases on WN18 compared to FB15K.

“However, different from the results on FB15K, the performance of DistMult significantly decreases on WN18.”

The ComplEx model does not work well on FB15k-237.

“the ComplEx model does not work well on FB15k-237, which requires modeling the composition pattern.”

Besides one-to-one relations, RotatE performs quite well on non-injective relations, especially many-to-many relations, on FB15k.

“We can see that besides the one-to-one relation, the RotatE model also performs quite well on the non-injective relations, especially on many-to-many relations.”

On the Countries dataset, TransE slightly outperforms RotatE on the S3 task.

“The results on the Countries dataset are a little bit different, where the TransE model slightly outperforms RoateE on the S3 task.”

RotatE's performance is significantly better than other methods on the S3 task of the Countries dataset.

“The performance of RotatE is significantly better than other methods on S3, which is the most difficult task.”

RotatE outperforms all existing state-of-the-art models on four large-scale benchmarks.

“Our experimental results show that the RotatE model outperforms all existing state-of-theart models on four large-scale benchmarks.”

The TransE model does not work well on the FB15k datasets.

“The TransE model does not work well on the FB15k datasets, which requires modeling the symmetric pattern;”

RotatE achieves state-of-the-art results on a benchmark explicitly designed for composition pattern inference and modeling.

“Moreover, RotatE also achieves state-of-the-art results on a benchmark that is explicitly designed for composition pattern inference and modeling.”

For a symmetric relation in WN18, the embedding phases of a 500-dimensional RotatE are either π (r_i = -1) or 0, 2π (r_i = 1).

“We can find that the embedding phases are either π ( r i = -1 ) or 0 , 2 π ( r i = 1 ).”

For r1 = hypernym and its inverse r2 = hyponym in WN18, all element-wise additive embedding phases are 0 or 2π.

“All the additive embedding phases are 0 or 2 π , which represents that r 1 = r -1 2 .”

Self-adversarial sampling is the most effective negative sampling technique among uniform sampling, self-adversarial sampling, and KBGAN on FB15k-237, WN18RR, and WN18.

“We can see that self-adversarial sampling is the most effective negative sampling technique.”

RotatE outperforms all previous models on the Countries dataset according to the AUC-PR metric.

“We can see that RotatE outperforms all the previous models.”

For the relation hypernym in WN18, the embedding of a general relation does not have the ±1 pattern.

“Figure 2b is the histogram of relation hypernym , which shows that the embedding of a general relation does not have such a ± 1 pattern.”

RotatE is the first model that achieves state-of-the-art performance on all the benchmarks.

“To the best of our knowledge, RotatE is the first model that achieves state-of-the-art performance on all the benchmarks.”

RotatE can infer the symmetry/antisymmetry pattern.

“Lemma 1. RotatE can infer the symmetry/antisymmetry pattern. (See proof in Appendix B)”

RotatE can infer the composition pattern.

“Lemma 3. RotatE can infer the composition pattern. (See proof in Appendix D)”

No existing approaches are capable of modeling all three relation patterns.

“We can see that no existing approaches are capable of modeling all the three relation patterns.”

• Also stated in: § 1 INTRODUCTION · p. 2 · plain

  “However, none of existing models is capable of modeling and inferring all the above patterns.”

Modulus is very important for modeling and inferring the composition pattern.

“This indicates that modulus is very important for modeling and inferring the composition pattern.”

RotatE can infer the inversion pattern.

“Lemma 2. RotatE can infer the inversion pattern. (See proof in Appendix C)”

RotatE is the only model that can model and infer all three relation patterns.

“We can see that the RotatE model is the only model that can model and infer all the three types of relation patterns.”

RotatE leverages the advantages of both TransE and ComplEx.

“The proposed RotatE model leverages the advantages of both.”

Composition pattern requires the embedding phases of the composed relation to be the addition of the other two relations.

“Composition pattern requires the embedding phases of the composed relation to be the addition of the other two relations.”

The performance of the compared models on different datasets is consistent with the analysis on the three relation patterns.

“Moreover, the performance of these models on different datasets is consistent with our analysis on the three relation patterns (Table 2):”

TransE does not suffer from its inability to model symmetric relations on the Countries dataset.

“Therefore, the TransE model does not suffer from its inability of modeling symmetric relations.”

• AssumptionRests on an assumption: The Countries dataset does not have symmetric relations between different regions.

TransE cannot model the symmetry pattern because it would yield r = 0 for symmetric relations.

“TransE cannot model the symmetry pattern because it would yield r = 0 for symmetric relations;”

For most relations in FB15K, the types of head entities and tail entities are different.

“The reason is that for most of the relations in FB15K, the types of head entities and tail entities are different.”

DistMult's good performance on FB15K is because for most relations head and tail entity types are different, so the opposition triplet is usually invalid.

“Although DistMult gives the same score to a true triplet ( h , r , t ) and its opposition triplet ( t , r , h ) , ( t , r , h ) is usually impossible to be valid since the entity type of t does not match the head entity type of h .”

ComplEx does not perform well on the Countries dataset because it cannot infer the composition pattern.

“For ComplEx, we can see that it does not perform well on Countries since it cannot infer the composition pattern.”

TransX cannot infer inversion and composition because g_r,1 and g_r,2 are invertible matrix multiplications.

“but cannot infer inversion and composition as g r, 1 and g r, 2 are invertible matrix multiplications;”

RotatE is able to simulate TransE.

“which indicates that RotatE is able to simulate TransE.”

FB15k-237 requires modeling the composition pattern.

“which requires modeling the composition pattern.”

TorusE can be regarded as a special case of RotatE.

“The TorusE model can be regarded as a special case of RotatE, where the modulus of embeddings are set fixed; our RotatE is defined on the entire complex space, which has much more representation capacity.”

A useful property for RotatE is that the inverse of a relation can be easily acquired by complex conjugate.

“A useful property for RotatE is that the inverse of a relation can be easily acquired by complex conjugate.”

ComplEx cannot infer the composition pattern.

“The reason is that, as discussed before, TransE is able to infer the composition pattern while ComplEx cannot infer the composition pattern.”

DistMult cannot model antisymmetry and inversion patterns.

“Surprisingly, DistMult achieves good performance on this dataset although it cannot model the antisymmetry and inversion patterns.”

DistMult is difficult to model the asymmetric and inversion pattern due to its symmetric nature.

“due to its symmetric nature, DistMult is difficult to model the asymmetric and inversion pattern;”

The FB15k datasets require modeling the symmetric pattern.

“which requires modeling the symmetric pattern;”

ComplEx cannot infer composition rules, since it does not model a bijection mapping from h to t via relation r.

“However, ComplEx cannot infer composition rules, since it does not model a bijection mapping from h to t via relation r .”

TransE is able to infer the composition pattern.

“The reason is that, as discussed before, TransE is able to infer the composition pattern while ComplEx cannot infer the composition pattern.”

RotatE infers and models the inversion pattern.

“This case shows that the inversion pattern is also inferred and modeled in the RotatE model.”

When the modulus m_h,i = m_t,i = C, the RotatE distance function is reduced to 2C ||sin((θ_h + θ_r - θ_t)/2)||.

“When we constrain the modulus m h,i = m t,i = C , the distance function is reduced to 2 C ∥ ∥ sin θ h + θ r -θ t 2 ∥ ∥ .”

Uniform negative sampling is inefficient because many samples are obviously false as training goes on and provide no meaningful information.

“Such a uniform negative sampling suffers the problem of inefficiency since many samples are obviously false as training goes on, which does not provide any meaningful information.”

RotatE defined on the entire complex space has much more representation capacity than TorusE.

“The TorusE model can be regarded as a special case of RotatE, where the modulus of embeddings are set fixed; our RotatE is defined on the entire complex space, which has much more representation capacity.”

RotatE can degenerate into TransE.

“Theorem 4. RotatE can degenerate into TransE.”

Inversion pattern requires the embeddings of a pair of inverse relations to be conjugate.

“Inversion pattern requires the embeddings of a pair of inverse relations to be conjugate.”

TransE cannot infer the symmetry pattern.

“We can see that ComplEx performs well while TransE does not perform well since ComplEx can infer both symmetry/antisymmetry and inversion patterns while TransE cannot infer symmetry pattern.”

• Also stated in: § 2 RELATED WORK · p. 3 · plain

  “but it cannot model symmetric relations”

All three tasks in the Countries dataset only require inferring the region for a given city.

“all the three tasks in the Countries datasets only require inferring the region for a given city.”

ComplEx can infer inversion rules because the complex conjugate of the solution to arg max_r Re(〈x, r, y〉) is exactly the solution to arg max_r Re(〈y, r, x〉).

“Moreover, it can infer inversion rules because the complex conjugate of the solution to arg max r Re( 〈 x , r , y 〉 ) is exactly the solution to arg max r Re( 〈 y , r , x 〉 ) .”

The greater representation capacity of RotatE over TorusE is critical for modeling and inferring composition patterns.

“Our experiments show that this is very critical for modeling and inferring the composition patterns.”

No existing models are capable of modeling all three relation patterns.

“No existing models are capable of modeling all the three relation patterns.”