STransE: a novel embedding model of entities and relationships in knowledge bases
Open accessAbstract
Knowledge bases of real-world facts about entities and their relationships are useful resources for a variety of natural language processing tasks. However, because knowledge bases are typically incomplete, it is useful to be able to perform link prediction or knowledge base completion, i.e., predict whether a relationship not in the knowledge base is likely to be true. This paper combines insights from several previous link prediction models into a new embedding model STransE that represents each entity as a low-dimensional vector, and each relation by two matrices and a translation vector. STransE is a simple combination of the SE and TransE models, but it obtains better link prediction performance on two benchmark datasets than previous embedding models. Thus, STransE can serve as a new baseline for the more complex models in the link prediction task.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
STransE, ConvKB and CapsE use transfer learning and require embeddings pre-trained on TransE.
“We also note that STransE [42], ConvKB [41] and CapsE [43] use transfer learning and require embeddings pre-trained on TransE [8].”
From Knowledge Graph Embedding for Link Prediction (2021) · cites this paper as [42]
Section 4.1 states that the authors initialized STransE’s entity and relation vectors with vectors produced by TransE, supporting the pre-training part for their experiments. The model description presents STransE’s structure without making TransE-pretrained embeddings a requirement, and this paper does not establish the claim about ConvKB or CapsE. (LLM, conf=0.85)
Experimental — accuracy not yet measuredSTransE associates to each relation two additional matrices to pre-multiply head and tail, alleviating TransE issues on 1-to-many, many-to-one and many-to-many relations.
“STransE [42], in addition to the 𝑑 -sized embeddings seen in TransE, associates to each relation 𝑟 two additional 𝑑 × 𝑑 independent matrices 𝑾 𝒉 𝒓 and 𝑾 𝒕 𝒓 . When computing the score of a fact ⟨ ℎ, 𝑟, 𝑡 ⟩ , before operating the usual translation, 𝒉 is pre-multiplied by 𝑾 𝒉 𝒓 and 𝒕 by 𝑾 𝒕 𝒓 ; this amounts to use relation-specific embeddings for the head and tail, alleviating the issues suffered by”
From Knowledge Graph Embedding for Link Prediction (2021) · cites this paper as [42]
The paper does not include pre-training times in its measurements for STransE, ConvKB and CapsE.
“As already mentioned, STransE [42], ConvKB [41] and CapsE [43] require embeddings pre-trained on TransE [8]; we do not include pre-training times in our measurements.”
From Knowledge Graph Embedding for Link Prediction (2021) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 37
- Gabor Angeli and Christopher Manning. 2013. Philoso- phers are Mortal: Inferring the Truth of Unseen Facts. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 133-142.
- Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A Col- laboratively Created Graph Database for Structuring Human Knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pages 1247-1250.
- Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning Structured Em- beddings of Knowledge Bases. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelli- gence, pages 301-306.
- Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2012. A Semantic Matching Energy Function for Learning with Multi-relational Data. Ma- chine Learning, 94(2):233-259.
- Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems 26, pages 2787-2795.
- John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. The Journal of Machine Learning Research, 12:2121-2159.
- Christiane D. Fellbaum. 1998. WordNet: An Electronic Lexical Database. MIT Press.
- Alberto Garcia-Duran, Antoine Bordes, and Nicolas Usunier. 2015a. Composing Relationships with Translations. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Process- ing, pages 286-290.
- Alberto Garcia-Duran, Antoine Bordes, Nicolas Usunier, and Yves Grandvalet. 2015b. Combining Two And Three-Way Embeddings Models for Link Prediction in Knowledge Bases. CoRR, abs/1506.00999. · arXiv
- Matt Gardner and Tom Mitchell. 2015. Efficient and Ex- pressive Knowledge Base Completion Using Subgraph Feature Extraction. In Proceedings of the 2015 Con- ference on Empirical Methods in Natural Language Processing, pages 1488-1498.
- Kelvin Guu, John Miller, and Percy Liang. 2015. Traversing Knowledge Graphs in Vector Space. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 318- 327.
- Shizhu He, Kang Liu, Guoliang Ji, and Jun Zhao. 2015. Learning to Represent Knowledge Graphs with Gaus- sian Embedding. In Proceedings of the 24th ACM In- ternational on Conference on Information and Knowl- edge Management, pages 623-632.
- Rodolphe Jenatton, Nicolas L. Roux, Antoine Bordes, and Guillaume R Obozinski. 2012. A latent factor model for highly multi-relational data. In Advances in Neural Information Processing Systems 25, pages 3167-3175.
- Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge Graph Embedding via Dy- namic Mapping Matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Pa- pers), pages 687-696.
- Denis Krompa, Stephan Baier, and Volker Tresp. 2015. Type-Constrained Representation Learning in Knowl- edge Graphs. In Proceedings of the 14th International Semantic Web Conference, pages 640-655.
- Ni Lao, Tom Mitchell, and William W. Cohen. 2011. Random Walk Inference and Learning in a Large Scale Knowledge Base. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 529-539.
- Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N. Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, Sören Auer, and Christian Bizer. 2015. DBpedia -A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia. Semantic Web, 6(2):167-195.
- Yankai Lin, Zhiyuan Liu, Huanbo Luan, Maosong Sun, Siwei Rao, and Song Liu. 2015a. Modeling Rela- tion Paths for Representation Learning of Knowledge Bases. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 705-714.
- Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015b. Learning Entity and Relation Em- beddings for Knowledge Graph Completion. In Pro- ceedings of the Twenty-Ninth AAAI Conference on Ar- tificial Intelligence Learning, pages 2181-2187.
- D. C. Liu and J. Nocedal. 1989. On the Limited Memory BFGS Method for Large Scale Optimization. Mathe- matical Programming, 45(3):503-528.
- Yuanfei Luo, Quan Wang, Bin Wang, and Li Guo. 2015. Context-Dependent Knowledge Graph Embed- ding. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1656-1661.
- Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013. Linguistic Regularities in Continuous Space Word Representations. In Proceedings of the 2013 Conference of the North American Chapter of the As- sociation for Computational Linguistics: Human Lan- guage Technologies, pages 746-751.
- Arvind Neelakantan, Benjamin Roth, and Andrew Mc- Callum. 2015. Compositional Vector Space Models for Knowledge Base Completion. In Proceedings of the 53rd Annual Meeting of the Association for Com- putational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 156-166.
- Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A Three-Way Model for Collective Learning on Multi-Relational Data. In Proceedings of the 28th International Conference on Machine Learn- ing, pages 809-816.
- Maximilian Nickel, Kevin Murphy, Volker Tresp, and Ev- geniy Gabrilovich. 2015. A Review of Relational Ma- chine Learning for Knowledge Graphs. Proceedings of the IEEE, to appear.
- Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M. Marlin. 2013. Relation Extraction with Matrix Factorization and Universal Schemas. In Pro- ceedings of the 2013 Conference of the North Ameri- can Chapter of the Association for Computational Lin- guistics: Human Language Technologies, pages 74- 84.
- Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. 2013. Reasoning With Neural Ten- sor Networks for Knowledge Base Completion. In Ad- vances in Neural Information Processing Systems 26, pages 926-934.
- Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. YAGO: A Core of Semantic Knowl- edge. In Proceedings of the 16th International Con- ference on World Wide Web, pages 697-706.
- Ben Taskar, Ming fai Wong, Pieter Abbeel, and Daphne Koller. 2004. Link Prediction in Relational Data. In Advances in Neural Information Processing Systems 16, pages 659-666.
- Kristina Toutanova and Danqi Chen. 2015. Observed Versus Latent Features for Knowledge Base and Text Inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Composi- tionality, pages 57-66.
- Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. 2015. Representing Text for Joint Embedding of Text and Knowledge Bases. In Proceedings of the 2015 Con- ference on Empirical Methods in Natural Language Processing, pages 1499-1509.
- Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014a. Knowledge Graph and Text Jointly Em- bedding. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1591-1601.
- Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014b. Knowledge Graph Embedding by Translating on Hyperplanes. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelli- gence, pages 1112-1119.
- Robert West, Evgeniy Gabrilovich, Kevin Murphy, Shao- hua Sun, Rahul Gupta, and Dekang Lin. 2014. Knowledge Base Completion via Search-based Ques- tion Answering. In Proceedings of the 23rd Interna- tional Conference on World Wide Web, pages 515-526.
- Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In Proceedings of the International Conference on Learn- ing Representations.
- Matthew D. Zeiler. 2012. ADADELTA: An Adaptive Learning Rate Method. CoRR, abs/1212.5701. · arXiv
- Yu Zhao, Sheng Gao, Patrick Gallinari, and Jun Guo. 2015. Knowledge Base Completion by Learn- ing Pairwise-Interaction Differentiated Embeddings. Data Mining and Knowledge Discovery, 29(5):1486- 1504.
Cited by
Sorted by citations
RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
Sun, Z., Deng, Z.-H., Nie, J.-Y., & Tang, J. (2019) · arXiv · doi:10.48550/arxiv.1902.10197 · Cited by 1,414 · see citing works
Knowledge Graph Embedding for Link Prediction
Rossi, A., Barbosa, D., Firmani, D., Matinata, A., & Merialdo, P. (2021) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3424672 · Cited by 601 · see citing works
- Authors
- Nguyen, D. Q., Sirts, K., Qu, L., & Johnson, M.
- Affiliations
- Macquarie University (AU)
- Data61 (AU)
- ORCID
- Johnson, Mark
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- Pages
- 460-466
- arXiv
- 1606.08140
- OpenAlex
- W2463781041
- Cited by
- 181
- References
- 0
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 9 · 2023: 13 · 2022: 19 · 2021: 24 · 2020: 28 · 2019: 37 · 2018: 23 · 2017: 16 · 2016: 4
- FWCI
- 14.10
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Australian Government
- Australian Research Council (DP160102156)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
- Keywords
- knowledge base embedding, entity embeddings, knowledge graphs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
