CoChat
Sign Up
Cited by 731Topic ModelingNatural Language Processing TechniquesAdvanced Graph Neural Networks

Matching the Blanks: Distributional Similarity for Relation Learning

Open access
Baldini Soares, Livio, FitzGerald, Nicholas, Ling, Jeffrey, Kwiatkowski, Tom·2019·Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
DOI 10.18653/v1/p19-1279
Open this paper in CoChat — free accountOpen PDF

Abstract

General purpose relation extractors, which can model arbitrary relations, are a core aspiration in information extraction.Efforts have been made to build general purpose extractors that represent relations with their surface forms, or which jointly embed surface forms with relations from an existing knowledge graph.However, both of these approaches are limited in their ability to generalize.In this paper, we build on extensions of Harris' distributional hypothesis to relations, as well as recent advances in learning text representations (specifically, BERT), to build task agnostic relation representations solely from entity-linked text.We show that these representations significantly outperform previous work on exemplar based relation extraction (FewRel) even without using any of that task's training data.We also show that models initialized with our task agnostic representations, and then tuned on supervised relation extraction datasets, significantly outperform the previous methods on Se-mEval 2010 Task 8, KBP37, and TACRED.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/p19-1279

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A previous study [53] examined various input encoding and output representation of using BERT for relation extraction, and concluded that using representations aggregated from the start position entity markers was the best practice.

    “A previous study [53] examined various input encoding and output representation of using BERT for relation extraction, and concluded that using representations aggregated from the start position entity markers (eg, [S1] and [S2] in Figure 3C) was the best practice.”

    From Extracting Family History of Patients From Clinical Narratives: Exploring an End-to-End Solution With Deep Learning Models (2020) · cites this paper as [53]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 29

  1. Gabor Angeli, Melvin Jose Johnson Premkumar, and Christopher D Manning. 2015. Leveraging linguis- tic structure for open domain information extraction. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Lan- guage Processing, volume 1, pages 344-354.
  2. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hin- ton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450. · arXiv
  3. Michele Banko, Michael J. Cafarella, Stephen Soder- land, Matt Broadhead, and Oren Etzioni. 2007. Open information extraction from the web. In Pro- ceedings of the 20th International Joint Conference on Artifical Intelligence, IJCAI'07, pages 2670- 2676, San Francisco, CA, USA.
  4. Ivan Bilan and Benjamin Roth. 2018. Position-aware self-attention with relative positional encodings for slot filling. CoRR, abs/1807.03052. · arXiv
  5. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A col- laboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Man- agement of Data, SIGMOD '08, pages 1247-1250, New York, NY, USA. ACM. · DOI · Paper page
  6. Antoine Bordes, Nicolas Usunier, Alberto Garcia- Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi- relational data. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 2787-2795.
  7. Claire Cardie. 1997. Empirical methods in information extraction. AI Magazine, 18(4):65-80.
  8. Anthony Fader, Stephen Soderland, and Oren Etzioni. 2011. Identifying relations for open information ex- traction. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Process- ing, pages 1535-1545, Edinburgh, Scotland, UK. Association for Computational Linguistics.
  9. Zhou GuoDong, Su Jian, Zhang Jie, and Zhang Min. 2005. Exploring various knowledge in relation ex- traction. In Proceedings of the 43rd annual meeting on association for computational linguistics, pages 427-434. Association for Computational Linguis- tics.
  10. Michael U Gutmann and Aapo Hyvärinen. 2012. Noise-contrastive estimation of unnormalized sta- tistical models, with applications to natural image statistics. Journal of Machine Learning Research, 13(Feb):307-361.
  11. Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2018. Fewrel: A large-scale supervised few-shot relation classifica- tion dataset with state-of-the-art evaluation. In Pro- ceedings of the 2018 Conference on Empirical Meth- ods in Natural Language Processing, pages 4803- 4809.
  12. Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2009. Semeval-2010 task 8: Multi-way classification of semantic relations be- tween pairs of nominals. In Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions, pages 94-99. Association for Computational Linguistics.
  13. Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S Weld. 2011. Knowledge- based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computa- tional Linguistics: Human Language Technologies- Volume 1, pages 541-550. Association for Compu- tational Linguistics.
  14. Nanda Kambhatla. 2004. Combining lexical, syntactic, and semantic features with maximum entropy mod- els for extracting relations. In Proceedings of the ACL 2004 on Interactive poster and demonstration sessions, page 22. Association for Computational Linguistics.
  15. Dekang Lin and Patrick Pantel. 2001. DIRT: Discovery of Inference Rules from Text. In Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'01), pages 323-328, New York, NY, USA. ACM Press. · DOI
  16. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119.
  17. Mike Mintz, Steven Bills, Rion Snow, and Daniel Ju- rafsky. 2009. Distant supervision for relation ex- traction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 1003-1011, Suntec, Singapore. Association for Computational Linguistics.
  18. Andriy Mnih and Koray Kavukcuoglu. 2013. Learning word embeddings efficiently with noise-contrastive estimation. In Advances in neural information pro- cessing systems, pages 2265-2273.
  19. Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. 2016. Holographic embeddings of knowl- edge graphs. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI'16, pages 1955-1961. AAAI Press.
  20. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word repre- sentations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), volume 1, pages 2227-2237.
  21. Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M Marlin. 2013. Relation extraction with matrix factorization and universal schemas. In Pro- ceedings of the 2013 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 74-84.
  22. Gabriel Stanovsky, Julian Michael, Luke Zettlemoyer, and Ido Dagan. 2018. Supervised open information extraction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 885- 895, New Orleans, Louisiana. Association for Com- putational Linguistics.
  23. Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoi- fung Poon, Pallavi Choudhury, and Michael Gamon. 2015. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1499-1509, Lisbon, Portu- gal. Association for Computational Linguistics.
  24. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems, pages 5998-6008.
  25. Patrick Verga and Andrew McCallum. 2016. Row-less universal schema. In Proceedings of the 5th Work- shop on Automated Knowledge Base Construction, pages 63-68, San Diego, CA. Association for Com- putational Linguistics.
  26. Linlin Wang, Zhu Cao, Gerard de Melo, and Zhiyuan Liu. 2016. Relation classification via multi-level at- tention cnns. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin- guistics, pages 1298-1307. Association for Compu- tational Linguistics. · DOI
  27. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via con- volutional deep neural network. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 2335-2344. Dublin City University and As- sociation for Computational Linguistics.
  28. Dongxu Zhang and Dong Wang. 2015. Relation clas- sification via recurrent neural network. CoRR, abs/1508.01006. · arXiv
  29. Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor An- geli, and Christopher D Manning. 2017. Position- aware attention and supervised data improve slot fill- ing. In Proceedings of the 2017 Conference on Em- pirical Methods in Natural Language Processing, pages 35-45.

Cited by

Sorted by citations

Authors
Baldini Soares, L., FitzGerald, N., Ling, J., & Kwiatkowski, T.
Affiliations
Google (United States) (US)
Venue
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Publisher
Association for Computational Linguistics
Published
2019
Pages
2895-2905
ISSN
0736-587X (ISSN-L)
DOI
10.18653/v1/p19-1279
OpenAlex
W2948131150
Cited by
731
References
3
Citations by year
2026: 29 · 2025: 58 · 2024: 96 · 2023: 137 · 2022: 133 · 2021: 170 · 2020: 98 · 2019: 10
FWCI
39.31
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
Keywords
relation extraction, distributional similarity, distributional hypothesis, BERT, few-shot relation extraction, knowledge graphs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review