The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification
Open accessAbstract
The existing methods for relation classification (RC) primarily rely on distant supervision (DS) because large-scale supervised training datasets are not readily available. Although DS automatically annotates adequate amounts of data for model training, the coverage of this data is still quite limited, and meanwhile many long-tail relations still suffer from data sparsity. Intuitively, people can grasp new knowledge by learning few instances. We thus provide a different view on RC by formalizing RC as a few-shot learning (FSL) problem. However, the current FSL models mainly focus on low-noise vision tasks, which makes them hard to directly deal with the diversity and noise of text. In this paper, we propose hybrid attention-based prototypical networks for the problem of noisy few-shot RC. We design instancelevel and feature-level attention schemes based on prototypical networks to highlight the crucial instances and features respectively, which significantly enhances the performance and robustness of RC models in a noisy FSL scenario. Besides, our attention schemes accelerate the convergence speed of RC models. Experimental results demonstrate that our hybrid attention-based models require fewer training iterations and outperform the state-of-the-art baseline models. The code and datasets are released on https://github.com/thunlp/ HATT-Proto.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Prototypical Networks for Few-shot Learning
Snell, J., Swersky, K., & Zemel, R. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.05175 · Cited by 4,755
Optimization as a Model for Few-Shot Learning
Ravi, S., & Larochelle, H. (2017) · International Conference on Learning Representations · Cited by 4,067
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · cited 1×
“Many works apply transfer learning methods to finetune pre-trained models for FSL, which transfer latent information from the common classes containing adequate instances to the uncommon classes with only few instances (Caruana 1995; Bengio 2012; Donahue et al. 2014).”
MethodSiamese Neural Networks for One-shot Image Recognition
Koch, G. E., Zemel, R. S., & Salakhutdinov, R. (2015) · Cited by 3,740
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
Distant supervision for relation extraction without labeled data
Mintz, M., Bills, S., Snow, R., & Jurafsky, D. (2009) · Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - ACL-IJCNLP '09 · doi:10.3115/1690219.1690287 · Cited by 2,752
Deep Learning of Representations for Unsupervised and Transfer Learning.
Bengio, Y. (2011) · Cited by 1,702
Meta-learning with memory-augmented neural networks
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016) · International Conference on Machine Learning · Cited by 1,504
Relation Classification via Convolutional Deep Neural Network
Zeng, D., Liu, K., Lai, S., Zhou, G., & Zhao, J. (2015) · Cited by 1,467
Modeling Relations and Their Mentions without Labeled Text
Riedel, S., Yao, L., & McCallum, A. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15939-8_10 · Cited by 1,270
Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks
Zeng, D., Liu, K., Chen, Y., & Zhao, J. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1203 · Cited by 1,149
Neural Relation Extraction with Selective Attention over Instances
Lin, Y., Shen, S., Liu, Z., Luan, H., & Sun, M. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1200 · Cited by 1,021
Self-citationKnowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., & Weld, D. S. (2011) · Cited by 874
A Simple Neural Attentive Meta-Learner
Mishra, N., Rohaninejad, M., Chen, X., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.03141 · Cited by 802
Exploring various knowledge in relation extraction
GuoDong, Z., Jian, S., Jie, Z., & Min, Z. (2005) · Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL '05 · doi:10.3115/1219840.1219893 · Cited by 764
Multi-instance Multi-label Learning for Relation Extraction
Surdeanu, M., Tibshirani, J., Nallapati, R. M., & Manning, C. D. (2012) · Cited by 643
Han, X., Zhu, H., Yu, P., Wang, Z., Yao, Y., Liu, Z., & Sun, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1514 · Cited by 613
Self-citationKernel methods for relation extraction
Zelenko, D., Aone, C., & Richardella, A. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118703 · Cited by 573
Classifying Relations by Ranking with Convolutional Neural Networks
dos Santos, C., Xiang, B., & Zhou, B. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1061 · Cited by 498
Zero-Shot Relation Extraction via Reading Comprehension
Levy, O., Seo, M., Choi, E., & Zettlemoyer, L. (2017) · arXiv · doi:10.48550/arxiv.1706.04115 · Cited by 482 · cited 1×
“Note that there are also some works attempting to handle RC in a zero-shot scenario (Levy et al. 2017), incorporating extra information to classify relations never appearing in training sets.”
BackgroundZero-Shot Relation Extraction via Reading Comprehension
Levy, O., Seo, M., Choi, E., & Zettlemoyer, L. (2017) · Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017) · doi:10.18653/v1/k17-1034 · Cited by 482 · cited 1×
“Note that there are also some works attempting to handle RC in a zero-shot scenario (Levy et al. 2017), incorporating extra information to classify relations never appearing in training sets.”
BackgroundDistant Supervision for Relation Extraction with Sentence-Level Attention and Entity Descriptions
Ji, G., Liu, K., He, S., & Zhao, J. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10953 · Cited by 357
Reinforcement Learning for Relation Classification From Noisy Data
Feng, J., Huang, M., Zhao, L., Yang, Y., & Zhu, X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12063 · Cited by 353
Learning Many Related Tasks at the Same Time with Backpropagation
Caruana, R. (1994) · Cited by 242
Diverse Few-Shot Text Classification with Multiple Metrics
Yu, M., Guo, X., Yi, J., Chang, S., Potdar, S., Cheng, Y., Tesauro, G., Wang, H., & Zhou, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1109 · Cited by 225
Adversarial Training for Relation Extraction
Wu, Y., Bamman, D., & Russell, S. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1187 · Cited by 224
Improved Relation Extraction with Feature-Rich Compositional Embedding Models
Gormley, M. R., Yu, M., & Dredze, M. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1205 · Cited by 191
Neural Knowledge Acquisition via Mutual Attention Between Knowledge Graph and Text
Han, X., Liu, Z., & Sun, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11927 · Cited by 161
Self-citationDeep Residual Learning for Weakly-Supervised Relation Extraction
Huang, Y. Y., & Wang, W. Y. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1191 · Cited by 118
Multilingual Relation Extraction using Compositional Universal Schema
Verga, P. W., Belanger, D., Strubell, E., Roth, B., & McCallum, A. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1103 · Cited by 112
Large Scaled Relation Extraction With Reinforcement Learning
Zeng, X., He, S., Liu, K., & Zhao, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11950 · Cited by 99
Adversarial Multi-lingual Neural Relation Extraction
Wang, X., Han, X., Lin, Y., Liu, Z., & Sun, M. (2018) · International Conference on Computational Linguistics · Cited by 34
Self-citationVerga, P. W., & McCallum, A. (2016) · Automated Knowledge Base Construction · doi:10.18653/v1/w16-1312 · Cited by 25
Diverse Few-Shot Text Classification with Multiple MetricsPreprint
Yu, M., Guo, X., Yi, J., Chang, S., Potdar, S., Cheng, Y., Tesauro, G. J., Wang, H., & Zhou, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07513 · Cited by 23
Cited by
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications
Ji, S., Pan, S., Cambria, E., Marttinen, P., & Yu, P. S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3070843 · Cited by 2,947
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities
Song, Y., Wang, T., Cai, P., Mondal, S. K., & Sahoo, J. P. (2023) · ACM Computing Surveys · doi:10.1145/3582688 · Cited by 721
“HATT-Proto [184] AAAI’19 Episodic Training Euclidean Distance Designs hybrid networks of attention for instance and prototype features”
BackgroundA Survey on Text Classification: From Traditional to Deep Learning
Li, Q., Peng, H., Li, J., Xia, C., Yang, R., Sun, L., Yu, P. S., & He, L. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3495162 · Cited by 494
Finding Task-Relevant Features for Few-Shot Learning by Category Traversal
Li, H., Eigen, D., Dodge, S., Zeiler, M., & Wang, X. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00009 · Cited by 400
KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction
Chen, X., Zhang, N., Xie, X., Deng, S., Yao, Y., Tan, C., Huang, F., Si, L., & Chen, H. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3511998 · Cited by 389
Metadata
- Authors
- Gao, T., Han, X., Liu, Z., & Sun, M.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2019-07-17
- Volume
- 33
- Issue
- 01
- Pages
- 6407-6414
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2905471643
- Cited by
- 379
- References
- 36
- FWCI
- 19.63
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.