The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation
Open accessAbstract
We present a Few-Shot Relation Classification Dataset (FewRel), consisting of 70, 000 sentences on 100 relations derived from Wikipedia and annotated by crowdworkers.The relation of each sentence is first recognized by distant supervision methods, and then filtered by crowdworkers.We adapt the most recent state-of-the-art few-shot learning methods for relation classification and conduct thorough evaluation of these methods.Empirical results show that even the most competitive few-shot learning models struggle on this task, especially as compared with humans.We also show that a range of different reasoning skills are needed to solve our task.These results indicate that few-shot relation classification remains an open problem and still requires further research.Our detailed analysis points multiple directions for future research.All details and resources about the dataset and baselines are released on http://zhuhao.me/ fewrel.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Prototypical Networks for Few-shot Learning
Snell, J., Swersky, K., & Zemel, R. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.05175 · Cited by 4,755
Optimization as a Model for Few-Shot Learning
Ravi, S., & Larochelle, H. (2017) · International Conference on Learning Representations · Cited by 4,067
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · cited 2×
“The early works (Caruana, 1995; Bengio, 2012; Donahue et al., 2014) apply transfer learning methods to finetune pre-trained models from the common classes containing adequate instances to the uncommon classes with only few instances.”
BackgroundSiamese Neural Networks for One-shot Image Recognition
Koch, G. E., Zemel, R. S., & Salakhutdinov, R. (2015) · Cited by 3,740
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
Distant supervision for relation extraction without labeled data
Mintz, M., Bills, S., Snow, R., & Jurafsky, D. (2009) · Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - ACL-IJCNLP '09 · doi:10.3115/1690219.1690287 · Cited by 2,752
Meta-learning with memory-augmented neural networks
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016) · International Conference on Machine Learning · Cited by 1,504
Relation Classification via Convolutional Deep Neural Network
Zeng, D., Liu, K., Lai, S., Zhou, G., & Zhao, J. (2015) · Cited by 1,467
Modeling Relations and Their Mentions without Labeled Text
Riedel, S., Yao, L., & McCallum, A. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15939-8_10 · Cited by 1,270
Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks
Zeng, D., Liu, K., Chen, Y., & Zhao, J. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1203 · Cited by 1,149
Neural Relation Extraction with Selective Attention over Instances
Lin, Y., Shen, S., Liu, Z., Luan, H., & Sun, M. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1200 · Cited by 1,021
Self-citationKnowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., & Weld, D. S. (2011) · Cited by 874
A Simple Neural Attentive Meta-Learner
Mishra, N., Rohaninejad, M., Chen, X., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.03141 · Cited by 802
Position-aware Attention and Supervised Data Improve Slot Filling
Zhang, Y., Zhong, V., Chen, D., Angeli, G., & Manning, C. D. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1004 · Cited by 798
Munkhdalai, T., & Yu, H. Q. (2017) · International Conference on Machine Learning · Cited by 771
Representation Learning of Knowledge Graphs with Entity Descriptions
Xie, R., Liu, Z., Jia, J., Luan, H., & Sun, M. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10329 · Cited by 725
Self-citationMulti-instance Multi-label Learning for Relation Extraction
Surdeanu, M., Tibshirani, J., Nallapati, R. M., & Manning, C. D. (2012) · Cited by 643
Kernel methods for relation extraction
Zelenko, D., Aone, C., & Richardella, A. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118703 · Cited by 573
Subsequence Kernels for Relation Extraction
Mooney, R. J., & Bunescu, R. (2005) · neural information processing systems · Cited by 493
SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals
Hendrickx, I., Kim, S. N., Kozareva, Z., Nakov, P., Séaghdha, D. Ó., Padó, S., Marco Pennacchiotti, B., Romano, L., & Śzpakowicz, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.10422 · Cited by 444
Randolph, J. (2005) · Cited by 399
Distant Supervision for Relation Extraction with Sentence-Level Attention and Entity Descriptions
Ji, G., Liu, K., He, S., & Zhao, J. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10953 · Cited by 357
Reinforcement Learning for Relation Classification From Noisy Data
Feng, J., Huang, M., Zhao, L., Yang, Y., & Zhu, X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12063 · Cited by 353
Learning Many Related Tasks at the Same Time with Backpropagation
Caruana, R. (1994) · Cited by 242
Diverse Few-Shot Text Classification with Multiple Metrics
Yu, M., Guo, X., Yi, J., Chang, S., Potdar, S., Cheng, Y., Tesauro, G., Wang, H., & Zhou, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1109 · Cited by 225
Adversarial Training for Relation Extraction
Wu, Y., Bamman, D., & Russell, S. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1187 · Cited by 224
Improved Relation Extraction with Feature-Rich Compositional Embedding Models
Gormley, M. R., Yu, M., & Dredze, M. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1205 · Cited by 191
Label Embedding for Zero-shot Fine-grained Named Entity Typing.
Ma, Y., Cambria, E., & Gao, S. (2016) · International Conference on Computational Linguistics · Cited by 119
Deep Residual Learning for Weakly-Supervised Relation Extraction
Huang, Y. Y., & Wang, W. Y. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1191 · Cited by 118
Incorporating Relation Paths in Neural Relation Extraction
Zeng, W., Lin, Y., Liu, Z., & Sun, M. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1186 · Cited by 110
Self-citationLarge Scaled Relation Extraction With Reinforcement Learning
Zeng, X., He, S., Liu, K., & Zhao, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11950 · Cited by 99
Learning with Noise: Enhance Distantly Supervised Relation Extraction with Dynamic Transition Matrix
Luo, B., Feng, Y., Wang, Z., Zhu, Z., Huang, S., Yan, R., & Zhao, D. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1040 · Cited by 97
Self-citationCoupling semi-supervised learning of categories and relations
Carlson, A., Betteridge, J., Junior, E. R. H., & Mitchell, T. M. (2009) · doi:10.3115/1621829.1621830 · Cited by 74
A survey of noise reduction methods for distant supervision
Roth, B., Barth, T., Wiegand, M., & Klakow, D. (2013) · doi:10.1145/2509558.2509571 · Cited by 55
Strassel, S. M., Przybocki, M. A., Peterson, K., Song, Z., & Maeda, K. (2008) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/23nqo4rf4ozi · Cited by 43
Put It Back: Entity Typing with Language Model Enhancement
Xin, J., Zhu, H., Han, X., Liu, Z., & Sun, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1121 · Cited by 24
Self-citation
Cited by
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications
Ji, S., Pan, S., Cambria, E., Marttinen, P., & Yu, P. S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3070843 · Cited by 2,947
Self-citationERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1139 · Cited by 1,482
Self-citationMaking Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.295 · Cited by 1,380
Generalizing from a Few Examples: A Survey on Few-Shot Learning
Wang, Y., Yao, Q., Kwok, J. T.-Y., & Ni, L. M.-S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05046 · Cited by 772
Matching the Blanks: Distributional Similarity for Relation Learning
Baldini Soares, L., FitzGerald, N., Ling, J., & Kwiatkowski, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1279 · Cited by 731
Metadata
- Authors
- Han, X., Zhu, H., Yu, P., Wang, Z., Yao, Y., Liu, Z., & Sun, M.
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 4803-4809
- OpenAlex
- W2963777632
- Cited by
- 613
- References
- 37
- FWCI
- 21.95
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.