Automatically Labeled Data Generation for Large Scale Event Extraction
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Modern models of event extraction for tasks like ACE are based on supervised learning of events from small hand-labeled data.However, hand-labeled training data is expensive to produce, in low coverage of event types, and limited in size, which makes supervised methods hard to extract large scale of events for knowledge base population.To solve the data labeling problem, we propose to automatically label training data for event extraction via world knowledge and linguistic knowledge, which can detect key arguments and trigger words for each event type and employ them to label events in texts automatically.The experimental results show that the quality of our large scale automatically labeled data is competitive with elaborately human-labeled data.And our automatically labeled data can incorporate with human-labeled data, then improve the performance of models learned from these data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · cited 1×
“Distant supervision have been used in relation extraction for automatically labeling training data (Mintz et al., 2009; Hinton et al., 2012; Krause et al., 2012; Krishnamurthy and Mitchell, 2012; Berant et al., 2013; Surdeanu et al., 2012; Zeng et al., 2015).”
MethodADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322 · cited 2×
“…) , we can define the objective function using cross-entropy at the bag level as follows: where j is constrained as follows: To compute the network parameter θ , we maximize the log likelihood J ( θ ) through stochastic gradient descent over mini-batches with the Adadelta (Zeiler, 2012) update rule.”
MethodFreebase: a collaboratively created graph database for structuring human knowledge
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376746 · Cited by 4,852 · cited 1×
“com/acl2017submission/event-data Freebase is a semantic knowledge base (Bollacker et al., 2008), which makes use of mediators (also called compound value types, CVTs) to merge multiple values into a single value.”
BackgroundDistant supervision for relation extraction without labeled data
Mintz, M., Bills, S., Snow, R., & Jurafsky, D. (2009) · Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - ACL-IJCNLP '09 · doi:10.3115/1690219.1690287 · Cited by 2,752 · cited 3×
“And DS for RE assumes that if two entities have a relationship in a known knowledge base, then all sentences that mention these two entities will express that relationship in some way (Mintz et al., 2009).”
BackgroundMethodBaker, C. F., Fillmore, C. J., & Lowe, J. B. (1998) · Proceedings of the 36th annual meeting on Association for Computational Linguistics - · doi:10.3115/980845.980860 · Cited by 2,518 · cited 1×
“FrameNet 3 is a linguistic resource storing information about lexical and predicate argument semantics (Baker et al., 1998).”
Background(2010) · Cited by 1,906
Semantic Parsing on Freebase from Question-Answer Pairs
Berant, J., Chou, A., Frostig, R., & Liang, P. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1160 · Cited by 1,517 · cited 2×
“, 2012; Krishnamurthy and Mitchell, 2012; Berant et al., 2013; Surdeanu et al., 2012; Zeng et al., 2015). But DS for RE cannot directly use for EE. For the reasons that an event is more complicated than a relation and the task of EE is more difficult than RE. The best reported supervised RE and EE system got a F1-scor…”
BackgroundMethodInfluentialDistant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks
Zeng, D., Liu, K., Chen, Y., & Zhao, J. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1203 · Cited by 1,149 · cited 1×
“Distant supervision have been used in relation extraction for automatically labeling training data (Mintz et al., 2009; Hinton et al., 2012; Krause et al., 2012; Krishnamurthy and Mitchell, 2012; Berant et al., 2013; Surdeanu et al., 2012; Zeng et al., 2015).”
MethodSelf-citationEvent Extraction via Dynamic Multi-Pooling Convolutional Neural Networks
Chen, Y., Xu, L., Liu, K., Zeng, D., & Zhao, J. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1017 · Cited by 906 · cited 6×
“Following (Nguyen et al., 2016; Chen et al., 2015; Li et al., 2013), we used the same test set with 40 newswire articles and the same development set with 30 documents and the rest 529 documents are used for ACE training set.”
BackgroundMethodInfluentialSelf-citationKnowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., & Weld, D. S. (2011) · Cited by 874
Joint Event Extraction via Recurrent Neural Networks
Nguyen, T. H., Cho, K., & Grishman, R. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1034 · Cited by 647 · cited 5×
“Following (Nguyen et al., 2016; Chen et al., 2015; Li et al., 2013), we used the same test set with 40 newswire articles and the same development set with 30 documents and the rest 529 documents are used for ACE training set.”
BackgroundMethodInfluentialMulti-instance Multi-label Learning for Relation Extraction
Surdeanu, M., Tibshirani, J., Nallapati, R. M., & Manning, C. D. (2012) · Cited by 643
The stages of event extraction
Ahn, D. (2006) · doi:10.3115/1629235.1629236 · Cited by 589 · cited 1×
“(Ahn, 2006; Ji and Grishman, 2008; Hong et al., 2011; McClosky et al., 2011; Li et al., 2013, 2014; Chen et al., 2015; Nguyen and Grishman, 2015; Nguyen et al., 2016).”
Relation Classification via Multi-Level Attention CNNs
Wang, L., Cao, Z., de Melo, G., & Liu, Z. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1123 · Cited by 487 · cited 2×
“0% (Wang et al., 2016) and 55 .”
Joint Event Extraction via Structured Prediction with Global Features
Li, Q., Ji, H., & Huang, L. (2013) · Cited by 479
Event Detection and Domain Adaptation with Convolutional Neural Networks
Nguyen, T. H., & Grishman, R. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.3115/v1/p15-2060 · Cited by 408 · cited 1×
“(Ahn, 2006; Ji and Grishman, 2008; Hong et al., 2011; McClosky et al., 2011; Li et al., 2013, 2014; Chen et al., 2015; Nguyen and Grishman, 2015; Nguyen et al., 2016).”
Refining Event Extraction through Cross-Document Inference
Ji, H., & Grishman, R. (2008) · Cited by 400
Using Cross-Entity Inference to Improve Event Extraction
Hong, Y., Zhang, J., Ma, B., Yao, J., Zhou, G., & Zhu, Q. (2011) · Cited by 239
Event Extraction as Dependency Parsing
McClosky, D., Surdeanu, M., & Manning, C. D. (2011) · Cited by 218
Liberal Event Extraction and Event Schema Induction
Huang, L., Cassidy, T., Feng, X., Ji, H., Voss, C. R., Han, J., & Sil, A. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1025 · Cited by 161 · cited 1×
“Un-supervised methods can extract large numbers of events without using labeled data (Chambers and Jurafsky, 2011; Cheung et al., 2013; Huang et al., 2016).”
BackgroundWeakly Supervised Training of Semantic Parsers
Krishnamurthy, J., & Mitchell, T. M. (2012) · KiltHub Repository · doi:10.1184/r1/6476447 · Cited by 145
Leveraging FrameNet to Improve Automatic Event Detection
Liu, S., Chen, Y., He, S., Liu, K., & Zhao, J. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1201 · Cited by 126 · cited 1×
“Each frame of FrameNet can be taken as a semantic frame of a type of events (Liu et al., 2016).”
BackgroundSelf-citationProbabilistic Frame InductionPreprint
Cheung, J. C. K., Poon, H., & Vanderwende, L. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4813 · Cited by 116 · cited 2×
“Un-supervised methods can extract large numbers of events without using labeled data (Chambers and Jurafsky, 2011; Cheung et al., 2013; Huang et al., 2016).”
BackgroundConstructing Information Networks Using One Single Model
Li, Q., Ji, H., HONG, Y., & Li, S. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1198 · Cited by 114 · cited 2×
“(Ahn, 2006; Ji and Grishman, 2008; Hong et al., 2011; McClosky et al., 2011; Li et al., 2013, 2014; Chen et al., 2015; Nguyen and Grishman, 2015; Nguyen et al., 2016).”
MethodLarge-Scale Learning of Relation-Extraction Rules with Distant Supervision from the Web
Krause, S., Li, H., Uszkoreit, H., & Xu, F. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35176-1_17 · Cited by 75 · cited 1×
“Distant supervision have been used in relation extraction for automatically labeling training data (Mintz et al., 2009; Hinton et al., 2012; Krause et al., 2012; Krishnamurthy and Mitchell, 2012; Berant et al., 2013; Surdeanu et al., 2012; Zeng et al., 2015).”
MethodEvent Extraction Using Distant Supervision
Reschke, K. S., Jankowiak, M., Surdeanu, M., Manning, C. D., & Jurafsky, D. S. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4czcapr6ez74 · Cited by 37 · cited 1×
“Reschke et al. (2014) extended the distant supervision approach to fill slots in plane crash.”
Background
Cited by
Graph Convolutional Networks With Argument-Aware Pooling for Event Detection
Nguyen, T., & Grishman, R. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12039 · Cited by 383
Exploring Pre-trained Language Models for Event Extraction and Generation
Yang, S., Feng, D., Qiao, L., Kan, Z., & Li, D. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1522 · Cited by 348
Event Extraction as Machine Reading Comprehension
Liu, J., Chen, Y., Liu, K., Bi, W., & Liu, X. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.128 · Cited by 303
Self-citation
Metadata
- Authors
- Chen, Y., Liu, S., Zhang, X., Liu, K., & Zhao, J.
- Venue
- Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2017
- Pages
- 409-419
- OpenAlex
- W2618285232
- Cited by
- 163
- References
- 26
- FWCI
- 8.89
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.