Overview of BioNLP'09 shared task on event extraction
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
The paper presents the design and implementation of the BioNLP'09 Shared Task, and reports the final results with analysis. The shared task consists of three sub-tasks, each of which addresses bio-molecular event extraction at a different level of specificity. The data was developed based on the GENIA event corpus. The shared task was run over 12 weeks, drawing initial interest from 42 teams. Of these teams, 24 submitted final results. The evaluation results are encouraging, indicating that state-of-the-art performance is approaching a practically applicable level and revealing some remaining challenges.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Generating Typed Dependency Parses from Phrase Structure Parses
de Marneffe, M.-C., MacCartney, B., & Manning, C. D. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3wtkaidk6ea8 · Cited by 2,258
MINT: the Molecular INTeraction database
Chatr-aryamontri, A., Ceol, A., Palazzi, L. M., Nardelli, G., Schneider, M. V., Castagnoli, L., & Cesareni, G. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkl950 · Cited by 1,026
The Gene Ontology Annotation (GOA) Database: sharing knowledge in Uniprot with Gene Ontology
Camon, E. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh021 · Cited by 927
Coarse-to-fine n-best parsing and MaxEnt discriminative reranking
Charniak, E., & Johnson, M. (2005) · Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL '05 · doi:10.3115/1219840.1219862 · Cited by 872
Introduction to the bio-entity recognition task at JNLPBA
Kim, J.-D., Ohta, T., Tsuruoka, Y., Tateisi, Y., & Collier, N. (2004) · Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications - JNLPBA '04 · doi:10.3115/1567594.1567610 · Cited by 573
Self-citationBioInfer: a corpus for information extraction in the biomedical domain
Pyysalo, S., Ginter, F., Heimonen, J., Björne, J., Boberg, J., Järvinen, J., & Salakoski, T. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-50 · Cited by 485
Self-citationPathguide: a Pathway Resource List
Bader, G. D. (2006) · Nucleic Acids Research · doi:10.1093/nar/gkj126 · Cited by 435
Corpus annotation for mining biomedical events from literature
Kim, J.-D., Ohta, T., & Tsujii, J. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-10 · Cited by 395
Self-citationIntricacies of Collins' Parsing Model
Bikel, D. M. (2004) · Computational Linguistics · doi:10.1162/0891201042544929 · Cited by 265
TREC 2004 Genomics Track Overview
Hersh, W., Bhuptiraju, R. T., Ross, L., Johnson, P., Cohen, A., & Kraemer, D. F. (2003) · Cited by 259
Cited by
Position-aware Attention and Supervised Data Improve Slot Filling
Zhang, Y., Zhong, V., Chen, D., Angeli, G., & Manning, C. D. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1004 · Cited by 798
DNorm: disease name normalization with pairwise learning to rank
Leaman, R., Islamaj Doğan, R., & Lu, Z. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt474 · Cited by 484
SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals
Hendrickx, I., Kim, S. N., Kozareva, Z., Nakov, P., Ó Séaghdha, D., Padó, S., Pennacchiotti, M., Romano, L., & Szpakowicz, S. (2009) · Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions - DEW '09 · doi:10.3115/1621969.1621986 · Cited by 477
SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals
Hendrickx, I., Kim, S. N., Kozareva, Z., Nakov, P., Séaghdha, D. Ó., Padó, S., Marco Pennacchiotti, B., Romano, L., & Śzpakowicz, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.10422 · Cited by 444
Cross-Sentence N-ary Relation Extraction with Graph LSTMs
Peng, N., Poon, H., Quirk, C., Toutanova, K., & Yih, W.-T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00049 · Cited by 443
LINNAEUS: A species name identification system for biomedical literature
Gerner, M., Nenadic, G., & Bergman, C. M. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-85 · Cited by 335
Bidirectional RNN for Medical Event Detection in Electronic Health Records
Jagannatha, A. N., & Yu, H. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1056 · Cited by 314
“Many named entity recognition systems in the biomedical domain have been driven by the Shared tasks of BioNLP (Kim et al., 2009), BioCreAtivE (Hirschman et al., 2005) i2b2 shared NLP tasks (Li et al., 2009) and ShARe/CLEF evaluation tasks (Pradhan et al., 2014”
MethodTaggerOne: joint named entity recognition and normalization with semi-Markov Models
Leaman, R., & Lu, Z. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw343 · Cited by 305
Bravo, À., Piñero, J., Queralt-Rosinach, N., Rautschka, M., & Furlong, L. I. (2015) · BMC Bioinformatics · doi:10.1186/s12859-015-0472-9 · Cited by 264
Biomedical Named Entity Recognition with Multilingual BERT
Hakala, K., & Pyysalo, S. (2019) · Proceedings of The 5th Workshop on BioNLP Open Shared Tasks · doi:10.18653/v1/d19-5709 · Cited by 96
“The task data is distributed in the simple stand-off format first introduced for the BioNLP Shared Task 2009 (Kim et al., 2009).”
MethodSelf-citationBiomedical text mining for research rigor and integrity: tasks, challenges, directions
Kilicoglu, H. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbx057 · Cited by 61
Prediction of robust scientific facts from literature
Belikov, A. V., Rzhetsky, A., & Evans, J. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00474-8 · Cited by 30
From POS tagging to dependency parsing for biomedical event extraction
Nguyen, D. Q., & Verspoor, K. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2604-0 · Cited by 24
BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing
Fries, J. A., Weber, L., Seelam, N., Altay, G., Datta, D., Garda, S., Kang, M., Su, R., Kusa, W., Cahyawijaya, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.15076 · Cited by 17
Tri-Train: Automatic Pre-Fine Tuning between Pre-Training and Fine-Tuning for SciNER
Zeng, Q., Yu, W., Yu, M., Jiang, T., Weninger, T., & Jiang, M. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.429 · Cited by 10
Explanatory argumentation in natural language for correct and incorrect medical diagnoses
Molinet, B., Marro, S., Cabrio, E., & Villata, S. (2024) · Journal of Biomedical Semantics · doi:10.1186/s13326-024-00306-1 · Cited by 4
“The AnatEM corpus [38] and some of the BioNLP Shared Tasks [39, 40] concentrate on entities and relations, while other approaches [41–46] specifically address gene, protein, and species entities.”
Method
Metadata
- Authors
- Kim, J.-D., Ohta, T., Pyysalo, S., Kano, Y., & Tsujii, J.
- Venue
- Proceedings of the Workshop on BioNLP Shared Task - BioNLP '09
- Published
- 2009
- Pages
- 1
- OpenAlex
- W2032566933
- Cited by
- 588
- References
- 17
- Type
- conference abstract
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.