Proceedings of the 18th BioNLP Workshop and Shared Task
Open accessThe DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Abstract
Recent years have seen an explosion of workshops, community challenges, corpora and publicly available tools in the biomedical and clinical language processing domain.That trend continues in 2019.In a significant advance, this year the original BioNLP-ST challenge matured into an open platform capable of providing technical support and sustaining any group that is interested in organizing a biomedical language processing challenge [1], while the BioNLP Special Interest Group continues supporting Shared Tasks in emerging areas of research through the annual meeting.This year, BioNLP-ST presents research directions explored by 72 teams for inference and entailment in the medical domain, and their contribution to domain-specific information retrieval and question answering systems [2].The BioNLP meeting has now been ongoing for 18 years.BioNLP continues to stay the flagship and the generalist meeting in biomedical language processing, accepting noteworthy work independently of the tasks and sublanguages studied.BioNLP also continues promoting research in languages other than English, this year presenting work in Romanian, Portuguese, Spanish, and Chinese [3, 4, 5, 6], primarily covering development of resources for these languages.The quality of submissions continues to impress the program committee and the organizers.BioNLP 2019 received 72 submissions to the workshop, and 21 for the Shared Task.Of the work submitted to the workshop, 14 papers were accepted for oral presentation and 24 as poster presentations.This year, various deep learning architectures are explored in all papers, with continuing focus on interesting new models and in-depth exploration of the state-of-the-art publicly available tools.Most of the work uses BERT [7] or BERT models trained on PubMed, with one paper exploring BERT and ELMo on ten biomedical benchmarking datasets [8] and many others using and exploring embeddings and neural networks for chemical recognition [9], concept extraction and coding [10], relation extraction [11, 12, 13], and phenotyping [14].As for the past several years, the themes in this year's papers and posters continue to focus equally on clinical text and biological language processing.They also reveal sustained interest in social media and consumer language processing [15].As it has been for the past 18 years, the workshop is truly a community-wide effort of the authors producing high quality work that is already contributing to acceleration of foundational biomedical research [16,17,18,19] and clinical practice [20,21,22,23] through improvements in information retrieval and extraction, question answering, diagnosis and clinical decision support [24].We are equally happy to see sustained contributions from those who started forming the field of BioNLP research, and first-time contributions that show the increasing interest in the domain.We are particularly indebted to our reviewers who reviewed a higher than usual workload in a very short time.Their judgments resulted in a program that will undoubtedly advance both the BioNLP research and the practical areas that it serves.Due to space and time constraints, we could only accept the papers that were recommended for acceptance by at least two reviewers.We hope that the authors of the papers that could not be accepted received good feedback that will help them improve their work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
DeepWalk: online learning of social representations
Perozzi, B., Al-Rfou, R., & Skiena, S. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623732 · Cited by 10,123
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Geurts, P., Ernst, D., & Wehenkel, L. (2006) · Machine Learning · doi:10.1007/s10994-006-6226-1 · Cited by 9,300
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Modeling Relational Data with Graph Convolutional Networks
Schlichtkrull, M., Kipf, T. N., Bloem, P., van den Berg, R., Titov, I., & Welling, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93417-4_38 · Cited by 5,570
LINE: Large-scale Information Network Embedding
Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., & Mei, Q. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741093 · Cited by 5,416
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Bai, S., Kolter, J. Z., & Koltun, V. (2018) · arXiv · doi:10.48550/arxiv.1803.01271 · Cited by 4,655
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1607.01759 · Cited by 4,542
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377
The probabilistic relevance framework: BM25 and beyond
Robertson, S., & Zaragoza, H. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000019 · Cited by 3,374
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
Tai, K. S., Socher, R., & Manning, C. D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1150 · Cited by 3,081
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2017) · arXiv · doi:10.48550/arxiv.1707.08114 · Cited by 2,910
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567
Towards Federated Learning at Scale: System Design
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.01046 · Cited by 2,334
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967
An overview of MetaMap: historical perspective and recent advances
Aronson, A. R., & Lang, F.-M. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.002733 · Cited by 1,519
ERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1139 · Cited by 1,482
The SIDER database of drugs and side effects
Kuhn, M., Letunic, I., Jensen, L. J., & Bork, P. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1075 · Cited by 1,451
Cer, D., Yang, Y., Kong, S.-Y., Hua, N., Limtiaco, N., John, R. S., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.11175 · Cited by 1,267
Graph Convolution over Pruned Dependency Trees Improves Relation Extraction
Zhang, Y., Qi, P., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1244 · Cited by 1,048
Very Deep Convolutional Networks for Text Classification
Conneau, A., Schwenk, H., Barrault, L., & Lecun, Y. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers · doi:10.18653/v1/e17-1104 · Cited by 1,046
CoQA: A Conversational Question Answering Challenge
Reddy, S., Chen, D., & Manning, C. D. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00266 · Cited by 975
Siamese Recurrent Architectures for Learning Sentence Similarity
Mueller, J., & Thyagarajan, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10350 · Cited by 935
Peng, Y., Yan, S., & Lu, Z. (2019) · Proceedings of the 18th BioNLP Workshop and Shared Task · doi:10.18653/v1/w19-5006 · Cited by 899
A side effect resource to capture phenotypic effects of drugs
Kuhn, M., Campillos, M., Letunic, I., Jensen, L. J., & Bork, P. (2010) · Molecular Systems Biology · doi:10.1038/msb.2009.98 · Cited by 899
Hovy, E., Marcus, M., Palmer, M., Ramshaw, L., & Weischedel, R. (2006) · Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers on XX - NAACL '06 · doi:10.3115/1614049.1614064 · Cited by 806
DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
Shen, T., Zhou, T., Long, G., Jiang, J., Pan, S., & Zhang, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11941 · Cited by 779
(2006) · doi:10.7551/mitpress/7503.003.0029 · Cited by 753
Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text
Sun, H., Dhingra, B., Zaheer, M., Mazaitis, K., Salakhutdinov, R., & Cohen, W. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1455 · Cited by 716
Nogueira, R., & Cho, K. (2019) · arXiv · doi:10.48550/arxiv.1901.04085 · Cited by 615
Explainable Prediction of Medical Codes from Clinical Text
Mullenbach, J., Wiegreffe, S., Duke, J., Sun, J., & Eisenstein, J. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1100 · Cited by 597
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2018) · arXiv · doi:10.48550/arxiv.1804.07461 · Cited by 562
Friedman, C., Rubin, J., Brown, J., Buntin, M., Corn, M., Etheredge, L., Gunter, C., Musen, M., Platt, R., Stead, W., et al. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-002977 · Cited by 509
An efficient framework for learning sentence representations
Logeswaran, L., & Lee, H. (2018) · arXiv · doi:10.48550/arxiv.1803.02893 · Cited by 427
Methods for Causality Assessment of Adverse Drug Reactions: A Systematic Review
Agbabiaka, T. B., Savović, J., & Ernst, E. (2008) · Drug Safety · doi:10.2165/00002018-200831010-00003 · Cited by 366
SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
Dunn, M., Sagun, L., Higgins, M., Guney, V. U., Cirik, V., & Cho, K. (2017) · arXiv · doi:10.48550/arxiv.1704.05179 · Cited by 359
Enhancing clinical concept extraction with contextual embeddings
Si, Y., Wang, J., Xu, H., & Roberts, K. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz096 · Cited by 322
Chronic myeloid leukemia, version 1.2019, NCCN Clinical Practice Guidelines in Oncology
Radich, J. P., Deininger, M., Abboud, C. N., Altman, J. K., Berman, E., Bhatia, R., Bhatnagar, B., Curtin, P., DeAngelo, D. J., Gotlib, J., et al. (2018) · Journal of the National Comprehensive Cancer Network · doi:10.6004/jnccn.2018.0071 · Cited by 277
Embeddings for Word Sense Disambiguation: An Evaluation Study
Iacobacci, I., Pilehvar, M. T., & Navigli, R. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1085 · Cited by 253
Wei, C.-H., Peng, Y., Leaman, R., Davis, A. P., Mattingly, C. J., Li, J., Wiegers, T. C., & Lu, Z. (2016) · Database · doi:10.1093/database/baw032 · Cited by 198
emrQA: A Large Corpus for Question Answering on Electronic Medical Records
Pampari, A., Raghavan, P., Liang, J., & Peng, J. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1258 · Cited by 191
Dr Google in the ED: Searching for online health information by adult emergency department patients
Cocco, A. M., Zordan, R., Taylor, D. M., Weiland, T. J., Dilley, S. J., Kant, J., Dombagolla, M., Hendarto, A., Lai, F., & Hutton, J. (2018) · Medical Journal of Australia · doi:10.5694/mja17.00889 · Cited by 114
Fadl Federated-autonomous deep learning for distributed electronic health record
Liu, D., Miller, T., Sayeed, R., & Mandl, K. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11400 · Cited by 65
Learning Patient Representations from Text
Dligach, D., & Miller, T. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2014 · Cited by 13
Cited by
A review of applications in federated learning
Li, L., Fan, Y., Tse, M., & Lin, K.-Y. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.106854 · Cited by 1,559
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150
An introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools
Lauriola, I., Lavelli, A., & Aiolli, F. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.05.103 · Cited by 691
Neural Natural Language Processing for unstructured data in electronic health records: A review
Li, I., Pan, J., Goldwasser, J., Verma, N., Wong, W. P., Nuzumlalı, M. Y., Rosand, B., Li, Y., Zhang, M., Chang, D., et al. (2022) · Computer Science Review · doi:10.1016/j.cosrev.2022.100511 · Cited by 254
Clinical concept extraction: A methodology review
Fu, S., Chen, D., He, H., Liu, S., Moon, S., Peterson, K. J., Shen, F., Wang, L., Wang, Y., Wen, A., et al. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103526 · Cited by 183
Metadata
- Published
- 2019
- OpenAlex
- W4250526791
- Cited by
- 97
- References
- 53
- Type
- paratext
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.