The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
TaggerOne: joint named entity recognition and normalization with semi-Markov Models
Open accessAbstract
Motivation: Text mining is increasingly used to manage the accelerating pace of the biomedical literature. Many text mining applications depend on accurate named entity recognition (NER) and normalization (grounding). While high performing machine learning methods trainable for many entity types exist for NER, normalization methods are usually specialized to a single entity type. NER and normalization systems are also typically used in a serial pipeline, causing cascading errors and limiting the ability of the NER system to directly exploit the lexical information provided by the normalization. Methods: We propose the first machine learning model for joint NER and normalization during both training and prediction. The model is trainable for arbitrary entity types and consists of a semi-Markov structured linear classifier, with a rich feature approach for NER and supervised semantic indexing for normalization. We also introduce TaggerOne, a Java implementation of our model as a general toolkit for joint NER and normalization. TaggerOne is not specific to any entity type, requiring only annotated training data and a corresponding lexicon, and has been optimized for high throughput. Results: We validated TaggerOne with multiple gold-standard corpora containing both mention- and concept-level annotations. Benchmarking results show that TaggerOne achieves high performance on diseases (NCBI Disease corpus, NER f-score: 0.829, normalization f-score: 0.807) and chemicals (BioCreative 5 CDR corpus, NER f-score: 0.914, normalization f-score 0.895). These results compare favorably to the previous state of the art, notwithstanding the greater flexibility of the model. We conclude that jointly modeling NER and normalization greatly improves performance. Availability and Implementation: The TaggerOne source code and an online demonstration are available at: http://www.ncbi.nlm.nih.gov/bionlp/taggerone Contact: zhiyong.lu@nih.gov Supplementary information: Supplementary data are available at Bioinformatics online.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An algorithm for suffix stripping
Porter, M. F. (1980) · Program · doi:10.1108/eb046814 · Cited by 8,096 · cited 2×
“• Stemming: diseases use the Porter stemmer (Porter, 1980) while chemicals only remove plurals (Hartman, 1991).”
BackgroundOn the algorithmic implementation of multiclass kernel-based vector machines
Crammer, K., & Singer, Y. (2002) · Journal of Machine Learning Research · Cited by 2,310
On the algorithmic implementation of multiclass kernel-based vector machines
CrammerKoby, & SingerYoram (2002) · Journal of Machine Learning Research · doi:10.5555/944790.944813 · Cited by 874
NCBI disease corpus: A resource for disease name recognition and concept normalization
Doğan, R. I., Leaman, R., & Lu, Z. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.12.006 · Cited by 820
Self-citationOverview of BioNLP'09 shared task on event extraction
Kim, J.-D., Ohta, T., Pyysalo, S., Kano, Y., & Tsujii, J. (2009) · Proceedings of the Workshop on BioNLP Shared Task - BioNLP '09 · doi:10.3115/1572340.1572342 · Cited by 588
Overview of BioCreAtIvE: critical assessment of information extraction for biology
Hirschman, L., Yeh, A., Blaschke, C., & Valencia, A. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-s1-s1 · Cited by 530
(2007) · The MIT Press eBooks · doi:10.7551/mitpress/7443.001.0001 · Cited by 525
DNorm: disease name normalization with pairwise learning to rank
Leaman, R., Islamaj Doğan, R., & Lu, Z. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt474 · Cited by 484
Self-citationBANNER: AN EXECUTABLE SURVEY OF ADVANCES IN BIOMEDICAL NAMED ENTITY RECOGNITION
LEAMAN, R., & GONZALEZ, G. (2007) · Biocomputing 2008 · doi:10.1142/9789812776136_0062 · Cited by 479
Self-citationHarman, D. (1991) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199101)42:1<7::aid-asi2>3.0.co;2-p · Cited by 381
Overview of BioCreative II gene normalization
Morgan, A. A., Lu, Z., Wang, X., Cohen, A. M., Fluck, J., Ruch, P., Divoli, A., Fundel, K., Leaman, R., Hakenberg, J., et al. (2008) · Genome Biology · doi:10.1186/gb-2008-9-s2-s3 · Cited by 327
Self-citationChemSpot: a hybrid system for chemical named entity recognition
Rocktäschel, T., Weidlich, M., & Leser, U. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts183 · Cited by 269
tmChem: a high performance approach for chemical named entity recognition and normalization
Leaman, R., Wei, C.-H., & Lu, Z. (2015) · Journal of Cheminformatics · doi:10.1186/1758-2946-7-s1-s3 · Cited by 268
Self-citationCHEMDNER: The drugs and chemical names extraction challenge
Krallinger, M., Leitner, F., Rabal, O., Vazquez, M., Oyarzabal, J., & Valencia, A. (2015) · Journal of Cheminformatics · doi:10.1186/1758-2946-7-s1-s1 · Cited by 259
A Joint Model for Entity Analysis: Coreference, Typing, and Linking
Durrett, G., & Klein, D. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00197 · Cited by 234
Cohen, W. W., & Sarawagi, S. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014065 · Cited by 223
Ultraconservative Online Algorithms for Multiclass Problems
Crammer, K., & Singer, Y. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_7 · Cited by 221
OSCAR4: a flexible architecture for chemical text-mining
Jessop, D. M., Adams, S. E., Willighagen, E. L., Hawizy, L., & Murray-Rust, P. (2011) · Journal of Cheminformatics · doi:10.1186/1758-2946-3-41 · Cited by 213
GNormPlus: An Integrative Approach for Tagging Genes, Gene Families, and Protein Domains
Wei, C.-H., Kao, H.-Y., & Lu, Z. (2015) · BioMed Research International · doi:10.1155/2015/918710 · Cited by 203
Self-citationWei, C.-H., Peng, Y., Leaman, R., Davis, A. P., Mattingly, C. J., Li, J., Wiegers, T. C., & Lu, Z. (2016) · Database · doi:10.1093/database/baw032 · Cited by 198
Self-citationChallenges in clinical natural language processing for automated disorder normalization
Leaman, R., Khare, R., & Lu, Z. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.07.010 · Cited by 187
Self-citationJoint parsing and named entity recognition
Finkel, J. R., & Manning, C. D. (2009) · Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics on - NAACL '09 · doi:10.3115/1620754.1620802 · Cited by 185
Evaluating the state of the art in disorder recognition and normalization of the clinical narrative
Pradhan, S., Elhadad, N., South, B. R., Martínez, D., Christensen, L. M., Vogel, A., Suominen, H., Chapman, W. W., & Savova, G. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002544 · Cited by 157
Predicting gene function in a hierarchical context with an ensemble of classifiers
Guan, Y., Myers, C. L., Hess, D. C., Barutçuoğlu, Z., Caudy, A. A., & Troyanskaya, O. G. (2008) · Genome biology · doi:10.1186/gb-2008-9-s1-s3 · Cited by 150
Detection of IUPAC and IUPAC-like chemical names
Klinger, R., Kolářik, C., Fluck, J., Hofmann‐Apitius, M., & Friedrich, C. M. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn181 · Cited by 138
Using rule-based natural language processing to improve disease normalization in biomedical text
Kang, N., Singh, B., Afzal, Z., van Mulligen, E. M., & Kors, J. A. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001173 · Cited by 132
Chemical named entities recognition: a review on approaches and applications
Eltahier, S., & Salim, N. B. (2014) · Journal of Cheminformatics · doi:10.1186/1758-2946-6-17 · Cited by 128
Assessment of disease named entity recognition on a corpus of annotated sentences
Yepes, A. J., Jiménez-Ruiz, E., Lee, V., Gaudan, S., Berlanga, R., & Rebholz‐Schuhmann, D. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-s3-s3 · Cited by 123
PKDE4J: Entity and relation extraction for public knowledge discovery
Song, M., Kim, W. C., Lee, D., Heo, G. E., & Kang, K. Y. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.08.008 · Cited by 120
Abbreviation definition identification based on automatic precision estimates
Sohn, S., Comeau, D. C., Kim, W. B., & Wilbur, W. J. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-402 · Cited by 120
Learning to rank with (a lot of) word features
Bai, B., Weston, J., Grangier, D., Collobert, R., Sadamasa, K., Qi, Y., Chapelle, O., & Weinberger, K. Q. (2009) · Information Retrieval · doi:10.1007/s10791-009-9117-9 · Cited by 118
Rebholz‐Schuhmann, D., Yepes, A. J., van Mulligen, E. M., KANG, N., Kors, J. A., Milward, D., Corbett, P. T., Buyko, E., Beißwanger, E., & Hahn, U. (2010) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s0219720010004562 · Cited by 98
Text Mining for Drugs and Chemical Compounds: Methods, Tools and Applications
Vázquez, M., Krallinger, M., Leitner, F., & Valencia, A. (2011) · Molecular Informatics · doi:10.1002/minf.201100005 · Cited by 87
Anatomical entity mention recognition at literature scale
Pyysalo, S., & Ananiadou, S. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt580 · Cited by 79
Mapping proteins to disease terminologies: from UniProt to MeSH
Mottaz, A., Yip, Y. L., Ruch, P., & Veuthey, A. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-s5-s3 · Cited by 73
A modular framework for biomedical concept recognition
Campos, D., Matos, S., & Oliveira, J. L. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-281 · Cited by 70
Chemical Names: Terminological Resources and Corpora Annotation
Kolářik, C., Klinger, R., Friedrich, C. M., et al. (2008) · PUB – Publications at Bielefeld University (Bielefeld University) · Cited by 66
(2008) · Technometrics · doi:10.1198/tech.2008.s913 · Cited by 45
Finkel, J. R., & Manning, C. D. (2010) · Cited by 43
SimConcept: A Hybrid Approach for Simplifying Composite Named Entities in Biomedical Text
Wei, C.-H., Leaman, R., & Lu, Z. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2422651 · Cited by 25
Self-citationQuantifying the Impact and Extent of Undocumented Biomedical Synonymy
Blair, D. R., Wang, K., Nestorov, S., Evans, J. A., & Rzhetsky, A. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003799 · Cited by 12
Cited by
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
Self-citationDeep learning with word embeddings improves biomedical named entity recognition
Habibi, M., Weber, L., Neves, M., Wiegandt, D. L., & Leser, U. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx228 · Cited by 542
PubTator central: automated concept annotation for biomedical full text articles
Wei, C.-H., Allot, A., Leaman, R., & Lu, Z. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz389 · Cited by 400
Self-citationAn attention-based BiLSTM-CRF approach to document-level chemical named entity recognition
Luo, L., Yang, Z., Yang, P., Zhang, Y., Wang, L., Lin, H., & Wang, J. (2018) · Bioinformatics · doi:10.1093/bioinformatics/btx761 · Cited by 417
AI in Health: State of the Art, Challenges, and Future Directions
Wang, F., & Preininger, A. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677908 · Cited by 398
Metadata
- Authors
- Leaman, R., & Lu, Z.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-06-09
- Volume
- 32
- Issue
- 18
- Pages
- 2839-2846
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2414378847
- Cited by
- 305
- References
- 41
- FWCI
- 14.60
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.