The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
DNorm: disease name normalization with pairwise learning to rank
Open accessAbstract
Motivation: Despite the central role of diseases in biomedical research, there have been much fewer attempts to automatically determine which diseases are mentioned in a text—the task of disease name normalization (DNorm)—compared with other normalization tasks in biomedical text mining research. Methods: In this article we introduce the first machine learning approach for DNorm, using the NCBI disease corpus and the MEDIC vocabulary, which combines MeSH® and OMIM. Our method is a high-performing and mathematically principled framework for learning similarities between mentions and concept names directly from training data. The technique is based on pairwise learning to rank, which has not previously been applied to the normalization task but has proven successful in large optimization problems for information retrieval. Results: We compare our method with several techniques based on lexical normalization and matching, MetaMap and Lucene. Our algorithm achieves 0.782 micro-averaged F-measure and 0.809 macro-averaged F-measure, an increase over the highest performing baseline method of 0.121 and 0.098, respectively. Availability: The source code for DNorm is available at http://www.ncbi.nlm.nih.gov/CBBresearch/Lu/Demo/DNorm, along with a web-based demonstration and links to the NCBI disease corpus. Results on PubMed abstracts are available in PubTator: http://www.ncbi.nlm.nih.gov/CBBresearch/Lu/Demo/PubTator Contact: zhiyong.lu@nih.gov
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836
Advances in Large-Margin Classifiers
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/1113.001.0001 · Cited by 1,869
2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text
Uzuner, Ö., South, B. R., Shen, S., & DuVall, S. L. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000203 · Cited by 1,252
Large Margin Rank Boundaries for Ordinal Regression
Herbrich, R., Graepel, T., & Obermayer, K. (2000) · Advances in Large-Margin Classifiers · doi:10.7551/mitpress/1113.003.0010 · Cited by 1,074
Disease Ontology: a backbone for disease semantic integration
Schriml, L. M., Arze, C., Nadendla, S., Chang, Y.-W. W., Mazaitis, M., Felix, V., Feng, G., & Kibbe, W. A. (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr972 · Cited by 929
Proceedings of the AMIA Symposium
Schulz, S., Daumke, P., Smith, B., & Hahn, U. (2005) · Cited by 606
Overview of BioNLP'09 shared task on event extraction
Kim, J.-D., Ohta, T., Pyysalo, S., Kano, Y., & Tsujii, J. (2009) · Proceedings of the Workshop on BioNLP Shared Task - BioNLP '09 · doi:10.3115/1572340.1572342 · Cited by 588
PubMed and beyond: a survey of web tools for searching biomedical literature
Lu, Z. (2011) · Database · doi:10.1093/database/baq036 · Cited by 571
Self-citationProceedings of the Eighteenth International Conference on Machine Learning
Brodley, C. E., & Danyluk, A. (2001) · Cited by 568
PubTator: a web-based text mining tool for assisting biocuration
Wei, C.-H., Kao, H.-Y., & Lu, Z. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt441 · Cited by 566
Self-citationOverview of BioCreAtIvE: critical assessment of information extraction for biology
Hirschman, L., Yeh, A., Blaschke, C., & Valencia, A. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-s1-s1 · Cited by 530
Collins, M., & Duffy, N. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073128 · Cited by 506
BANNER: AN EXECUTABLE SURVEY OF ADVANCES IN BIOMEDICAL NAMED ENTITY RECOGNITION
LEAMAN, R., & GONZALEZ, G. (2007) · Biocomputing 2008 · doi:10.1142/9789812776136_0062 · Cited by 479
Self-citationSNOMED clinical terms: overview of the development process and project status.
Stearns, M. Q., Price, C., Spackman, K. A., & Wang, A. Y. (2001) · Europe PMC (PubMed Central) · Cited by 448
Overview of BioCreative II gene normalization
Morgan, A. A., Lu, Z., Wang, X., Cohen, A. M., Fluck, J., Ruch, P., Divoli, A., Fundel, K., Leaman, R., Hakenberg, J., et al. (2008) · Genome Biology · doi:10.1186/gb-2008-9-s2-s3 · Cited by 327
Self-citationWhat is a disease?: Disease, disability and their definitions
Scully, J. L. (2004) · The EMBO Reports · doi:10.1038/sj.embor.7400195 · Cited by 195
The gene normalization task in BioCreative III
Lu, Z., Kao, H.-Y., Wei, C.-H., Huang, M., Liu, J., Kuo, C.-J., Hsu, C.-N., Tsai, R. T.-H., Dai, H.-J., Okazaki, N., et al. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-s8-s2 · Cited by 181
Self-citationMEDIC: a practical disease vocabulary used at the Comparative Toxicogenomics Database
Davis, A. P., Wiegers, T. C., Rosenstein, M. C., & Mattingly, C. J. (2012) · Database · doi:10.1093/database/bar065 · Cited by 166
Overview of BioCreAtIvE task 1B: normalized gene lists
Hirschman, L., Colosimo, M., Morgan, A., & Yeh, A. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-s1-s11 · Cited by 164
Recommending MeSH terms for annotating biomedical articles
Huang, M., Névéol, A., & Lu, Z. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2010-000055 · Cited by 158
Self-citationUsing rule-based natural language processing to improve disease normalization in biomedical text
Kang, N., Singh, B., Afzal, Z., van Mulligen, E. M., & Kors, J. A. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001173 · Cited by 132
Assessment of disease named entity recognition on a corpus of annotated sentences
Yepes, A. J., Jiménez-Ruiz, E., Lee, V., Gaudan, S., Berlanga, R., & Rebholz‐Schuhmann, D. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-s3-s3 · Cited by 123
Abbreviation definition identification based on automatic precision estimates
Sohn, S., Comeau, D. C., Kim, W. B., & Wilbur, W. J. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-402 · Cited by 120
Learning to rank with (a lot of) word features
Bai, B., Weston, J., Grangier, D., Collobert, R., Sadamasa, K., Qi, Y., Chapelle, O., & Weinberger, K. Q. (2009) · Information Retrieval · doi:10.1007/s10791-009-9117-9 · Cited by 118
Overview of the TREC 2012 Medical Records Track.
Voorhees, E. M., & Hersh, W. (2012) · Text REtrieval Conference · Cited by 116
Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium
Luo, G., Liu, J., & Yang, C. C. (2012) · doi:10.1145/2110363 · Cited by 112
High-performance gene name normalization with G e N o
Wermter, J., Tomanek, K., & Hahn, U. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp071 · Cited by 101
Wei, C.-H., Harris, B. R., Li, D., Berardini, T., Huala, E., Kao, H.-Y., & Lu, Z. (2012) · Database · doi:10.1093/database/bas041 · Cited by 96
Self-citationGeneTUKit: a software for document-level gene normalization
Huang, M., Liu, J., & Zhu, X. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr042 · Cited by 71
An improved corpus of disease mentions in PubMed citations
Islamaj, R., & Lu, Z. (2012) · Cited by 65
Self-citationA SNPshot of PubMed to associate genetic variants with drugs, diseases, and adverse reactions
Hakenberg, J., Voronov, D., Nguyên, V. H., Liang, S., Anwar, S., Lumpkin, B., Leaman, R., Tari, L., & Baral, C. R. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2012.04.006 · Cited by 48
Self-citationBioNLP: Proceedings of the 2012 Workshop on Biomedical Natural Language Processing
Cohen, K. B., Demner‐Fushman, D., Ananiadou, S., Webber, B., Tsujii, J., & Pestian, J. P. (2012) · Cited by 47
Collaborative biocuration--text-mining development task for document prioritization for curation
Wiegers, T. C., Davis, A. P., & Mattingly, C. J. (2012) · Database · doi:10.1093/database/bas037 · Cited by 40
Biesecker, L. (2005) · Clinical Genetics · doi:10.1111/j.1399-0004.2005.00509.x · Cited by 34
An Inference Method for Disease Name Normalization.
Islamaj, R., & Lu, Z. (2012) · National Conference on Artificial Intelligence · Cited by 24
Self-citationKim, S., Kim, W., Wei, C.-H., Lu, Z., & Wilbur, W. J. (2012) · Database · doi:10.1093/database/bas042 · Cited by 14
Self-citationLinking multiple disease-related resources through UMLS
Névéol, A., Li, J., & Lu, Z. (2012) · International Health Informatics Symposium · doi:10.1145/2110363.2110455 · Cited by 11
Self-citation(2001) · doi:10.1002/pds.4850 · Cited by 7
Cited by
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
Self-citationBioCreative V CDR task corpus: a resource for chemical disease relation extraction
Li, J., Sun, Y., Johnson, R. J., Sciaky, D., Wei, C.-H., Leaman, R., Davis, A. P., Mattingly, C. J., Wiegers, T. C., & Lu, Z. (2016) · Database · doi:10.1093/database/baw068 · Cited by 870
Self-citationNCBI disease corpus: A resource for disease name recognition and concept normalization
Doğan, R. I., Leaman, R., & Lu, Z. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.12.006 · Cited by 820
Self-citationA survey of current trends in computational drug repositioning
Li, J., Zheng, S., Chen, B., Butte, A. J., Swamidass, S. J., & Lu, Z. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbv020 · Cited by 555
Self-citationThe Comparative Toxicogenomics Database: update 2017
Davis, A. P., Grondin, C. J., Johnson, R. J., Sciaky, D., King, B. L., McMorran, R., Wiegers, J., Wiegers, T. C., & Mattingly, C. J. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkw838 · Cited by 544
Metadata
- Authors
- Leaman, R., Islamaj Doğan, R., & Lu, Z.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2013-08-21
- Volume
- 29
- Issue
- 22
- Pages
- 2909-2917
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2142016317
- Cited by
- 484
- References
- 43
- FWCI
- 13.99
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.