Text mining genotype-phenotype relationships from biomedical literature for database curation and precision medicine
Open accessAbstract
The practice of precision medicine will ultimately require databases of genes and mutations for healthcare providers to reference in order to understand the clinical implications of each patient's genetic makeup. Although the highest quality databases require manual curation, text mining tools can facilitate the curation process, increasing accuracy, coverage, and productivity. However, to date there are no available text mining tools that offer high-accuracy performance for extracting such triplets from biomedical literature. In this paper we propose a high-performance machine learning approach to automate the extraction of disease-gene-variant triplets from biomedical literature. Our approach is unique because we identify the genes and protein products associated with each mutation from not just the local text content, but from a global context as well (from the Internet and from all literature in PubMed). Our approach also incorporates protein sequence validation and disease association using a novel text-mining-based machine learning approach. We extract disease-gene-variant triplets from all abstracts in PubMed related to a set of ten important diseases (breast cancer, prostate cancer, pancreatic cancer, lung cancer, acute myeloid leukemia, Alzheimer's disease, hemochromatosis, age-related macular degeneration (AMD), diabetes mellitus, and cystic fibrosis). We then evaluate our approach in two ways: (1) a direct comparison with the state of the art using benchmark datasets; (2) a validation study comparing the results of our approach with entries in a popular human-curated database (UniProt) for each of the previously mentioned diseases. In the benchmark comparison, our full approach achieves a 28% improvement in F1-measure (from 0.62 to 0.79) over the state-of-the-art results. For the validation study with UniProt Knowledgebase (KB), we present a thorough analysis of the results and errors. Across all diseases, our approach returned 272 triplets (disease-gene-variant) that overlapped with entries in UniProt and 5,384 triplets without overlap in UniProt. Analysis of the overlapping triplets and of a stratified sample of the non-overlapping triplets revealed accuracies of 93% and 80% for the respective categories (cumulative accuracy, 77%). We conclude that our process represents an important and broadly applicable improvement to the state of the art for curation of disease-gene-variant relationships.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Patient clinical and omics data, as part of health-related big data, is now more widely accessible.
“Health-based big data including linked information for patients, such as their clinical data (for example gender, age, pathological and physiological history) and omics data (such as genetics, proteomics and metabolomics) has now become more widely available (Canuel et al., 2015;Singhal et al., 2016).”
From Using machine learning approaches for multi-omics data analysis: A review (2021) · cites this paper as Singhal et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A New Initiative on Precision Medicine
Collins, F. S., & Varmus, H. (2015) · New England Journal of Medicine · doi:10.1056/nejmp1500523 · Cited by 5,202 · see citing works
UniProt: a hub for protein information
The UniProt Consortium (2015) · Nucleic Acids Research · doi:10.1093/nar/gku989 · Cited by 5,106 · see citing works
Pruitt, K. D., Tatusova, T. A., & Maglott, D. R. (2006) · Nucleic Acids Research · doi:10.1093/nar/gkl842 · Cited by 4,441 · see citing works
The Universal Protein Resource (UniProt)
Bairoch, A. (2004) · Nucleic Acids Research · doi:10.1093/nar/gki070 · Cited by 3,774 · see citing works
Patterns of somatic mutation in human cancer genomes
Greenman, C. D., Stephens, P., Smith, R., Dalgliesh, G. L., Hunter, C., Bignell, G. R., Davies, H. R., Teague, J. W., Butler, A. P., Stevens, C., et al. (2007) · Nature · doi:10.1038/nature05610 · Cited by 3,062 · see citing works
Amberger, J. S., Bocchini, C. A., Schiettecatte, F., Scott, A. F., & Hamosh, A. (2015) · Nucleic Acids Research · doi:10.1093/nar/gku1205 · Cited by 2,691 · see citing works
HGVS Recommendations for the Description of Sequence Variants: 2016 Update
den Dunnen, J. T., Dalgleish, R. W. M., Maglott, D. R., Hart, R. K., Greenblatt, M. S., McGowan‐Jordan, J., Roux, A., Smith, T. D., Antonarakis, S. E., & Taschner, P. E. M. (2016) · Human Mutation · doi:10.1002/humu.22981 · Cited by 1,662 · see citing works
Clinical assessment incorporating a personal genome
Ashley, E. A., Butte, A. J., Wheeler, M. T., Chen, R., Klein, T. E., Dewey, F. E., Dudley, J. T., Ormond, K. E., Pavlovic, A., Morgan, A. A., et al. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60452-7 · Cited by 674 · see citing works
The human gene mutation database
Cooper, D. (1998) · Nucleic Acids Research · doi:10.1093/nar/26.1.285 · Cited by 604 · see citing works
PubTator: a web-based text mining tool for assisting biocuration
Wei, C.-H., Kao, H. Y., & Lu, Z. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt441 · Cited by 567 · see citing works
Self-citationDNorm: disease name normalization with pairwise learning to rank
Leaman, R., Islamaj, R., & Lu, Z. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt474 · Cited by 484 · see citing works
Self-citationBANNER: AN EXECUTABLE SURVEY OF ADVANCES IN BIOMEDICAL NAMED ENTITY RECOGNITION
Leaman, R., & Gonzalez‐Hernandez, G. (2007) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789812776136_0062 · Cited by 482 · see citing works
Clinical Interpretation and Implications of Whole-Genome Sequencing
Dewey, F., Grove, M. E., Pan, C., Goldstein, B., Bernstein, J., Chaib, H., Merker, J., Goldfeder, R., Enns, G., David, S., et al. (2014) · Journal of the American Medical Association (JAMA) · doi:10.1001/jama.2014.1717 · Cited by 456 · see citing works
Soussi, T., Dehouche, K., & Béroud, C. (2000) · Human Mutation · doi:10.1002/(sici)1098-1004(200001)15:1 · Cited by 269 · see citing works
The Comparative Toxicogenomics Database (CTD).
Mattingly, C. J., Colby, G. T., Forrest, J. N., & Boyer, J. L. (2003) · Environmental Health Perspectives · doi:10.1289/ehp.6028 · Cited by 253 · see citing works
Manual curation is not sufficient for annotation of genomic databases
Baumgartner, W. A., Cohen, K. B., Fox, L. M., Acquaah-Mensah, G. K., & Hunter, L. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm229 · Cited by 229 · see citing works
GNormPlus: An Integrative Approach for Tagging Genes, Gene Families, and Protein Domains
Wei, C.-H., Kao, H., & Lu, Z. (2015) · BioMed Research International · doi:10.1155/2015/918710 · Cited by 208 · see citing works
Self-citationtmVar: a text mining approach for extracting sequence variants in biomedical literature
Wei, C.-H., Harris, B., Kao, H., & Lu, Z. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt156 · Cited by 168 · see citing works
Self-citationMutationFinder: a high-performance system for extracting point mutation mentions from text
Caporaso, J. G., Baumgartner, W. A., Randolph, D. A., Cohen, K. B., & Hunter, L. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm235 · Cited by 159 · see citing works
Abbreviation definition identification based on automatic precision estimates
Sohn, S., Comeau, D. C., Kim, W. B., & Wilbur, W. J. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-402 · Cited by 120 · see citing works
Biological Databases for Human Research
Zou, D., Ma, L., Yu, J. M., & Zhang, Z. (2015) · Genomics Proteomics & Bioinformatics · doi:10.1016/j.gpb.2015.01.006 · Cited by 118 · see citing works
Winnenburg, R., Wachter, T., Plake, C., Doms, A., & Schroeder, M. (2008) · Briefings in Bioinformatics · doi:10.1093/bib/bbn043 · Cited by 108 · see citing works
Doughty, E., Kertész‐Farkas, A., Bodenreider, O., Thompson, G., Adadey, A., Peterson, T., & Kann, M. G. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq667 · Cited by 105 · see citing works
Cross-species gene normalization by species inference
Wei, C.-H., & Kao, H. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-s8-s5 · Cited by 88 · see citing works
SR4GN: A Species Recognition Software Tool for Gene Normalization
Wei, C.-H., Kao, H., & Lu, Z. (2012) · PLoS ONE · doi:10.1371/journal.pone.0038460 · Cited by 87 · see citing works
Self-citationAnnotating the biomedical literature for the human variome
Verspoor, K. M., Yepes, A. J., Cavedon, L., McIntosh, T., Herten-Crabb, A., Thomas, Z., & Plazzer, J. (2013) · Database · doi:10.1093/database/bat019 · Cited by 79 · see citing works
Singhal, A., Simmons, M., & Lu, Z. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw041 · Cited by 75 · see citing works
Self-citationA SNPshot of PubMed to associate genetic variants with drugs, diseases, and adverse reactions
Hakenberg, J., Voronov, D., Nguyên, V. H., Liang, S., Anwar, S., Lumpkin, B., Leaman, R., Tari, L., & Baral, C. R. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2012.04.006 · Cited by 48 · see citing works
Genetic Variations and Diseases in UniProtKB/Swiss‐Prot: The Ins and Outs of Expert Manual Curation
Famiglietti, M. L., Estreicher, A., Gos, A., Bolleman, J. T., Géhant, S., Breuza, L., Bridge, A. J., Poux, S., Redaschi, N., Bougueleret, L., et al. (2014) · Human Mutation · doi:10.1002/humu.22594 · Cited by 45 · see citing works
Yepes, A. J., & Verspoor, K. M. (2014) · Database · doi:10.1093/database/bau003 · Cited by 45 · see citing works
Hybrid curation of gene–mutation relations combining automated extraction and crowdsourcing
Burger, J. D., Doughty, E., Khare, R., Wei, C.-H., Mishra, R., Aberdeen, J. S., Tresner‐Kirsch, D., Wellner, B., Kann, M. G., Lu, Z., et al. (2014) · Database · doi:10.1093/database/bau094 · Cited by 44 · see citing works
Self-citationLee, K., Lee, S., Park, S., et al. (2016) · Database · doi:10.1093/database/baw043 · Cited by 43 · see citing works
Algorithms and semantic infrastructure for mutation impact extraction and grounding
Laurila, J. B., Naderi, N., Witte, R., et al. (2010) · BMC Genomics · doi:10.1186/1471-2164-11-s4-s24 · Cited by 35 · see citing works
Yepes, A. J., & Verspoor, K. M. (2014) · F1000Research · doi:10.12688/f1000research.3-18.v2 · Cited by 32 · see citing works
APPLICATION OF AUTOMATIC MUTATION–GENE PAIR EXTRACTION TO DISEASES
Erdoǧmuş, M., & Sezerman, O. U. (2007) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s021972000700317x · Cited by 31 · see citing works
Associating disease-related genetic variants in intergenic regions to the genes they impact
Macintyre, G., Yepes, A. J., Ong, C. S., et al. (2014) · PeerJ · doi:10.7717/peerj.639 · Cited by 27 · see citing works
OSIRIS: a tool for retrieving literature about sequence variants
Bonis, J., Furlong, L. I., & Sanz, F. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl421 · Cited by 26 · see citing works
SimConcept: A Hybrid Approach for Simplifying Composite Named Entities in Biomedical Text
Wei, C.-H., Leaman, R., & Lu, Z. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2422651 · Cited by 24 · see citing works
Self-citationYepes, A. J., & Verspoor, K. M. (2014) · F1000Research · doi:10.12688/f1000research.3-18.v1 · Cited by 24 · see citing works
Yeniterzi, S., & Sezerman, O. U. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-s8-s2 · Cited by 22 · see citing works
Novel tools for extraction and validation of disease-related mutations applied to fabry disease
Kuipers, R. K. P., van den Bergh, T., Joosten, H., Deprez, R. H. L., Mannens, M. M. A. M., & Schaap, P. J. (2010) · Human Mutation · doi:10.1002/humu.21317 · Cited by 22 · see citing works
Hakenberg, J., Cheng, W., Thomas, P. E., Wang, Y., Uzilov, A. V., & Chen, R. (2016) · BMC Bioinformatics · doi:10.1186/s12859-015-0865-9 · Cited by 21 · see citing works
Petsko, G. A. (2002) · Genome biology · doi:10.1186/gb-2002-3-4-comment1005 · Cited by 6 · see citing works
Finding your Way Through Pneumocystis Sequences in the NCBI Gene Database
Weissenbacher‐Lang, C., Nedorost, N., & Weißenböck, H. (2014) · Journal of Eukaryotic Microbiology · doi:10.1111/jeu.12132 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Using machine learning approaches for multi-omics data analysis: A review
Reel, P. S., Reel, S., Pearson, E., Trucco, E., & Jefferson, E. (2021) · Biotechnology Advances · doi:10.1016/j.biotechadv.2021.107739 · Cited by 903 · see citing works
PubTator central: automated concept annotation for biomedical full text articles
Wei, C.-H., Allot, A., Leaman, R., & Lu, Z. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz389 · Cited by 400 · see citing works
Self-citationAI in Health: State of the Art, Challenges, and Future Directions
Wang, F., & Preininger, A. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677908 · Cited by 398 · see citing works
CancerMine: a literature-mined resource for drivers, oncogenes and tumor suppressors in cancer
Lever, J., Zhao, E. Y., Grewal, J., Jones, M. R., & Jones, S. J. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0422-y · Cited by 205 · see citing works
Recent advances in biomedical literature mining
Zhao, S., Su, C., Lu, Z., & Wang, F. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa057 · Cited by 134 · see citing works
Self-citationAMELIE speeds Mendelian diagnosis by matching patient phenotype and genotype to primary literature
Birgmeier, J., Haeussler, M., Deisseroth, C. A., Steinberg, E. H., Jagadeesh, K. A., Ratner, A. J., Guturu, H., Wenger, A. M., Diekhans, M. E., Stenson, P. D., et al. (2020) · Science Translational Medicine · doi:10.1126/scitranslmed.aau9113 · Cited by 118 · see citing works
- Authors
- Singhal, A., Simmons, M., & Lu, Z.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Lu, Zhiyong
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-11-30
- Volume
- 12
- Issue
- 11
- Pages
- e1005017
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC5130168
- OpenAlex
- W2559034752
- Cited by
- 96
- References
- 44
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 10 · 2023: 3 · 2022: 10 · 2021: 8 · 2020: 16 · 2019: 19 · 2018: 10 · 2017: 15 · 2016: 2
- FWCI
- 7.97
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- creativecommons.org/publicdomain/zero/1.0/ (vor)
- Locations
- PLoS Computational Biology (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Foundation for the National Institutes of Health
- Howard Hughes Medical Institute
- Doris Duke Charitable Foundation
- Colgate-Palmolive Company
- American Association for Dental, Oral, and Craniofacial Research
- National Institutes of Health
- U.S. National Library of Medicine
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genomics and Rare Diseases
- Keywords
- genotype-phenotype correlation, biomedical text mining, database curation, precision medicine, gene-disease association, PubMed abstracts, cancer genes, genetic variants
- MeSH terms
- Database Management Systems, Humans, Natural Language Processing, Periodicals as Topic, Genome, Human, Genetic Predisposition to Disease, Databases, Genetic, Data Mining, Precision Medicine, High-Throughput Nucleotide Sequencing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
