CoChat
Sign Up
Cited by 231Biomedical Text Mining and OntologiesTopic ModelingMachine Learning in Healthcare

A neural network multi-task learning approach to biomedical named entity recognition

Open access
Crichton, Gamal, Pyysalo, Sampo, Chiu, Billy, Korhonen, Anna·2017·BMC Bioinformatics
DOI 10.1186/s12859-017-1776-8PMC PMC5558737
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Named Entity Recognition (NER) is a key task in biomedical text mining. Accurate NER systems require task-specific, manually-annotated datasets, which are expensive to develop and thus limited in size. Since such datasets contain related but different information, an interesting question is whether it might be possible to use them together to improve NER performance. To investigate this, we develop supervised, multi-task, convolutional neural network models and apply them to a large number of varied existing biomedical named entity datasets. Additionally, we investigated the effect of dataset size on performance in both single- and multi-task settings. RESULTS: We present a single-task model for NER, a Multi-output multi-task model and a Dependent multi-task model. We apply the three models to 15 biomedical datasets containing multiple named entities including Anatomy, Chemical, Disease, Gene/Protein and Species. Each dataset represent a task. The results from the single-task model and the multi-task models are then compared for evidence of benefits from Multi-task Learning. With the Multi-output multi-task model we observed an average F-score improvement of 0.8% when compared to the single-task model from an average baseline of 78.4%. Although there was a significant drop in performance on one dataset, performance improves significantly for five datasets by up to 6.3%. For the Dependent multi-task model we observed an average improvement of 0.4% when compared to the single-task model. There were no significant drops in performance on any dataset, and performance improves significantly for six datasets by up to 1.1%. The dataset size experiments found that as dataset size decreased, the multi-output model's performance increased compared to the single-task model's. Using 50, 25 and 10% of the training data resulted in an average drop of approximately 3.4, 8 and 16.7% respectively for the single-task model but approximately 0.2, 3.0 and 9.8% for the multi-task model. CONCLUSIONS: Our results show that, on average, the multi-task models produced better NER results than the single-task models trained on a single NER dataset. We also found that Multi-task Learning is beneficial for small datasets. Across the various settings the improvements are significant, demonstrating the benefit of Multi-task Learning for this task.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12859-017-1776-8

Cites

71 of 71 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Machine learning methods, particularly deep learning, typically employ neural networks and first-generation transformer-based large language models (e.g., BERT).

    “Machine learning methods11–14, particularly deep learning, typically employ neural networks and the first generation of transformer-based large language models (e.g., BERT).”

    From A critical assessment of using ChatGPT for extracting structured data from clinical notes (2024) · cites this paper as [11]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. [5]Wei CH Peng Y Leaman R Davis AP Mattingly CJ Li J Wiegers TC Lu Z Overview of the Biocreative V chemical disease relation (CDR) task Proceedings of the BioCreative 5 Workshop 2015 Sevilla The Fifth BioCreative challenge evaluation workshop 2015
  2. [36]Leaman R Miller C Gonzalez G Enabling recognition of diseases in biomedical text with machine learning: corpus and benchmark Proceedings of the 2009 Symposium on Languages in Biology and Medicine, vol. 82 2009 Jeju Island The 3rd International Symposium on Languages in Biology and Medicine
  3. [39]Ando RK Biocreative ii gene mention tagging system at ibm watson Proceedings of the Second BioCreative Challenge Evaluation Workshop, vol. 23 2007 Madrid Centro Nacional de Investigaciones Oncologicas (CNIO)
  4. [42]Li H Chen Q Chen K Tang B HITSZ CDR system for disease and chemical named entity recognition and relation extraction Proceedings of the Fifth BioCreative Challenge Evaluation Workshop 2015 Sevilla The fifth BioCreative challenge evaluation workshop 2015

Cited by

Sorted by citations

Related

Authors
Crichton, G., Pyysalo, S., Chiu, B., & Korhonen, A.
Affiliations
University of Cambridge (GB)
Corresponding author
Crichton, Gamal
ORCID
Venue
BMC Bioinformatics
Publisher
Springer Science and Business Media LLC
Published
2017-08-15
Volume
18
Issue
1
Pages
368-368
ISSN
1471-2105 (ISSN-L)
DOI
10.1186/s12859-017-1776-8
PMC
PMC5558737
OpenAlex
W2743028754
Cited by
229
References
71
Citations by year
2026: 8 · 2025: 6 · 2024: 23 · 2023: 37 · 2022: 32 · 2021: 44 · 2020: 32 · 2019: 36 · 2018: 9 · 2017: 2
FWCI
8.82
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • BMC Bioinformatics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Apollo (University of Cambridge) (submittedVersion, cc-by)
  • Apollo
Funders
  • Medical Research Council (MR/M013049/1)
  • Cambridge Commonwealth, European and International Trust
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
biomedical named entity recognition, multi-task learning, convolutional neural networks, multi-output models, dataset size effects, limited training data, biomedical text mining, chemical entity, disease entity
MeSH terms
Machine Learning, Models, Theoretical, Databases, Factual, Neural Networks, Computer, Data Mining

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review