CoChat

Clinical Context–Aware Biomedical Text Summarization Using Deep Neural Network: Model Development and Validation.

Open access
Afzal, Muhammad, Alam, Fakhare, Malik, Khalid Mahmood, Malik, Ghaus M · 2020 · Journal of Medical Internet Research

Published version of a preprint

The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.

Abstract

Background: Automatic text summarization (ATS) enables users to retrieve meaningful evidence from big data of biomedical repositories to make complex clinical decisions. Deep neural and recurrent networks outperform traditional machine-learning techniques in areas of natural language processing and computer vision; however, they are yet to be explored in the ATS domain, particularly for medical text summarization. Objective: Traditional approaches in ATS for biomedical text suffer from fundamental issues such as an inability to capture clinical context, quality of evidence, and purpose-driven selection of passages for the summary. We aimed to circumvent these limitations through achieving precise, succinct, and coherent information extraction from credible published biomedical resources, and to construct a simplified summary containing the most informative content that can offer a review particular to clinical needs. Methods: In our proposed approach, we introduce a novel framework, termed Biomed-Summarizer, that provides quality-aware Patient/Problem, Intervention, Comparison, and Outcome (PICO)-based intelligent and context-enabled summarization of biomedical text. Biomed-Summarizer integrates the prognosis quality recognition model with a clinical context–aware model to locate text sequences in the body of a biomedical article for use in the final summary. First, we developed a deep neural network binary classifier for quality recognition to acquire scientifically sound studies and filter out others. Second, we developed a bidirectional long-short term memory recurrent neural network as a clinical context–aware classifier, which was trained on semantically enriched features generated using a word-embedding tokenizer for identification of meaningful sentences representing PICO text sequences. Third, we calculated the similarity between query and PICO text sequences using Jaccard similarity with semantic enrichments, where the semantic enrichments are obtained using medical ontologies. Last, we generated a representative summary from the high-scoring PICO sequences aggregated by study type, publication credibility, and freshness score. Results: Evaluation of the prognosis quality recognition model using a large dataset of biomedical literature related to intracranial aneurysm showed an accuracy of 95.41% (2562/2686) in terms of recognizing quality articles. The clinical context–aware multiclass classifier outperformed the traditional machine-learning algorithms, including support vector machine, gradient boosted tree, linear regression, K-nearest neighbor, and naïve Bayes, by achieving 93% (16127/17341) accuracy for classifying five categories: aim, population, intervention, results, and outcome. The semantic similarity algorithm achieved a significant Pearson correlation coefficient of 0.61 (0-1 scale) on a well-known BIOSSES dataset (with 100 pair sentences) after semantic enrichment, representing an improvement of 8.9% over baseline Jaccard similarity. Finally, we found a highly positive correlation among the evaluations performed by three domain experts concerning different metrics, suggesting that the automated summarization is satisfactory. Conclusions: By employing the proposed method Biomed-Summarizer, high accuracy in ATS was achieved, enabling seamless curation of research evidence from the biomedical literature to use for clinical decision-making.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/19810

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Proposed Biomed-Summarizer architecture with four major components: data preprocessing, quality recognition, context identification, and summary construction. PQR: prognosis quality recognition; CCA: clinical context-aware; PICO: Population/Problem, Intervention, Comparison, Outcome.
Figure 1 Proposed Biomed-Summarizer architecture with four major components: data preprocessing, quality recognition, context identification, and summary construction. PQR: prognosis quality recognition; CCA: clinical context-aware; PICO: Population/Problem, Intervention, Comparison, Outcome.CC BY · PubMed Central
Figure 2 Process steps of proposed prognosis quality recognition (PQR) model training and testing.
Figure 2 Process steps of proposed prognosis quality recognition (PQR) model training and testing.CC BY · PubMed Central
Figure 3 Clinical context–aware (CCA) classifier trained on 250-dimension feature vectors, 100 nodes at the embedding layer, 100 memory units of the long short-term memory (LSTM) layer logical hidden layers, and 5 classification nodes.
Figure 3 Clinical context–aware (CCA) classifier trained on 250-dimension feature vectors, 100 nodes at the embedding layer, 100 memory units of the long short-term memory (LSTM) layer logical hidden layers, and 5 classification nodes.CC BY · PubMed Central
Figure 4 Step-by-step scenario of query execution, retrieval of documents, quality checking, clinical context-aware (CCA) classification, semantic similarity, ranking, and summary creation. A: Aim; P: Population/Patients/Problem; I: Intervention; R: Results; O: Outcome; PICO: Patient/Problem, Intervention, Comparison, Outcome.
Figure 4 Step-by-step scenario of query execution, retrieval of documents, quality checking, clinical context-aware (CCA) classification, semantic similarity, ranking, and summary creation. A: Aim; P: Population/Patients/Problem; I: Intervention; R: Results; O: Outcome; PICO: Patient/Problem, Intervention, Comparison, Outcome.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 46 references

Research works printed but not linked: 12

  1. SchulzeFNevesMEntity-Supported Summarization of Biomedical AbstractsThe COLING 2016 Organizing CommitteeProceedings of the Fifth Workshop on Building and Evaluating Resources for Biomedical Text Mining (BioTxtM2016)December 2016Osaka, Japan4049
  2. Allahyari, M, 2017, arXiv, 1
  3. Khatri, C, 2018, arxiv, 1807.08000 · arXiv
  4. MikolovTYihWZweigGLinguistic Regularities in Continuous Space Word Representations Internet201306Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language TechnologiesJune 2013Atlanta, GeorgiaAssociation for Computational Linguistics746751
  5. Moradi, M, 2019, arXiv, 1903.02861 · arXiv
  6. ZlabingerMAnderssonLHanburyAAnderssonMQuasnikVBrasseyJMedical entity corpus with pico elements and sentiment analysisEuropean Language Resources Association (ELRA)201805Eleventh International Conference on Language Resources and Evaluation (LREC 2018)2018Miyazaki, Japan
  7. Wallace, B, 2016, J Mach Learn Res, 17, 132
  8. Cao, Z, 2016, arXiv, 1604.00125 · arXiv
  9. Nallapati, R, 2016, arxiv, 1611.04230 · arXiv
  10. Cheng, J, 2016, arXiv, 1603.07252 · arXiv
  11. Loper, E, 2002, arXiv, 0205028
  12. Pedregosa, F, 2011, J Machine Learn Res, 2825

Cited by

See all 66

Metadata

Authors
Afzal, M., Alam, F., Malik, K. M., & Malik, G. M.
Affiliations
  • Oakland University (US)
  • Henry Ford Health (US)
Corresponding author
Malik, Khalid Mahmood
Venue
Journal of Medical Internet Research
Published
2020-10-23
Volume
22
Issue
10
Pages
e19810
ISSN
1438-8871 (ISSN-L)
OpenAlex
W3086767710
Cited by
65
References
46
Citations by year
2026: 11 · 2025: 8 · 2024: 16 · 2023: 14 · 2022: 12 · 2021: 4
FWCI
2.43
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Brain Aneurysm Foundation
Topics
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
biomedical text summarization, PICO framework, bidirectional LSTM, semantic enrichment, medical ontologies, Jaccard similarity, query-document matching, intracranial aneurysm, clinical decision support, word embeddings, clinical decision support systems, evidence-based medicine
MeSH terms
Humans, Reproducibility of Results, Information Storage and Retrieval, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.