CoChat
Sign Up
Cited by 121Biomedical Text Mining and OntologiesMachine Learning in HealthcareComputational and Text Analysis Methods

The DOI matches its Crossref record; 60 of the 61 references it cites were found in a registry or an index.

Redundancy in electronic health record corpora: analysis, impact on text mining performance and mitigation strategies

Open access
Cohen, Raphael, Elhadad, Michael, Elhadad, Noémie·2013·BMC Bioinformatics
DOI 10.1186/1471-2105-14-10PMC PMC3599108
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: The increasing availability of Electronic Health Record (EHR) data and specifically free-text patient notes presents opportunities for phenotype extraction. Text-mining methods in particular can help disease modeling by mapping named-entities mentions to terminologies and clustering semantically related terms. EHR corpora, however, exhibit specific statistical and linguistic characteristics when compared with corpora in the biomedical literature domain. We focus on copy-and-paste redundancy: clinicians typically copy and paste information from previous notes when documenting a current patient encounter. Thus, within a longitudinal patient record, one expects to observe heavy redundancy. In this paper, we ask three research questions: (i) How can redundancy be quantified in large-scale text corpora? (ii) Conventional wisdom is that larger corpora yield better results in text mining. But how does the observed EHR redundancy affect text mining? Does such redundancy introduce a bias that distorts learned models? Or does the redundancy introduce benefits by highlighting stable and important subsets of the corpus? (iii) How can one mitigate the impact of redundancy on text mining? RESULTS: We analyze a large-scale EHR corpus and quantify redundancy both in terms of word and semantic concept repetition. We observe redundancy levels of about 30% and non-standard distribution of both words and concepts. We measure the impact of redundancy on two standard text-mining applications: collocation identification and topic modeling. We compare the results of these methods on synthetic data with controlled levels of redundancy and observe significant performance variation. Finally, we compare two mitigation strategies to avoid redundancy-induced bias: (i) a baseline strategy, keeping only the last note for each patient in the corpus; (ii) removing redundant notes with an efficient fingerprinting-based algorithm. (a)For text mining, preprocessing the EHR corpus with fingerprinting yields significantly better results. CONCLUSIONS: Before applying text-mining techniques, one must pay careful attention to the structure of the analyzed corpora. While the importance of data cleaning has been known for low-level text characteristics (e.g., encoding and spelling), high-level and difficult-to-quantify corpus characteristics, such as naturally occurring redundancy, can also hurt text mining. Fingerprinting enables text-mining techniques to leverage available data in the EHR corpus, while avoiding the bias introduced by redundancy.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/1471-2105-14-10

Cites

60 of 60 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Distribution of similarity levels across pairs of same-patient informative notes in the corpus.
Figure 1 Distribution of similarity levels across pairs of same-patient informative notes in the corpus.CC BY · PubMed Central
Figure 2 Concept-distribution. Distribution of UMLS concept occurrences in corpora with different levels of redundancy. The All Notes (a) and All Informative Notes (b) corpora have inherent redundancy, while the Last Informative Note (c) corpus does not. The shapes of the distributions of concepts differ depending on the presence of redundancy in the corpus.
Figure 2 Concept-distribution. Distribution of UMLS concept occurrences in corpora with different levels of redundancy. The All Notes (a) and All Informative Notes (b) corpora have inherent redundancy, while the Last Informative Note (c) corpus does not. The shapes of the distributions of concepts differ depending on the presence of redundancy in the corpus.CC BY · PubMed Central
Figure 3 Model fit as function of number of topics on the EHR corpora.
Figure 3 Model fit as function of number of topics on the EHR corpora.CC BY · PubMed Central
Figure 4 Model fit as function of number of topics on the WSJ corpora. In (a) we compare the effect of size on LDA, bigger corpora yield better fit. In (b) we examine the effect of redundancy: the doubled/trebled corpora reduce fit slightly while the noisier WSJs5 performs almost as badly as training on the smaller WSJ-600 corpus.
Figure 4 Model fit as function of number of topics on the WSJ corpora. In (a) we compare the effect of size on LDA, bigger corpora yield better fit. In (b) we examine the effect of redundancy: the doubled/trebled corpora reduce fit slightly while the noisier WSJs5 performs almost as badly as training on the smaller WSJ-600 corpus.CC BY · PubMed Central
Figure 5 Model fit as function of number of topics. Patient notes corpora, including the “Reduced Informative” corpus.
Figure 5 Model fit as function of number of topics. Patient notes corpora, including the “Reduced Informative” corpus.CC BY · PubMed Central
Figure 6 Pseudo Code of greedy controlled redundancy sub-corpus construction algorithm.
Figure 6 Pseudo Code of greedy controlled redundancy sub-corpus construction algorithm.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 5

  1. Joshi M, Pedersen T, Maclin R Proceedings of the 2nd Indian International Conference on Artificial Intelligence (IICAI’05): 2005. A comparative study of support vector machines applied to the supervised word sense disambiguation problem in the medical domain 2005, 3449-3468.
  2. Zeng QT, Crowell J: Semantic classification of consumer health content. MEDNET Retrieved May 2008, 2006: 19.
  3. Yoshimasa Tsuruoka YT, Jin-Dong K, Tomoko O, Sophia A, Jun’ichi T Lecture Notes in Computer Science. Developing a Robust Part-of-Speech Tagger for Biomedical Text 2005.
  4. Baldridge J, Morton T, Bierner G Technical report, SourceForge. The opennlp maximum entropy package 2002.
  5. McCallum AK: Mallet: A machine learning for language toolkit. 2002.

Cited by

Sorted by citations

See all 121

Related

See all

Metadata

Authors
Cohen, R., Elhadad, M., & Elhadad, N.
Affiliations
  • Ben-Gurion University of the Negev (IL)
  • Columbia University (US)
Corresponding author
Cohen, Raphael
ORCID
Venue
BMC Bioinformatics
Publisher
Springer Science and Business Media LLC
Published
2013-01-16
Volume
14
Issue
1
Pages
10-10
ISSN
1471-2105 (ISSN-L)
DOI
10.1186/1471-2105-14-10
PMC
PMC3599108
OpenAlex
W2146408445
Cited by
121
References
61
Citations by year
2026: 3 · 2025: 6 · 2024: 10 · 2023: 6 · 2022: 13 · 2021: 7 · 2020: 10 · 2019: 10 · 2018: 9 · 2017: 8 · 2016: 10 · 2015: 18 · 2014: 8 · 2013: 2
FWCI
5.68
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Bioinformatics (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
U.S. National Library of Medicine
Topics
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Keywords
electronic health records, clinical text, collocation identification, topic modeling, fingerprinting, data cleaning, named entity recognition, phenotype extraction, longitudinal patient records
MeSH terms
Algorithms, Humans, Semantics, Health Records, Personal, Data Mining, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review