CoChat

Mining fall-related information in clinical notes: Comparison of rule-based and novel word embedding-based machine learning approaches

Open access
Topaz, Maxim, Murga, Ludmila, Gaddis, Katherine M., McDonald, Margaret V., Bar-Bachar, Ofrit, Goldberg, Yoav, Bowles, Kathryn H. · 2019 · Journal of Biomedical Informatics
Cited by 109

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Natural language processing (NLP) of health-related data is still an expertise demanding, and resource expensive process. We created a novel, open source rapid clinical text mining system called NimbleMiner. NimbleMiner combines several machine learning techniques (word embedding models and positive only labels learning) to facilitate the process in which a human rapidly performs text mining of clinical narratives, while being aided by the machine learning components. OBJECTIVE: This manuscript describes the general system architecture and user Interface and presents results of a case study aimed at classifying fall-related information (including fall history, fall prevention interventions, and fall risk) in homecare visit notes. METHODS: We extracted a corpus of homecare visit notes (n = 1,149,586) for 89,459 patients from a large US-based homecare agency. We used a gold standard testing dataset of 750 notes annotated by two human reviewers to compare the NimbleMiner's ability to classify documents regarding whether they contain fall-related information with a previously developed rule-based NLP system. RESULTS: NimbleMiner outperformed the rule-based system in almost all domains. The overall F- score was 85.8% compared to 81% by the rule based-system with the best performance for identifying general fall history (F = 89% vs. F = 85.1% rule-based), followed by fall risk (F = 87% vs. F = 78.7% rule-based), fall prevention interventions (F = 88.1% vs. F = 78.2% rule-based) and fall within 2 days of the note date (F = 83.1% vs. F = 80.6% rule-based). The rule-based system achieved slightly better performance for fall within 2 weeks of the note date (F = 81.9% vs. F = 84% rule-based). DISCUSSION & CONCLUSIONS: NimbleMiner outperformed other systems aimed at fall information classification, including our previously developed rule-based approach. These promising results indicate that clinical text mining can be implemented without the need for large labeled datasets necessary for other types of machine learning. This is critical for domains with little NLP developments, like nursing or allied health professions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jbi.2019.103103

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 17

  1. Murdoch, 2013, The inevitable application of big data to health care, JAMA, 309, 1351, 10.1001/jama.2013.393 · DOI · Paper page
  2. Topaz, 2016, Creating a natural language processing algorithm to identify wound information in narrative Clinical Notes, Int. J. Nurs. Stud. [IN Press], 10.1016/j.ijnurstu.2016.09.013 · DOI · Paper page
  3. Halpern, 2016, Electronic medical record phenotyping using the anchor and learn framework, J. Am. Med. Informatics Assoc., 23, 731, 10.1093/jamia/ocw011 · DOI · Paper page
  4. O. Levy, Y. Goldberg, Neural word embedding as implicit matrix factorization, in: NIPS'14 Proceedings of the 27th International Conference on Neural Information Processing Systems, 2014, pp. 2177–2185.
  5. T. Mikolov, G. Corrado, K. Chen, J. Dean, Efficient estimation of word representations in vector space, in: Proc. Int. Conf. Learn. Represent. (ICLR 2013), 2013, pp. 1–12. doi:https://doi.org/10.1162/153244303322533223. · DOI · Paper page
  6. Minarro-Giménez, 2014, Exploring the application of deep learning techniques on medical text corpora, Stud. Health Technol. Inform., 205, 584
  7. T. Mikolov, I. Sutskever, K. Chen, G.S. Corrado, J. Dean, Distributed representations of words and phrases and their compositionality, 2013, pp. 3111–3119. <http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality> (accessed October 2, 2017).
  8. Reker, 2015, Active-learning strategies in computer-assisted drug discovery, Drug Discov. Today., 20, 458, 10.1016/j.drudis.2014.12.004 · DOI · Paper page
  9. S. Marsland, Machine Learning: An Algorithmic Perspective (Google eBook), 2014, p. 457. doi:https://doi.org/10.1145/242224.242229. · DOI · Paper page
  10. Zecevic, 2006, Defining a fall and reasons for falling: comparisons among the views of seniors, health care providers, and the research literature, Gerontologist, 46, 367, 10.1093/geront/46.3.367 · DOI · Paper page
  11. B.F. Miller, Encyclopedia & dictionary of medicine, nursing, and allied health., Saunders, 2003. <https://www.elsevier.com/books/miller-keane-encyclopedia-and-dictionary-of-medicine-nursing-and-allied-health/miller-keane/978-0-7216-9791-8> (accessed April 12, 2018).
  12. Chapman, 2001, A simple algorithm for identifying negated findings and diseases in discharge summaries, J. Biomed. Inform., 34, 301, 10.1006/jbin.2001.1029 · DOI · Paper page
  13. Wood, 2004, Statistical inference using bootstrap confidence intervals, Significance, 1, 180, 10.1111/j.1740-9713.2004.00067.x · DOI
  14. Toyabe, 2012, Detecting inpatient falls by using natural language processing of electronic medical records, BMC Health Serv. Res., 12, 448, 10.1186/1472-6963-12-448 · DOI
  15. Shiner, 2016, Identification of inpatient falls using automated review of text-based medical records, J. Patient Saf., 1
  16. M. Kholghi, L. De Vine, L. Sitbon, G. Zuccon, A. Nguyen, The benefits of word embeddings features for active learning in clinical information extraction, 2016. http://arxiv.org/abs/1607.02810 (accessed April 12, 2018). · arXiv
  17. Zhu, 2017, Semantic relatedness and similarity of biomedical terms: examining the effects of recency, size, and section of biomedical publications on the performance of word2vec, BMC Med. Inform. Decis. Mak., 17, 95, 10.1186/s12911-017-0498-1 · DOI

Cited by

See all 109

Metadata

Authors
Topaz, M., Murga, L., Gaddis, K. M., McDonald, M. V., Bar-Bachar, O., Goldberg, Y., & Bowles, K. H.
Venue
Journal of Biomedical Informatics
Publisher
Elsevier BV
Published
2019-02
Volume
90
Pages
103103
ISSN
1532-0464 (ISSN-L)
OpenAlex
W2909322947
Cited by
109
References
3
FWCI
5.64
Type
journal article
Language
en
Open access
hybrid
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.