Mining fall-related information in clinical notes: Comparison of rule-based and novel word embedding-based machine learning approaches
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Natural language processing (NLP) of health-related data is still an expertise demanding, and resource expensive process. We created a novel, open source rapid clinical text mining system called NimbleMiner. NimbleMiner combines several machine learning techniques (word embedding models and positive only labels learning) to facilitate the process in which a human rapidly performs text mining of clinical narratives, while being aided by the machine learning components. OBJECTIVE: This manuscript describes the general system architecture and user Interface and presents results of a case study aimed at classifying fall-related information (including fall history, fall prevention interventions, and fall risk) in homecare visit notes. METHODS: We extracted a corpus of homecare visit notes (n = 1,149,586) for 89,459 patients from a large US-based homecare agency. We used a gold standard testing dataset of 750 notes annotated by two human reviewers to compare the NimbleMiner's ability to classify documents regarding whether they contain fall-related information with a previously developed rule-based NLP system. RESULTS: NimbleMiner outperformed the rule-based system in almost all domains. The overall F- score was 85.8% compared to 81% by the rule based-system with the best performance for identifying general fall history (F = 89% vs. F = 85.1% rule-based), followed by fall risk (F = 87% vs. F = 78.7% rule-based), fall prevention interventions (F = 88.1% vs. F = 78.2% rule-based) and fall within 2 days of the note date (F = 83.1% vs. F = 80.6% rule-based). The rule-based system achieved slightly better performance for fall within 2 weeks of the note date (F = 81.9% vs. F = 84% rule-based). DISCUSSION & CONCLUSIONS: NimbleMiner outperformed other systems aimed at fall information classification, including our previously developed rule-based approach. These promising results indicate that clinical text mining can be implemented without the need for large labeled datasets necessary for other types of machine learning. This is critical for domains with little NLP developments, like nursing or allied health professions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Interrater reliability: The kappa statistic
McHugh, M. L. (2012) · Biochemia Medica · doi:10.11613/bm.2012.031 · Cited by 19,877
Savova, G. K., Kipper-Schuler, K. C., Hurdle, J. F., & Meystre, S. M. (2008) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638592 · Cited by 895
Hospital Readmission and Social Risk Factors Identified from Physician Notes
Navathe, A. S., Zhong, F., Lei, V. J., Chang, F. Y., Sordo, M., Topaz, M., Navathe, S. B., Rocha, R. A., & Zhou, L. (2018) · Health Services Research · doi:10.1111/1475-6773.12670 · Cited by 161
Self-citation
Research works printed but not linked: 17
- Murdoch, 2013, The inevitable application of big data to health care, JAMA, 309, 1351, 10.1001/jama.2013.393 · DOI · Paper page
- Topaz, 2016, Creating a natural language processing algorithm to identify wound information in narrative Clinical Notes, Int. J. Nurs. Stud. [IN Press], 10.1016/j.ijnurstu.2016.09.013 · DOI · Paper page
- Halpern, 2016, Electronic medical record phenotyping using the anchor and learn framework, J. Am. Med. Informatics Assoc., 23, 731, 10.1093/jamia/ocw011 · DOI · Paper page
- O. Levy, Y. Goldberg, Neural word embedding as implicit matrix factorization, in: NIPS'14 Proceedings of the 27th International Conference on Neural Information Processing Systems, 2014, pp. 2177–2185.
- T. Mikolov, G. Corrado, K. Chen, J. Dean, Efficient estimation of word representations in vector space, in: Proc. Int. Conf. Learn. Represent. (ICLR 2013), 2013, pp. 1–12. doi:https://doi.org/10.1162/153244303322533223. · DOI · Paper page
- Minarro-Giménez, 2014, Exploring the application of deep learning techniques on medical text corpora, Stud. Health Technol. Inform., 205, 584
- T. Mikolov, I. Sutskever, K. Chen, G.S. Corrado, J. Dean, Distributed representations of words and phrases and their compositionality, 2013, pp. 3111–3119. <http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality> (accessed October 2, 2017).
- Reker, 2015, Active-learning strategies in computer-assisted drug discovery, Drug Discov. Today., 20, 458, 10.1016/j.drudis.2014.12.004 · DOI · Paper page
- S. Marsland, Machine Learning: An Algorithmic Perspective (Google eBook), 2014, p. 457. doi:https://doi.org/10.1145/242224.242229. · DOI · Paper page
- Zecevic, 2006, Defining a fall and reasons for falling: comparisons among the views of seniors, health care providers, and the research literature, Gerontologist, 46, 367, 10.1093/geront/46.3.367 · DOI · Paper page
- B.F. Miller, Encyclopedia & dictionary of medicine, nursing, and allied health., Saunders, 2003. <https://www.elsevier.com/books/miller-keane-encyclopedia-and-dictionary-of-medicine-nursing-and-allied-health/miller-keane/978-0-7216-9791-8> (accessed April 12, 2018).
- Chapman, 2001, A simple algorithm for identifying negated findings and diseases in discharge summaries, J. Biomed. Inform., 34, 301, 10.1006/jbin.2001.1029 · DOI · Paper page
- Wood, 2004, Statistical inference using bootstrap confidence intervals, Significance, 1, 180, 10.1111/j.1740-9713.2004.00067.x · DOI
- Toyabe, 2012, Detecting inpatient falls by using natural language processing of electronic medical records, BMC Health Serv. Res., 12, 448, 10.1186/1472-6963-12-448 · DOI
- Shiner, 2016, Identification of inpatient falls using automated review of text-based medical records, J. Patient Saf., 1
- M. Kholghi, L. De Vine, L. Sitbon, G. Zuccon, A. Nguyen, The benefits of word embeddings features for active learning in clinical information extraction, 2016. http://arxiv.org/abs/1607.02810 (accessed April 12, 2018). · arXiv
- Zhu, 2017, Semantic relatedness and similarity of biomedical terms: examining the effects of recency, size, and section of biomedical publications on the performance of word2vec, BMC Med. Inform. Decis. Mak., 17, 95, 10.1186/s12911-017-0498-1 · DOI
Cited by
Ronquillo, C. E., Peltonen, L., Pruinelli, L., Chu, C. H., Bakken, S., Beduschi, A., Cato, K., Hardiker, N., Junger, A., Michalowski, M., et al. (2021) · Journal of Advanced Nursing · doi:10.1111/jan.14855 · Cited by 453
Self-citationArtificial Intelligence -based technologies in nursing: A scoping literature review of the evidence
von Gerich, H., Moen, H., Block, L. J., Chu, C. H., DeForest, H., Hobensack, M., Michalowski, M., Mitchell, J., Nibber, R., Olalia, M. A., et al. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2021.104153 · Cited by 360
“…et al., 2015 , Hur et al., 2019 , Ivanov et al., 2021 , North et al., 2014 , Safavi et al., 2019 , Sullivan et al., 2019 , Topaz et al., 2016 , Topaz et al., 2019b 12.9 Purposeful sampling Annapragada et al., 2021 , Alshurafa et al., 2017 , Chun et al., 2021 , Gangavarapu et al., 2019 ,…”
InfluentialSelf-citationChallenges and opportunities beyond structured data in analysis of electronic health records
Tayefi, M., Ngo, P., Chomutare, T., Dalianis, H., Salvi, E., Budrionis, A., & Godtliebsen, F. (2021) · WIREs Computational Statistics · doi:10.1002/wics.1549 · Cited by 290
Integration of artificial intelligence into nursing practice
Abuzaid, M. M., Elshami, W., & Fadden, S. M. (2022) · Health and Technology · doi:10.1007/s12553-022-00697-0 · Cited by 172
Changing role of EMS –analyses of non-conveyed and conveyed patients in Finland
Paulin, J., Kurola, J., Salanterä, S., Moen, H., Guragain, N., Koivisto, M., Käyhkö, N., Aaltonen, V., & Iirola, T. (2020) · Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine · doi:10.1186/s13049-020-00741-w · Cited by 78
“This approach is comparable to the interactive rapid vocabulary exploration used in a recent study [26].”
Result
Metadata
- Authors
- Topaz, M., Murga, L., Gaddis, K. M., McDonald, M. V., Bar-Bachar, O., Goldberg, Y., & Bowles, K. H.
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2019-02
- Volume
- 90
- Pages
- 103103
- ISSN
- 1532-0464 (ISSN-L)
- OpenAlex
- W2909322947
- Cited by
- 109
- References
- 3
- FWCI
- 5.64
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (vor, 365 d)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.