CoChat
Sign Up
Cited by 40Machine Learning in HealthcareBiomedical Text Mining and OntologiesNursing Diagnosis and Documentation

Mining clinical phrases from nursing notes to discover risk factors of patient deterioration

Open access
Korach, Zfania Tom, Yang, Jie, Rossetti, Sarah Collins, Cato, Kenrick D., Kang, Min-Jeoung, Knaplund, Christopher, Schnock, Kumiko O., Garcia, Jose P. +3 more·2020·International Journal of Medical Informatics
DOI 10.1016/j.ijmedinf.2019.104053PMC PMC7103062
Open this paper in CoChat — free account

Abstract

Objective Early identification and treatment of patient deterioration is crucial to improving clinical outcomes. To act, hospital rapid response (RR) teams often rely on nurses' clinical judgement typically documented narratively in the electronic health record (EHR). We developed a data-driven, unsupervised method to discover potential risk factors of RR events from nursing notes. Methods We applied multiple natural language processing methods, including language modelling, word embeddings, and two phrase mining methods (TextRank and NC-Value), to identify quality phrases that represent clinical entities from unannotated nursing notes. TextRank was used to determine the important word-sequences in each note. NC-Value was then used to globally rank the locally-important sequences across the whole corpus. We evaluated our method both on its accuracy compared to human judgement and on the ability of the mined phrases to predict a clinical outcome, RR event hazard. Results When applied to 61,740 hospital encounters with 1,067 RR events and 778,955 notes, our method achieved an average precision of 0.590 to 0.764 (when excluding numeric tokens). Time-dependent covariates Cox model using the phrases achieved a concordance index of 0.739. Clustering the phrases revealed clinical concepts significantly associated with RR event hazard. Discussion Our findings demonstrate that our minimal-annotation, unsurprised method can rapidly mine quality phrases from a large amount of nursing notes, and these identified phrases are useful for downstream tasks, such as clinical outcome predication and risk factor identification.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.ijmedinf.2019.104053

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • AI is able to interpret nursing documentation to recognize patient types and predict clinical deterioration.

    “AI is able to interpret nursing documentation to recognize patient types and predict clinical deterioration.43–47”

    From Potential uses of AI for perioperative nursing handoffs: a qualitative study (2023) · cites this paper as [44]

    The Discussion reports that phrases mined from nursing notes can be used to predict rapid-response event risk, supporting the deterioration-prediction part of the claim. The paper describes identifying phrases and risk factors, not recognizing patient types. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 33 · Cited by 40

Research works printed but not linked: 24

  1. [1]Lyons PG, Edelson DP, Churpek MM, Rapid response systems, Resuscitation 128 (2018) 191– 197, 10.1016/j.resuscitation.2018.05.013. 29777740 PMC6147149 · DOI
  2. [2]Winters BD, Weaver SJ, Pfoh ER, Yang T, Pham JC, Dy SM, Rapid-response systems as a patient safety strategy: a systematic review, Ann. Intern. Med 158 (2013) 417– 425, 10.7326/0003-4819-158-5-201303051-00009. 23460099 PMC4695999 · DOI · Paper page
  3. [4]Bellomo R, Goldsmith D, Uchino S, Buckmaster J, Hart G, Opdam H, Silvester W, Doolan L, Gutteridge G, Prospective controlled trial of effect of medical emergency team on postoperative morbidity and mortality rates, Crit. Care Med 32 (2004) 916– 921. 15071378
  4. [5]Collins SA, Cato K, Albers D, Scott K, Stetson PD, Bakken S, Vawdrey DK, Relationship between nursing documentation and patients’ mortality, Am. J. Crit. Care 22 (2013) 306– 313, 10.4037/ajcc2013426. 23817819 PMC3771321 · DOI · Paper page
  5. [6]Parr MJ, Hadfield JH, Flabouris A, Bishop G, Hillman K, The Medical Emergency Team: 12 month analysis of reasons for activation, immediate outcome and not-for-resuscitation orders, Resuscitation. 50 (2001) 39– 44. 11719127
  6. [7]Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J, Burges CJC, Bottou L, Welling M, Ghahramani Z, Weinberger KQ (Eds.), Distributed Representations of Words and Phrases and Their Compositionality, Curran Associates, Inc., 2013, pp. 3111– 3119 (Accessed September 1, 2017), http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf.
  7. [9]Frantzi K, Ananiadou S, Mima H, Automatic recognition of multi-word terms:. the C-value/NC-value method, Int J Digit Libr. 3 (2000) 115– 130, 10.1007/s007999900023. · DOI
  8. [10]Park Y, Byrd RJ, Boguraev BK, Automatic glossary extraction: beyond terminology identification, Proceedings of the 19th International Conference on Computational Linguistics - Volume 1, Association for Computational Linguistics, Stroudsburg, PA, USA, 2002, pp. 1– 7, 10.3115/1072228.1072370. · DOI
  9. [11]Zhang Z, Iria J, Brewster C, Ciravegna F, A Comparative Evaluation of Term Recognition Algorithms, LREC, 2008.
  10. [12]Chen K, Chen H-H, Extracting noun phrases from large-scale texts: a hybrid approach and its automatic evaluation, Proceedings of the 32Nd Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics, Stroudsburg, PA, USA, 1994, pp. 234– 241, 10.3115/981732.981764. · DOI
  11. [13]Punyakanok V, Roth D, The use of classifiers in sequential inference, in: Leen TK, Dietterich TG, Tresp V (Eds.), Advances in Neural Information Processing Systems 13, MIT Press, 2001, pp. 995– 1001 http://papers.nips.cc/paper/1817-the-use-of-classifiers-in-sequential-inference.pdf.
  12. [14]Xun E, Huang C, Zhou M, A unified statistical model for the identification of English baseNP, Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics, Stroudsburg, PA, USA, 2000, pp. 109– 116, 10.3115/1075218.1075233. · DOI
  13. [15]Koo TK, Carreras X, Collins M, Simple Semi-supervised Dependency Parsing, ACL, 2008.
  14. [16]Kageura K, Umino B, Methods of Automatic Term Recognition - a Review, (1996).
  15. [17]Liu W, Chung BC, Wang R, Ng J, Morlet N, A genetic algorithm enabled ensemble for unsupervised medical term extraction from clinical letters, Health Inf. Sci. Syst 3 (2015) 5, 10.1186/s13755-015-0013-y. 26664724 PMC4674942 · DOI
  16. [18]Mortazavi-Asl H Pinto UD, Mining sequential patterns by pattern-growth: the PrefixSpan approach, IEEE Trans. Knowl. Data Eng 16 (2004) 1424– 1440, 10.1109/TKDE.2004.77. · DOI
  17. [19]Jiang M, Denny JC, Tang B, Cao H, Xu H, Extracting semantic lexicons from discharge summaries using machine learning and the C-Value method, AMIA Annu. Symp. Proc 2012 (2012) 409– 416. 23304311 PMC3540581
  18. [20]Liu J, Shang J, Wang C, Ren X, Han J, Mining quality phrases from massive text corpora, Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, ACM, New York, NY, USA (2015) 1729– 1744, 10.1145/2723372.2751523. PMC4688018 26705375 · DOI
  19. [21]Liu J, Shang J, Han J, Phrase mining from massive text and its applications, Synth. Lect. Data Min. Knowl. Discov 9 (2017) 1– 89, 10.2200/S00759ED1V01Y201702DMK013. · DOI
  20. [22]Tsuruoka Y, Tateishi Y, Kim J-D, Ohta T, McNaught J, Ananiadou S, Tsujii J, Developing a robust part-of-speech tagger for biomedical text, in: Bozanis P, Houstis EN (Eds.), Advances in Informatics, Springer, Berlin Heidelberg, 2005, pp. 382– 392.
  21. [23]Bojanowski P, Grave E, Joulin A, Mikolov T, Enriching Word Vectors With Subword Information, CoRR. abs/1607.04606, (2016) http://arxiv.org/abs/1607.04606. · arXiv
  22. [24]Scheepers T, Kanoulas E, Gavves E, Improving word embedding compositionality using lexicographic definitions, Proceedings of the 2018 World Wide Web Conference, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland (2018) 1083– 1093, 10.1145/3178876.3186007. · DOI
  23. [27]Voorhees EM, Harman D, Overview of the Eighth Text REtrieval Conference (TREC-8), Trec, 1999, p. 103.
  24. [28]Uzuner Ö, South BR, Shen S, DuVall SL, 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text, J. Am. Med. Inform. Assoc 18 (2011) 552– 556, 10.1136/amiajnl-2011-000203. 21685143 PMC3168320 · DOI

Authors
Korach, Z. T., Yang, J., Rossetti, S. C., Cato, K. D., Kang, M.-J., Knaplund, C., Schnock, K. O., Garcia, J. P., Jia, H., Schwartz, J. M., & Zhou, L.
Affiliations
  • Brigham and Women's Hospital (US)
  • Columbia University (US)
Corresponding author
Korach, Zfania Tom
ORCID
Venue
International Journal of Medical Informatics
Publisher
Elsevier BV
Published
2020-03
Volume
135
Pages
104053
ISSN
1386-5056 (ISSN-L)
DOI
10.1016/j.ijmedinf.2019.104053
PMC
PMC7103062
OpenAlex
W2995776595
Cited by
40
References
33
Citations by year
2026: 4 · 2025: 5 · 2024: 8 · 2023: 4 · 2022: 11 · 2021: 7 · 2020: 1
FWCI
2.14
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Medical Informatics (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Institute of Nursing Research
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
  • Health Sciences › Nursing › Issues, ethics and legal aspects › Nursing Diagnosis and Documentation
Keywords
nursing notes, clinical deterioration, risk factors, TextRank, word embeddings, clinical entity extraction, clinical outcome prediction, electronic health records
MeSH terms
Adult, Aged, Female, Humans, Male, Middle Aged, Natural Language Processing, Nurses, Risk Factors, Data Mining, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review