Identifying encephalopathy in patients admitted to an intensive care unit: Going beyond structured information using natural language processing
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Background: Encephalopathy is a severe co-morbid condition in critically ill patients that includes different clinical constellation of neurological symptoms. However, even for the most recognised form, delirium, this medical condition is rarely recorded in structured fields of electronic health records precluding large and unbiased retrospective studies. We aimed to identify patients with encephalopathy using a machine learning-based approach over clinical notes in electronic health records. Methods: We used a list of ICD-9 codes and clinical concepts related to encephalopathy to define a cohort of patients from the MIMIC-III dataset. Clinical notes were annotated with MedCAT and vectorized with a bag-of-word approach or word embedding using clinical concepts normalised to standard nomenclatures as features. Machine learning algorithms (support vector machines and random forest) trained with clinical notes from patients who had a diagnosis of encephalopathy (defined by ICD-9 codes) were used to classify patients with clinical concepts related to encephalopathy in their clinical notes but without any ICD-9 relevant code. A random selection of 50 patients were reviewed by a clinical expert for model validation. Results: Among 46,520 different patients, 7.5% had encephalopathy related ICD-9 codes in all their admissions (group 1, definite encephalopathy), 45% clinical concepts related to encephalopathy only in their clinical notes (group 2, possible encephalopathy) and 38% did not have encephalopathy related concepts neither in structured nor in clinical notes (group 3, non-encephalopathy). Length of stay, mortality rate or number of co-morbid conditions were higher in groups 1 and 2 compared to group 3. The best model to classify patients from group 2 as patients with encephalopathy (SVM using embeddings) had F1 of 85% and predicted 31% patients from group 2 as having encephalopathy with a probability >90%. Validation on new cases found a precision ranging from 92% to 98% depending on the criteria considered. Conclusions: Natural language processing techniques can leverage relevant clinical information that might help to identify patients with under-recognised clinical disorders such as encephalopathy. In the MIMIC dataset, this approach identifies with high probability thousands of patients that did not have a formal diagnosis in the structured information of the EHR.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2022) · doi:10.1176/appi.books.9780890425787 · Cited by 23,476
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · cited 1×
“We analysed the MIMIC-III dataset (21), a freely accessible critical care database of 46,520 patients admitted between 2001 and 2012 in a single centre in the United States.”
MethodThe unified medical language system (UMLS): integrating biomedical terminology
Bodenreider, O. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh061 · Cited by 4,356 · cited 6×
“From an initial list of signs, symptoms, and ICD-9 diagnostic codes indicative of encephalopathy selected by a clinical expert, these concepts were mapped into a standardized vocabulary, Uni fi ed Medical Language System (UMLS) (22)), and further expanded to include child concepts of the terms from the initial list usi…”
MethodInfluentialEly, E. W., Inouye, S. K., Bernard, G. R., Gordon, S., Francis, J., May, L., Truman, B., Speroff, T., Gautam, S., Margolin, R., et al. (2001) · JAMA · doi:10.1001/jama.286.21.2703 · Cited by 3,210
Ely, E. W., Margolin, R., Francis, J., May, L., Truman, B., Dittus, R., Speroff, T., Gautam, S., Bernard, G. R., & Inouye, S. K. (2001) · Critical Care Medicine · doi:10.1097/00003246-200107000-00012 · Cited by 2,422
Wilson, J. E., Mart, M. F., Cunningham, C., Shehabi, Y., Girard, T. D., MacLullich, A. M. J., Slooter, A. J. C., & Ely, E. W. (2020) · Nature Reviews Disease Primers · doi:10.1038/s41572-020-00223-4 · Cited by 1,200
Boogaard, M. V. D., Pickkers, P., Slooter, A. J. C., Kuiper, M. A., Spronk, P. E., Voort, P. H. J. V. D., Hoeven, J. G. V. D., Donders, R., Achterberg, T. V., & Schoonhoven, L. (2012) · BMJ · doi:10.1136/bmj.e420 · Cited by 447
Updated nomenclature of delirium and acute encephalopathy: statement of ten Societies
Slooter, A. J. C., Otte, W. M., Devlin, J. W., Arora, R. C., Bleck, T. P., Claassen, J., Duprey, M. S., Ely, E. W., Kaplan, P. W., Latronico, N., et al. (2020) · Intensive Care Medicine · doi:10.1007/s00134-019-05907-4 · Cited by 358
McCormick, N., Lacaille, D., Bhole, V., & Avina-Zubieta, J. A. (2014) · PLoS ONE · doi:10.1371/journal.pone.0104519 · Cited by 234 · cited 1×
“This con fi rms that ICD codes, that were designed for billing purposes, are not a good criteria for cohort de fi nition used in retrospective research studies (27, 28).”
BackgroundKraljevic, Z., Searle, T., Shek, A., Roguski, L., Noor, K., Bean, D., Mascio, A., Zhu, L., Folarin, A. A., Roberts, A., et al. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102083 · Cited by 220
Self-citationClinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data
Beam, A. L., Kompa, B., Schmaltz, A., Fried, I., Weber, G., Palmer, N., Shi, X., Cai, T., & Kohane, I. S. (2019) · Biocomputing 2020 · doi:10.1142/9789811215636_0027 · Cited by 176 · cited 2×
“Here, we used cui2vec, an embedding model pre-trained on a large collection of multimodal medical data (26).”
BackgroundMethodWong, A., Young, A. T., Liang, A. S., Gonzales, R., Douglas, V. C., & Hadley, D. (2018) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.1018 · Cited by 152 · cited 1×
“In fact, previous studies have been focused on detecting and predicting either delirium (3, 4), a syndrome with well-characterized clinical criteria (5) and specific screening tools (6), or coma, the most severe form of encephalopathy.”
MethodAccuracy and Completeness of Clinical Coding Using ICD-10 for Ambulatory Visits.
Horsky, J., Drucker, E. A., & Ramelson, H. Z. (2017) · PubMed Central · Cited by 138
Bishara, A. M., Chiu, C., Whitlock, E. L., Douglas, V. C., Lee, S., Butte, A. J., Leung, J. M., & Donovan, A. L. (2022) · BMC Anesthesiology · doi:10.1186/s12871-021-01543-y · Cited by 129
Evaluation of algorithms to identify delirium in administrative claims and drug utilization database
Kim, D. H., Lee, J., Kim, C. A., Huybrechts, K. F., Bateman, B. T., Patorno, E., & Marcantonio, E. R. (2017) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4226 · Cited by 128
Natural language processing in clinical neuroscience and psychiatry: A review
Crema, C., Attardi, G., Sartiano, D., & Redolfi, A. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.946387 · Cited by 89 · cited 1×
“…text (not only those related to the delirium semiology) and a more complex language model representation to capture semantic similarities (word embeddings instead of string matching) (19, 20) could yield better results to detect patients with different phenotypes of encephalopathy besides delirium.”
ResultPrediction of Incident Delirium Using a Random Forest classifier
Corradi, J., Thompson, S., Mather, J. F., Waszynski, C. M., & Dicks, R. S. (2018) · Journal of Medical Systems · doi:10.1007/s10916-018-1109-0 · Cited by 80 · cited 1×
“used a Random Forest model based on demographic data, comorbidities, medications, procedures, and physiological measures to predict ED (15), and Racine et al.”
MethodThe Language of Delirium: Keywords for Identifying Delirium from Medical Records
Puelle, M. R., Kosar, C. M., Xu, G., Schmitt, E. M., Jones, R. N., Marcantonio, E. R., Cooper, Z. R., Inouye, S. K., & Saczynski, J. S. (2015) · Journal of Gerontological Nursing · doi:10.3928/00989134-20150723-01 · Cited by 79
Bulic, D., Bennett, M. H., Georgousopoulou, E. N., Shehabi, Y., Pham, T. O., Looi, J. C., & van Haren, F. M. P. (2020) · Annals of Intensive Care · doi:10.1186/s13613-020-00723-2 · Cited by 76
van Walraven, C. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyx253 · Cited by 68 · cited 1×
“This con fi rms that ICD codes, that were designed for billing purposes, are not a good criteria for cohort de fi nition used in retrospective research studies (27, 28).”
BackgroundRacine, A. M., Tommet, D., D’Aquila, M., Fong, T. G., Gou, Y., Tabloski, P. A., Metzger, E. D., Hshieh, T. T., Schmitt, E. M., Vasunilashorn, S. M., et al. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-06238-7 · Cited by 67 · cited 1×
“showed that machine learning methods can be used to identify patients at high risk of developing delirium after surgery (16).”
BackgroundClinical Knowledge Graph Integrates Proteomics Data into Clinical Decision-Making
Santos, A., Colaço, A. R., Nielsen, A. B., Niu, L., Geyer, P. E., Coscia, F., Albrechtsen, N. J. W., Mundt, F., Jensen, L. J., & Mann, M. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.05.09.084897 · Cited by 58 · cited 2×
“…codes indicative of encephalopathy selected by a clinical expert, these concepts were mapped into a standardized vocabulary, Uni fi ed Medical Language System (UMLS) (22)), and further expanded to include child concepts of the terms from the initial list using Clinical Knowledge Graph (23).”
MethodYoung, M., Holmes, N. E., Kishore, K., Marhoon, N., Amjad, S., Neto, A. S., & Bellomo, R. (2022) · Intensive Care Medicine · doi:10.1007/s00134-022-06650-z · Cited by 36
Bendayan, R., Kraljević, Ž., Shaari, S., Das‐Munshi, J., Leipold, L., Chaturvedi, J., Mirza, L., Aldelemi, S., Searle, T., Chance, N., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-054414 · Cited by 33
Self-citationChen, P., Chen, K., Liao, W.-C., Lai, F., He, T.-L., Lin, S.-C., Chen, W.-J., Yang, C.-Y., Lin, Y., Tsai, I.-C., et al. (2022) · JMIR Medical Informatics · doi:10.2196/37557 · Cited by 20 · cited 1×
“…text (not only those related to the delirium semiology) and a more complex language model representation to capture semantic similarities (word embeddings instead of string matching) (19, 20) could yield better results to detect patients with different phenotypes of encephalopathy besides delirium.”
ResultA novel model to label delirium in an intensive care unit from clinician actions
Coombes, C. E., Coombes, K. R., & Fareed, N. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01461-6 · Cited by 17 · cited 4×
“To address the issues of misclassi fi cations, imprecision and omissions in diagnostic codes, recent studies have exploited machine learning methodologies to detect risk of encephalopathy and delirium (ED) based on patient information recorded in electronic health records (EHRs) (10, 11, 13 – 16).”
BackgroundMethodInfluentialA machine learning approach to identifying delirium from electronic health records
Kim, J. H., Hua, M. S., Whittington, R. A., Lee, J. H., Liu, C., Ta, C., Marcantonio, E. R., Goldberg, T. E., & Weng, C. (2022) · JAMIA Open · doi:10.1093/jamiaopen/ooac042 · Cited by 8
Cited by
Mikołajewska, E., & Masiak, J. (2025) · Electronics · doi:10.3390/electronics14102024 · Cited by 18
Zarantonello, F., De Cassai, A., Pettenuzzo, T., Sella, N., Mormando, G., Bolzon, A., Fulvio, G. A., Bertoncello, C. A., & Boscolo, A. (2026) · Korean journal of anesthesiology · doi:10.4097/kja.251142 · Cited by 3
“User satisfaction was formally assessed in only 9.8% of studies [16,18,30,50,54,60,63,80,93, 88,93], implementation barriers in 17.9% [17,18,23,28,41,47,54,57 ,69,71,74,76,77,84,86,87,91,93,103,106], and workflow efficiency in 9.8% [11,38,39,41,55,60,63,80,83,91,104], indicating that algorithmic…”
InfluentialNatural Language Processing Applied to Psychiatric Clinical Notes: Scoping Review
Rao, S., Chen, X. R., Deng, G., Xie, J., Jiang, T., Li, T., Zhang, Y., & Jiang, H. (2026) · JMIR Medical Informatics · doi:10.2196/91249 · Cited by 2
AZZOUZI, I. E., CHAKER, A., Talbi, H., & Hijri, A. E. (2026) · Scholars Journal of Applied Medical Sciences · doi:10.36347/sjams.2026.v14i07.006
Metadata
- Authors
- Ariño, H., Bae, S. K., Chaturvedi, J., Wang, T., & Roberts, A.
- Affiliations
- King's College London (GB)
- Yonsei University (KR)
- National Health Service (GB)
- Corresponding author
- Wang, Tao
- Venue
- Frontiers in Digital Health
- Publisher
- Frontiers Media SA
- Published
- 2023-01-23
- Volume
- 5
- Pages
- 1085602-1085602
- ISSN
- 2673-253X (ISSN-L)
- PMC
- PMC9899891
- OpenAlex
- W4317717879
- Cited by
- 2
- References
- 27
- Citations by year
- 2026: 1 · 2025: 1
- FWCI
- 0.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Research Portal (King's College London) (submittedVersion)
- Funders
- South London and Maudsley NHS Foundation Trust
- Maudsley Charity
- UK Research and Innovation
- National Institute for Health and Care Research
- Department of Health and Social Care
- King's College London
- European Commission (100010434, 754550, HORIZON2020, ID 100010434)
- Fundación Bancaria Caixa d'Estalvis i Pensions de Barcelona (ID 100010434, 100010434)
- Health Data Research UK
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Intensive Care Unit Cognitive Disorders
- Keywords
- encephalopathy, delirium, intensive care unit, clinical notes, natural language processing, electronic health records, clinical text, MedCAT, word embeddings, ICD-9 codes, MIMIC-III, cohort identification, critical illness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.