Mobility functional status ascertainment in electronic health records using large language models
Open accessPublished version of a preprint
The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index; 3 of the 17 sources its claims cite were checked for support.
Abstract
This study explores using Large Language Models (LLMs) to extract and standardize mobility status from unstructured EHR data (i.e., clinical notes). We annotated 600 clinical notes from three healthcare institutions located in southeastern Minnesota and west-central Wisconsin, focusing on expressions of mobility and associated impairment. Leveraging the open-source Llama 3 model, we tested various prompting strategies, including zero-shot, few-shot, and task decomposition, and evaluated their performance. Our final model achieved a patient-level micro-average F1-score of 0.876 [95% CI 0.858–0.894] for Mobility Extraction and 0.897 [95% CI 0.878–0.917] for Impairment Classification. A local, deterministic setup improved trustworthiness by ensuring consistent outputs, safeguarding privacy, and demonstrating cross-institution generalizability. These findings highlight the feasibility of LLM-based solutions for extracting mobility functional status from unstructured EHR data, supporting both clinical applications and research.
Overview
CoChat Summary
TL;DR
This study used the open-source Llama 3 model with prompt engineering to extract and classify mobility functional status from 600 annotated clinical notes across three healthcare institutions. The best standalone configuration was error-informed prompt refinement, which reached patient-level micro-average F1-scores of 0.876 for Mobility Extraction and 0.897 for Impairment Classification.
Key findings
- The dataset included 600 clinical notes from 256 unique patients; the cohort had a mean age of 75.6 years (SD 13.2) and was 53.5% female (137 patients).
- Under the baseline zero-shot setup, section-level micro-average F1-scores for Mobility Extraction were 0.663, 0.638, and 0.516 for Sites 1–3; for Impairment Classification, they were 0.888, 0.857, and 0.833.
- The best configuration was Error-Informed Prompt Refinement; its section-level micro-average F1-scores for Mobility Extraction were 0.729, 0.723, and 0.608 at Sites 1–3, and for Impairment Classification were 0.832, 0.814, and 0.785.
- At the patient level, the final model achieved micro-average F1-scores of 0.876 [95% CI 0.858–0.894] for Mobility Extraction and 0.897 [95% CI 0.878–0.917] for Impairment Classification; under the paper’s relaxed evaluation, these rose to 0.962 and 0.948, respectively.
Methods
- Design: Cross-institution evaluation of an LLM-based annotation pipeline using Llama 3 with zero-shot, few-shot, error-informed prompt refinement, and task decomposition strategies.
- Data/sample: 600 notes from three institutions in southeastern Minnesota and west-central Wisconsin; after sectioning, the dataset comprised 3,810 sections (1,153 Site 1, 1,075 Site 2, 1,582 Site 3). The notes included 200 PT/OT notes each from Sites 1 and 2 and 200 unrestricted clinical notes from Site 3.
- Main analysis: Two tasks were evaluated: Mobility Extraction (Mentioned vs Not Mentioned) and Impairment Classification (Impaired vs Unimpaired). Performance was measured with precision, recall, F1-score, micro-averages, and 95% bootstrap confidence intervals, at section-, note-, and patient-levels.
Limitations
- The paper states there was no direct comparison with conventional clinical NLP systems such as cTAKES or CLAMP.
- The paper states the model was not fine-tuned on the annotated dataset; this was a deliberate choice because the dataset was relatively small and fine-tuning could overfit or be computationally expensive.
- Evident from the text: the development/prompt refinement was done on Site 1, so generalizability beyond the two validation sites is promising but still limited to the specific institutions and note types studied.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
3 Valid · 1 Not valid · 7 Not checked · 107 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Furthermore, manual chart review, the traditional method for extracting information from unstructured text, is prohibitively time-consuming and expensive for large-scale research or routine clinical use13.”
§ Mobility functional status ascertainment in electronic health records using large language models > Introduction
What the sources say
- Partly supportedcites [13] Kreimeyer, K. et al. Natural Language processing systems for capturing and standardizing unstructured clinical information: a systematic review. J. Biomedical Inf. 73, 14–29 (2017). sourceExperimental — accuracy not yet measured
The Introduction says converting unstructured clinical text is time-consuming and that reducing manual expert-review time benefits clinical practice and large-scale data processing. It does not say manual chart review is expensive or establish that it is prohibitive for large-scale research or routine clinical use. (LLM, conf=0.87)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“However, a rich source of information on patient mobility also resides within electronic health records (EHRs)11,12.”
§ Mobility functional status ascertainment in electronic health records using large language models > Introduction
What the sources say
- Not checked — no full text of the sourcecites [11] Kimia, A. A., Savova, G., Landschaft, A. & Harper, M. B. An introduction to natural Language processing: how you can get more from those electronic notes you are generating. Pediatr. Emerg. Care. 31, 536–541 (2015). source
- Supportedcites [12] Fu, S. et al. Quality assessment of functional status Documentation in EHRs across different healthcare institutions. Front. Digit. Heal. 4, 958539 (2022). source
The Results section, “Identification of instrument-assisted documentation” (page not provided), gives an EHR documentation example that records whether patients can walk, use transportation, and get in and out of bed. This supports the claim that EHRs contain information about patient mobility. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The micro-average F1-scores across all mobility classes are 0.663, 0.638, and 0.516 for Site 1, Site 2, and Site 3, respectively.”
§ Mobility functional status ascertainment in electronic health records using large language models > Results > Baseline performance with zero-shot learning
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Regarding Task 2 (Impairment Classification), the class “Walking and moving” demonstrated best performance with F1-scores ranging from 0.930 to 0.964 across three sites.”
§ Mobility functional status ascertainment in electronic health records using large language models > Results > Baseline performance with zero-shot learning
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Inference errors are a common source of false positives.”
§ Mobility functional status ascertainment in electronic health records using large language models > Results > Error analyses
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
44 more claims
Create a free account to see all 118 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 36
- It may state two separate claims as one. 44
Citations
Cites
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“LLMs, with their advanced capabilities in natural language understanding and generation, offer a powerful means of processing and interpreting complex textual data 14,15 .”
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944 · cited 1×
“LLMs, with their advanced capabilities in natural language understanding and generation, offer a powerful means of processing and interpreting complex textual data 14,15 .”
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967 · cited 1×
“A limitation of this study is the lack of a direct comparison with conventional clinical NLP systems like cTAKES 27 or CLAMP 28 .”
Self-citationToward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734
Kreimeyer, K., Foster, M., Pandey, A., Arya, N., Halford, G., Jones, S. F., Forshee, R., Walderhaug, M., & Botsis, T. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.07.012 · Cited by 616 · cited 1×
“traditional method for extracting information from unstructured text, is prohibitively time-consuming and expensive for large-scale research or routine clinical use 13 .”
Mobility of older people and their quality of life
Metz, D. H. (2000) · Transport Policy · doi:10.1016/s0967-070x(00)00004-4 · Cited by 531
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505
CLAMP – a toolkit for efficiently building customized clinical natural language processing pipelines
Soysal, E., Wang, J., Jiang, M., Wu, Y., Pakhomov, S., Liu, H., & Xu, H. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx132 · Cited by 360
Narain, P., Rubenstein, L. Z., Wieland, G. D., Rosbrook, B., Strome, L. S., Pietruszka, F., & Morley, J. E. (1988) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1988.tb04259.x · Cited by 333
Zaretsky, J., Kim, J. M., Baskharoun, S., Zhao, Y., Austrian, J., Aphinyanaphongs, Y., Gupta, R., Blecker, S. B., & Feldman, J. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.0357 · Cited by 303
Research works printed but not linked: 1
- Goel, A. et al. Llms accelerate annotation for medical information extraction. In Machine Learning for Health (ML4H) 82–100 (PMLR, 2023).
Cited by
Liu, X., Garg, M., Jeon, E., Jia, H., Crowson, C. S., Sauver, J. L. S., Pagali, S. R., & Sohn, S. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag139 · Cited by 1
Self-citationZhong, R., Liu, D., Dai, Y., Li, Z., Song, Z., Li, Z., Zhang, L., Chen, S., Su, Y., Sun, Y., et al. (2026) · Chinese Medicine · doi:10.1186/s13020-026-01512-y · Cited by 1
“Large language models (LLMs) have demonstrated substantial potential in medical question answering [4], clinical document processing [5, 6], and decision support [7].”
Metadata
- Authors
- Liu, X., Garg, M., Jia, H., Sauver, J. S., Pagali, S. R., & Sohn, S.
- Affiliations
- Mayo Clinic (US)
- Corresponding author
- Liu, Xingyi
- Venue
- Scientific Reports
- Published
- 2026-01-23
- Volume
- 16
- Issue
- 1
- Pages
- 6045-6045
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC12902038
- OpenAlex
- W7125499897
- Cited by
- 2
- References
- 21
- Citations by year
- 2026: 2
- FWCI
- 6.73
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor, 20 d)
- Locations
- Scientific Reports (publishedVersion, cc-by-nc-nd)
- Research Square (acceptedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed Central (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institutes of Health (R01 AG068007)
- Eric and Wendy Schmidt Fund for AI Research and Innovation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Keywords
- large language models, electronic health records, clinical notes, clinical text, Llama 3, prompting strategies, zero-shot learning, few-shot learning, task decomposition, clinical annotation, aging, precision medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Mobility functional status ascertainment in electronic health records using large language models. https://app.cochat.ai/papers/10.1038/s41598-026-37025-9
- The paper
- Liu, X., Garg, M., Jia, H., Sauver, J. S., Pagali, S. R., & Sohn, S. (2026). Mobility functional status ascertainment in electronic health records using large language models. Scientific Reports. https://doi.org/10.1038/s41598-026-37025-9
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.