Cited by 157Nursing Diagnosis and DocumentationMachine Learning in HealthcareHealthcare Technology and Patient Monitoring

Relationship Between Nursing Documentation and Patients' Mortality

Open access
Collins, S. A., Cato, K., Albers, D., Scott, K., Stetson, P. D., Bakken, S., Vawdrey, D. K.·2013·American Journal of Critical Care

Abstract

BACKGROUND: Nurses alter their monitoring behavior as a patient's clinical condition deteriorates, often detecting and documenting subtle changes before physiological trends are apparent. It was hypothesized that a nurse's behavior of recording optional documentation (beyond what is required) reflects concern about a patient's status and that mining data from patients' electronic health records for the presence of these features could help predict patients' mortality. METHODS: Data-mining methods were used to analyze electronic nursing documentation from a 15-month period at a large, urban academic medical center. Mortality rates and the frequency of vital sign measurements (beyond required) and optional nursing comment documentation were analyzed for a random set of patients and patients who experienced a cardiac arrest during their hospitalization. Patients were stratified by age-adjusted Charlson comorbidity index. RESULTS: A total of 15,000 acute care patients and 145 cardiac arrest patients were studied. Patients who died had a mean of 0.9 to 1.5 more optional comments and 6.1 to 10 more vital signs documented within 48 hours than did patients who survived. A higher frequency of comment and vital sign documentation was also associated with a higher likelihood of cardiac arrest. Of patients who had a cardiac arrest, those with more documented comments were more likely to die. CONCLUSIONS: For the first time, nursing documentation patterns have been linked to patients' mortality. Findings were consistent with the hypothesis that some features of nursing documentation within electronic health records can be used to predict mortality. With future work, these associations could be used in real time to establish a threshold of concern indicating a risk for deterioration in a patient's condition.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.4037/ajcc2013426

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Patterns of additional data that nurses document in the EHR to highlight concerning patient changes are not explicitly evident to other members of the care team.

    “As part of surveillance, nurses document additional data in the EHR to highlight concerning patient changes, but the patterns of these additional data are not explicitly evident to other members of the care team12,13.”

    From Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial (2025) · cites this paper as [13]

    The paper’s discussion of prior work says nurses use optional EHR flowsheet documentation to highlight abnormal data and record concerns during increased surveillance. Its Discussion also says important information can be difficult to find among routine notes, but it does not establish that patterns of additional documentation are not explicitly evident to other care-team members. The indexed passages provide section labels but no page numbers. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • The team has published on statistically significant signals detected in nursing data using this metadata modeling approach since 2012, demonstrating its resilience as EHR functionality and documentation policies change over time and across multiple sites.

    “Our team has published on statistically significant signals detected in nursing data using this metadata modeling approach since 2012 (refs. 13,21), demonstrating its resilience, first, as EHR functionality and documentation policies change over time and across multiple sites.”

    From Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial (2025) · cites this paper as [13]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured
  • CONCERN EWS's methodological choices, including robust modeling of temporal data patterns and use of nursing assessment and observational data, allowed it to have accurate and robust predictive power and sufficient lead time to alter a patient's clinical trajectory.

    “Several methodological choices allowed CONCERN EWS to have accurate and robust predictive power and sufficient lead time to alter a patient’s clinical trajectory as we observed in this trial, including: (1) robust modeling of temporal data patterns40–42 (for example, time of day, day of week and patient hospital day) for health care processes (for example, nurses’ surveillance and medication”

    From Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial (2025) · cites this paper as [13]

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured
  • CONCERN EWS overcomes the limitations of physiological measurements by leveraging nursing surveillance patterns and an existing expert knowledge base.

    “By leveraging nursing surveillance patterns rather than physiological measures10,44, CONCERN EWS overcomes the limitations of physiological measurements8 and leverages an existing expert knowledge base: nurses’ autonomous decisions to take greater action and document more observations or notes than mandated by policy in direct response to concerning changes in a patient’s clinical state10,13.”

    From Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial (2025) · cites this paper as [13]

    The Discussion (page not provided) says increased optional nursing documentation may signal nurses’ concern and could help predict deterioration or death. It also says further work is needed to determine whether documented concerns outperform existing risk scores. The paper does not evaluate CONCERN EWS or establish that it overcomes the limitations of physiological measurements. (LLM, conf=0.87)

    Experimental — accuracy not yet measured
  • Nurses play an important role in collecting data that might eventually be used by AI tools.

    “Nurses play an important role in collecting data that might be eventually used by AI tools, as evidenced by work that has linked the nature and patterns of nursing documentation practices with patients' mortality (Collins et al., 2013).”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. [1]Funk D Sebat F Kumar A A systems approach to the early recognition and rapid administration of best practice therapy in sepsis and septic shock Curr Opin Crit Care 2009 15 4 301 307 19561493
  2. [2]Aiken LH Clarke SP Sloane DM Hospital staffing, organization, and quality of care: cross-national findings Nurs Outlook 2002 50 5 187 194 12386653
  3. [4]Ranji SR Auerbach AD Hurd CJ O’Rourke K Shojania KG Effects of rapid response systems on clinical outcomes: systematic review and meta-analysis J Hosp Med 2007 2 6 422 432 18081187
  4. [5]Chan PS Jain R Nallmothu BK Berg R Sasson C Rapid response teams: a systematic review and meta-analysis Arch Intern Med 2010 170 1 18 26 20065195
  5. [6]Chen J Flabouris A Bellomo R Hillman K Finfer S The Medical Emergency Team System and not-for-resuscitation orders: results from the MERIT study Resuscitation 2008 79 3 391 397 18952354
  6. [7]Hodgetts TJ Kenward G Vlachonikolis IG Payne S Castle N The identification of risk factors for cardiac arrest and formulation of activation criteria to alert a medical emergency team Resuscitation 2002 54 2 125 131 12161291
  7. [8]Litvak E Pronovost PJ Rethinking rapid response teams JAMA 2010 304 12 1375 1376 20858881
  8. [9]Devita MA Bellomo R Hillman K Findings of the first consensus conference on medical emergency teams Crit Care Med 2006 34 9 2463 2478 16878033
  9. [10]Welton JM Implications of Medicare reimbursement changes related to inpatient nursing care quality J Nurs Admin 2008 38 7–8 325 330 18690122
  10. [11]Jansen JO Cuthbertson BH Detecting critical illness outside the ICU: the role of track and trigger systems Curr Opin Crit Care 2010 16 3 184 190 20305556
  11. [12]Saeed M Mark R A novel method for the efficient retrieval of similar multiparameter physiologic time series using wavelet-based symbolic representations AMIA Annu Symp Proc 2006 679 683 17238427 PMC1839671
  12. [13]Horn PS Feng L Li Y Pesce AJ Effect of outliers and non-healthy individuals on reference interval estimation Clin Chem 2001 47 12 2137 2145 11719478
  13. [14]Albers DJ Hripcsak G A statistical dynamics approach to the study of human health data: resolving population scale diurnal variation in laboratory data Phys Lett A 2010 374 9 1159 1164 20544004 PMC2882798
  14. [15]Gao H McDonnell A Harrison DA Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward Intensive Care Med 2007 33 4 667 679 17318499
  15. [16]Cioffi J Recognition of patients who require emergency assistance: a descriptive study Heart Lung 2000 29 4 262 268 10900063
  16. [17]Odell M Victor C Oliver D Nurses’ role in detecting deterioration in ward patients: systematic literature review J Adv Nurs 2009 65 10 1992 2006 20568317
  17. [18]Collins SA Vawdrey DK “Reading between the lines” of flowsheet data: nurses’ optional documentation associated with cardiac arrest outcomes Appl Nurs Res 2012 25 4 251 257 22079746 PMC3288795
  18. [19]Smith SK An analysis of the phenomenon of deterioration in the critically ill Image J Nurs Scholarsh 1988 20 1 12 15 3350553
  19. [20]Classen J Is failure to rescue really failure to communicate? Champion the move from reactive process to proactive model Nurs Manage 2010 41 7 38 41 20581600
  20. [21]Franklin C Mathew J Developing strategies to prevent inhospital cardiac arrest: analyzing responses of physicians and nurses in the hours before the event Crit Care Med 1994 22 2 244 247 8306682
  21. [22]Brown P Borowitz SM Novicoff W Information exchange in the NICU: what sources of patient data do physicians prefer to use? Int J Med Inform 2004 73 4 349 355 15135753
  22. [25]Donabedian A The quality of care: how can it be assessed? JAMA 1988 260 12 1743 1748 3045356
  23. [26]Davidoff F Heterogeneity is not always noise: lessons from improvement JAMA 2009 302 23 2580 2586 20009058
  24. [27]Extermann M Measurement and impact of comorbidity in older cancer patients Crit Rev Oncol Hematol 2000 35 3 181 200 10960800
  25. [28]Charlson ME Pompei P Ales KL MacKenzie CR A new method of classifying prognostic comorbidity in longitudinal studies: development and validation J Chronic Dis 1987 40 5 373 383 3558716
  26. [30]Ghali W Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data J Clin Epidemiol 1996 49 3 273 278 8676173
  27. [31]Charlson ME Sax FL MacKenzie CR Assessing illness severity: does clinical judgment work? J Chronic Dis 1986 39 6 439 452 3086355
  28. [32]Törnvall E Wilhelmsson S Nursing documentation for communicating and evaluating care J Clin Nurs 2008 17 16 2116 2124 18710374
  29. [33]Brown P Guerlain S Gordon P Bauer D Variations in faculty assessment of NICU flowsheet data: implications for electronic data display Int J Med Inform 2011 80 7 529 532 21511520
  30. [35]Ash JS Sittig DF Poon EG The extent and importance of unintended consequences related to computerized provider order entry J Am Med Inform Assoc 2007 14 4 415 423 17460127 PMC2244906
  31. [36]Bauer D Guerlain S Brown P The design and evaluation of a graphical display for laboratory data J Am Med Inform Assoc 2010 17 4 416 424 20595309 PMC2995657
  32. [37]Stetson PD Morrison FP Bakken S Johnson SB Preliminary development of the physician documentation quality instrument J Am Med Inform Assoc 2008 15 4 534 541 18436914 PMC2442259
  33. [38]Hripcsak G Vawdrey DK Fred MR Bostwick SB Use of electronic clinical documentation: time spent and team interactions J Am Med Inform Assoc 2011 18 2 112 117 21292706 PMC3116265

Cited by

Sorted by citations

Authors
Collins, S. A., Cato, K., Albers, D., Scott, K., Stetson, P. D., Bakken, S., & Vawdrey, D. K.
Affiliations
  • Columbia University (US)
  • NewYork–Presbyterian Hospital (US)
Venue
American Journal of Critical Care
Published
2013-07-01
Volume
22
Issue
4
Pages
306-313
ISSN
  • 1062-3264 (ISSN-L)
  • 1937-710X
OpenAlex
W2110497403
Cited by
157
References
4
Citations by year
2026: 5 · 2025: 12 · 2024: 9 · 2023: 10 · 2022: 15 · 2021: 27 · 2020: 15 · 2019: 22 · 2018: 10 · 2017: 4 · 2016: 7 · 2015: 5 · 2014: 13 · 2013: 2
FWCI
45.29
Type
journal article
Language
en
Open access
Locations
  • American Journal of Critical Care (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • National Cancer Institute (R24CA092656, U24CA092656)
  • National Institute of Nursing Research (T32NR007969)
  • U.S. National Library of Medicine (R01:LM06910, R01LM006910, T15LM007079)
Topics
  • Health Sciences › Nursing › Issues, ethics and legal aspects › Nursing Diagnosis and Documentation
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
Keywords
nursing documentation, patient mortality, electronic health records, vital signs monitoring, cardiac arrest, mortality prediction, clinical deterioration, Charlson Comorbidity Index
MeSH terms
Aged, Documentation, Female, Humans, Length of Stay, Male, Middle Aged, Monitoring, Physiologic, Nursing Staff, Hospital, Hospital Mortality, Vital Signs, Data Mining, Electronic Health Records, Patient Acuity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review