Identifying Barriers to The Implementation of Communicating Narrative Concerns Entered by Registered Nurses, An Early Warning System SmartApp
Open accessAbstract
Background Nurses are at the frontline of detecting patient deterioration. We developed Communicating Narrative Concerns Entered by Registered Nurses (CONCERN), an early warning system for clinical deterioration that generates a risk prediction score utilizing nursing data. CONCERN was implemented as a randomized clinical trial at two health systems in the Northeastern United States. Following the implementation of CONCERN, our team sought to develop the CONCERN Implementation Toolkit to enable other hospital systems to adopt CONCERN. Objective The aim of this study was to identify the optimal resources needed to implement CONCERN and package these resources into the CONCERN Implementation Toolkit to enable the spread of CONCERN to other hospital sites. Methods To accomplish this aim, we conducted qualitative interviews with nurses, prescribing providers, and information technology experts in two health systems. We recruited participants from July 2022 to January 2023. We conducted thematic analysis guided by the Donabedian model. Based on the results of the thematic analysis, we updated the α version of the CONCERN Implementation Toolkit. Results There was a total of 32 participants included in our study. In total, 12 themes were identified, with four themes mapping to each domain in Donabedian's model (i.e., structure, process, and outcome). Eight new resources were added to the CONCERN Implementation Toolkit. Conclusions This study validated the α version of the CONCERN Implementation Toolkit. Future studies will focus on returning the results of the Toolkit to the hospital sites to validate the β version of the CONCERN Implementation Toolkit. As the development of early warning systems continues to increase and clinician workflows evolve, the results of this study will provide considerations for research teams interested in implementing early warning systems in the acute care setting.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Through its use, nurses and prescribing providers perceive that their situational awareness is enhanced.
“through its use, nurses and prescribing providers perceive that their situational awareness is enhanced23.”
From Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial (2025) · cites this paper as [23]
Theme 3 reports that clinicians said CONCERN increased the interdisciplinary team's situational awareness, supporting the claimed perceived benefit. The Discussion says difficulty recruiting prescribing providers limited the authors' ability to assess their perspectives, so the paper does not clearly establish that prescribing providers themselves perceived this benefit. No page number is available in our copy. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 16 · Cited by 3
Research works printed but not linked: 14
- Morgan RW, Fitzgerald JC, Weiss SL, Nadkarni VM, Sutton RM, Berg RA. Sepsis-associated in-hospital cardiac arrest: epidemiology, pathophys- iology, and potential therapies. J Crit Care 2017;40:128-135
- Song MJ, Lee DS, Choi YY, et al. Incidence of preventable cardio- pulmonary arrest in a mature part-time rapid response system: a prospective cohort study. PLoS One 2022;17(02):e0264272
- Rhee C, Jones TM, Hamad Y, et al; Centers for Disease Control and Prevention (CDC) Prevention Epicenters Program. Prevalence, underlying causes, and preventability of sepsis-associated mor- tality in US acute care hospitals. JAMA Netw Open 2019;2(02): e187571-e187571
- Schnock KO, Kang MJ, Rossetti SC, et al. Identifying nursing documentation patterns associated with patient deterioration and recovery from deterioration in critical and acute care settings. Int J Med Inform 2021;153:104525
- Rossetti SC, Dykes PC, Knaplund C, et al. The Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) clinical decision support early warning system: protocol for a cluster randomized pragmatic clinical trial. JMIR Res Protoc 2021; 10(12):e30238
- Davis MM, Howk S, Spurlock M, McGinnis PB, Cohen DJ, Fagnan LJ. A qualitative study of clinic and community member perspectives on intervention toolkits: "Unless the toolkit is used it won't help solve the problem". BMC Health Serv Res 2017;17(01):497
- Thoele K, Ferren M, Moffat L, Keen A, Newhouse R. Development and use of a toolkit to facilitate implementation of an evidence- based intervention: a descriptive case study. Implement Sci Commun 2020;1(01):86
- Dykes PC, Khasnabish S, Burns Z, et al. Development and valida- tion of a fall prevention efficiency scale. J Patient Saf 2022;18(02): 94-101
- Murray E, May C, Mair F. Development and formative evaluation of the e-Health Implementation Toolkit (e-HIT). BMC Med Inform Decis Mak 2010;10(01):61
- Donabedian A. Evaluating the quality of medical care. 1966. Milbank Q 2005;83(04):691-729
- Moorman LP. Principles for real-world implementation of bedside predictive analytics monitoring. Appl Clin Inform 2021;12(04): 888-896
- Kurtzman ET, Ghazal LV, Girouard S, et al. Nursing workforce challenges in the postpandemic world. J Nurs Regul 2022;13(02): 49-60
- McDermott C. Reimagining the preceptor role. Nurs Adm Q 2023; 47(03):227-233
- Birt L, Scott S, Cavers D, Campbell C, Walter F. Member checking: a tool to enhance trustworthiness or merely a nod to validation? Qual Health Res 2016;26(13):1802-1811
- Authors
- Hobensack, M., Withall, J., Douthit, B., Cato, K., Dykes, P., Cho, S., Lowenthal, G., Ivory, C., Yen, P.-Y., & Rossetti, S.
- Affiliations
- Icahn School of Medicine at Mount Sinai (US)
- Columbia University (US)
- Vanderbilt University (US)
- University of Pennsylvania (US)
- Brigham and Women's Hospital (US)
- Newton Wellesley Hospital (US)
- Vanderbilt University Medical Center (US)
- Washington University in St. Louis (US)
- Corresponding author
- Hobensack, Mollie
- ORCID
- Venue
- Applied Clinical Informatics
- Publisher
- Georg Thieme Verlag KG
- Published
- 2024-03
- Volume
- 15
- Issue
- 02
- Pages
- 295-305
- ISSN
- 1869-0327 (ISSN-L)
- PMC
- PMC11023711
- OpenAlex
- W4394895453
- Cited by
- 3
- References
- 16
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 1
- FWCI
- 1.76
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Applied Clinical Informatics (publishedVersion)
- PubMed Central (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Jonas Philanthropies
- National Institute of Nursing Research (T32NR007969, R01NR016941)
- National Institute on Minority Health and Health Disparities
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Keywords
- concern, early warning systems, clinical deterioration, implementation barriers, implementation toolkit, nursing data, risk scoring, Donabedian model, nurses' concerns, acute care, implementation resources
- MeSH terms
- Humans, Nurses
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
