Diffusing an Innovation: Clinician Perceptions of Continuous Predictive Analytics Monitoring in Intensive Care
Open accessAbstract
Background The purpose of this article is to describe neonatal intensive care unit clinician perceptions of a continuous predictive analytics technology and how those perceptions influenced clinician adoption. Adopting and integrating new technology into care is notoriously slow and difficult; realizing expected gains remain a challenge. Methods Semistructured interviews from a cross-section of neonatal physicians (n = 14) and nurses (n = 8) from a single U.S. medical center were collected 18 months following the conclusion of the predictive monitoring technology randomized control trial. Following qualitative descriptive analysis, innovation attributes from Diffusion of Innovation Theory-guided thematic development. Results Results suggest that the combination of physical location as well as lack of integration into work flow or methods of using data in care decisionmaking may have delayed clinicians from routinely paying attention to the data. Once data were routinely collected, documented, and reported during patient rounds and patient handoffs, clinicians came to view data as another vital sign. Through clinicians' observation of senior physicians and nurses, and ongoing dialogue about data trends and patient status, clinicians learned how to integrate these data in care decision making (e.g., differential diagnosis) and came to value the technology as beneficial to care delivery. Discussion The use of newly created predictive technologies that provide early warning of illness may require implementation strategies that acknowledge the risk–benefit of treatment clinicians must balance and take advantage of existing clinician training methods.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The lack of adoption typically translates to low to moderate impact on clinical processes.
“As expected, the lack of adoption typically translates to low to moderate impact on clinical processes 24,51 .”
From Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing (2022) · cites this paper as [51]
The Background and Significance section links implementation difficulties and failure to use a screening tool to limited effects on care and study findings. But the paper does not establish that lack of adoption typically produces a low-to-moderate impact on clinical processes; it neither measures that impact nor characterizes its magnitude that way. Page numbers were not available in our copy. (LLM, conf=0.82)
Experimental — accuracy not yet measuredAdoption of CDS has been studied across a wide range of clinical applications.
“Adoption of CDS has been studied across a wide range of clinical applications 24,51 .”
From Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing (2022) · cites this paper as [51]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Research in clinical reasoning: past history and current trends
Norman, G. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02127.x · Cited by 1,018 · see citing works
Koppel, R., Wetterneck, T., Telles, J. L., & Karsh, B.-T. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2616 · Cited by 685 · see citing works
Evaluating the impact of prediction models: lessons learned, challenges, and recommendations
Kappen, T. H., van Klei, W. A., van Wolfswinkel, L., Kalkman, C. J., Vergouwe, Y., & Moons, K. G. M. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0033-6 · Cited by 215 · see citing works
Clinical prediction rules: a systematic review of healthcare provider opinions and preferences
Kennedy, G., & Gallego, B. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.12.003 · Cited by 39 · see citing works
Flagged first, then by citations
Research works printed but not linked: 50
- [2]Seely A Newman K Herry C Monitoring variability and complexity at the bedside New York Springer 2016
- [3]Green G C Bradley B Bravi A Seely A JE Continuous multiorgan variability analysis to track severity of organ failure in critically ill patients J Crit Care 2013 28 05 8790 8.79E13 23726387
- [4]Henry K E Hager D N Pronovost P J Saria S A targeted real-time early warning score (TREWScore) for septic shock Sci Transl Med 2015 7 299 299ra122 26246167
- [7]Nakamura F Nakai M Prediction models - why are they used or not used? Circ J 2017 81 12 1766 1767 29129875
- [8]Shelov E Muthu N Wolfe H Design and implementation of a pediatric ICU acuity scoring tool as clinical decision support Appl Clin Inform 2018 9 03 576 587 30068013 PMC6070387
- [9]Chalmers J D Singanayagam A Akram A R Severity assessment tools for predicting mortality in hospitalised patients with community-acquired pneumonia. Systematic review and meta-analysis Thorax 2010 65 10 878 883 20729231
- [10]Hosein F S Bobrovitz N Berthelot S Zygun D Ghali W A Stelfox H T A systematic review of tools for predicting severe adverse events following patient discharge from intensive care units Crit Care 2013 17 03 R102 23718698 PMC4056089
- [11]Breslow M J Badawi O Severity scoring in the critically ill: part 1--interpretation and accuracy of outcome prediction scoring systems Chest 2012 141 01 245 252 22215834
- [12]van den Boogaard M Schoonhoven L Maseda E Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study Intensive Care Med 2014 40 03 361 369 24441670
- [13]Wassenaar A van den Boogaard M van Achterberg T Multinational development and validation of an early prediction model for delirium in ICU patients Intensive Care Med 2015 41 06 1048 1056 25894620 PMC4477716
- [14]Seely A J Bravi A Herry C Do heart and respiratory rate variability improve prediction of extubation outcomes in critically ill patients? Crit Care 2014 18 02 R65 24713049 PMC4057494
- [15]Müller-Riemenschneider F Holmberg C Rieckmann N Barriers to routine risk-score use for healthy primary care patients: survey and qualitative study Arch Intern Med 2010 170 08 719 724 20421559
- [17]Barber N Cornford T Klecun E Qualitative evaluation of an electronic prescribing and administration system Qual Saf Health Care 2007 16 04 271 278 17693675 PMC2464937
- [18]Bar-Lev S Harrison M I Negotiating time scripts during implementation of an electronic medical record Health Care Manage Rev 2006 31 01 11 17 16493268
- [21]Lau F Penn A Wilson D Noseworthy T Vincent D Doze S The diffusion of an evidence-based disease guidance system for managing stroke Int J Med Inform 1998 51(2-3): 107 116 9794327
- [22]Littlejohns P Wyatt J C Garvican L Evaluating computerised health information systems: hard lessons still to be learnt BMJ 2003 326(7394): 860 863 12702622 PMC153476
- [23]Koppel R Metlay J P Cohen A Role of computerized physician order entry systems in facilitating medication errors JAMA 2005 293 10 1197 1203 15755942
- [24]Novak L Brooks J Gadd C Anders S Lorenzi N Mediating the intersections of organizational routines during the introduction of a health IT system Eur J Inf Syst 2012 21 05 552 569 PMC3866022 24357898
- [26]Kappen T H Moons K G van Wolfswinkel L Kalkman C J Vergouwe Y van Klei W A Impact of risk assessments on prophylactic antiemetic prescription and the incidence of postoperative nausea and vomiting: a cluster-randomized trial Anesthesiology 2014 120 02 343 354 24105403
- [27]Kappen T H van Loon K Kappen M A Barriers and facilitators perceived by physicians when using prediction models in practice J Clin Epidemiol 2016 70 136 145 26399905
- [28]Griffin M P Moorman J R Toward the early diagnosis of neonatal sepsis and sepsis-like illness using novel heart rate analysis Pediatrics 2001 107 01 97 104 11134441
- [29]Moorman J R Carlo W A Kattwinkel J Mortality reduction by heart rate characteristic monitoring in very low birth weight neonates: a randomized trial J Pediatr 2011 159 06 900 906 21864846 PMC3215822
- [30]Sullivan B A Grice S M Lake D E Moorman J R Fairchild K D Infection and other clinical correlates of abnormal heart rate characteristics in preterm infants J Pediatr 2014 164 04 775 780 24412138 PMC3962693
- [31]Goel N Chakraborty M Watkins W J Banerjee S Predicting extubation outcomes-a model incorporating heart rate characteristics index J Pediatr 2018 195 53 58 29329913
- [32]Clark M T Vergales B D Paget-Brown A O Predictive monitoring for respiratory decompensation leading to urgent unplanned intubation in the neonatal intensive care unit Pediatr Res 2013 73 01 104 110 23138402 PMC5321074
- [33]Yap T L Kennerly S Corazzini K Porter K Toles M Anderson R AEvaluation of Cueing Innovation for Pressure Ulcer Prevention Using Staff Focus Groups. Paper presented at: Healthcare 2014 PMC4934592 27429278
- [34]Dharampal N Cameron C Dixon E Ghali W Quan M L Attitudes and beliefs about the surgical safety checklist: just another tick box? Can J Surg 2016 59 04 268 275 27454839 PMC4961490
- [35]Federman A D Sanchez-Munoz A Jandorf L Salmon C Wolf M S Kannry J Patient and clinician perspectives on the outpatient after-visit summary: a qualitative study to inform improvements in visit summary design J Am Med Inform Assoc 2017 24(e1): e61 e68 27497793 PMC7651937
- [36]Vedel I Akhlaghpour S Vaghefi I Bergman H Lapointe L Health information technologies in geriatrics and gerontology: a mixed systematic review J Am Med Inform Assoc 2013 20 06 1109 1119 23666776 PMC3822120
- [37]Rogers E Diffusion of Innovations. 5th ed New York Free Press 2003
- [38]Simunovic M Coates A Smith A Thabane L Goldsmith C H Levine M N Uptake of an innovation in surgery: observations from the cluster-randomized Quality Initiative in Rectal Cancer trial Can J Surg 2013 56 06 415 421 24284150 PMC3859785
- [39]Mikkelsen M E Gaieski D F Goyal M Factors associated with nonadherence to early goal-directed therapy in the ED Chest 2010 138 03 551 558 20173053 PMC2939882
- [40]Bunkenborg G Poulsen I Samuelson K Ladelund S Åkeson J Mandatory early warning scoring--implementation evaluated with a mixed-methods approach Appl Nurs Res 2016 29 168 176 26856510
- [42]Sandelowski M Whatever happened to qualitative description? Res Nurs Health 2000 23 04 334 340 10940958
- [44]Szabo V Strang V R Secondary analysis of qualitative data ANS Adv Nurs Sci 1997 20 02 66 74 9398940
- [45]Hsieh H-F Shannon S E Three approaches to qualitative content analysis Qual Health Res 2005 15 09 1277 1288 16204405
- [46]Miles M B Huberman A M Saldaña J Qualitative Data Analysis: A Methods Sourcebook Thousand Oaks, CA SAGE Publications, Incorporated 2013
- [47]Griffin M P Lake D E Bissonette E A Harrell F E Jr O'Shea T M Moorman J R Heart rate characteristics: novel physiomarkers to predict neonatal infection and death Pediatrics 2005 116 05 1070 1074 16263991
- [48]Ford E W McAlearney A S Phillips M T Menachemi N Rudolph B Predicting computerized physician order entry system adoption in US hospitals: can the federal mandate be met? Int J Med Inform 2008 77 08 539 545 18053762
- [49]Gardner-Thorpe J Love N Wrightson J Walsh S Keeling N The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational study Ann R Coll Surg Engl 2006 88 06 571 575 17059720 PMC1963767
- [50]Priestley G Watson W Rashidian A Introducing critical care outreach: a ward-randomised trial of phased introduction in a general hospital Intensive Care Med 2004 30 07 1398 1404 15112033
- [53]Greenhalgh T Robert G Macfarlane F Bate P Kyriakidou O Diffusion of innovations in service organizations: systematic review and recommendations Milbank Q 2004 82 04 581 629 15595944 PMC2690184
- [54]Rehr C A Wong A Seger D L Bates D W Determining inappropriate medication alerts from “inaccurate warning” overrides in the intensive care unit Appl Clin Inform 2018 9 02 268 274 29695013 PMC5916798
- [55]Moss T J Lake D E Calland J F Signatures of subacute potentially catastrophic illness in the ICU: model development and validation Crit Care Med 2016 44 09 1639 1648 27452809 PMC4987175
- [58]Cruess S R Cruess R L Steinert Y Role modelling--making the most of a powerful teaching strategy BMJ 2008 336(7646): 718 721 18369229 PMC2276302
- [59]Masterson Creber R M Dayan P S Kuppermann N Applying the RE-AIM framework for the evaluation of a clinical decision support tool for pediatric head trauma: a mixed-methods study Appl Clin Inform 2018 9 03 693 703 30184559 PMC6125135
- [60]Menachemi N Burke D E Ayers D J Factors affecting the adoption of telemedicine--a multiple adopter perspective J Med Syst 2004 28 06 617 632 15615290
- [62]Subbe C P Kruger M Rutherford P Gemmel L Validation of a modified Early Warning Score in medical admissions QJM 2001 94 10 521 526 11588210
- [63]Heitz C R Gaillard J P Blumstein H Case D Messick C Miller C D Performance of the maximum modified early warning score to predict the need for higher care utilization among admitted emergency department patients J Hosp Med 2010 5 01 E46 E52 20063289
- [64]Keim-Malpass J Kitzmiller R R Skeeles-Worley A Advancing continuous predictive analytics monitoring: Moving from implementation to clinical action in a learning health system Crit Care Nurs Clin North Am 2018 30 02 273 287 29724445
Cited by
Sorted by citations
Lambert, S. I., Madi, M., Sopka, S., Lenes, A., Stange, H., Buszello, C.-P., & Stephan, A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00852-5 · Cited by 404 · see citing works
Henry, K. E., Adams, R., Parent, C., Soleimani, H., Sridharan, A., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., Klein, E. Y., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01895-z · Cited by 127 · see citing works
Principles for Real-World Implementation of Bedside Predictive Analytics Monitoring
Moorman, L. P. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1735183 · Cited by 26 · see citing works
Wan, Y.-K. J., Wright, M. C., McFarland, M. M., Dishman, D., Nies, M. A., Rush, A., Madaras-Kelly, K., Jeppesen, A., & Del Fiol, G. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad203 · Cited by 11 · see citing works
- Authors
- Kitzmiller, R., Vaughan, A., Skeeles-Worley, A., Keim-Malpass, J., Yap, T., Lindberg, C., Kennerly, S., Mitchell, C., Tai, R., Sullivan, B., Anderson, R., & Moorman, J.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- University of Virginia (US)
- School of Nursing, University of Virginia, Charlottesville, Virginia, United States
- Duke University (US)
- Billings Clinic (US)
- East Carolina University (US)
- ORCID
- Venue
- Applied Clinical Informatics
- Publisher
- Georg Thieme Verlag KG
- Published
- 2019-03
- Volume
- 10
- Issue
- 02
- Pages
- 295-306
- ISSN
- 1869-0327 (ISSN-L)
- PMC
- PMC6494616
- OpenAlex
- W2943193432
- Cited by
- 25
- References
- 4
- Citations by year
- 2025: 3 · 2024: 4 · 2023: 3 · 2022: 5 · 2021: 8 · 2020: 2
- FWCI
- 3.97
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Applied Clinical Informatics (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- The Scholarship East Carolina University's Institutional Repository (East Carolina University) (submittedVersion)
- Funders
- National Institutes of Health (KL2TR001109)
- National Center for Advancing Translational Sciences
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- predictive analytics, continuous monitoring, neonatal intensive care, clinician perceptions, technology adoption, diffusion of innovations, early warning systems, workflow integration, clinical decision-making, implementation strategies
- MeSH terms
- Attitude of Health Personnel, Critical Care, Heart Rate, Humans, Monitoring, Physiologic, Physicians, Inventions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review