Cited by 25Electronic Health Records SystemsHealthcare Technology and Patient MonitoringArtificial Intelligence in Healthcare and Education

Diffusing an Innovation: Clinician Perceptions of Continuous Predictive Analytics Monitoring in Intensive Care

Open access
Kitzmiller, Rebecca, Vaughan, Ashley, Skeeles-Worley, Angela, Keim-Malpass, Jessica, Yap, Tracey, Lindberg, Curt, Kennerly, Susan, Mitchell, Claire +4 more·2019·Applied Clinical Informatics

Abstract

Background The purpose of this article is to describe neonatal intensive care unit clinician perceptions of a continuous predictive analytics technology and how those perceptions influenced clinician adoption. Adopting and integrating new technology into care is notoriously slow and difficult; realizing expected gains remain a challenge. Methods Semistructured interviews from a cross-section of neonatal physicians (n = 14) and nurses (n = 8) from a single U.S. medical center were collected 18 months following the conclusion of the predictive monitoring technology randomized control trial. Following qualitative descriptive analysis, innovation attributes from Diffusion of Innovation Theory-guided thematic development. Results Results suggest that the combination of physical location as well as lack of integration into work flow or methods of using data in care decisionmaking may have delayed clinicians from routinely paying attention to the data. Once data were routinely collected, documented, and reported during patient rounds and patient handoffs, clinicians came to view data as another vital sign. Through clinicians' observation of senior physicians and nurses, and ongoing dialogue about data trends and patient status, clinicians learned how to integrate these data in care decision making (e.g., differential diagnosis) and came to value the technology as beneficial to care delivery. Discussion The use of newly created predictive technologies that provide early warning of illness may require implementation strategies that acknowledge the risk–benefit of treatment clinicians must balance and take advantage of existing clinician training methods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1055/s-0039-1688478

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 50

  1. [2]Seely A Newman K Herry C Monitoring variability and complexity at the bedside New York Springer 2016
  2. [3]Green G C Bradley B Bravi A Seely A JE Continuous multiorgan variability analysis to track severity of organ failure in critically ill patients J Crit Care 2013 28 05 8790 8.79E13 23726387
  3. [4]Henry K E Hager D N Pronovost P J Saria S A targeted real-time early warning score (TREWScore) for septic shock Sci Transl Med 2015 7 299 299ra122 26246167
  4. [7]Nakamura F Nakai M Prediction models - why are they used or not used? Circ J 2017 81 12 1766 1767 29129875
  5. [8]Shelov E Muthu N Wolfe H Design and implementation of a pediatric ICU acuity scoring tool as clinical decision support Appl Clin Inform 2018 9 03 576 587 30068013 PMC6070387
  6. [9]Chalmers J D Singanayagam A Akram A R Severity assessment tools for predicting mortality in hospitalised patients with community-acquired pneumonia. Systematic review and meta-analysis Thorax 2010 65 10 878 883 20729231
  7. [10]Hosein F S Bobrovitz N Berthelot S Zygun D Ghali W A Stelfox H T A systematic review of tools for predicting severe adverse events following patient discharge from intensive care units Crit Care 2013 17 03 R102 23718698 PMC4056089
  8. [11]Breslow M J Badawi O Severity scoring in the critically ill: part 1--interpretation and accuracy of outcome prediction scoring systems Chest 2012 141 01 245 252 22215834
  9. [12]van den Boogaard M Schoonhoven L Maseda E Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study Intensive Care Med 2014 40 03 361 369 24441670
  10. [13]Wassenaar A van den Boogaard M van Achterberg T Multinational development and validation of an early prediction model for delirium in ICU patients Intensive Care Med 2015 41 06 1048 1056 25894620 PMC4477716
  11. [14]Seely A J Bravi A Herry C Do heart and respiratory rate variability improve prediction of extubation outcomes in critically ill patients? Crit Care 2014 18 02 R65 24713049 PMC4057494
  12. [15]Müller-Riemenschneider F Holmberg C Rieckmann N Barriers to routine risk-score use for healthy primary care patients: survey and qualitative study Arch Intern Med 2010 170 08 719 724 20421559
  13. [17]Barber N Cornford T Klecun E Qualitative evaluation of an electronic prescribing and administration system Qual Saf Health Care 2007 16 04 271 278 17693675 PMC2464937
  14. [18]Bar-Lev S Harrison M I Negotiating time scripts during implementation of an electronic medical record Health Care Manage Rev 2006 31 01 11 17 16493268
  15. [21]Lau F Penn A Wilson D Noseworthy T Vincent D Doze S The diffusion of an evidence-based disease guidance system for managing stroke Int J Med Inform 1998 51(2-3): 107 116 9794327
  16. [22]Littlejohns P Wyatt J C Garvican L Evaluating computerised health information systems: hard lessons still to be learnt BMJ 2003 326(7394): 860 863 12702622 PMC153476
  17. [23]Koppel R Metlay J P Cohen A Role of computerized physician order entry systems in facilitating medication errors JAMA 2005 293 10 1197 1203 15755942
  18. [24]Novak L Brooks J Gadd C Anders S Lorenzi N Mediating the intersections of organizational routines during the introduction of a health IT system Eur J Inf Syst 2012 21 05 552 569 PMC3866022 24357898
  19. [26]Kappen T H Moons K G van Wolfswinkel L Kalkman C J Vergouwe Y van Klei W A Impact of risk assessments on prophylactic antiemetic prescription and the incidence of postoperative nausea and vomiting: a cluster-randomized trial Anesthesiology 2014 120 02 343 354 24105403
  20. [27]Kappen T H van Loon K Kappen M A Barriers and facilitators perceived by physicians when using prediction models in practice J Clin Epidemiol 2016 70 136 145 26399905
  21. [28]Griffin M P Moorman J R Toward the early diagnosis of neonatal sepsis and sepsis-like illness using novel heart rate analysis Pediatrics 2001 107 01 97 104 11134441
  22. [29]Moorman J R Carlo W A Kattwinkel J Mortality reduction by heart rate characteristic monitoring in very low birth weight neonates: a randomized trial J Pediatr 2011 159 06 900 906 21864846 PMC3215822
  23. [30]Sullivan B A Grice S M Lake D E Moorman J R Fairchild K D Infection and other clinical correlates of abnormal heart rate characteristics in preterm infants J Pediatr 2014 164 04 775 780 24412138 PMC3962693
  24. [31]Goel N Chakraborty M Watkins W J Banerjee S Predicting extubation outcomes-a model incorporating heart rate characteristics index J Pediatr 2018 195 53 58 29329913
  25. [32]Clark M T Vergales B D Paget-Brown A O Predictive monitoring for respiratory decompensation leading to urgent unplanned intubation in the neonatal intensive care unit Pediatr Res 2013 73 01 104 110 23138402 PMC5321074
  26. [33]Yap T L Kennerly S Corazzini K Porter K Toles M Anderson R AEvaluation of Cueing Innovation for Pressure Ulcer Prevention Using Staff Focus Groups. Paper presented at: Healthcare 2014 PMC4934592 27429278
  27. [34]Dharampal N Cameron C Dixon E Ghali W Quan M L Attitudes and beliefs about the surgical safety checklist: just another tick box? Can J Surg 2016 59 04 268 275 27454839 PMC4961490
  28. [35]Federman A D Sanchez-Munoz A Jandorf L Salmon C Wolf M S Kannry J Patient and clinician perspectives on the outpatient after-visit summary: a qualitative study to inform improvements in visit summary design J Am Med Inform Assoc 2017 24(e1): e61 e68 27497793 PMC7651937
  29. [36]Vedel I Akhlaghpour S Vaghefi I Bergman H Lapointe L Health information technologies in geriatrics and gerontology: a mixed systematic review J Am Med Inform Assoc 2013 20 06 1109 1119 23666776 PMC3822120
  30. [37]Rogers E Diffusion of Innovations. 5th ed New York Free Press 2003
  31. [38]Simunovic M Coates A Smith A Thabane L Goldsmith C H Levine M N Uptake of an innovation in surgery: observations from the cluster-randomized Quality Initiative in Rectal Cancer trial Can J Surg 2013 56 06 415 421 24284150 PMC3859785
  32. [39]Mikkelsen M E Gaieski D F Goyal M Factors associated with nonadherence to early goal-directed therapy in the ED Chest 2010 138 03 551 558 20173053 PMC2939882
  33. [40]Bunkenborg G Poulsen I Samuelson K Ladelund S Åkeson J Mandatory early warning scoring--implementation evaluated with a mixed-methods approach Appl Nurs Res 2016 29 168 176 26856510
  34. [42]Sandelowski M Whatever happened to qualitative description? Res Nurs Health 2000 23 04 334 340 10940958
  35. [44]Szabo V Strang V R Secondary analysis of qualitative data ANS Adv Nurs Sci 1997 20 02 66 74 9398940
  36. [45]Hsieh H-F Shannon S E Three approaches to qualitative content analysis Qual Health Res 2005 15 09 1277 1288 16204405
  37. [46]Miles M B Huberman A M Saldaña J Qualitative Data Analysis: A Methods Sourcebook Thousand Oaks, CA SAGE Publications, Incorporated 2013
  38. [47]Griffin M P Lake D E Bissonette E A Harrell F E Jr O'Shea T M Moorman J R Heart rate characteristics: novel physiomarkers to predict neonatal infection and death Pediatrics 2005 116 05 1070 1074 16263991
  39. [48]Ford E W McAlearney A S Phillips M T Menachemi N Rudolph B Predicting computerized physician order entry system adoption in US hospitals: can the federal mandate be met? Int J Med Inform 2008 77 08 539 545 18053762
  40. [49]Gardner-Thorpe J Love N Wrightson J Walsh S Keeling N The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational study Ann R Coll Surg Engl 2006 88 06 571 575 17059720 PMC1963767
  41. [50]Priestley G Watson W Rashidian A Introducing critical care outreach: a ward-randomised trial of phased introduction in a general hospital Intensive Care Med 2004 30 07 1398 1404 15112033
  42. [53]Greenhalgh T Robert G Macfarlane F Bate P Kyriakidou O Diffusion of innovations in service organizations: systematic review and recommendations Milbank Q 2004 82 04 581 629 15595944 PMC2690184
  43. [54]Rehr C A Wong A Seger D L Bates D W Determining inappropriate medication alerts from “inaccurate warning” overrides in the intensive care unit Appl Clin Inform 2018 9 02 268 274 29695013 PMC5916798
  44. [55]Moss T J Lake D E Calland J F Signatures of subacute potentially catastrophic illness in the ICU: model development and validation Crit Care Med 2016 44 09 1639 1648 27452809 PMC4987175
  45. [58]Cruess S R Cruess R L Steinert Y Role modelling--making the most of a powerful teaching strategy BMJ 2008 336(7646): 718 721 18369229 PMC2276302
  46. [59]Masterson Creber R M Dayan P S Kuppermann N Applying the RE-AIM framework for the evaluation of a clinical decision support tool for pediatric head trauma: a mixed-methods study Appl Clin Inform 2018 9 03 693 703 30184559 PMC6125135
  47. [60]Menachemi N Burke D E Ayers D J Factors affecting the adoption of telemedicine--a multiple adopter perspective J Med Syst 2004 28 06 617 632 15615290
  48. [62]Subbe C P Kruger M Rutherford P Gemmel L Validation of a modified Early Warning Score in medical admissions QJM 2001 94 10 521 526 11588210
  49. [63]Heitz C R Gaillard J P Blumstein H Case D Messick C Miller C D Performance of the maximum modified early warning score to predict the need for higher care utilization among admitted emergency department patients J Hosp Med 2010 5 01 E46 E52 20063289
  50. [64]Keim-Malpass J Kitzmiller R R Skeeles-Worley A Advancing continuous predictive analytics monitoring: Moving from implementation to clinical action in a learning health system Crit Care Nurs Clin North Am 2018 30 02 273 287 29724445

Cited by

Sorted by citations

Authors
Kitzmiller, R., Vaughan, A., Skeeles-Worley, A., Keim-Malpass, J., Yap, T., Lindberg, C., Kennerly, S., Mitchell, C., Tai, R., Sullivan, B., Anderson, R., & Moorman, J.
Affiliations
  • University of North Carolina at Chapel Hill (US)
  • University of Virginia (US)
  • School of Nursing, University of Virginia, Charlottesville, Virginia, United States
  • Duke University (US)
  • Billings Clinic (US)
  • East Carolina University (US)
Venue
Applied Clinical Informatics
Published
2019-03
Volume
10
Issue
02
Pages
295-306
ISSN
1869-0327 (ISSN-L)
OpenAlex
W2943193432
Cited by
25
References
4
Citations by year
2025: 3 · 2024: 4 · 2023: 3 · 2022: 5 · 2021: 8 · 2020: 2
FWCI
3.97
Type
journal article
Language
en
Open access
Locations
  • Applied Clinical Informatics (publishedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • The Scholarship East Carolina University's Institutional Repository (East Carolina University) (submittedVersion)
Funders
  • National Institutes of Health (KL2TR001109)
  • National Center for Advancing Translational Sciences
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
predictive analytics, continuous monitoring, neonatal intensive care, clinician perceptions, technology adoption, diffusion of innovations, early warning systems, workflow integration, clinical decision-making, implementation strategies
MeSH terms
Attitude of Health Personnel, Critical Care, Heart Rate, Humans, Monitoring, Physiologic, Physicians, Inventions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review