Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review
Open accessAbstract
Objective: Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes. Materials and methods: The scoping review followed Arksey and O’Malley’s framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants—clinicians in inpatient settings; concepts—intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus. Results: Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome. Conclusions: Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users’ trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Other EWSs have influenced in-hospital mortality, sepsis or LOS, but not across the breadth of outcomes observed in this study.
“Other EWSs have influenced in-hospital mortality, sepsis or LOS, but not across the breadth of outcomes we observed4,5,26,27.”
From Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial (2025) · cites this paper as [4]
Most published studies target a particular condition or hospital setting, and only a minority of EWSs have been evaluated in randomized controlled trials.
“Most published studies target a particular condition (for example, sepsis) or hospital setting (for example, ICU)4 and only a minority of EWSs have been evaluated in randomized controlled trials4,6.”
From Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial (2025) · cites this paper as [4]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565 · see citing works
Scoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742 · see citing works
To Err is Human: Building a Safer Health System
Hendee, W. R. (2001) · Journal of Vascular and Interventional Radiology · doi:10.1016/s1051-0443(01)70072-3 · Cited by 10,122 · see citing works
Guidance for conducting systematic scoping reviews
Peters, M. D. J., Godfrey, C. M., Khalil, H., McInerney, P., Parker, D., & Soares, C. B. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000050 · Cited by 7,472 · see citing works
PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement
McGowan, J., Sampson, M., Salzwedel, D. M., Cogo, E., Foerster, V., & Lefebvre, C. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.01.021 · Cited by 5,700 · see citing works
JBI Manual for Evidence Synthesis
(2020) · JBI eBooks · doi:10.46658/jbimes-20-01 · Cited by 4,165 · see citing works
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., PRISMA-S Group, Blunt, H., Brigham, T., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-020-01542-z · Cited by 3,714 · see citing works
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., & Koffel, J. (2019) · doi:10.31219/osf.io/sfc38 · Cited by 3,696 · see citing works
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703 · see citing works
(2020) · JBI Manual for Evidence Synthesis · doi:10.46658/jbimes-20-12 · Cited by 2,545 · see citing works
Variation in Hospital Mortality Associated with Inpatient Surgery
Ghaferi, A. A., Birkmeyer, J. D., & Dimick, J. B. (2009) · New England Journal of Medicine · doi:10.1056/nejmsa0903048 · Cited by 1,467 · see citing works
Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1809 · Cited by 1,117 · see citing works
Smith, G. B., Prytherch, D. R., Meredith, P., Schmidt, P. E., & Featherstone, P. I. (2013) · Resuscitation · doi:10.1016/j.resuscitation.2012.12.016 · Cited by 1,077 · see citing works
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · see citing works
Rapid response systems: a systematic review and meta-analysis
Maharaj, R., Raffaele, I., & Wendon, J. A. (2015) · Critical Care · doi:10.1186/s13054-015-0973-y · Cited by 606 · see citing works
Gerry, S., Bonnici, T., Birks, J., Kirtley, S., Virdee, P. S., Watkinson, P., & Collins, P. G. S. (2020) · BMJ · doi:10.1136/bmj.m1501 · Cited by 398 · see citing works
Shimabukuro, D. W., Barton, C. W., Feldman, M. D., Mataraso, S. J., & Das, R. (2017) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2017-000234 · Cited by 382 · see citing works
Wilson, F. P., Shashaty, M. G. S., Testani, J. M., Aqeel, I., Borovskiy, Y., Ellenberg, S. S., Feldman, H. I., Fernández, H., Gitelman, Y., Lin, J. J., et al. (2015) · The Lancet · doi:10.1016/s0140-6736(15)60266-5 · Cited by 353 · see citing works
Automated Identification of Adults at Risk for In-Hospital Clinical Deterioration
Escobar, G. J., Liu, V. X., Schuler, A., Lawson, B., Greene, J. D., & Kipnis, P. (2020) · New England Journal of Medicine · doi:10.1056/nejmsa2001090 · Cited by 314 · see citing works
Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision
Sittig, D. F., Wright, A. T., & Middleton, B. (2016) · Yearbook of Medical Informatics · doi:10.15265/iys-2016-s034 · Cited by 276 · see citing works
Giannini, H. M., Ginestra, J. C., Chivers, C., Draugelis, M., Hanish, A., Schweickert, W. D., Fuchs, B. D., Meadows, L., Lynch, M., Donnelly, P. J., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003891 · Cited by 269 · see citing works
Strengths and limitations of early warning scores: A systematic review and narrative synthesis
Downey, C. L., Tahir, W., Randell, R., Brown, J. M., & Jayne, D. G. (2017) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2017.09.003 · Cited by 260 · see citing works
Colpaert, K. E., Hoste, E. A. J., Steurbaut, K., Benoît, D., Hoecke, S. V., Turck, F. D., & Decruyenaere, J. M. A. (2011) · Critical Care Medicine · doi:10.1097/ccm.0b013e3182387a6b · Cited by 246 · see citing works
Electronic health record alerts for acute kidney injury: multicenter, randomized clinical trial
Wilson, F. P., Martin, M., Yamamoto, Y., Partridge, C., Moreira, E., Arora, T., Biswas, A., Feldman, H., Garg, A. X., Greenberg, J. H., et al. (2021) · BMJ · doi:10.1136/bmj.m4786 · Cited by 200 · see citing works
McNeill, G., & Bryden, D. (2013) · Resuscitation · doi:10.1016/j.resuscitation.2013.08.006 · Cited by 190 · see citing works
Machine Learning–Based Early Warning Systems for Clinical Deterioration: Systematic Scoping Review
Muralitharan, S., Nelson, W., Di, S., McGillion, M. H., Devereaux, P. J., Barr, N. G., & Petch, J. (2020) · Journal of Medical Internet Research · doi:10.2196/25187 · Cited by 179 · see citing works
Drug interaction alert override rates in the Meaningful Use era: No evidence of progress
Bryant, A. D., Fletcher, G. S., & Payne, T. H. (2014) · Applied Clinical Informatics · doi:10.4338/aci-2013-12-ra-0103 · Cited by 172 · see citing works
Implementation of a real-time computerized sepsis alert in nonintensive care unit patients*
Sawyer, A. M., Deal, E. N., Labelle, A. J., Witt, C., Thiel, S. W., Heard, K., Reichley, R. M., Micek, S. T., & Kollef, M. H. (2011) · Critical Care Medicine · doi:10.1097/ccm.0b013e318205df85 · Cited by 169 · see citing works
Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study
Sandhu, S., Lin, A. L., Brajer, N., Sperling, J., Ratliff, W., Bedoya, A. D., Balu, S., O'Brien, C., & Sendak, M. P. (2020) · Journal of Medical Internet Research · doi:10.2196/22421 · Cited by 168 · see citing works
Development, implementation, and impact of an automated early warning and response system for sepsis
Umscheid, C. A., Betesh, J., VanZandbergen, C., Hanish, A., Tait, G., Mikkelsen, M. E., French, B. C., & Fuchs, B. D. (2014) · Journal of Hospital Medicine · doi:10.1002/jhm.2259 · Cited by 167 · see citing works
Minimal Impact of Implemented Early Warning Score and Best Practice Alert for Patient Deterioration*
Bedoya, A. D., Clement, M. E., Phelan, M., Steorts, R. C., O’Brien, C., & Goldstein, B. A. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003439 · Cited by 162 · see citing works
Ginestra, J. C., Giannini, H. M., Schweickert, W. D., Meadows, L., Lynch, M. J., Pavan, K., Chivers, C. J., Draugelis, M., Donnelly, P. J., Fuchs, B. D., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003803 · Cited by 159 · see citing works
McCoy, A., & Das, R. (2017) · BMJ Open Quality · doi:10.1136/bmjoq-2017-000158 · Cited by 148 · see citing works
Wright, A. T., Sittig, D. F., Ash, J. S., Feblowitz, J. C., Meltzer, S. N., McMullen, C. K., Guappone, K. P., Carpenter, J., Richardson, J. E., Simonaitis, L., et al. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000113 · Cited by 147 · see citing works
Cardiorespiratory instability before and after implementing an integrated monitoring system*
Hravnak, M., DeVita, M. A., CLONTZ, A. S., Edwards, L., Valenta, C., & Pinsky, M. R. (2010) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181fb7b1c · Cited by 136 · see citing works
Lee, T. C., Shah, N. U., Haack, A., & Baxter, S. L. (2020) · Informatics · doi:10.3390/informatics7030025 · Cited by 133 · see citing works
Clinical deterioration: A concept analysis
Padilla, R. M., & Mayo, A. M. (2017) · Journal of Clinical Nursing · doi:10.1111/jocn.14238 · Cited by 123 · see citing works
Bailey, T. C., Chen, Y., Mao, Y., Lu, C., Hackmann, G., Micek, S. T., Heard, K. M., Faulkner, K. M., & Kollef, M. H. (2013) · Journal of Hospital Medicine · doi:10.1002/jhm.2009 · Cited by 121 · see citing works
Hooper, M. H., Weavind, L., Wheeler, A. P., Martin, J. B., Gowda, S. S., Semler, M. W., Hayes, R. M., Albert, D. W., Deane, N. B., Nian, H., et al. (2012) · Critical Care Medicine · doi:10.1097/ccm.0b013e318250a887 · Cited by 121 · see citing works
Defining AMIA’s artificial intelligence principles
Solomonides, A. E., Koski, E., Atabaki, S. M., Weinberg, S., McGreevey, J. D., Kannry, J. L., Petersen, C., & Lehmann, C. U. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac006 · Cited by 121 · see citing works
Downing, N. L., Rolnick, J., Poole, S. F., Hall, E., Wessels, A. J., Heidenreich, P., & Shieh, L. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008765 · Cited by 112 · see citing works
van Galen, L. S., Struik, P. W., Driesen, B. E. J. M., Merten, H., Ludikhuize, J., van der Spoel, J. I., Kramer, M. H. H., & Nanayakkara, P. W. B. (2016) · PLoS ONE · doi:10.1371/journal.pone.0161393 · Cited by 111 · see citing works
Chua, W. L., See, M. T. A., Legido‐Quigley, H., Jones, D. A., Tee, A. K. H., & Liaw, S. Y. (2017) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzx149 · Cited by 107 · see citing works
Jones, S. L., Ashton, C. M., Kiehne, L., Gigliotti, E., Bell-Gordon, C., Disbot, M., Masud, F. N., Shirkey, B. A., & Wray, N. P. (2015) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(15)41063-3 · Cited by 106 · see citing works
The five "rights" of clinical decision support.
Campbell, R. J. (2013) · PubMed · Cited by 100 · see citing works
Kollef, M. H., Chen, Y., Heard, K. M., LaRossa, G. N., Lu, C., Martin, N. R., Martin, N., Micek, S. T., & Bailey, T. C. (2014) · Journal of Hospital Medicine · doi:10.1002/jhm.2193 · Cited by 91 · see citing works
Manaktala, S., & Claypool, S. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw056 · Cited by 89 · see citing works
An Electronic Tool for the Evaluation and Treatment of Sepsis in the ICU
Semler, M. W., Weavind, L., Hooper, M. H., Rice, T. W., Gowda, S. S., Nádas, A., Song, Y., Martin, J. B., Bernard, G. R., & Wheeler, A. P. (2015) · Critical Care Medicine · doi:10.1097/ccm.0000000000001020 · Cited by 87 · see citing works
Lagadec, D. L., & Dwyer, T. (2016) · Australian Critical Care · doi:10.1016/j.aucc.2016.10.003 · Cited by 84 · see citing works
Burdick, H., Pino, E., Gabel-Comeau, D., McCoy, A., Gu, C., Roberts, J., Le, S., Slote, J., Pellegrini, E., Green-Saxena, A., et al. (2020) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2019-100109 · Cited by 80 · see citing works
Afshar, M., Arain, E., Ye, C., Gilbert, E. R., Xie, M., Lee, J., Churpek, M. M., Durazo-Arvizú, R., Markossian, T. W., & Joyce, C. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003919 · Cited by 72 · see citing works
Kawamoto, K., & McDonald, C. J. (2020) · Annals of Internal Medicine · doi:10.7326/m19-0875 · Cited by 72 · see citing works
Central Fetal Monitoring With and Without Computer Analysis: A Randomized Controlled Trial
Nunes, I., Ayres-de-Campos, D., Ugwumadu, A., Amin, P., Banfield, P., Nicoll, A., Cunningham, S., Sousa, P., Costa-Santos, C., Bernardes, J., et al. (2017) · Obstetrics & Gynecology · doi:10.1097/aog.0000000000001799 · Cited by 71 · see citing works
Hodgson, L. E., Roderick, P. J., Venn, R. M., Yao, G., Dimitrov, B. D., & Forni, L. G. (2018) · PLoS ONE · doi:10.1371/journal.pone.0200584 · Cited by 70 · see citing works
Machine learning techniques for mortality prediction in emergency departments: a systematic review
Naemi, A., Schmidt, T., Mansourvar, M., Naghavi‐Behzad, M., Ebrahimi, A., & Wiil, U. K. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-052663 · Cited by 67 · see citing works
O’Neill, S. M., Clyne, B., Bell, M., Casey, A., Leen, B., Smith, S. M., Ryan, M., & O'Neill, M. (2021) · BMC Emergency Medicine · doi:10.1186/s12873-021-00403-9 · Cited by 66 · see citing works
Predicting Patient Deterioration: A Review of Tools in the Digital Hospital Setting
Mann, K. D., Good, N. M., Fatehi, F., Khanna, S., Campbell, V. K., Conway, R., Sullivan, C. M., Staib, A. N., Joyce, C., & Cook, D. (2021) · Journal of Medical Internet Research · doi:10.2196/28209 · Cited by 66 · see citing works
Burke, J., Downey, C. L., & Almoudaris, A. M. (2022) · PubMed · doi:10.1097/pts.0000000000000720 · Cited by 60 · see citing works
Strasberg, H. R., Rhodes, B., Fiol, G. D., Jenders, R. A., Haug, P. J., & Kawamoto, K. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab070 · Cited by 59 · see citing works
Kawamoto, K., Kukhareva, P., Weir, C. R., Flynn, M. C., Nanjo, C. J., Martin, D., Warner, P. B., Shields, D. E., Rodriguez-Loya, S., Bradshaw, R. L., et al. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab041 · Cited by 59 · see citing works
Automated detection of physiologic deterioration in hospitalized patients
Evans, R. S., Kuttler, K. G., Simpson, K. J., HOWE, S. E., Crossno, P. F., Johnson, K. V., Schreiner, M. N., Lloyd, J. F., Tettelbach, W. H., Keddington, R. K., et al. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-002816 · Cited by 58 · see citing works
Austrian, J. S., Jamin, C. T., Doty, G., & Blecker, S. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx072 · Cited by 57 · see citing works
Use of a modified early warning score system to reduce the rate of in-hospital cardiac arrest
Nishijima, I., Oyadomari, S., Maedomari, S., 當間梨沙, Igei, C., Kobata, S., Koyama, J., Tomori, R., Kawamitsu, N., Yamamoto, Y., et al. (2016) · Journal of Intensive Care · doi:10.1186/s40560-016-0134-7 · Cited by 56 · see citing works
The Five Rights of Clinical Decision Support: CDS Tools Helpful for Meeting Meaningful Use
Campbell, R. J. (2013) · Journal of AHIMA · Cited by 55 · see citing works
Ruminski, C., Clark, M., Lake, D. E., Kitzmiller, R. R., Keim‐Malpass, J., Robertson, M. P., Simons, T. R., Moorman, J. R., & Calland, J. F. (2018) · Journal of Clinical Monitoring and Computing · doi:10.1007/s10877-018-0194-4 · Cited by 54 · see citing works
Lei, N., Zhang, X., Wei, M., Lao, B., Xu, X., Zhang, M., Chen, H., Xu, Y., Xia, B., Zhang, D., et al. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01951-1 · Cited by 53 · see citing works
Implementing Machine Learning in the Electronic Health Record: Checklist of Essential Considerations
Kawamoto, K., Finkelstein, J., & Fiol, G. D. (2023) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2023.01.013 · Cited by 51 · see citing works
Wright, M. C., Borbolla, D. A., Waller, R. G., Fiol, G. D., Reese, T., Nesbitt, P., & Segall, N. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.yjbinx.2019.100041 · Cited by 48 · see citing works
Self-citationNovel displays of patient information in critical care settings: a systematic review
Waller, R. G., Wright, M. C., Segall, N., Nesbitt, P., Reese, T. S., Borbolla, D. A., & Fiol, G. D. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy193 · Cited by 43 · see citing works
Self-citationPrevention of inpatient hypoglycemia with a real‐time informatics alert
Kilpatrick, C. R., Elliott, M., Pratt, E., Schafers, S. J., Blackburn, M. C., Heard, K. M., McGill, J. B., Thoelke, M. S., & Tobin, G. S. (2014) · Journal of Hospital Medicine · doi:10.1002/jhm.2221 · Cited by 41 · see citing works
Veldhuis, L. I., Woittiez, N. J. C., Nanayakkara, P. W. B., & Ludikhuize, J. (2022) · Critical Care Explorations · doi:10.1097/cce.0000000000000744 · Cited by 41 · see citing works
Klumpner, T. T., Kountanis, J. A., Langen, E. S., Smith, R. D., & Tremper, K. K. (2018) · BMC Anesthesiology · doi:10.1186/s12871-018-0540-6 · Cited by 40 · see citing works
SEND: a system for electronic notification and documentation of vital sign observations
Wong, D., Bonnici, T., Knight, J., Morgan, L. J., Coombes, P., & Watkinson, P. (2015) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-015-0186-y · Cited by 39 · see citing works
Honeyford, K., Cooke, G., Kinderlerer, A. R., Williamson, E., Gilchrist, M. J., Holmes, A. H., Room, T. S. B., Glampson, B., Mulla, A., & Costelloe, C. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz186 · Cited by 36 · see citing works
Lin, Y. L., Guerguerian, A., Tomasi, J., Laussen, P. C., & Trbovich, P. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0520-7 · Cited by 36 · see citing works
Ju, T., Al‐Mashat, M., Rivas, L., & Sarani, B. (2018) · Critical Care Clinics · doi:10.1016/j.ccc.2017.12.004 · Cited by 35 · see citing works
Blythe, R. D., Parsons, R., White, N. M., Cook, D., & McPhail, S. (2022) · BMJ Quality & Safety · doi:10.1136/bmjqs-2021-014527 · Cited by 35 · see citing works
Dewan, M. L., Soberano, B., Sosa, T., Zackoff, M., Hagedorn, P., Brady, P. W., Chima, R. S., Stalets, E. L., Moore, L., Britto, M., et al. (2021) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000002816 · Cited by 35 · see citing works
Effect of a Real-Time Electronic Dashboard on a Rapid Response System
Fletcher, G. S., Aaronson, B. A., White, A. A., & Julka, R. (2017) · Journal of Medical Systems · doi:10.1007/s10916-017-0858-5 · Cited by 34 · see citing works
Menon, S., Tarrago, R., Carlin, K. E., Wu, H., & Yonekawa, K. E. (2020) · Pediatric Research · doi:10.1038/s41390-020-1046-8 · Cited by 31 · see citing works
Helman, S., Terry, M. A., Pellathy, T., Williams, A., Dubrawski, A., Clermont, G., Pinsky, M. R., Al-Zaiti, S., & Hravnak, M. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104643 · Cited by 31 · see citing works
Bourdeaux, C., Ghosh, E., Atallah, L., Palanisamy, K., Patel, P., Thomas, M., Gould, T., Warburton, J., Rivers, J., & Hadfield, J. (2020) · Critical Care · doi:10.1186/s13054-020-03343-1 · Cited by 28 · see citing works
Design and Implementation of a Pediatric ICU Acuity Scoring Tool as Clinical Decision Support
Shelov, E. D., Muthu, N., Wolfe, H. A., Traynor, D., Craig, N., Bonafide, C. P., Nadkarni, V., Davis, D. H., & Dewan, M. L. (2018) · Applied Clinical Informatics · doi:10.1055/s-0038-1667122 · Cited by 27 · see citing works
Amland, R. C., Lyons, J., Greene, T. L., & Haley, J. M. (2015) · JRSM Open · doi:10.1177/2054270415609004 · Cited by 26 · see citing works
Evaluation of User-Interface Alert Displays for Clinical Decision Support Systems for Sepsis
Long, D., Capan, M., Mascioli, S., Weldon, D. L. M., Arnold, R. C., & Miller, K. (2018) · Critical Care Nurse · doi:10.4037/ccn2018352 · Cited by 26 · see citing works
Kitzmiller, R., Vaughan, A., Skeeles-Worley, A., Keim-Malpass, J., Yap, T., Lindberg, C., Kennerly, S., Mitchell, C., Tai, R., Sullivan, B., et al. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1688478 · Cited by 25 · see citing works
Westra, B. L., Landman, S. R., Yadav, P., & Steinbach, M. S. (2017) · Applied Clinical Informatics · doi:10.4338/aci-2016-07-ra-0112 · Cited by 24 · see citing works
Lang, A. R., Simmonds, M., Pinchin, J., Sharples, S., Dunn, L., Clarke, S., Bennett, O., Wood, S., & Swinscoe, C. (2019) · JMIR Medical Informatics · doi:10.2196/11678 · Cited by 23 · see citing works
Automated alert and activation of medical emergency team using early warning score
Na, S. J., Ko, R., Ko, M. G., & Jeon, K. (2021) · Journal of Intensive Care · doi:10.1186/s40560-021-00588-y · Cited by 23 · see citing works
Wu, C., Kuo, C.-T., Shih, S., Chen, J.-C., Lo, Y.-C., Yu, H.-H., Huang, M.-D., Sheu, W. H., & Liu, S. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18094550 · Cited by 22 · see citing works
Howarth, M., Bhatt, M., Benterud, E., Wolska, A., Minty, E. P., Choi, K.-Y., Devrome, A. N., Harrison, T. G., Baylis, B. W., Dixon, E., et al. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01303-x · Cited by 22 · see citing works
Sooner is better: use of a real-time automated bedside dashboard improves sepsis care
Jung, A. D., Baker, J. E., Droege, C. A., Nomellini, V., Johannigman, J. A., Holcomb, J. B., Goodman, M. D., & Pritts, T. A. (2018) · Journal of Surgical Research · doi:10.1016/j.jss.2018.05.078 · Cited by 21 · see citing works
Design and Development of a Proactive Rapid Response System
Heal, M., Silvest-Guerrero, S., & Kohtz, C. (2016) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000292 · Cited by 18 · see citing works
Aakre, C. A., Kitson, J., Li, M., & Herasevich, V. (2017) · JMIR Human Factors · doi:10.2196/humanfactors.7567 · Cited by 17 · see citing works
Accuracy of Machine Learning Models to Predict In-hospital Cardiac Arrest
Moffat, L., & Xu, D. (2021) · Clinical Nurse Specialist · doi:10.1097/nur.0000000000000644 · Cited by 17 · see citing works
Dawes, T., Cheek, E., Bewick, V., Dennis, M. S., Duckitt, R., Walker, J., Forni, L. G., & Venn, R. M. (2014) · British Journal of Anaesthesia · doi:10.1093/bja/aeu107 · Cited by 17 · see citing works
Haynes, R. B., Fiol, G. D., Michelson, M., & Iorio, A. (2020) · Annals of Internal Medicine · doi:10.7326/m19-0874 · Cited by 17 · see citing works
Reduction in Time to First Action as a Result of Electronic Alerts for Early Sepsis Recognition
Kurczewski, L., Sweet, M. A., McKnight, R., & Halbritter, K. (2015) · Critical Care Nursing Quarterly · doi:10.1097/cnq.0000000000000060 · Cited by 15 · see citing works
Reese, T. J., Liu, S., Steitz, B. D., McCoy, A. B., Russo, E. M., Koh, B., Ancker, J. S., & Wright, A. T. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac089 · Cited by 15 · see citing works
Testing modes of computerized sepsis alert notification delivery systems
Dziadzko, M. A., Harrison, A. M., Tiong, I., Pickering, B. W., Franco, P. M., & Herasevich, V. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0396-y · Cited by 15 · see citing works
Scott, J., Finch, T. L., Bevan, M. T., Maniatopoulos, G., Gibbins, C., Yates, B., Narayanan, K., Sheerin, N., & Kanagasundaram, N. S. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-032925 · Cited by 14 · see citing works
Ryu, A. J., Ayanian, S., Qian, R., Core, M. A., Heaton, H. A., Lamb, M. W., Parikh, R. S., Boyum, J. P., Garza, E. L., Condon, J. L., et al. (2023) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2022.11.019 · Cited by 13 · see citing works
Evaluation of a Sepsis Alert in the Pediatric Acute Care Setting
Gibbs, K. D., Shi, Y. P., Sanders, N., Bodnar, A., Brown, T. L., Shah, M. D., & Hess, L. M. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1730027 · Cited by 12 · see citing works
Development of an Obstetric Vital Sign Alert to Improve Outcomes in Acute Care Obstetrics
Behling, D. J., & Renaud, M. (2015) · Nursing for Women s Health · doi:10.1111/1751-486x.12185 · Cited by 11 · see citing works
You, S., Jung, S., Lee, H. J., Kim, S., Yang, E., & SAVER team (2021) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/s13049-021-00979-y · Cited by 11 · see citing works
Implementation of a Vital Sign Alert System to Improve Outcomes
Huff, S., Stephens, K., Whiteman, K. F., Swanson‐Biearman, B., & Mori, C. (2018) · Journal of Nursing Care Quality · doi:10.1097/ncq.0000000000000384 · Cited by 10 · see citing works
Emmanuel, J. M., & Torres, A. (2018) · Cureus · doi:10.7759/cureus.3395 · Cited by 8 · see citing works
Mau, K. A., Fink, S., Hicks, B., Brookhouse, A., Flannery, A. M., & Siedlecki, S. L. (2019) · Applied Nursing Research · doi:10.1016/j.apnr.2019.07.001 · Cited by 7 · see citing works
Perlin, J. B., Jackson, E., Hall, C., Mindick, A., Berberich, G., Korwek, K., Poland, R. E., Roach, J., Novak, A., & Guy, J. S. (2020) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/j.jcjq.2020.04.006 · Cited by 7 · see citing works
Parrish, W. M., Hravnak, M., Dudjak, L. A., & Guttendorf, J. (2017) · PubMed · Cited by 6 · see citing works
Esmaeilzadeh, S., Lane, C. M., Gerberi, D. J., Wakeam, E., Pickering, B. W., Herasevich, V., & Hyder, J. A. (2022) · Critical Care Explorations · doi:10.1097/cce.0000000000000644 · Cited by 5 · see citing works
Linnen, D. T., Hu, X., & Stephens, C. E. (2020) · Nursing Administration Quarterly · doi:10.1097/naq.0000000000000438 · Cited by 3 · see citing works
Design and Development of a Proactive Rapid Response System
(2017) · CIN Computers Informatics Nursing · doi:10.1097/01.ncn.0000513144.27362.9c · Cited by 3 · see citing works
Development of an early detection and monitoring system for deteriorating patients.
Hwang, M., Jo, E.-M., Yoon, H., & Suh, D. I. (2013) · PubMed · Cited by 1 · see citing works
Wan, Y.-K. J., Fiol, G. D., McFarland, M., & Wright, M. C. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-055525 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
2 of the 117 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Morgan RW, Wright MM. An early warning scoring system for detecting developing critical illness. Clin Intensive Care. 1997;8(2):100.
Cited by
Sorted by citations
Rossetti, S. C., Dykes, P. C., Knaplund, C., Cho, S., Withall, J., Lowenthal, G., Albers, D., Lee, R. Y., Jia, H., Bakken, S., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03609-7 · Cited by 54 · see citing works
“…algorithm-based early warning systems (EWSs) positively impact patient outcomes, few randomized controlled trials have demonstrated an impact 4,5 , and many predictions are focused on one particular event type rather than a broad set of outcomes, such as in-hospital mortality, length of…”
InfluentialJayawardena, T. D., Baysari, M. T., & Bamgboje‐Ayodele, A. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105946 · Cited by 5 · see citing works
Butler, J. M., Doubleday, A., Sattar, U., Nies, M. A., Jeppesen, A., Wright, M. C., Reese, T., Kawamoto, K., Fiol, G. D., & Madaras‐Kelly, K. (2025) · Applied Clinical Informatics · doi:10.1055/a-2505-7743 · Cited by 5 · see citing works
Self-citationMultisite Pragmatic Cluster-Randomized Controlled Trial of the CONCERN Early Warning SystemPreprint
Rossetti, S. C., Dykes, P. C., Knaplund, C., Cho, S., Withall, J. B., Lowenthal, G., Albers, D. J., Lee, R. Y., Jia, H., Henry, S. B., et al. (2024) · medRxiv · doi:10.1101/2024.06.04.24308436 · Cited by 5 · see citing works
Yu, Y., Cao, L., Sun, H., Sun, S., Wang, K., Xiao, Y., Luo, H.-Y., Liu, B., Li, J., Wang, L., et al. (2025) · Journal of Craniofacial Surgery · doi:10.1097/scs.0000000000011758 · Cited by 3 · see citing works
Raszke, P., Giebel, G. D., Wasem, J., Adamzik, M., Nowak, H., Palmowski, L., Heinz, P., Timmesfeld, N., Tokic, M. C., Brunkhorst, F. M., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/76772 · Cited by 3 · see citing works
User interfaces remain an important area of study
Henry, S. B. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad246 · Cited by 1 · see citing works
Bamgboje‐Ayodele, A., Boscolo, A., Dao, M., Burger, M., Newton, N., McPhail, S., & Baysari, M. T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5150513
Winterbottom, F. A. (2026) · Critical Care Nursing Clinics of North America · doi:10.1016/j.cnc.2026.01.002
Saikali, M., Baysari, M. T., Lichtner, V., Sylvia Pelayo, A., Carland, J. E., & Marcilly, R. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106369
Wan, Y.-K. J., Abdelrahman, S., Facelli, J. C., Madaras‐Kelly, K., Kawamoto, K., Dishman, D., Cato, K., Rossetti, S. C., & Fiol, G. D. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag077
Self-citation
- Authors
- Wan, Y.-K. J., Wright, M. C., McFarland, M. M., Dishman, D., Nies, M. A., Rush, A., Madaras-Kelly, K., Jeppesen, A., & Del Fiol, G.
- Affiliations
- University of Utah (US)
- Idaho State University (US)
- The University of Texas Health Science Center at Houston (US)
- College of Pharmacy, Idaho State University , Meridian, ID 83642, United States
- Department of Biomedical Informatics, University of Utah , Salt Lake City, UT 84108, United States
- ORCID
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-10-17
- Volume
- 31
- Issue
- 1
- Pages
- 256-273
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC10746326
- OpenAlex
- W4387693565
- Cited by
- 11
- References
- 117
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 1 · 2023: 1
- FWCI
- 0.71
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of General Medical Sciences
- National Institute of Health (R01GM137083)
- National Center for Advancing Translational Sciences (UL1TR002538)
- Topics
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Keywords
- automated surveillance algorithm, clinical decision support, information visualization, clinical alerts, sepsis prediction, algorithmic transparency, clinical workflow integration, trust in AI
- MeSH terms
- Algorithms, Data Display, Hospitals, Humans, Inpatients, Sepsis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
