Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry System
Abstract
Objective. In response to the landmark 1999 report by the Institute of Medicine and safety initiatives promoted by the Leapfrog Group, our institution implemented a commercially sold computerized physician order entry (CPOE) system in an effort to reduce medical errors and mortality. We sought to test the hypothesis that CPOE implementation results in reduced mortality among children who are transported for specialized care. Methods. Demographic, clinical, and mortality data were collected of all children who were admitted via interfacility transport to our regional, academic, tertiary-care level children’s hospital during an 18-month period. A commercially sold CPOE program that operated within the framework of a general, medical-surgical clinical application platform was rapidly implemented hospital-wide over 6 days during this period. Retrospective analyses of pre-CPOE and post-CPOE implementation time periods (13 months before and 5 months after CPOE implementation) were subsequently performed. Results. Among 1942 children who were referred and admitted for specialized care during the study period, 75 died, accounting for an overall mortality rate of 3.86%. Univariate analysis revealed that mortality rate significantly increased from 2.80% (39 of 1394) before CPOE implementation to 6.57% (36 of 548) after CPOE implementation. Multivariate analysis revealed that CPOE remained independently associated with increased odds of mortality (odds ratio: 3.28; 95% confidence interval: 1.94–5.55) after adjustment for other mortality covariables. Conclusions. We have observed an unexpected increase in mortality coincident with CPOE implementation. Although CPOE technology holds great promise as a tool to reduce human error during health care delivery, our unanticipated finding suggests that when implementing CPOE systems, institutions should continue to evaluate mortality effects, in addition to medication error rates, for children who are dependent on time-sensitive therapies.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Published typologies have not covered how the level of clinical-decision-support automation affects safety flaws and their chance of causing an incident.
“In addition, the published typologies have not addressed such critical issues as the effect of the level of clinical-decision-support automation on safety flaws and their likelihood of contributing to a safety incident.14,47”
From EHR Safety: The Way Forward to Safe and Effective Systems (2008) · cites this paper as [14]
Conflicting recent reports question how much CPOE truly improves care, noting that these systems have created new error types, harmed patient outcomes, and raised costs for adopting institutions.
“Recent, conflicting reports about the role of CPOE in the reduction of medication errors and associated costs have cast some doubt on the actual scale of improvements to be gained as CPOE systems have generated new kinds of medical errors, negatively affected patient outcomes, and resulted in higher overall medical costs for those institutions implementing them.9,[11][12][13]”
From Computerized Provider Order Entry Adoption: Implications for Clinical Workflow (2009) · cites this paper as [12]
eHealth may compromise patient safety in a number of ways.
“eHealth may compromise patient safety in a number of ways.”
From Evaluating eHealth Interventions: The Need for Continuous Systemic Evaluation (2009) · cites this paper as [13]
An unexpected increase in mortality was observed after the implementation of a commercially sold computerised physician order entry system; this increase may have been due to "system integration failure" and/or "human-machine interface flaws".
“An unexpected increase in mortality was observed after the implementation of a commercially sold computerised physician order entry system; this increase may have been due to “system integration failure” and/or “human-machine interface flaws” [13].”
From Evaluating eHealth Interventions: The Need for Continuous Systemic Evaluation (2009) · cites this paper as [13]
Relatively little time, thought, or resources have been devoted to assessing the potential risks associated with eHealth interventions.
“it is of concern that, in this rush, relatively little time, thought, or resources have been devoted to assessing the potential risks associated with eHealth interventions [11]–[13],[33]–[36].”
From Evaluating eHealth Interventions: The Need for Continuous Systemic Evaluation (2009) · cites this paper as [13]
Cites
Institute of Medicine (2000) · National Academies Press eBooks · doi:10.17226/9728 · Cited by 14,174 · see citing works
Early goal-directed therapy in the treatment of severe sepsis and septic shock
Rivers, E., Nguyen, B., Havstad, S., Ressler, J., Muzzin, A., Knoblich, B., Peterson, E., & Tomlanovich, M. (2001) · New England Journal of Medicine · doi:10.1056/nejmoa010307 · Cited by 10,743 · see citing works
Surviving Sepsis Campaign guidelines for management of severe sepsis and septic shock
Dellinger, R. P., Carlet, J. M., Masur, H., Gerlach, H., Calandra, T., Cohen, J., Gea‐Banacloche, J., Keh, D., Marshall, J. C., Parker, M. M., et al. (2004) · Intensive Care Medicine · doi:10.1007/s00134-004-2210-z · Cited by 3,298 · see citing works
Surviving Sepsis Campaign guidelines for management of severe sepsis and septic shock
Dellinger, R. P., Carlet, J. M., Masur, H., Gerlach, H., Calandra, T., Cohen, J., Gea‐Banacloche, J., Keh, D., Marshall, J. C., Parker, M. M., et al. (2004) · Critical Care Medicine · doi:10.1097/01.ccm.0000117317.18092.e4 · Cited by 3,012 · see citing works
Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Koppel, R. (2005) · JAMA · doi:10.1001/jama.293.10.1197 · Cited by 2,126 · see citing works
Bates, D. W. (1998) · JAMA · doi:10.1001/jama.280.15.1311 · Cited by 2,107 · see citing works
Dellinger, R. P. (2004) · Critical Care Medicine · Cited by 1,851 · see citing works
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789 · see citing works
Classen, D. C. (1997) · JAMA · doi:10.1001/jama.277.4.301 · Cited by 1,781 · see citing works
Bates, D. W. (1997) · JAMA · doi:10.1001/jama.277.4.307 · Cited by 1,678 · see citing works
COMMITTEE ON QUALITY OF HEALTH CARE IN AMERICA
Kohn, L. T., Corrigan, J. M., & Donaldson, M. S. (2000) · Cited by 1,508 · see citing works
Pediatric risk of mortality (PRISM) score
Pollack, M. M., Ruttimann, U. E., & Getson, P. R. (1988) · Critical Care Medicine · doi:10.1097/00003246-198811000-00006 · Cited by 1,396 · see citing works
Brown, A. B., & Patterson, D. A. (2001) · Cited by 1,396 · see citing works
The Impact of Computerized Physician Order Entry on Medication Error Prevention
Bates, D. W., Teich, J. M., Lee, J., Seger, D., Kuperman, G. J., Ma'Luf, N., Boyle, D., & Leape, L. (1999) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1999.00660313 · Cited by 1,260 · see citing works
Han, Y. Y., Carcillo, J. A., Dragotta, M. A., Bills, D. M., Watson, R. S., Westerman, M. E., & Orr, R. A. (2003) · PEDIATRICS · doi:10.1542/peds.112.4.793 · Cited by 636 · see citing works
Self-citationCarcillo, J. A., & Fields, A. I. (2002) · Critical Care Medicine · doi:10.1097/00003246-200206000-00040 · Cited by 609 · see citing works
Self-citationPhysician inpatient order writing on microcomputer workstations. Effects on resource utilization
Tierney, W. M. (1993) · JAMA · doi:10.1001/jama.269.3.379 · Cited by 561 · see citing works
Computer Physician Order Entry: Benefits, Costs, and Issues
Kuperman, G. J., & Gibson, R. F. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-139-1-200307010-00010 · Cited by 438 · see citing works
Computerized Physician Order Entry and Medication Errors in a Pediatric Critical Care Unit
Potts, A. L., Barr, F. E., Gregory, D. F., Wright, L., & Patel, N. R. (2004) · Pediatrics · doi:10.1542/peds.113.1.59 · Cited by 410 · see citing works
Mekhjian, H. S., Kumar, R. R., Kuehn, L. A., Bentley, T. D., Teater, P., Thomas, A., Payne, B. P., & Ahmad, A. (2002) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1038 · Cited by 357 · see citing works
King, W. J., Paice, N., Rangrej, J., Forestell, G. J., & Swartz, R. (2003) · Pediatrics · doi:10.1542/peds.112.3.506 · Cited by 281 · see citing works
Aarts, J., Doorewaard, H., & Berg, M. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1372 · Cited by 251 · see citing works
Computerized Physician Order Entry: Helpful or Harmful?
Berger, R. G. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1411 · Cited by 223 · see citing works
Arias, Y., Taylor, D. S., & Marcin, J. P. (2004) · PEDIATRICS · doi:10.1542/peds.113.6.e530 · Cited by 122 · see citing works
Upperman, J. S., Staley, P., Friend, K., Neches, W. H., Kazimer, D., Benes, J., & Wiener, E. S. (2005) · Journal of Pediatric Surgery · doi:10.1016/j.jpedsurg.2004.09.024 · Cited by 121 · see citing works
Comparison of Time Spent Writing Orders on Paper with Computerized Physician Order Entry
Shu, K. M., Boyle, D., Spurr, C. D., Horsky, J., Heiman, H. L., Paula, O., John, L., & Bates, D. W. (2001) · Studies in health technology and informatics · doi:10.3233/978-1-60750-928-8-1207 · Cited by 108 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Technology Acceptance Model: Its past and its future in health care
Holden, R. J., & Karsh, B.-T. (2010) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.07.002 · Cited by 2,669 · see citing works
Artificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · see citing works
Sittig, D. F., & Singh, H. (2010) · Quality and Safety in Health Care · doi:10.1136/qshc.2010.042085 · Cited by 897 · see citing works
Harrison, M. I., Koppel, R., & Bar-Lev, S. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2384 · Cited by 691 · see citing works
Jaspers, M. W. M. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.10.002 · Cited by 649 · see citing works
LUDWICK, D., & DOUCETTE, J. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.06.005 · Cited by 668 · see citing works
GREENHALGH, T., POTTS, H. W. W., WONG, G., BARK, P., & SWINGLEHURST, D. (2009) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2009.00578.x · Cited by 541 · see citing works
The impact of electronic health records on healthcare quality: a systematic review and meta-analysis
Campanella, P., Lovato, E., Marone, C., Fallacara, L., Mancuso, A., Ricciardi, W., & Specchia, M. L. (2016) · The European Journal of Public Health · doi:10.1093/eurpub/ckv122 · Cited by 434 · see citing works
Evaluating eHealth Interventions: The Need for Continuous Systemic Evaluation
Catwell, L., & Sheikh, A. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000126 · Cited by 307 · see citing works
TURF: Toward a unified framework of EHR usability
Zhang, J., & Walji, M. F. (2011) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.08.005 · Cited by 301 · see citing works
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A’Court, C., Hinder, S., Procter, R., & Shaw, S. (2018) · BMC Medicine · doi:10.1186/s12916-018-1050-6 · Cited by 232 · see citing works
Kim, M. O., Coiera, E., & Magrabi, F. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw154 · Cited by 263 · see citing works
An analysis of electronic health record-related patient safety concerns
Meeks, D. W., Smith, M. W., Taylor, L., Sittig, D. F., Scott, J. M., & Singh, H. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002578 · Cited by 207 · see citing works
Hospital Computing and the Costs and Quality of Care: A National Study
Himmelstein, D. U., Wright, A., & Woolhandler, S. (2010) · The American Journal of Medicine · doi:10.1016/j.amjmed.2009.09.004 · Cited by 212 · see citing works
Health information exchange and patient safety
Kaelber, D. C., & Bates, D. W. (2007) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.08.011 · Cited by 206 · see citing works
Patient outcomes at 26 months in the patient-centered medical home National Demonstration Project
Jaen, C. R., Ferrer, R. L., Miller, W. L., Palmer, R. F., Wood, R., Davila, M., Stewart, E. E., Crabtree, B. F., Nutting, P. A., & Stange, K. C. (2010) · The Annals of Family Medicine · doi:10.1370/afm.1121 · Cited by 194 · see citing works
Primary Care Physicians’ Use of an Electronic Medical Record System: A Cognitive Task Analysis
Shachak, A., Hadas-Dayagi, M., Ziv, A., & Reis, S. (2009) · Journal of General Internal Medicine · doi:10.1007/s11606-008-0892-6 · Cited by 188 · see citing works
Del Beccaro, M. A., Jeffries, H. E., Eisenberg, M. A., & Harry, E. D. (2006) · Pediatrics · doi:10.1542/peds.2006-0367 · Cited by 181 · see citing works
Magrabi, F., Ong, M.-S., Runciman, W., & Coiera, E. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.002444 · Cited by 162 · see citing works
TSIKNAKIS, M., & KOUROUBALI, A. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.07.001 · Cited by 167 · see citing works
Frequency of Failure to Inform Patients of Clinically Significant Outpatient Test Results
Casalino, L. P., Dunham, D., Chin, M. H., Bielang, R., Kistner, E. O., Karrison, T. G., Ong, M. K., Sarkar, U., McLaughlin, M. A., & Meltzer, D. O. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.130 · Cited by 159 · see citing works
Computerization Can Create Safety Hazards: A Bar-Coding Near Miss
McDonald, C. J. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-7-200604040-00010 · Cited by 146 · see citing works
Computerized Provider Order Entry Adoption: Implications for Clinical Workflow
Campbell, E. M., Guappone, K. P., Sittig, D. F., Dykstra, R. H., & Ash, J. S. (2009) · Journal of General Internal Medicine · doi:10.1007/s11606-008-0857-9 · Cited by 139 · see citing works
Sittig, D. F., Ash, J. S., Zhang, J., Osheroff, J. A., & Shabot, M. M. (2006) · Pediatrics · doi:10.1542/peds.2005-3132 · Cited by 130 · see citing works
Holden, R. J., & Karsh, B.-T. (2009) · Behaviour & Information Technology · doi:10.1080/01449290601138245 · Cited by 128 · see citing works
EHR Safety: The Way Forward to Safe and Effective Systems
Walker, J. M., Carayon, P., Leveson, N., Paulus, R. A., Tooker, J., Chin, H., Bothe, A., & Stewart, W. F. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2618 · Cited by 113 · see citing works
The unintended consequences of health information technology revisited
Ash, J., Berg, M., & Coiera, E. (2016) · Yearbook of Medical Informatics · doi:10.15265/iy-2016-014 · Cited by 101 · see citing works
Guideline for good evaluation practice in health informatics (GEP-HI)
Nykänen, P., Brender, J., Talmon, J., de Keizer, N., Rigby, M., Beuscart-Zephir, M.-C., & Ammenwerth, E. (2011) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2011.08.004 · Cited by 105 · see citing works
Magrabi, F., Baker, M., Sinha, I., Ong, M.-S., Harrison, S., Kidd, M. R., Runciman, W. B., & Coiera, E. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.12.003 · Cited by 100 · see citing works
Complexity and the science of implementation in health IT—Knowledge gaps and future visions
Abbott, P. A., Foster, J., Marin, H. D. F., & Dykes, P. C. (2014) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2013.10.009 · Cited by 95 · see citing works
Teixeira, L., Ferreira, C., & Santos, B. S. (2012) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2010.10.007 · Cited by 76 · see citing works
Clinical decision support systems in the pediatric intensive care unit*
Mack, E. H., Wheeler, D. S., & Embi, P. J. (2009) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0b013e3181936b23 · Cited by 76 · see citing works
Local Investment in Training Drives Electronic Health Record User Satisfaction
Longhurst, C., Davis, T., Maneker, A., Eschenroeder, H., Dunscombe, R., Reynolds, G., Clay, B., Moran, T., Graham, D., Dean, S., et al. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1688753 · Cited by 76 · see citing works
PIRNEJAD, H., NIAZKHANI, Z., VANDERSIJS, H., BERG, M., & BAL, R. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.04.001 · Cited by 73 · see citing works
Myers, R. B., Jones, S. L., & Sittig, D. F. (2011) · Applied Clinical Informatics · doi:10.4338/aci-2010-11-ra-0064 · Cited by 67 · see citing works
Holmgren, A. J., Adler-Milstein, J., & McCullough, J. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx135 · Cited by 63 · see citing works
Mazur, L. M., Mosaly, P. R., Moore, C., Comitz, E., Yu, F., Falchook, A. D., Eblan, M. J., Hoyle, L. M., Tracton, G., Chera, B. S., et al. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw016 · Cited by 45 · see citing works
Computerized Provider Order Entry and Patient Safety
Lehmann, C. U., & Kim, G. R. (2006) · Pediatric Clinics of North America · doi:10.1016/j.pcl.2006.09.006 · Cited by 45 · see citing works
Lessons Learned from Pilot Site Implementation of an Ambulatory Electronic Health Record
Fullerton, C., Aponte, P., Hopkins, R., Bragg, D., & Ballard, D. J. (2006) · Baylor University Medical Center Proceedings · doi:10.1080/08998280.2006.11928188 · Cited by 40 · see citing works
Publication bias in clinical trials of electronic health records
Vawdrey, D. K., & Hripcsak, G. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2012.08.007 · Cited by 38 · see citing works
Rosenbloom, S. T., Harrell, F. E., Lehmann, C. U., Schneider, J. H., Spooner, S. A., & Johnson, K. B. (2006) · Pediatrics · doi:10.1542/peds.2005-3163 · Cited by 38 · see citing works
Nyame-Asiamah, F. (2020) · Information Technology for Development · doi:10.1080/02681102.2019.1650326 · Cited by 28 · see citing works
Resilient practices in maintaining safety of health information technologies
Smith, M. W., Ash, J. S., Sittig, D. F., & Singh, H. (2014) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343414534242 · Cited by 26 · see citing works
Gamm, L., Kash, B., & Bolin, J. (2007) · Journal of Ambulatory Care Management · doi:10.1097/01.jac.0000290397.58790.b9 · Cited by 24 · see citing works
Get To The Point! Problem-Based Curated Data Views To Augment Care For Critically Ill Patients
Zhang, M., Ehrmann, D., Mazwi, M., Eytan, D., Ghassemi, M., & Chevalier, F. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501887 · Cited by 16 · see citing works
A model curriculum for an emergency medicine residency rotation in clinical informatics
Baker, C., Maniam, N., Schnapp, B., Genes, N., Nielson, J., Mohan, V., Hersh, W., & Slovis, B. (2022) · Journal of Education and Teaching in Emergency Medicine · doi:10.5070/m57459214 · Cited by 6 · see citing works
- Authors
- Han, Y. Y., Carcillo, J. A., Venkataraman, S. T., Clark, R. S. B., Watson, R. S., Nguyen, T. C., Bayir, H., & Orr, R. A.
- Affiliations
- Children's Hospital of Pittsburgh (US)
- University of Pittsburgh (US)
- Departments of Critical Care Medicine
- ORCID
- Venue
- Pediatrics
- Publisher
- American Academy of Pediatrics (AAP)
- Published
- 2005-12-01
- Volume
- 116
- Issue
- 6
- Pages
- 1506-1512
- ISSN
- 0031-4005 (ISSN-L)
- 1098-4275
- OpenAlex
- W2162040788
- Cited by
- 1,149
- References
- 26
- Citations by year
- 2026: 9 · 2025: 7 · 2024: 11 · 2023: 20 · 2022: 20 · 2021: 25 · 2020: 24 · 2019: 47 · 2018: 22 · 2017: 30 · 2016: 61 · 2015: 86 · 2014: 75 · 2013: 89 · 2012: 94
- FWCI
- 168.54
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- PEDIATRICS (publishedVersion)
- PubMed (publishedVersion)
- Funders
- U.S. Public Health Service (1-MCH-4240030-01-0)
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (T32HD040686)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
- Health Sciences › Medicine › Emergency Medicine › Hospital Admissions and Outcomes
- Keywords
- computerized physician order entry, CPOE implementation, child mortality, interfacility transport, medical errors, patient safety
- MeSH terms
- Child, Preschool, Female, Humans, Infant, Infant Mortality, Male, Outcome and Process Assessment, Health Care, Patient Transfer, Child Mortality, Medical Order Entry Systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review