Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems
Open accessAbstract
The 1999 Institute of Medicine report raised public awareness of the frequency and cost of adverse drug events in medicine. In response, in November 2000 a coalition of healthcare purchasers announced the formation of the Leapfrog Group, an organization dedicated to making "great leaps forward" in the safety and quality of health care in America. Their first target-computerized physician order entry (CPOE)-was selected specifically for its potential to reduce harm to patients from medications. The Leapfrog inpatient CPOE standard included a requirement that the organization operating CPOE should demonstrate via a test that their inpatient CPOE system can alert physicians to at least 50% of common serious prescribing errors. This paper outlines the development of this test which evaluates the ability of implemented CPOE systems to prevent the occurrence of medication errors that have a high likelihood of leading to adverse drug events. A framework was developed to include 12 different categories of CPOE based decision support that could prevent prescribing errors leading to adverse drug events. A scoring system was developed based on the known frequency and severity of adverse drug events. Simulated test patients and accompanying simulated test medication orders were developed to evaluate the ability of a CPOE system to intercept prescribing errors in all 12 decision support categories. The test was validated at a number of inpatient sites using both commercially available and custom developed CPOE systems. A web based application was developed to allow hospitals to self-administer the evaluation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The health care literature shows that EHRs can contribute to safety incidents and patient harm.
“Complicating matters, the health care literature documents the potential of EHRs to contribute to safety incidents and patient harm.1,23,30 -33”
From EHR Safety: The Way Forward to Safe and Effective Systems (2008) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Institute of Medicine (2000) · National Academies Press eBooks · doi:10.17226/9728 · Cited by 14,174 · see citing works
To Err is Human: Building a Safer Health System
Hendee, W. R. (2001) · Journal of Vascular and Interventional Radiology · doi:10.1016/s1051-0443(01)70072-3 · Cited by 10,122 · see citing works
The Nature of Adverse Events in Hospitalized Patients
Leape, L. L., Brennan, T. A., Laird, N. M., Lawthers, A. G., Localio, A. R., Barnes, B. A., Hebert, L. E., Newhouse, J. P., Weiler, P. C., & Hiatt, H. H. (1991) · New England Journal of Medicine · doi:10.1056/nejm199102073240605 · Cited by 3,766 · see citing works
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936 · see citing works
Incidence of Adverse Drug Events and Potential Adverse Drug Events: Implications for Prevention
Bates, D. W. (1995) · JAMA · doi:10.1001/jama.1995.03530010043033 · Cited by 2,127 · see citing works
Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Koppel, R. (2005) · JAMA · doi:10.1001/jama.293.10.1197 · Cited by 2,126 · see citing works
Bates, D. W. (1998) · JAMA · doi:10.1001/jama.280.15.1311 · Cited by 2,107 · see citing works
Medication Errors and Adverse Drug Events in Pediatric Inpatients
Kaushal, R. (2001) · JAMA · doi:10.1001/jama.285.16.2114 · Cited by 1,917 · see citing works
Classen, D. C. (1997) · JAMA · doi:10.1001/jama.277.4.301 · Cited by 1,781 · see citing works
Bates, D. W. (1997) · JAMA · doi:10.1001/jama.277.4.307 · Cited by 1,678 · see citing works
A Computer-Assisted Management Program for Antibiotics and Other Antiinfective Agents
Evans, R. S., Pestotnik, S. L., Classen, D. C., Clemmer, T. P., Weaver, L. K., Orme, J. F., Lloyd, J. F., & Burke, J. P. (1998) · New England Journal of Medicine · doi:10.1056/nejm199801223380406 · Cited by 1,060 · see citing works
A COMPUTER-ASSISTED MANAGEMENT PROGRAM FOR ANTIBIOTICS AND OTHER ANTIINFECTIVE AGENTS
Vans, R. S. C. E., Estotnik, S. T. L. P., Lassen, D. A. C. C., Lemmer, T., Eaver, L. I. K. W., Rme, J. A. F. O., Loyd, J. A. F. L., & Urke, J. O. P. B. (1998) · Cited by 692 · see citing works
Computerized Surveillance of Adverse Drug Events in Hospital Patients
Classen, D. C. (1991) · JAMA · doi:10.1001/jama.1991.03470200059035 · Cited by 643 · see citing works
Physician inpatient order writing on microcomputer workstations. Effects on resource utilization
Tierney, W. M. (1993) · JAMA · doi:10.1001/jama.269.3.379 · Cited by 561 · see citing works
Incidence of adverse drug events and potential adverse drug events
Bates, D. W., Cullen, D. J., & Laird, N. (1995) · International Journal of Trauma Nursing · doi:10.1016/s1075-4210(05)80011-2 · Cited by 537 · see citing works
Jha, A. K., Kuperman, G. J., Teich, J. M., Leape, L. L., Shea, B. S., Rittenberg, E., Burdick, E., Seger, D. L., Vliet, M. V., & Bates, D. W. (1998) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1998.0050305 · Cited by 514 · see citing works
Adverse drug events in hospitalized patients
van den Bemt, P. M. L. A., Egberts, T. C. G., Lenderink, A. W., Verzijl, J. M., Simons, K. A., van der Pol, W. S. C. J. M., & Leufkens, H. G. M. (1999) · European Journal of Clinical Pharmacology · doi:10.1007/s002280050611 · Cited by 460 · see citing works
Adverse drug event trigger tool: a practical methodology for measuring medication related harm
Rozich, J. D. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.3.194 · Cited by 439 · see citing works
Computer Physician Order Entry: Benefits, Costs, and Issues
Kuperman, G. J., & Gibson, R. F. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-139-1-200307010-00010 · Cited by 438 · see citing works
Leape, L. L. (1992) · American Journal of Infection Control · doi:10.1016/s0196-6553(05)80240-8 · Cited by 411 · see citing works
A Randomized Trial of "Corollary Orders" to Prevent Errors of Omission
Overhage, J. M., Tierney, W. M., Zhou, X., & McDonald, C. J. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040364 · Cited by 375 · see citing works
Computerized surveillance of adverse drug events in hospital patients.
Classen, D. C., Pestotnik, S. L., Evans, R. S., & Burke, J. P. (1991) · PubMed · Cited by 281 · see citing works
Bagian, J. P., Lee, C., Gosbee, J. W., DeRosier, J., Stalhandske, E. J., Eldridge, N., Williams, R., & Burkhardt, M. E. (2001) · The Joint Commission Journal on Quality Improvement · doi:10.1016/s1070-3241(01)27046-1 · Cited by 267 · see citing works
Stockman, J. A. (2006) · Yearbook of Pediatrics · doi:10.1016/s0084-3954(07)70181-1 · Cited by 225 · see citing works
Computerized surveillance of adverse drug events in hospital patients
Classen, D. C. (2005) · BMJ Quality & Safety · doi:10.1136/qshc.2002.002972 · Cited by 211 · see citing works
Medication Errors and Adverse Drug Events in Pediatric Inpatients
Bailey, C. R. (2002) · Survey of Anesthesiology · doi:10.1097/00132586-200206000-00041 · Cited by 191 · see citing works
Preventing Adverse Drug Events in Hospitalized Patients
Evans, R. S., Pestotnik, S. L., Classen, D. C., Horn, S. D., Bass, S. B., & Burke, J. P. (1994) · Annals of Pharmacotherapy · doi:10.1177/106002809402800417 · Cited by 160 · see citing works
Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Koppel, R., Metlay, J. P., & Cohen, A. L. (2005) · ACC Current Journal Review · doi:10.1016/j.accreview.2005.06.019 · Cited by 155 · see citing works
Bennett, B. S., & Lipman, A. G. (1977) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/34.9.931 · Cited by 58 · see citing works
Effect of a computerized prescriber-order-entry system on reported medication errors
Spencer, D. C., Leininger, A., Daniels, R., Granko, R. P., & Coeytaux, R. R. (2005) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/62.4.416 · Cited by 58 · see citing works
Adverse drug event reporting. Improving the low US reporting rates
Edlavitch, S. A. (1988) · Archives of Internal Medicine · doi:10.1001/archinte.148.7.1499 · Cited by 42 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Phansalkar, S., Edworthy, J., Hellier, E., Seger, D. L., Schedlbauer, A., Avery, A. J., & Bates, D. W. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.005264 · Cited by 226 · see citing works
A Clinical Case of Electronic Health Record Drug Alert Fatigue: Consequences for Patient Outcome
Carspecken, C. W., Sharek, P. J., Longhurst, C., & Pageler, N. M. (2013) · Pediatrics · doi:10.1542/peds.2012-3252 · Cited by 142 · see citing works
EHR Safety: The Way Forward to Safe and Effective Systems
Walker, J. M., Carayon, P., Leveson, N., Paulus, R. A., Tooker, J., Chin, H., Bothe, A., & Stewart, W. F. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2618 · Cited by 113 · see citing works
Safe Electronic Health Record Use Requires a Comprehensive Monitoring and Evaluation Framework
Sittig, D. F. (2010) · JAMA · doi:10.1001/jama.2010.61 · Cited by 76 · see citing works
- Authors
- Kilbridge, P. M.
- Affiliations
- Duke University Health System (US)
- Corresponding author
- Kilbridge, P M
- Venue
- Quality and Safety in Health Care
- Publisher
- BMJ
- Published
- 2006-04-01
- Volume
- 15
- Issue
- 2
- Pages
- 81-84
- ISSN
- 1475-3898 (ISSN-L)
- 1475-3901
- PMC
- PMC2464830
- OpenAlex
- W2125872723
- Cited by
- 72
- References
- 31
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 3 · 2023: 2 · 2021: 3 · 2020: 4 · 2019: 1 · 2017: 2 · 2016: 3 · 2015: 4 · 2014: 4 · 2013: 6 · 2012: 5
- FWCI
- 13.13
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- BMJ Quality & Safety (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Keywords
- Leapfrog Group, computerized physician order entry, prescribing errors, medication errors, adverse drug reactions, clinical decision support, inpatient medication safety, hospital quality assessment, web-based evaluation tool
- MeSH terms
- Certification, Diffusion of Innovation, Humans, Medication Errors, Medication Systems, Hospital, Organizations, Pharmacy Service, Hospital, Quality Assurance, Health Care, Quality Control, United States, Program Evaluation, Safety Management, Decision Support Systems, Clinical, Medical Order Entry Systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
