The impact of electronic medical records data sources on an adverse drug event quality measure
Open accessAbstract
OBJECTIVE: To examine the impact of billing and clinical data extracted from an electronic medical record system on the calculation of an adverse drug event (ADE) quality measure approved for use in The Joint Commission's ORYX program, a mandatory national hospital quality reporting system. DESIGN: The Child Health Corporation of America's "Use of Rescue Agents-ADE Trigger" quality measure uses medication billing data contained in the Pediatric Health Information Systems (PHIS) data warehouse to create The Joint Commission-approved quality measure. Using a similar query, we calculated the quality measure using PHIS plus four data sources extracted from our electronic medical record (EMR) system: medications charged, medication orders placed, medication orders with associated charges (orders charged), and medications administered. MEASUREMENTS: Inclusion and exclusion criteria were identical for all queries. Denominators and numerators were calculated using the five data sets. The reported quality measure is the ADE rate (numerator/denominator). RESULTS: Significant differences in denominators, numerators, and rates were calculated from different data sources within a single institution's EMR. Differences were due to both common clinical practices that may be similar across institutions and unique workflow practices not likely to be present at any other institution. The magnitude of the differences would significantly alter the national comparative ranking of our institution compared to other PHIS institutions. CONCLUSIONS: More detailed clinical information may result in quality measures that are not comparable across institutions due institution-specific workflow, differences that are exposed using EMR-derived data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
EHRs typically are optimized for efficient patient care and nonclinical administrative requirements, resulting in great variation in clinical documentation practices, even among users of the same systems.
“EHRs typically are optimized for efficient patient care and nonclinical administrative requirements, resulting in great variation in clinical documentation practices, even among users of the same systems.38–40”
From A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data (2016) · cites this paper as [38]
The Discussion (page unavailable) describes differences in clinical documentation and billing workflows, including different medication-recording practices in the operating room and other care settings. It also says institution-specific workflows affect EMR-derived data. The paper does not establish that EHRs are typically optimized for efficient care and administrative requirements, or that documentation varies among users of the same system. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Measuring Diagnoses: ICD Code Accuracy
O’Malley, K. J., Cook, K. F., Price, M. D., Wildes, K. R., Hurdle, J. F., & Ashton, C. M. (2005) · Health Services Research · doi:10.1111/j.1475-6773.2005.00444.x · Cited by 1,123 · see citing works
Computerized Surveillance of Adverse Drug Events in Hospital Patients
Classen, D. C. (1991) · JAMA · doi:10.1001/jama.1991.03470200059035 · Cited by 643 · see citing works
Jollis, J. G., Ancukiewicz, M., DeLong, E. R., Pryor, D. B., Muhlbaier, L. H., & Mark, D. B. (1993) · Annals of Internal Medicine · doi:10.7326/0003-4819-119-8-199310150-00011 · Cited by 626 · see citing works
Measuring errors and adverse events in health care
Thomas, E. J., & Petersen, L. A. (2003) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2003.20147.x · Cited by 456 · see citing works
Adverse drug event trigger tool: a practical methodology for measuring medication related harm
Rozich, J. D. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.3.194 · Cited by 439 · see citing works
Methodology and rationale for the measurement of harm with trigger tools
Resar, R. K. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.suppl_2.ii39 · Cited by 312 · see citing works
A National Profile Of Patient Safety In U.S. Hospitals
Romano, P. S., Geppert, J., Davies, S. M., Miller, M. R., Elixhauser, A., & McDonald, K. M. (2003) · Health Affairs · doi:10.1377/hlthaff.22.2.154 · Cited by 267 · see citing works
Takata, G. S., Mason, W. H., Taketomo, C. K., Logsdon, T. R., & Sharek, P. J. (2008) · PEDIATRICS · doi:10.1542/peds.2007-1779 · Cited by 249 · see citing works
Tang, P. C., Ralston, M., Arrigotti, M. F., Qureshi, L., & Graham, J. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2198 · Cited by 193 · see citing works
Evaluating the Patient Safety Indicators
Rosen, A. K., Rivard, P. E., Zhao, S., Loveland, S. A., Tsilimingras, D., Christiansen, C. L., Elixhauser, A., & Romano, P. S. (2005) · Medical Care · doi:10.1097/01.mlr.0000173561.79742.fb · Cited by 114 · see citing works
Baker, D. W., Persell, S. D., Thompson, J. A., Soman, N. S., Burgner, K. M., Liss, D. T., & Kmetik, K. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-4-200702200-00006 · Cited by 89 · see citing works
Sedman, A. B., Harris, J. M., Schulz, K. A., Schwalenstocker, E., Remus, D., Scanlon, M. C., & Bahl, V. (2004) · PEDIATRICS · doi:10.1542/peds.2004-1083 · Cited by 88 · see citing works
Evaluation of the Agency for Healthcare Research and Quality Pediatric Quality Indicators
Scanlon, M. C., Harris, J. M., Levy, F. H., & Sedman, A. B. (2008) · PEDIATRICS · doi:10.1542/peds.2007-3247 · Cited by 81 · see citing works
Persell, S. D. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.20.2272 · Cited by 78 · see citing works
Ford, E. W., McAlearney, A. S., Phillips, M., Menachemi, N., & Rudolph, B. A. (2007) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2007.10.009 · Cited by 48 · see citing works
A comparison of hospital adverse events identified by three widely used detection methods
Naessens, J. M., Campbell, C. R., Huddleston, J. M., Berg, B. P., Lefante, J. J., Williams, A. R., & Culbertson, R. A. (2009) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzp027
Bahl, V., Thompson, M. A., Kau, T.-Y., Hu, H. M., & Campbell, D. A. (2008) · Medical Care · doi:10.1097/mlr.0b013e31815f537f
Flagged first, then by citations
Research works printed but not linked: 5
- Joint Commission on Accreditation of Healthcare Organizations. Special reporteORYX: the next evolution in accreditation. Joint Commission introduces new requirements to integrate performance measurement into accreditation process. Jt Comm Perspect 1997;17:26e7, 9.
- Rexe K, Vigneault S, Thornhill J. Public reporting: one piece of the quality improvement puzzle. Healthc Q 2008;11:18e19.
- Felt-Lisk S, Barrett A, Nyman R. Public reporting of quality information on Medicaid health plans. Health Care Financ Rev 2007;28:5e16.
- Classen DC, Pestotnik SL, Evans RS, et al. Description of a computerized adverse drug event monitor using a hospital information system. Hosp Pharm 1992;27:774, 6e9, 83.
- Yu FB, Allison JJ, Berner ES, et al. CPOE in the United States. Health Aff (Millwood) 2006;25:567.
Cited by
Sorted by citations
Kahn, M. G., Callahan, T. J., Barnard, J., Bauck, A. E., Brown, J., Davidson, B. N., Estiri, H., Goerg, C., Holve, E., Johnson, S. G., et al. (2016) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1244 · Cited by 636 · see citing works
Self-citationHersh, W. R., Weiner, M. G., Embi, P. J., Logan, J. R., Payne, P. R. O., Bernstam, E. V., Lehmann, H. P., Hripcsak, G., Hartzog, T. H., Cimino, J. J., et al. (2013) · Medical Care · doi:10.1097/mlr.0b013e31829b1dbd · Cited by 567 · see citing works
- Authors
- Kahn, M. G., & Ranade, D.
- Affiliations
- Children's Hospital Colorado (US)
- Corresponding author
- Kahn, Michael G
- ORCID
- Kahn, Michael G
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2010-03
- Volume
- 17
- Issue
- 2
- Pages
- 185-191
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC3000787
- OpenAlex
- W2104006906
- Cited by
- 31
- References
- 17
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 1 · 2023: 1 · 2020: 1 · 2019: 1 · 2018: 3 · 2017: 2 · 2016: 3 · 2015: 3 · 2014: 2 · 2013: 5 · 2012: 3
- FWCI
- 6.57
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Georgia Clinical and Translational Science Alliance
- National Institutes of Health
- National Center for Research Resources (UL1RR025780)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Keywords
- adverse drug reactions, quality metrics, electronic health records, billing data, medication orders, medication administration, hospital quality reporting, data source comparability, children's hospitals, data warehouse
- MeSH terms
- Humans, Joint Commission on Accreditation of Healthcare Organizations, Quality Assurance, Health Care, United States, Information Storage and Retrieval, Adverse Drug Reaction Reporting Systems, Workflow, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
