Using Electronic Health Records to Measure Physician Performance for Acute Conditions in Primary Care: Empirical Evaluation of the Community-Acquired Pneumonia Clinical Quality Measure Set

Abstract

Background Electronic health records (EHRs) have the potential to facilitate performance measurement for acute conditions. Objective To evaluate the reliability and feasibility-of-use of a performance measure set for community-acquired pneumonia in an ambulatory EHR. Design Retrospective, cross-sectional electronic chart review. Setting Primary Care Clinics. Patients Adults with an administrative claims diagnosis of pneumonia during a 14-month period. Measurements Two reviewers independently examined data in the EHR to determine if (1) the encounter was a visit for acute pneumonia; (2) there was documentation for each of 12 performance measures; and (3) such information was in coded form. Results Of 688 encounters with a claim diagnosis of pneumonia, 210 (31%) were identified by either reviewer as a primary care acute pneumonia visit. The 2 reviewers agreed that 198 encounters to 71 different clinicians were visits for acute pneumonia [kappa, 0.96; 95% confidence interval (CI), 0.93-0.98]. Measure performance ranged from 10% for providing location of care rationale to 91% for documenting blood pressure, averaging 52% across all 12 measures. Inter-rater reliability ranged from 0.66 (95% CI, 0.47-0.84) for providing a location of care rationale to 0.97 (95% CI, 0.91-1.0) for vital sign assessment. The proportion of data that was in coded form ranged from 0% for mental status, hydration status, chest x-ray performance, and location of care to 100% for medications and immunizatio…