The validity of self-reported diabetes quality of care measures

Abstract

OBJECTIVE: First, to test the validity of self-reported quality of care and treatment measures compared with medical records and administrative data for: eye examinations, hemoglobin A1C tests, and use of insulin and oral agents for adult patients with diabetes; and secondly to assess the consistency between medical record information and administrative data for the same measures plus microalbumin testing. DESIGN: Cross-sectional study using data from telephone survey, primary care medical and eye records, and administrative claims. SETTING: Statewide health maintenance organization in Minnesota, USA, 1995. STUDY PARTICIPANTS: Four hundred and forty adults with diabetes, aged 31-64 years. MAIN OUTCOME MEASURES: Validity++ of self-reported diabetes quality of care measures compared with a criterion standard combining information from primary care and eye records with information from administrative data; and reliability of medical record information compared with administrative data. RESULTS: Although the sensitivity of self-reported eye examination was high (89%), the specificity was low (65%). Self-report of hemoglobin A1C also had high sensitivity (99%) and a lower specificity than that of eye examination (28%). The two information sources (medical records and claims) used in the criterion standard each contained complementary and non-overlapping information. Reliability was highest for microalbumin testing (kappa, 0.75) and lowest for eye examination (kappa, 0.37). CONCLUSIONS: Quality of care measures for diabetes are often drawn from a variety of sources. To the extent that data sources are biased, the measures can be misleading. Self-report is likely to lead to an overestimate of eye screening and the measurement of hemoglobin A1C. Reported rates of quality of care should be inspected carefully. The 'same' rate taken from different sources may vary.