Validating Risk-Adjusted Surgical Outcomes: Site Visit Assessment of Process and Structure1

Abstract

BACKGROUND: Risk-adjusted mortality and morbidity rates are often used as measures of the quality of surgical care. This study was conducted to determine the validity of risk-adjusted surgical morbidity and mortality rates as measures of quality of care by assessing the process and structure of care in surgical services with higher-than-expected and lower-than-expected risk-adjusted 30-day mortality and morbidity rates. STUDY DESIGN: A structural survey of 44 Veterans Affairs Medical Center surgical services and site visits to 20 surgical services with higher-than-expected and lower-than-expected risk-adjusted outcomes were conducted. Main outcome measures included assessment of technology and equipment, technical competence of staff, leadership, relationship with other services, monitoring of quality of care, coordination of work, relationship with affiliated institutions, and overall quality of care. RESULTS: Surgical services with lower-than-expected risk-adjusted surgical morbidity and mortality rates had significantly more equipment available in surgical intensive care units than did services with higher-than-expected outcomes (4.3 versus 2.9, p < 0.05). Site-visitor ratings of overall quality of care were significantly higher for surgical services with lower-than-expected morbidity and mortality rates (6.1 versus 4.5 for high outliers, p < 0.05); technology and equipment were rated significantly better among low-outlier services (7.1 versus 4.8 for high outliers, p < 0.001). Masked site-visit teams correctly predicted the outlier status (high versus low) of 17 of the 20 surgical services visited (p < 0.001). CONCLUSIONS: Significant differences in several dimensions of process and structure of the delivery of surgical care are associated with differences in risk-adjusted surgical morbidity and mortality rates among 44 Veterans Affairs Medical Centers.