Are Guidelines Following Guidelines?: The Methodological Quality of Clinical Practice Guidelines in the Peer-Reviewed Medical Literature

Abstract

CONTEXT: Practice guidelines play an important role in medicine. Methodological principles have been formulated to guide their development. OBJECTIVE: To determine whether practice guidelines in peer-reviewed medical literature adhered to established methodological standards for practice guidelines. DESIGN: Structured review of guidelines published from 1985 through June 1997 identified by a MEDLINE search. MAIN OUTCOME MEASURES: Mean number of standards met based on a 25-item instrument and frequency of adherence. RESULTS: We evaluated 279 guidelines, published from 1985 through June 1997, produced by 69 different developers. Mean overall adherence to standards by each guideline was 43.1% (10.77/25). Mean (SD) adherence to methodological standards on guideline development and format was 51.1% (25.3%); on identification and summary of evidence, 33.6% (29.9%); and on the formulation of recommendations, 46% (45%). Mean adherence to standards by each guideline improved from 36.9% (9.2/25) in 1985 to 50.4% (12.6/25) in 1997 (P<.001). However, there was little improvement over time in adherence to standards on identification and summary of evidence from 34.6% prior to 1990 to 36.1 % after 1995 (P = .11). There was no difference in the mean number of standards satisfied by guidelines produced by subspecialty medical societies, general medical societies, or government agencies (P = .55). Guideline length was positively correlated with adherence to methodological standards (P = .001). CONCLUSION: Guidelines published in the peer-reviewed medical literature during the past decade do not adhere well to established methodological standards. While all areas of guideline development need improvement, greatest improvement is needed in the identification, evaluation, and synthesis of the scientific evidence.