Words Without Action? The Production, Dissemination, and Impact of Consensus Recommendations

Abstract

When existing evaluations find little or no evidence of consensus recommendations leading to action, one can justifiably ask why so much of this review was dedicated to analyzing alternative ways of producing such "words without action." There are, however, at least two reasons why consensus recommendations should be produced with care and attention to validity. First, recommendations do sometimes have an impact on behavior as a consequence of mere dissemination activity--the Dutch program, for instance, was more successful than most. This success may occur when the target audience is already particularly receptive to change and the message is timely and delivered by a credible source in a clinically relevant way. Thus, although "such a conjunction of favorable conditions is probably the exception rather than the rule for consensus topics" (46, 240) it does happen. Second, the output from consensus processes is increasingly a potential input to other processes. Consensus recommendations can be used as the criteria for evaluation and appraisal aimed at changing practice behavior, making administrative decisions on resource allocation, or defining research protocols. For instance, quality assurance activities, such as peer assessment, practitioner certification, or utilization review, are actively seeking criteria with which to make judgments and elicit changes in practice to improve the quality of care. Funding agencies are looking for information to help make reimbursement, capital expenditure, or fee-for-service decisions on cessation of insurance for particular procedures or approaches. These uses of the consensus criteria are potentially major and controversial. Therefore, even if dissemination rarely leads to action, consensus processes should still be done carefully and with valid techniques. The use of their recommendations embedded within other activities may well lead to (forced) changes in behavior. On ethical grounds alone, we should be as sure as possible that the behavior changes being implied and encouraged are indeed advisable. For these reasons, the review describes the decision points in the production process for consensus recommendations as a start on the development of a set of recognized standards. The review offers a critical appraisal of the various methodological choices available at each decision point. The seven decision points are selecting a topic, picking the consensus group, providing background preparation, identifying information inputs, choosing a group judgment process, defining the criteria for recommendations, and choosing a report preparation procedure and format. At least two important points emerged from this review. First, the research is often not well enough developed to give clear indications for many of the choices on what is the "best" alternative.(ABSTRACT TRUNCATED AT 400 WORDS)