Innovations in Experimental Design in Attitude Surveys

Abstract

In the last decade a revolution has occurred in the design of public opinion surveys. The principal breakthrough has been to combine the distinctive external validity advantages of the representative public opinion survey with the decisive internal validity strengths of the fully randomized, multifaceted experiment. The availability of computer-driven multifactorial, multivalent designs has encouraged a reorientation from narrowly methodological concerns to broader substantive issues. After a season in which the principal emphasis in survey-based experimentation was on standardization of measurement and methodological refinements, the emphasis now is on substantive discoveries and on innovation—new technology, new procedures, and new objectives. In this chapter, we survey the integration of experimental design and large-scale, representative, general population samples. After highlighting the limitations of the classic split-ballot experiment, we distinguish between nondirective and directive experimental variations, and, among the directive, between postdecisional and predecisional. We introduce a tripartite analytical scheme, sorting experimental variations as a function of whether they manipulate (i) the formulation of a choice, (ii) the context of choice, or (iii) the characteristics of the chooser. Finally we give an account of the experimental style now characteristic of general population attitude surveys, underlining its low emotional intensity and low cognitive demands, attributing both to features of (i) the interview site, (ii) the sample, (iii) the mode of interviewing, and (iv) considerations of ethics.