Survey Questions: Handcrafting the Standardized Questionnaire

Abstract

This book reviews the experimental literature on how survey questions behave as well as the lore of professional experience and culls from them those guiding principles and specific findings that have implications for writing survey questions. Mindful that the findings of experiments cannot be applied mechanically and that survey questions cannot be mass-produced they have suggested a number of ways in which to make pilot and pretest work more fruitful. The authors have arranged ideas about writing questions in 3 classes or concentric circles which progressively narrow to the specific design task and the book is written in 3 chapters to match. Chapter 1 bears on general strategies culled from examples or experience of question-crafters and the findings of empirical researchers. This material is a litany of cautions offering more general perspectives than specific procedures. Chapter 2 focuses on specific empirical findings. In recent years there has been renewed research into question design and question effects and much has been learned about how some questions tend to behave. But the implications of this research for actual practice are not always clear. Chapter 3 tries to zero in on the actual task at hand. It discusses pilot work pretesting and making use of the advice of experts--critics colleagues and especially interviewers. It is commonplace that all survey questions must be pretested but there is no commonly shared tradition about how to go about it. The authors are most familiar with questionnaires designed for use with the broad American public--national cross-sections drawn by the Institute for Social Research or samples of the greater Detroit metropolitan area by the Detroit Area Study so some of the cautions urged in chapter 1 may not apply to surveying the well educated or intensely motivated such as college students legislators social scientists political activists and medical patients.