Nonresponse and mode effects in self‐ and interviewer‐administered surveys

Abstract

We examine the quality of two probability-based polls, one interviewer administered (telephone) and one self-administered (Internet and mail mixed mode survey). The polls use the same sampling frame (registered voters) and the same questions. First, we examine the representativeness of both surveys using information known about the population, and although we find important differences between the two in terms of sampling and nonresponse bias, we also find that both surveys represent the underlying population despite low response rates. We also test for mode effects between surveys due to social desirability and how it influences nondifferentiation or satisficing. Using a variety of methods (t-tests, multivariate regression, and genetic propensity matching), we find evidence that the presence of an interviewer alters response patterns on ego-driven questions. The implications of our work are important, due to the increasing popularity of mixed mode surveys. Researchers need to be methodologically sensitive to these differences when analyzing surveys that allow for different response modes.