Comparing Data from Online and Face-to-face Surveys

Abstract

This paper explores some of the issues surrounding the use of internet-based methodologies, in particular the extent to which data from an online survey can be matched to data from a face-to-face survey. Some hypotheses about what causes differences in data from online panel surveys and nationally representative face-to-face surveys are discussed. These include: interviewer effect and social desirability bias in face-to-face methodologies; the mode effects of online and face-to-face survey methodologies, including how response scales are used; and differences in the profile of online panellists – both demographic and attitudinal. Parallel surveys were conducted using online panel and face-to-face (CAPI) methodologies, and data were compared before weighting, following demographic weighting and following ‘propensity score weighting ’ – a technique developed by Harris Interactive to correct for attitudinal differences typically found in online respondents. This paper looks at the differences in data from online and face-to-face surveys and puts forward some theories about why these differences might exist. The varying degrees of success of the weighting are also examined.