Nonresponse Rates are a Problematic Indicator of Nonresponse Bias in Survey Research

Abstract

Nonresponse Rates are a Problematic Indicator of Nonresponse Bias in Survey Research 1Survey researchers are rightly concerned with measuring the level of potential bias in estimates generated from the surveys. 2 Bias in estimates can result from measurement error, processing/editing error, coverage error, and nonresponse error (Federal Committee on Statistical Methodology [FCSM] 2001).It is nonresponse bias that is the focus of this editorial and it is also the subject of the paper by Halbesleben and Whitman (2013) that this editorial accompanies.Nonresponse bias is a perennial concern for survey researchers as not everyone we attempt to include in our surveys responds.And to the extent that nonrespondents are different from respondents on the key variables the survey was designed to study these differences could bias the very estimates the survey was designed to make.Because we often have very little information about those who do not respond, survey researchers have long focused on the response rate as a key indicator of survey quality.The assumption is that the more nonresponse there is in a survey, the higher the potential for nonresponse bias.Unfortunately, this assumption has served as a problematic diversion for survey research from the real concern of how survey nonresponse potentially biases survey estimates. WHAT IS KNOWN ABOUT RESPONSE RATESResponse rates lack both validity and reliability as a proxy measure of nonresponse bias.Response rates lack validity in that there is not even a moderate correlation with nonresponse bias (Groves 2006).Groves' analysis showed that there is a significant amount of variability in nonresponse bias from one