Estimating Nonresponse Bias in Mail Surveys
The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
Valid predictions for the direction of nonresponse bias were obtained from subjective estimates and extrapolations in an analysis of mail survey data from published studies. For estimates of the magnitude of bias, the use of extrapolations led to substantial improvements over a strategy of not using extrapolations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The t -test results did not find significant differences between the two respondent groups, suggesting an absence of non-response bias (Armstrong and Overton, 1977).”
From Understanding managers’ attitudes and behavioral intentions towards using artificial intelligence for organizational decision-making (2021) · cites this paper as Armstrong and Overton, 1977
In the citing paper
“Prior to data analysis, non-response bias was assessed through wave analysis, comparing the responses of the early respondent wave (first quartile) to those of the late respondent wave (fourth quartile) using two tailed tstatistics (Armstrong and Overton, 1977; Rogelberg and Stanton, 2007).”
From Examining the antecedents of the technology acceptance model within e-procurement (2018) · cites this paper as Armstrong and Overton, 1977
In the citing paper
“Non-response bias was estimated using the method suggested by Armstrong and Overton (1977). We compared early respondents with late respondents for all the constructs, including the control variables. None of the t-statistics for difference in means were statistically significant, suggesting that non-response bias was not an issue.”
From Trust, Satisfaction, and Online Repurchase Intention: The Moderating Role of Perceived Effectiveness of E-Commerce Institutional Mechanisms1 (2014) · cites this paper as Armstrong and Overton (1977)
In the citing paper
“We examined for the potential for non-response bias by comparing the data collected in two waves, following Armstrong and Overton's (1977) guidelines.”
From Big data analytics and artificial intelligence pathway to operational performance under the effects of entrepreneurial orientation and environmental dynamism: A study of manufacturing organisations (2020) · cites this paper as Armstrong and Overton's (1977)
Citations
Cites
Mail Surveys and Response Rates: A Literature Review
Kanuk, L., & Berenson, C. (1975) · Journal of Marketing Research · doi:10.2307/3151093 · Cited by 855
The Problem of Non-Response in Sample Surveys
Hansen, M. H., & Hurwitz, W. N. (1946) · Journal of the American Statistical Association · doi:10.1080/01621459.1946.10501894 · Cited by 599
A Critical Comparison of Three Strategies of Collecting Data from Households
Hochstim, J. R. (1967) · Journal of the American Statistical Association · doi:10.1080/01621459.1967.10500909 · Cited by 418
Stimulating Responses to Mailed Questionnaires: A Review
Linsky, A. S. (1975) · Public Opinion Quarterly · doi:10.1086/268201 · Cited by 398 · cited 1×
“Nonresponse can be kept under 30% in most situations if appropriate procedures are followed [25, 32, 44].”
BackgroundScott, C. (1961) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2984129 · Cited by 258 · cited 4×
“Nonresponse can be kept under 30% in most situations if appropriate procedures are followed [25, 32, 44].”
BackgroundInfluentialMail Surveys and Response Rates: A Literature Review
Kanuk, L., & Berenson, C. (1975) · Journal of Marketing Research · doi:10.1177/002224377501200408 · Cited by 248 · cited 1×
“Nonresponse can be kept under 30% in most situations if appropriate procedures are followed [25, 32, 44].”
BackgroundRosenthal, R. (1965) · Human Relations · doi:10.1177/001872676501800407 · Cited by 200 · cited 2×
“Rosenthal, Robert, “The volunteer subject,” Human Relations, 18 (1965), 389-406.”
BackgroundAgricultural Economics Research
Bolton, B. (1964) · Cited by 199
Social psychology : an experimental approach
Zajonc, R. B. (1966) · Cited by 156
Methodological Bias in Public Opinion Surveys
Wiseman, F. (1972) · Public Opinion Quarterly · doi:10.1086/267981 · Cited by 154 · cited 1×
“However, as the known values come from a different source instrument, differences may occur as a result of response bias [17, 50] rather than nonresponse bias.”
BackgroundEstimating Bias Due to Nonresponse in Mail Surveys
Filion, F. L. (1975) · Public Opinion Quarterly · doi:10.1086/268245 · Cited by 147 · cited 4×
“Filion [14] reanalyzed data from Ellis et al. [10] and concluded that, in fact, extrapolation did help.”
MethodInfluentialImplications of Nonresponse for the Interpretation of Mail Questionnaire Data
Donald, M. N. (1960) · Public Opinion Quarterly · doi:10.1086/266934 · Cited by 132 · cited 2×
“The "interest hypothesis" is another widely recommended basis for subjective estimates [2, 8, 9, 17].”
BackgroundHolmes, C., & Erdos, P. L. (1971) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2346638 · Cited by 128
Bills, D., & Erdos, P. L. (1977) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2065364 · Cited by 116
Factors influencing questionnaire returns from former university students.
Pace, C. (1939) · Journal of Applied Psychology · doi:10.1037/h0063286 · Cited by 112 · cited 1×
“Extrapolation methods are based on the assumption that subjects who respond less readily are more like nonrespondents [37].”
Controlling Bias in Mail Questionnaires
Clausen, J. A., & Ford, R. N. (1947) · Journal of the American Statistical Association · doi:10.1080/01621459.1947.10501943 · Cited by 98 · cited 2×
“from [6, 30, 44], two items from [34], three from [I], seven from [39], and 24 each from [8] and [23].”
BackgroundThe Use of Potential Nonrespondents for Studying Nonresponse Bias
Ellis, R. A., Endo, C. M., & Armer, J. M. (1970) · The Pacific Sociological Review · doi:10.2307/1388313 · Cited by 75
The Problem of Bias in Mail Returns: A Solution
Ferber, R. C. (1948) · Public Opinion Quarterly · doi:10.1086/266009 · Cited by 69 · cited 1×
“Tables for R and Q can be found in [12] and [18].”
BackgroundExploring and Correcting for Nonresponse Bias Using Follow-Ups of Nonrespondents
Filion, F. L. (1976) · The Pacific Sociological Review · doi:10.2307/1388756 · Cited by 68 · cited 5×
“Filion [14] reanalyzed data from Ellis et al.”
MethodInfluentialLansing, J. B., & Morgan, J. N. (1971) · Deep Blue (University of Michigan) · Cited by 63
A Case For - and Against - Mail Questionnaires
Wallace, D. (1954) · Public Opinion Quarterly · doi:10.1086/266484 · Cited by 62 · cited 2×
“Thus, if the initial data covered 70% of the sample, what part of the remaining 30% is covered by the criterion data? The method of data collection is important because respondents may answer differently in a mail survey than they would in a personal interview if the questions concern sensitive items [16, 23, 49].”
BackgroundSuchman, E. A., & McCandless, B. (1940) · Journal of Applied Psychology · doi:10.1037/h0063437 · Cited by 61 · cited 1×
“It involves the assumption that people who are more interested in the subject of a questionnaire respond more readily [1, 30, 40, 41, 47], and that nonresponse bias occurs on items in which the subject's answer is related to his interest in the questionnaire [4].”
BackgroundPersonal Follow-Up in a Mail Survey: Its Contribution and Its Cost
Hochstim, J. R., & Athanasopoulos, D. A. (1970) · Public Opinion Quarterly · doi:10.1086/267774 · Cited by 51 · cited 2×
“The authors found eight additional studies with 63 biased items – one item each from [6, 30, 44], two items from [34], three from [1], seven from [39], and 24 each from [8] and [23].”
BackgroundCan the Mail-Back Bias Contribute to a Study's Validity?
Larson, R. F., & Catton, W. R. (1959) · American Sociological Review · doi:10.2307/2089435 · Cited by 51
A Note on Nonresponse in a Mail Survey
Mayer, C. S., & Pratt, R. W. (1966) · Public Opinion Quarterly · doi:10.1086/267461 · Cited by 47 · cited 1×
“The authors found eight additional studies with 63 biased items – one item each from [6, 30, 44], two items from [34], three from [1], seven from [39], and 24 each from [8] and [23].”
BackgroundResponse Bias in a Mail Survey
Baur, E. J. (1947) · Public Opinion Quarterly · doi:10.1086/265895 · Cited by 45
The sophisticated poll watcher's guide
Gallup, G. H. (1976) · Medical Entomology and Zoology · Cited by 44
Notes on the validity of mail questionnaire returns.
Stanton, F. (1939) · Journal of Applied Psychology · doi:10.1037/h0061723 · Cited by 43 · cited 1×
“In total, however, there were only nine items-three items each from [1, 16], two from [44], and one from [45].”
BackgroundDifferences Between Persons Responding and Not Responding to a Mailed Questionnaire
Reuss, C. F. (1943) · American Sociological Review · doi:10.2307/2085805 · Cited by 43 · cited 1×
“It involves the assumption that people who are more interested in the subject of a questionnaire respond more readily [1, 30, 40, 41, 47], and that nonresponse bias occurs on items in which the subject's answer is related to his interest in the questionnaire [4].”
BackgroundWeighting to Adjust for Survey Nonresponse
Fuller, C. H. (1974) · Public Opinion Quarterly · doi:10.1086/268154 · Cited by 42 · cited 1×
“Tables for R and Q can be found in [12] and [18].”
BackgroundNonresponse in Sociological Surveys
Daniel, W. W. (1975) · Sociological Methods & Research · doi:10.1177/004912417500300303 · Cited by 42 · cited 1×
“Still another approach to the nonresponse problem, the one examined herein, is to estimate the effects of nonresponse [ 7 , 21].”
BackgroundMail Questionnaire as a Research Problem
Franzen, R. H., & Lazarsfeld, P. F. (1945) · The Journal of Psychology · doi:10.1080/00223980.1945.9917260 · Cited by 36
Objective Differences Among Various Types of Respondents to a Mailed Questionnaire
Edgerton, H. A., Britt, S. H., & Norman, R. D. (1947) · American Sociological Review · doi:10.2307/2087289 · Cited by 36
Characteristics of Willing and Reluctant Respondents
Pavalko, R. M., & Lutterman, K. G. (1973) · The Pacific Sociological Review · doi:10.2307/1388500 · Cited by 29
Some personality differences between responders and nonresponders to a survey questionnaire.
Lubin, B., Levitt, E. E., & Zuckerman, M. (1962) · Journal of Consulting Psychology · doi:10.1037/h0048697 · Cited by 29 · cited 1×
“For example, respondents generally are better educated than nonrespondents [6, 44, 49], and there may be differences in personality between respondents and nonrespondents [33, 48].”
BackgroundQuestionnaire-Return Bias in the Study of Blue-Collar Workers
Schwirian, K. P., & Blaine, H. R. (1966) · Public Opinion Quarterly · doi:10.1086/267463 · Cited by 28 · cited 1×
“Furthermore, they have been shown to have some validity in [43], where the direction of bias was correctly predicted for each of 17 items.”
BackgroundBENSON, L. E. (1946) · Public Opinion Quarterly · doi:10.1093/poq/10.2.234 · Cited by 24 · cited 1×
“The “interest hypothesis” is another widely recommended basis for subjective estimates [2, 8, 9, 17].”
BackgroundSocioeconomic Status and Familial Variables in Mail Questionnaire Responses
Vincent, C. E. (1964) · American Journal of Sociology · doi:10.1086/223695 · Cited by 22 · cited 2×
“For example, respondents generally are better educated than nonrespondents [6, 44, 49], and there may be differences in personality between respondents and nonrespondents [33, 48].”
BackgroundValidity of extrapolating nonresponse bias from mail questionnaire follow-ups.
Zimmer, H. (1956) · Journal of Applied Psychology · doi:10.1037/h0047262 · Cited by 19 · cited 3×
“Extrapolations across two waves were examined by Zimmer [52]; improved predictions were found for five of seven items.”
InfluentialSocial science data archives : progress and prospects
Bisco, R. L. (1967) · Social Science Information · doi:10.1177/053901846700600102 · Cited by 17 · cited 1×
“With the establishment of data archives [3, 24], the reanalysis of survey data is likely to increase in popularity.”
BackgroundCorrecting Nonresponse Bias in Mail Questionnaires
Ognibene, P. (1971) · Journal of Marketing Research · doi:10.1177/002224377100800213 · Cited by 16 · cited 1×
“Many researchers have concluded that it is not possible to obtain valid estimates [10, 23, 29, 36].”
BackgroundBias in Mail Surveys Cannot Be Controlled by One Mailing
Ford, R. N., & Zeisel, H. (1949) · Public Opinion Quarterly · doi:10.1086/266098 · Cited by 15 · cited 6×
“Ford and Zeisel [16] did not find such extrapolations to be valid.”
BackgroundMethodInfluentialRollins, M. (1940) · Journal of Applied Psychology · doi:10.1037/h0058353 · Cited by 12 · cited 1×
“It involves the assumption that people who are more interested in the subject of a questionnaire respond more readily [1, 30, 40, 41, 47], and that nonresponse bias occurs on items in which the subject's answer is related to his interest in the questionnaire [4].”
BackgroundKirchner, W. K., & Mousley, N. M. (1963) · Journal of Applied Psychology · doi:10.1037/h0043714 · Cited by 9
Just How Credible Are Your Market Estimates?
Brown, R. V. (1969) · Journal of Marketing · doi:10.1177/002224296903300308 · Cited by 7
Research works printed but not linked: 9
- 10.1086/265785 · doi:10.1086/265785
- Finkner A. L., 1950, North Carolina Agricultural Experiment Station Technical Bulletin, 91, 374
- Hyman Herbert, 1972, A Secondary Analysis of Sample Surveys.
- Kish Leslie, 1965, Survey Sampling.
- Kivlin Joseph E., 1965, Rural Sociology, 30, 332
- Lehman Edward C., 1963, Rural Sociology, 28, 284
- NewmanSheldon W. “Differences Between Early and Late Respondents to a Mailed Survey,” Journal of Advertising Research, 2 (June 1962), 37–9. · doi:10.1080/00218499.1962.12519677
- Reid Seerley, 1942, Educational Research Bulletin, 21, 87
- Stephan Frederick F., 1958, Sampling Opinions.
Cited by
Chaudhuri, A., & Holbrook, M. B. (2001) · Journal of Marketing · doi:10.1509/jmkg.65.2.81.18255 · Cited by 6,674
“Finally, the sample was split into early and late respondents (Armstrong and Overton 1977).”
BackgroundComputer Self-Efficacy: Development of a Measure and Initial Test
Compeau, D. R., & Higgins, C. A. (1995) · MIS Quarterly · doi:10.2307/249688 · Cited by 6,553
(2003) · International Journal of Electronic Commerce · doi:10.1080/10864415.2003.11044275 · Cited by 5,715
“html), and the early (65%) and late respondents (35%) [5].”
BackgroundIndex Construction with Formative Indicators: An Alternative to Scale Development
Diamantopoulos, A., & Winklhofer, H. M. (2001) · Journal of Marketing Research · doi:10.1509/jmkr.38.2.269.18845 · Cited by 3,748
Consumer trust, value, and loyalty in relational exchanges
Sirdeshmukh, D., Singh, J., & Sabol, B. (2002) · Journal of Marketing · doi:10.1509/jmkg.66.1.15.18449 · Cited by 4,196
Metadata
- Authors
- Armstrong, J. S., & Overton, T. S.
- Affiliations
- University of Pennsylvania (US)
- ORCID
- Armstrong, J. Scott
- Venue
- Journal of Marketing Research
- Publisher
- SAGE Publications
- Published
- 1977-08
- Volume
- 14
- Issue
- 3
- Pages
- 396-402
- ISSN
- 0022-2437 (ISSN-L)
- 1547-7193
- OpenAlex
- W2134884007
- Cited by
- 10,390
- References
- 45
- Citations by year
- 2026: 848 · 2025: 736 · 2024: 723 · 2023: 876 · 2022: 889 · 2021: 894 · 2020: 766 · 2019: 704 · 2018: 518 · 2017: 394 · 2016: 315 · 2015: 299 · 2014: 264 · 2013: 229 · 2012: 274
- FWCI
- 5.10
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Journal of Marketing Research (publishedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Census and Population Estimation
- Keywords
- nonresponse bias, mail surveys, extrapolation, survey data quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.