Do low survey response rates bias results? Evidence from Japan
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
BACKGROUNDIn developed countries, response rates have dropped to such low levels that many in the population field question whether the data can provide unbiased results. OBJECTIVEThe paper uses three Japanese surveys conducted in the 2000s to ask whether low survey response rates bias results.A secondary objective is to bring results reported in the survey response literature to the attention of the demographic research community. METHODSUsing a longitudinal survey as well as paradata from a cross-sectional survey, a variety of statistical techniques (chi square, analysis of variance (ANOVA), logistic regression, ordered probit or ordinary least squares regression (OLS), as appropriate) are used to examine response-rate bias. RESULTSEvidence of response-rate bias is found for the univariate distributions of some demographic characteristics, behaviors, and attitudinal items.But when examining relationships between variables in a multivariate analysis, controlling for a variety of background variables, for most dependent variables we do not find evidence of bias from low response rates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Systematic Social Observation of Public Spaces: A New Look at Disorder in Urban Neighborhoods
Sampson, R. J., & Raudenbush, S. W. (1999) · American Journal of Sociology · doi:10.1086/210356 · Cited by 2,485
Nonresponse rates and nonresponse bias in household surveys
Groves, R. M. (2006) · Public Opinion Quarterly · doi:10.1093/poq/nfl033 · Cited by 2,420 · cited 6×
“…these three variables we make the untestable assumption, which has been a common assumption in the survey research literature (e.g., Curtin, Presser, and Singer 2000, 2005; Groves 2006), that there is a continuum that ranges from more-willing 14 responders to reluctant responders to non-responders.”
BackgroundResultInfluentialNonresponse in Household Interview Surveys
Groves, R. M., & Couper, M. P. (1998) · doi:10.1002/9781118490082 · Cited by 761
Using Survey Participants to Estimate the Impact of Nonparticipation
Lin, I.-F., & Schaeffer, N. C. (1995) · Public Opinion Quarterly · doi:10.1086/269471 · Cited by 356
Cited by
Domestic division of labour and fertility preference in China, Japan, South Korea, and Taiwan
Kan, M.-Y., & Hertog, E. (2017) · Demographic Research · doi:10.4054/demres.2017.36.18 · Cited by 99
Students' privacy concerns in learning analytics: Model development
Mutimukwe, C., Viberg, O., Oberg, L., & Cerratto‐Pargman, T. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13234 · Cited by 90
Davis, M., Gilbar, O., & Padilla-Medina, D. (2020) · doi:10.1101/2020.06.08.20125914 · Cited by 61
Purpose in life and tobacco use among community-dwelling mothers of early adolescents
Morimoto, Y., Yamasaki, S., Ando, S., Koike, S., Fujikawa, S., Kanata, S., Endo, K., Nakanishi, M., Hatch, S. L., Richards, M., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-020586 · Cited by 22
“A recent study indicated that response rates of national cross-sectional surveys in Japan have declined in the recent decade (from around 70% to 50%).(64) One reason would be that many apartment buildings in Tokyo have recently adopted an automated system that enables residents talk with visitors at the locked gate th…”
Background
Metadata
- Authors
- Rindfuss, R. R., Choe, M. K., Tsuya, N. O., Bumpass, L. L., & Tamaki, E.
- Venue
- Demographic Research
- Published
- 2015-03-25
- Volume
- 32
- Pages
- 797-828
- ISSN
- 1435-9871 (ISSN-L)
- OpenAlex
- W2166558366
- Cited by
- 215
- References
- 6
- FWCI
- 47.62
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.