The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Selection Bias in Web Surveys
Open accessAbstract
Summary: At first sight, web surveys seem to be an interesting and attractive means of data collection. They provide simple, cheap, and fast access to a large group of potential respondents. However, web surveys are not without methodological problems. Specific groups in the populations are under‐represented because they have less access to Internet. Furthermore, recruitment of respondents is often based on self‐selection. Both under‐coverage and self‐selection may lead to biased estimates. This paper describes these methodological problems. It also explores the effect of various correction techniques (adjustment weighting and use of reference surveys). This all leads to the question whether properly design web surveys can be used for data collection. The paper attempts to answer this question. It concludes that under‐coverage problems may solve itself in the future, but that self‐selection leads to unreliable survey outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The reliability and validity of web-based self-reported mental disorder symptom clusters remains ambiguous,71 certainly for the current population”
From Mental Disorder Symptoms among Public Safety Personnel in Canada (2018) · cites this paper as [71]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In contrast, the current rates may be inflated because of a self-selection bias of motivated responding by clinically symptomatic persons.71”
From Mental Disorder Symptoms among Public Safety Personnel in Canada (2018) · cites this paper as [71]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,385
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008) · Cited by 5,345
A generalization of sampling without replacement from a finite universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483446 · Cited by 4,293
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
A Generalization of Sampling Without Replacement From a Finite Universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.2307/2280784 · Cited by 3,011
Couper, M. P. (2000) · Public Opinion Quarterly · doi:10.1086/318641 · Cited by 2,086
Neyman, J. (1934) · Journal of the Royal Statistical Society · doi:10.2307/2342192 · Cited by 1,086
The Effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies
Cochran, W. G. (1968) · Biometrics · doi:10.2307/2528036 · Cited by 961
Neyman, J. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_12 · Cited by 932
To mix or not to mix data collection modes in surveys.
de Leeuw, E. D. (2005) · Journal of Official Statistics · Cited by 808
Advantages and Disadvantages of Internet Research Surveys: Evidence from the Literature
Fricker, R. D., & Schonlau, M. (2002) · Field Methods · doi:10.1177/152582202237725 · Cited by 747
Conducting Research Surveys via E-mail and the Web
Schonlau, M., Fricker, R. D., & Elliott, M. N. (2002) · RAND Corporation eBooks · doi:10.7249/mr1480 · Cited by 603
Comparing Data from Online and Face-to-face Surveys
Duffy, B., Smith, K., Terhanian, G., & Bremer, J. (2005) · International Journal of Market Research · doi:10.1177/147078530504700602 · Cited by 569
Estimation in Surveys with Nonresponse
Särndal, C., & Lundström, S. (2005) · Wiley eBooks · doi:10.1002/0470011351 · Cited by 499
Designing Effective Web Surveys
Couper, M. P. (2008) · Cambridge University Press eBooks · doi:10.1017/cbo9780511499371 · Cited by 472
Sampling Methods for Censuses and Surveys.
Deming, W. E., & Yates, F. (1949) · Journal of the American Statistical Association · doi:10.2307/2279916 · Cited by 443
Dimensions of Internet science
Reips, U., & Bošnjak, M. (2001) · Cited by 319
Deming, W. E. (2010) · Andalas University Repository (Andalas University) · Cited by 292
Computer Assisted Survey Information Collection
(2000) · Technometrics · doi:10.1080/00401706.2000.10486039 · Cited by 202
C., A., & Deming, W. E. (1950) · Population · doi:10.2307/1523702 · Cited by 199
Representative Sampling, III: The Current Statistical Literature
Kruskal, W., & Mosteller, F. (1979) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1402647 · Cited by 194
Applied Survey Methods: A Statistical Perspective
Bethlehem, J. G. (2009) · Medical Entomology and Zoology · Cited by 187
Self-citationBethlehem, J. G. (2009) · Wiley eBooks · doi:10.1002/9780470494998 · Cited by 182
Self-citationA Comparison Between Responses From a Propensity-Weighted Web Survey and an Identical RDD Survey
Schonlau, M., Zapert, K., Simon, L. P., Haynes-Sanstad, K., Marcus, S. M., Adams, J., Spranca, M. D., Kan, H., Turner, R., & Berry, S. H. (2004) · Social Science Computer Review · doi:10.1177/0894439303256551 · Cited by 171
Weighting nonresponse adjustments based on auxiliary information
Bethlehem, J. G. (2002) · UvA-DARE (University of Amsterdam) · Cited by 143
Self-citationRepresentative Sampling, II: Scientific Literature, Excluding Statistics
Kruskal, W., & Mosteller, F. C. (1979) · International Statistical Review · doi:10.2307/1402564 · Cited by 141
Representative Sampling, I: Non-Scientific Literature
Kruskal, W., & Mosteller, F. C. (1979) · International Statistical Review · doi:10.2307/1403202 · Cited by 131
An Evaluation of the Effect of Response Formats on Data Quality in Web Surveys
Heerwegh, D., & Loosveldt, G. (2002) · Social Science Computer Review · doi:10.1177/089443902237323 · Cited by 89
Computer Assisted Survey Information Collection.
Morganstein, D. R. (1991) · Journal of the American Statistical Association · doi:10.2307/2290617 · Cited by 87
How Representative Are Online Panels? Problems of Coverage and Selection and Possible Solutions
Scherpenzeel, A. C. (2018) · Routledge eBooks · doi:10.4324/9780203844922-5 · Cited by 64
Bowley, A. L. (1906) · Journal Of The Royal Statistical Society · doi:10.2307/2339344 · Cited by 59
Access panels and online research, panacea or pitfall?
Stoop, I., & Wittenberg, M. (2008) · Data Archiving and Networked Services (DANS) · Cited by 20
Kendall, M. G. (1960) · Biometrika · doi:10.1093/biomet/47.3-4.447 · Cited by 18
Reducing the bias of web survey based estimates
Bethlehem, J. G. (2007) · Cited by 15
Self-citationAddress to the economic science and statistics section of the British Association
Giffen, R. (1887) · Printed by Spottiswoode eBooks · Cited by 1
Research works printed but not linked: 20
- Bethlehem J.G., 1988, Reduction of the nonresponse bias through regression estimation, J. Official Statist., 4, 251
- Bethlehem J.G., 2002, Survey Nonresponse
- Bethlehem J.G., 2009, The Rise of Survey Sampling
- Börsch‐Supan A., 2004, Correcting the Participation Bias in an Online Survey
- Bowley A.L., 1926, Measurement of the precision attained in sampling, Bull. Int. Statist. Inst., 6
- Cobben F., 2004, Nonresponse Correction Techniques in Household Surveys at Statistics Netherlands: a CAPI‐CATI Comparison
- Cochran W.G., 1953, Sampling Techniques
- Cochran W.G., 1977, Sampling Techniques
- De Haan J., 2006, Jaarboek ICT en Samenleving, De Digitale Generatie
- Eurostat, 2007, More than 40% of Households have Broadband Internet Access
- Hansen M.H., 1953, Survey Sampling Methods and Theory
- Kendall M.G., 1960, Where shall the history of statistics begin, Biometrika, 47, 447
- Kiaer A. N., 1895, Observations et Expériences Concernant des Dénombrements Représentatives, Bull. Internat. Statist. Inst., 176
- Kiaer A. N., 1997, Den Repräsentative Undersökelsesmetode, Christiania Videnskabsselskabets Skrifter. II. Historiskfilosofiske klasse, 4, 37
- Kish L., 1967, Survey Sampling
- Kuusela V., 2003, Proceedings of the 4th ASC International Conference, 317
- 10.1111/j.2397-2335.1934.tb04184.x · DOI
- Scherpenzeel A., 2010, Social Research and the Internet: Advances in applied Methods and New Research Strategies
- Terhanian G., 2001, Exploiting analytical advances: Minimizing the biases associated with Internet‐based surveys of non‐random samples, ESOMAR Publication Services, 248, 247
- Vonk T., 2006, The effects of panel recruitment and management on research results, a study among 19 online panels, ESOMAR Publication Services, 317, 79
Cited by
Sorted by citations
Expectations of brilliance underlie gender distributions across academic disciplines
Leslie, S.-J., Cimpian, A., Meyer, M., & Freeland, E. (2015) · Science · doi:10.1126/science.1261375 · Cited by 1,516
Mental Disorder Symptoms among Public Safety Personnel in Canada
Carleton, R. N., Afifi, T. O., Turner, S., Taillieu, T., Duranceau, S., LeBouthillier, D. M., Sareen, J., Ricciardelli, R., MacPhee, R. S., Groll, D., et al. (2018) · The Canadian Journal of Psychiatry · doi:10.1177/0706743717723825 · Cited by 558
“In contrast, the current rates may be inflated because of a self-selection bias of motivated responding by clinically symptomatic persons.(71) That argument might be countered by notions that particularly symptomatic persons may also be less likely to participate in research.”
BackgroundThe role of urban green space for human well-being
Bertram, C., & Katrin Rehdanz, A. (2015) · Ecological Economics · doi:10.1016/j.ecolecon.2015.10.013 · Cited by 515
Associations between green/blue spaces and mental health across 18 countries
White, M. P., Elliott, L. R., Grellier, J., Economou, T., Bell, S., Bratman, G. N., Cirach, M., Gascón, M., Lima, M. L., Lõhmus, M., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-87675-0 · Cited by 557
Krieger, N. (2020) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040119-094017 · Cited by 411
“, mainly highly educated and affluent persons) (18, 22, 201).”
Background
Metadata
- Authors
- Bethlehem, J.
- Affiliations
- Centraal Bureau voor de Statistiek (NL)
- Corresponding author
- Bethlehem, Jelke
- Venue
- International Statistical Review
- Publisher
- Wiley
- Published
- 2010-08
- Volume
- 78
- Issue
- 2
- Pages
- 161-188
- ISSN
- 0306-7734 (ISSN-L)
- 1751-5823
- OpenAlex
- W2129332419
- Cited by
- 1,315
- References
- 37
- Citations by year
- 2026: 228 · 2025: 173 · 2024: 151 · 2023: 149 · 2022: 122 · 2021: 144 · 2020: 109 · 2019: 47 · 2018: 38 · 2017: 54 · 2016: 20 · 2015: 32 · 2014: 19 · 2013: 14 · 2012: 6
- FWCI
- 29.95
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Statistical Review (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Census and Population Estimation
- Keywords
- online surveys, selection bias, undercoverage, self-selection, internet access, survey design, participant recruitment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
