Cited by 1,315Survey Methodology and NonresponseSurvey Sampling and Estimation TechniquesCensus and Population Estimation

The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.

Selection Bias in Web Surveys

Open access

Bethlehem, Jelke·2010·International Statistical Review

DOI 10.1111/j.1751-5823.2010.00112.x

Open this paper in CoChat — free account Open PDF

Abstract

Summary: At first sight, web surveys seem to be an interesting and attractive means of data collection. They provide simple, cheap, and fast access to a large group of potential respondents. However, web surveys are not without methodological problems. Specific groups in the populations are under‐represented because they have less access to Internet. Furthermore, recruitment of respondents is often based on self‐selection. Both under‐coverage and self‐selection may lead to biased estimates. This paper describes these methodological problems. It also explores the effect of various correction techniques (adjustment weighting and use of reference surveys). This all leads to the question whether properly design web surveys can be used for data collection. The paper attempts to answer this question. It concludes that under‐coverage problems may solve itself in the future, but that self‐selection leads to unreliable survey outcomes.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1111/j.1751-5823.2010.00112.x

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,385
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008) · Cited by 5,345
A generalization of sampling without replacement from a finite universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483446 · Cited by 4,293
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
A Generalization of Sampling Without Replacement From a Finite Universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.2307/2280784 · Cited by 3,011
Web Surveys
Couper, M. P. (2000) · Public Opinion Quarterly · doi:10.1086/318641 · Cited by 2,086
On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection
Neyman, J. (1934) · Journal of the Royal Statistical Society · doi:10.2307/2342192 · Cited by 1,086
The Effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies
Cochran, W. G. (1968) · Biometrics · doi:10.2307/2528036 · Cited by 961

Show all 37 references

Research works printed but not linked: 20

Bethlehem J.G., 1988, Reduction of the nonresponse bias through regression estimation, J. Official Statist., 4, 251
Bethlehem J.G., 2002, Survey Nonresponse
Bethlehem J.G., 2009, The Rise of Survey Sampling
Börsch‐Supan A., 2004, Correcting the Participation Bias in an Online Survey
Bowley A.L., 1926, Measurement of the precision attained in sampling, Bull. Int. Statist. Inst., 6
Cobben F., 2004, Nonresponse Correction Techniques in Household Surveys at Statistics Netherlands: a CAPI‐CATI Comparison
Cochran W.G., 1953, Sampling Techniques
Cochran W.G., 1977, Sampling Techniques
De Haan J., 2006, Jaarboek ICT en Samenleving, De Digitale Generatie
Eurostat, 2007, More than 40% of Households have Broadband Internet Access
Hansen M.H., 1953, Survey Sampling Methods and Theory
Kendall M.G., 1960, Where shall the history of statistics begin, Biometrika, 47, 447
Kiaer A. N., 1895, Observations et Expériences Concernant des Dénombrements Représentatives, Bull. Internat. Statist. Inst., 176
Kiaer A. N., 1997, Den Repräsentative Undersökelsesmetode, Christiania Videnskabsselskabets Skrifter. II. Historiskfilosofiske klasse, 4, 37
Kish L., 1967, Survey Sampling
Kuusela V., 2003, Proceedings of the 4th ASC International Conference, 317
10.1111/j.2397-2335.1934.tb04184.x · DOI
Scherpenzeel A., 2010, Social Research and the Internet: Advances in applied Methods and New Research Strategies
Terhanian G., 2001, Exploiting analytical advances: Minimizing the biases associated with Internet‐based surveys of non‐random samples, ESOMAR Publication Services, 248, 247
Vonk T., 2006, The effects of panel recruitment and management on research results, a study among 19 online panels, ESOMAR Publication Services, 317, 79

Sorted by citations

Items 301–400 of 1,000

High-resolution app data reveal sustained increases in recreational fishing effort in Europe during and after COVID-19 lockdowns
Audzijonytė, A., Mateos‐González, F., Dainys, J., Gundelund, C., Skov, C., DeWeber, J. T., Venturelli, P., Vienožinskis, V., & Smith, C. (2023) · Royal Society Open Science · doi:10.1098/rsos.230408 · Cited by 17
Predictors of transitions from single to multiple job holding: Results of a longitudinal study among employees aged 45‐64 in the Netherlands
Bouwhuis, S., Geuskens, G., Boot, C. R. L., Bongers, P. M., & van der Beek, A. J. (2017) · American Journal of Industrial Medicine · doi:10.1002/ajim.22738 · Cited by 18
“36 Response analyses of the baseline questionnaire showed that persons aged 60-64 years and those with a low level of education participated slightly less often in STREAM.”
Long term prevention and vector control of arboviral diseases: What does the future hold?
Cabral, B. P., Fonseca, M. D. G. D., & Mota, F. B. (2019) · International Journal of Infectious Diseases · doi:10.1016/j.ijid.2019.10.002 · Cited by 17
Antecedents and the underlying mechanism of customer intention of co-creating a dining experience
Im, J., Qu, H., & Beck, J. A. (2020) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2020.102715 · Cited by 17
Healthcare Workers’ Experiences and Views of Using Surgical Masks and Respirators, and Their Attitudes on the Sustainability: A Semi-Structured Survey Study during COVID-19
Venesoja, A., Grönman, K., Tella, S., Hiltunen, S., Koljonen, K., Butylina, S., Rotinen, L., Torkki, P. M., & Laatikainen, K. (2021) · Nursing Reports · doi:10.3390/nursrep11030059 · Cited by 18
“First, convenience sampling could have caused a self-selection bias [47].”
Background
What drives the changes in public transport use in the context of the COVID‐19 pandemic? Highlights from Lyon metropolitan area
Zein, A. E., Béziat, A., Pochet, P., Klein, O., & Vincent‐Geslin, S. (2022) · Regional Science Policy & Practice · doi:10.1111/rsp3.12519 · Cited by 18
“As a survey medium, the internet can lead to representativeness biases, because of unequal ease of access and self-selection biases (Bayart & Bonnel, 2012; Bethlehem, 2010).”
Background
Stakeholder engagement variability across public, private and public-private partnership projects: A data-driven network-based analysis
Uddin, S., Ong, S., & Matouš, P. (2023) · PLoS ONE · doi:10.1371/journal.pone.0279916 · Cited by 16
Mapping GPs’ motivation — it’s not all about the money: a nationwide cross-sectional survey study from Denmark
Yordanov, D., Oxholm, A. S., Gyrd‐Hansen, D., & Pedersen, L. B. (2023) · British Journal of General Practice · doi:10.3399/bjgp.2022.0563 · Cited by 19
“17 It”
Forecasting virus outbreaks with social media data via neural ordinary differential equations
Núñez, M., Barreiro, N. L., Barrio, R. A., & Rackauckas, C. (2023) · Scientific Reports · doi:10.1038/s41598-023-37118-9 · Cited by 21
Comparing responses from a paper-based survey with a web-based survey in environmental criminology
Ceccato, V. A., Gliori, G., Näsman, P., & Sundling, C. (2024) · Crime Prevention and Community Safety · doi:10.1057/s41300-024-00204-9 · Cited by 18
“The web-based survey may also be subject to self-selection bias, which occurs when individuals choose whether to participate, leading to a sample that may not represent the broader population (Anderson et al. 2022; Bethlehem 2010; Khazaal et al. 2014).”
What is a dog bite? Perceptions of UK dog bite victims
Oxley, J. A., Christley, R., & Westgarth, C. (2018) · Journal of Veterinary Behavior · doi:10.1016/j.jveb.2018.09.002 · Cited by 16
“…to this study, including the method of survey distribution being via social media, which has been found to potentially result in selection bias (e.g., self-selection and under coverage bias [Bethlehem, 2010]) and may account for the overrepresentation of female and dog-owning respondents.”
Method
The Cyber Self: Facebook as a Predictor of Well‐being
Whitman, C. N., & Gottdiener, W. H. (2015) · International Journal of Applied Psychoanalytic Studies · doi:10.1002/aps.1431 · Cited by 16
A survey of the current practice of intramuscular Botulinum toxin injections for hemiplegic shoulder pain in the UK
Holmes, R. J., & Connell, L. (2017) · Disability and Rehabilitation · doi:10.1080/09638288.2017.1400596 · Cited by 16
“There may have been a degree of self-selection bias as clinicians with a particular interest in the topic area may be more likely to opt in to the study, leading to under-coverage of the targeted sample [39].”
Background
Same-sex sexual behaviour among mammals is widely observed, yet seldomly reported: Evidence from an online expert survey
Anderson, K. A., Teichroeb, J. A., Ramsay, M. S., Bădescu, I., López‐Torres, S., & Gibb, J. K. (2024) · PLoS ONE · doi:10.1371/journal.pone.0304885 · Cited by 15
Consumer preferences for the design of a demand response quota scheme – Results of a choice experiment in Germany
Lehmann, N., Sloot, D., Ardone, A., & Fïchtner, W. (2022) · Energy Policy · doi:10.1016/j.enpol.2022.113023 · Cited by 15
“For example, researchers are unable to control for external influences, and self-selection bias may be present (see Bethlehem, 2010).”
Correcting Selection Bias in Big Data by Pseudo-Weighting
Liu, A.-C., Scholtus, S., & de Waal, T. (2022) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smac029 · Cited by 15
The Effect of the Covid-19 Pandemic on Domestic Violence in Germany: A Comparison of Three Representative Population Surveys
Kliem, S., von Thadden, A., Lohmann, A., Kröger, C. B., & Baier, D. (2023) · Journal of Interpersonal Violence · doi:10.1177/08862605221143194 · Cited by 15
“However, in this study, convenience samples (k = 23) or online samples (k = 6) were predominantly pooled, which is linked to several limitations (e.g., selection and response bias; Bethlehem, 2010).”
Background
Welcher Zweck heiligt die Mittel? Bemerkungen zur Repräsentativitätsdebatte in der Meinungsforschung
Köhler, U., & Post, J. C. (2023) · Zeitschrift für Soziologie · doi:10.1515/zfsoz-2023-2001 · Cited by 20
Illness invalidation and psychological distress in adults with chronic physical health symptoms
Woldhuis, T., & Gandy, M. (2024) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2024.10.001 · Cited by 15
Perceptions of Canadian Federal Policy Responses to COVID-19 among People with Disabilities and Chronic Health Conditions
Pettinicchio, D., Maroto, M. L., & Lukk, M. (2021) · Canadian Public Policy · doi:10.3138/cpp.2021-012 · Cited by 16
“In particular, we did not use poststrati ﬁ cation, a weighting method commonly used when analyzing data from web-based surveys to adjust for undercoverage, nonresponse, and self-selection in the sample ( Bethlehem 2010 ; Schonlau et al. 2009 ).”
Effect of Career Adaptability on Subjective Well-Being of Middle-Aged and Older Employees
Takao, M., & Ishiyama, N. (2021) · Sustainability · doi:10.3390/su13052570 · Cited by 19
“Internet surveys are known to have a particularly large bias in elderly samples [58], and a large bias is thus expected in the sample of those who have been rehired at retirement age.”
Background
Longitudinal evidence on mental health changes of college students with and without mental disorder background during the Argentina's lengthy mandatory quarantine
Steinmetz, L. C. L., Fong, S. B., & Godoy, J. C. (2021) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2021.110308 · Cited by 15
Impacts of new and emerging assistive technologies for ageing and disabled housing
Bridge, C. E., Zmudzki, F., Huang, T., Owen, C., & Faulkner, D. (2021) · AHURI Final Report · doi:10.18408/ahuri7122501 · Cited by 15
(Un)beliveable wages? An analysis of minimum wage policies in Europe from a living wage perspective
Fabo, B., & Belli, S. S. (2017) · IZA Journal of Labor Policy · doi:10.1186/s40173-017-0083-3 · Cited by 15
“There is a wide debate in the literature about the merit of such convenience sample-based surveys (Couper 2000; Dillman and Bowker 2001; Bethlehem 2010).”
Background
What Women Want: Employment Preference and Adjustment Among Mothers
Ciciolla, L. M., Curlee, A. S., & Luthar, S. S. (2017) · Journal of Family and Economic Issues · doi:10.1007/s10834-017-9534-7 · Cited by 19
“Sampling bias is possible due to our web-based data collection (Bethlehem 2010), and our focus on well-educated mothers limits the generalizability of the findings.”
Background
Intra-party democracy from members’ viewpoint: the case of left-wing parties in Portugal
Sanches, E. R., Lisi, M., Razzuoli, I., & Santo, P. E. (2017) · Acta Politica · doi:10.1057/s41269-017-0057-x · Cited by 15
“In fact, online surveys are often biased with regard to participants’ education levels and socioeconomic resources as it is individuals with more sophisticated cognitive skills and more regular users of new ICTs that tend to answer this type of survey (see, for instance, Bethlehem 2010).”
New data strategies: nonprobability sampling, mobile, big data
Link, M. (2018) · Quality Assurance in Education · doi:10.1108/qae-06-2017-0029 · Cited by 15
Understanding the Relationship between Intention and Cat Containment Behaviour: A Case Study of Kitten and Cat Adopters from RSPCA Queensland
McLeod, L. J., Evans, D., Jones, B., Paterson, M., & Zito, S. (2020) · Animals · doi:10.3390/ani10071214 · Cited by 14
Online self‐presentation and identity development: The moderating effect of neuroticism
Kawamoto, T. (2021) · PsyCh Journal · doi:10.1002/pchj.470 · Cited by 21
Gender Representation Among Contributors to Open-Source Infrastructure : An Analysis of 20 Package Manager Ecosystems
Qiu, H. S., Zhao, Z. H., Yu, T. K., Wang, J. L., Ma, A., Fang, H., Dabbish, L., & Vasilescu, B. (2023) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis58686.2023.00025 · Cited by 14
“However, survey data, albeit more reliable and accurate, are prone to selection bias [25].”
Background
Job crafting among school principals before and during COVID-19: Investigating the associations with work-related well-being and personal resources using variable- and person-oriented approaches
Toyama, H., Upadyaya, K., Hietajärvi, L., & Salmela‐Aro, K. (2023) · European Management Journal · doi:10.1016/j.emj.2023.07.006 · Cited by 16
Depression and Anxiety in Adolescents During the COVID-19 Pandemic in Relation to the Use of Digital Technologies: Longitudinal Cohort Study
(2024) · doi:10.2196/45114 · Cited by 14
“Participation in the T2 assessment might have been self-selected, given that data collection was mostly undertaken in a remote setting [27].”
Cross-sectional United Kingdom surveys demonstrate that owners and veterinary professionals differ in their perceptions of preventive and treatment healthcare needs in ageing dogs
Wallis, L., Radford, A. D., Belshaw, Z., Jackson, J., Kubinyi, E., German, A. J., & Westgarth, C. (2024) · Frontiers in Veterinary Science · doi:10.3389/fvets.2024.1358480 · Cited by 14
“Online surveys allow participants to misrepresent their eligibility to participate or mask their identity in order to participate multiple times (25), and offering incentives may increase fraudulent activity (26).”
Background
Workplace Responses and Psychologists’ Needs Following Client Suicide
Finlayson, M., & Simmonds, J. G. (2017) · OMEGA - Journal of Death and Dying · doi:10.1177/0030222817709693 · Cited by 14
Buyer and Nonbuyer Barriers to Purchasing Local Food
Qi, L., Rabinowitz, A. N., Liu, Y., & Campbell, B. L. (2017) · Agricultural and Resource Economics Review · doi:10.1017/age.2016.40 · Cited by 14
“However, a potential disadvantage to online surveys is that consumers without internet access are not included in the sample (Bethlehem 2010).”
What German dentists choose for their teeth: A Web-based survey of molar restorations and their longevity
Beyer, C., Schwahn, C., Meyer, G. F., & Söhnel, A. (2020) · Journal of Prosthetic Dentistry · doi:10.1016/j.prosdent.2020.01.034 · Cited by 14
Enhancing Planning Behavior during Retirement: Effects of a Time Perspective Based Training Intervention
Mooney, A., Tsotsoros, C. E., Earl, J. K., Hershey, D. A., & Mooney, C. H. (2021) · Social Sciences · doi:10.3390/socsci10080306 · Cited by 15
“The voluntary aspect of participation in psychological research has the potential for samples to under- or over-represent the target population (Bethlehem 2010).”
Background
Older Adults and the COVID-19 Pandemic, What About the Oldest Old? The PACOVID Population-Based Survey
Hernández‐Ruiz, V., Meillon, C., Ávila‐Funes, J. A., Bergua, V., Dartigues, J., Koleck, M., Letenneur, L., Ouvrard, C., Pérès, K., Rascle, N., et al. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.711583 · Cited by 14
“Indeed, while online surveys have the advantage of assessing large samples in a short period, they involve important selection bias particularly in older persons, as the included population tends to be higher educated, healthier, and to have more resources (27, 28).”
Background
Combining disparate surveys across time to study satisfaction with life: the effects of study context, sampling method, and transport attributes
Wang, X., Shaw, F. A., Mokhtarian, P. L., Circella, G., & Watkins, K. E. (2022) · Transportation · doi:10.1007/s11116-021-10252-x · Cited by 15
“, the self-selection bias of web surveys (Bethlehem 2010).”
Background
Validation of the Canadian French version of the fear of COVID‐19 scale in the general population of Quebec
Attieh, R., Koffi, K., Touré, M., Parr‐Labbé, É., Pakpour, A. H., & Poder, T. G. (2022) · Brain and Behavior · doi:10.1002/brb3.2550 · Cited by 14
“Third, in surveys, there is always a possibility of self-selection bias (Bethlehem, 2010), which may indicate an overrepresentation of specific subgroups within the general population.”
Background
Picky eating in children: Current clinical trends, practices, and observations within the Australian health‐care context
Chilman, L. B., Meredith, P. J., Kennedy‐Behr, A., Campbell, G., Frakking, T. T., Swanepoel, L., & Verdonck, M. (2023) · Australian Occupational Therapy Journal · doi:10.1111/1440-1630.12869 · Cited by 14
Understanding the longitudinal psychosocial impact of the COVID-19 pandemic in the United Kingdom; a methodological overview of The COVID-19 Psychological Wellbeing StudyPreprint
Armour, C., McGlinchey, E., Butter, S., McAloney‐Kocaman, K., & McPherson, K. E. (2020) · doi:10.31234/osf.io/9p4tv · Cited by 14
Political Values and Political Trust in the Digital Era: How Media Engagement Divides Chinese Netizens
Huang, Y., & Wang, L. (2021) · International Journal of Sociology · doi:10.1080/00207659.2021.1881870 · Cited by 15
“While probability sampling enables statistical analyses to produce accurate and unbiased estimates, self-selection can ‘lead to biased estimates, and therefore wrong conclusions are drawn from the collected data’ (Bethlehem 2010: 162).”
Background
Investigating Patients’ Preferences to Inform Drug Development Decisions: Novel Insights from a Discrete Choice Experiment in Migraine
Torbica, A., Rognoni, C., & Tarricone, R. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18094916 · Cited by 13
“First, involved patients were recruited via an online survey, and this population may be biased towards the ability to use a computer or a mobile device; moreover, only respondents with internet access were able to complete the questionnaire [43].”
Method
Inference with non-probability samples and survey data integration: a science mapping study
Salvatore, C. (2023) · METRON · doi:10.1007/s40300-023-00243-6 · Cited by 13
“For example, Bethlehem [12] discusses self-selection and undercoverage in web”
Background
An exploration of self-perceived competence in providing nutrition care among physiotherapists in Ireland: a cross-sectional study
Griffin, A., Conway, H., Chawke, J., Keane, M., Douglas, P., & Kelly, D. (2023) · Physiotherapy Theory and Practice · doi:10.1080/09593985.2023.2243624 · Cited by 13
“Additionally, the method of sampling used in this research may have contributed to self-selection bias, whereby physiotherapists with a particular interest in nutrition may have been more likely to complete the survey, skewing the results (Bethlehem, 2010).”
Spatiotemporal and multispecies comparisons between a citizen science platform and recall surveys in recreational fisheries
Gundelund, C., Venturelli, P., Hartill, B., Hyder, K., Olesen, H. J., & Skov, C. (2023) · Marine Policy · doi:10.1016/j.marpol.2023.105780 · Cited by 13
“Examples of self-selection include traditional diaries [22], web surveys [28,29], and citizen science programs [30,31].”
Background
A Multidimensional Approach to Understanding Food Deserts in Vulnerable Contexts
Cruz-Piedrahita, C., Martinez-Carranza, F.-J., & Delgado-Serrano, M. D. M. (2024) · Sustainability · doi:10.3390/su16031136 · Cited by 14
The dark side of salesperson ambidexterity: How salesperson ambidexterity increases felt stress
McClure, C. E., DeCarlo, T. E., & Hansen, J. D. (2024) · Industrial Marketing Management · doi:10.1016/j.indmarman.2024.08.008 · Cited by 15
Reducing Consumer–Brand Incongruity Through Corporate Social Responsibility and Brand Trust: Exploring Negative Word‐of‐Mouth ( NWOM )
Tosun, P., Çağlıyor, S., & Gürce, M. Y. (2024) · International Journal of Consumer Studies · doi:10.1111/ijcs.13099 · Cited by 13
Double-Bind on the Bench: Citizen Perceptions of Judge Gender and the Court
Nelson, K. (2015) · Politics & Gender · doi:10.1017/s1743923x15000021 · Cited by 14
“…with internet access and/or a device to access the internet, if needed, to preempt the issue of sampling bias through access (or lack thereof) (Bethlehem 2010).13 Finally, the nationally representative nature of the online sample precludes the usual concerns of using a “college sophomore”…”
Background
Comparability of web and telephone survey modes for the measurement of subjective well-being
Sarracino, F., Riillo, C. A. F., & Mikucka, M. (2016) · Survey research methods · doi:10.18148/srm/2017.v11i2.6740 · Cited by 13
“Some people are systematically more likely to answer the phone, and others are more likely to fill in a web-based survey (Bethlehem, 2010; Dever, Rafferty, & Valliant, 2008; Dillman et al., 2009; Jäckle et al., 2015).”
Background
Gardeners’ Perceptions of Northwestern U.S. Native Plants Are Influenced by Ecological Information and Garden Group Affiliation
Anderson, A. G., Messer, I., & Langellotto, G. (2021) · HortTechnology · doi:10.21273/horttech04770-20 · Cited by 13
Old Voters on New Dimensions: Why Do Voters Vote for Pensioners’ Parties? The Case of the Netherlands
Otjes, S., & Krouwel, A. P. M. (2017) · Journal of Aging & Social Policy · doi:10.1080/08959420.2017.1363589 · Cited by 13
“Pensioners, as “insiders” of the current welfare state, have an interest in maintaining their fixed state-provided income and public health care provisions.”
BackgroundInfluential
Global health education in United States anesthesiology residency programs: a survey of resident opportunities and program director attitudes
Kaur, G., Tabaie, S., Brar, J., Tangel, V. E., & Pryor, K. O. (2017) · BMC Medical Education · doi:10.1186/s12909-017-1056-3 · Cited by 13
“Bethlehem J. Selection bias in web surveys.”
MethodInfluential
Wchodzenie w dorosłość. Ogólnopolskie badanie młodych Polaków w wieku 18–29 lat
Wiszejko-Wierzbicka, D., & Kwiatkowska, A. (2018) · Studia Socjologiczne · doi:10.24425/122467 · Cited by 15
COVID-19 and a temporary ban on tobacco sales in South Africa: impact on smoking cessation
Saloojee, Y., & Mathee, A. (2021) · Tobacco Control · doi:10.1136/tobaccocontrol-2020-056293 · Cited by 13
Latvijas emigrantu kopienas: cerību diaspora
(2015) · doi:10.22364/lekcd · Cited by 13
Reel life vs. real life: how software developers share their daily life through vlogs
Chattopadhyay, S., Zimmermann, T., & Ford, D. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468599 · Cited by 13
“The survey may have been influenced by self-selection bias [15].”
Background
Living on the Edge: Institutional Supports and Perceptions of Economic Insecurity Among People with Disabilities and Chronic Health Conditions
Maroto, M. L., & Pettinicchio, D. (2022) · Sociological Inquiry · doi:10.1111/soin.12504 · Cited by 12
Population Norms for the EQ-5D-5L, PROPr and SF-6D in Hungary
Nikl, A., Janssen, M. F., Jenei, B., Brodszky, V., & Rencz, F. (2024) · PharmacoEconomics · doi:10.1007/s40273-024-01360-4 · Cited by 12
“First, collecting data from an online panel might be prone to selection bias, especially in the older generations, as they are less likely to have internet access or use the internet regularly [90–92].”
Background
Task implementation heterogeneity and wage dispersion
Visintin, S., Tijdens, K. G., Steinmetz, S., & de Pedraza, P. (2015) · IZA Journal of Labor Economics · doi:10.1186/s40172-015-0036-2 · Cited by 12
A pilot study of Australian pet owners who engage in social media and their use, experience and views of online pet health information
Kogan, L. R., Hazel, S. J., & Oxley, J. A. (2019) · Australian Veterinary Journal · doi:10.1111/avj.12870 · Cited by 12
The dynamics behind low posttraumatic growth in victims of Type III traumas: The case of Syrians and Palestinians.
Kanaan, A., Kira, I. A., Shuwiekh, H. A. M., Kucharska, J., & Alhuwailah, A. (2019) · Traumatology An International Journal · doi:10.1037/trm0000229 · Cited by 12
Schooling amidst a pandemic in the United States: Parents’ perceptions about reopening schools and anticipated challenges during COVID-19
Meghani, A., Agarwal, S., Zapf, A. J., Edwards, J. G., Labrique, A. B., & Gibson, D. G. (2022) · PLoS ONE · doi:10.1371/journal.pone.0268427 · Cited by 12
“While smartphone ownership has been increasing, and the digital divide has narrowed in the United States [42], it is possible our online survey may not captured the experiences of individuals without the resources and technology to access the internet [43], which may be associated with lower levels of education and lo…”
Background
Value dissonance in research(er) assessment: individual and perceived institutional priorities in review, promotion, and tenure
Ross‐Hellauer, T., Klebel, T., Knoth, P., & Pontika, N. (2023) · Science and Public Policy · doi:10.1093/scipol/scad073 · Cited by 20
“Our sample may also be subject to self-selection bias (Bethlehem 2010), in that academics particularly satisfied or dissatisfied with RPT processes may have been more likely to have responded.”
Impact of raising awareness of respondents on the measurement quality in a web survey
Revilla, M. (2015) · Quality & Quantity · doi:10.1007/s11135-015-0216-y · Cited by 12
“However, there are still problems with this data collection mode, especially with regards to coverage and non-response (Bethlehem 2010; Couper 2000; Dillman et al. 1999; Van Selm and Jankowski 2006).”
Background
Complaints of group-stalking (‘gang-stalking’): an exploratory study of their nature and impact on complainants
Sheridan, L. P., & James, D. (2015) · Journal of Forensic Psychiatry and Psychology · doi:10.1080/14789949.2015.1054857 · Cited by 12
“Self-selection may have skewed reported symptoms towards the more severe end of the spectrum (Bethlehem, 2010).”
Background
Social Workers, Intimate Partner Violence (IPV), and Client Financial Concerns
Hageman, S. A., & George, D. M. M. S. (2018) · Journal of Social Service Research · doi:10.1080/01488376.2018.1476288 · Cited by 12
“Thirdly, respondents in this study were self-selected, which may have biased the results because nonresponders may have differed from the survey respondents (Bethlehem, 2010).”
Result
Historical Drivers and Contemporary Perceptions of Wildfire in a Post-Industrial Rural Landscape
Saladyga, T., & Standlee, A. I. (2018) · Fire · doi:10.3390/fire1020033 · Cited by 11
Testing the waters — A cross-sectional survey of views about waterbirth among Swedish health professionals
Ulfsdottir, H., Saltvedt, S., & Georgsson, S. (2019) · Women and Birth · doi:10.1016/j.wombi.2019.04.003 · Cited by 12
Evaluación de la investigación con encuestas en artículos publicados en revistas del área de Biblioteconomía y Documentación
Oliván, J. A. S., Cuenca, G. M., & Avilés, R. A. (2021) · Revista española de Documentación Científica · doi:10.3989/redc.2021.2.1774 · Cited by 106
Further Validation of a Dutch Translation of the Sussex Oxford Compassion for the Self Scale in Samples of Crisis Line Volunteers, Military Personnel and Nursing Students
de Krijger, E., Willems, R., ten Klooster, P., Bakker, E., Miedema, H., Drossaert, C., & Bohlmeijer, E. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.895850 · Cited by 11
“Especially for the sample of military personnel there was a considerable amount of non-response, increasing the likelihood that the current results are influenced by self-selection bias (Bethlehem, 2010).”
Background
Personal safety perception of ride-share amongst young adults in Cape Town: The effect of gender, vehicle access and Covid-19
Baufeldt, J. L., & Vanderschuren, M. (2023) · Research in Transportation Economics · doi:10.1016/j.retrec.2023.101323 · Cited by 11
United States Value Set for the Functional Assessment of Cancer Therapy-General Eight Dimensions (FACT-8D), a Cancer-Specific Preference-Based Quality of Life Instrument
King, M., Revicki, D., Norman, R., Müller, F., Viney, R., Pickard, A., Cella, D., Shaw, J., & the MAUCa Consortium (2023) · PharmacoEconomics - Open · doi:10.1007/s41669-023-00448-5 · Cited by 10
Cohort profile: The ENTWINE iCohort study, a multinational longitudinal web-based study of informal care
Elayan, S., Bei, E., Ferraris, G., Fisher, O., Zarzycki, M., Angelini, V., Ansmann, L., Buskens, E., Hagedoorn, M., von Kutzleben, M., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0294106 · Cited by 10
The Efficacy of Propensity Score Matching for Separating Selection and Measurement Effects Across Different Survey Modes
Kibuchi, E., Sturgis, P., Durrant, G. B., & Maslovskaya, O. (2024) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smae017 · Cited by 13
“Differential sample composition between face-to-face and online surveys may arise due to both coverage and self-selection error (Bethlehem 2010).”
Performance-enhancement in esports – Players’ perspectives on prevalence, legitimacy, governance and regulations
Schubert, M., Güre, A., & Haller, N. (2024) · Performance Enhancement & Health · doi:10.1016/j.peh.2024.100290 · Cited by 11
Equine influenza vaccination as reported by horse owners and factors influencing their decision to vaccinate or not
Bambra, W., Daly, J. M., Kendall, N. R., Gardner, D. S., Brennan, M. L., & Kydd, J. H. (2020) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2020.105011 · Cited by 11
“454 This is a recognised failing of an electronic survey approach (Sax et al., 2003; 455 Bethlehem, 2010), but this method permits access by large numbers of people 456 ` easily; the number of individual UK households with internet access has increased 457 dramatically in recent years to 89% in…”
Method
Patient experience with NER1006 as a bowel preparation for colonoscopy: a prospective, multicenter US survey
Cash, B. D., Moncrief, M. B. C., Epstein, M. S., & Poppers, D. M. (2021) · BMC Gastroenterology · doi:10.1186/s12876-021-01605-y · Cited by 11
“Limitations of the current survey include parameters intrinsic to any survey-based study design, such as potential for sampling bias and/or recall bias [19, 20].”
Background
Human Evaluation of Creative NLG Systems: An Interdisciplinary Survey on Recent Papers
Hämäläinen, M. K., & Alnajjar, K. (2021) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2021.gem-1.9 · Cited by 11
Restaurant employees' attitudinal reactions to social distancing difficulties: a multi-wave study
Huo, M., Jiang, Z., Cheng, Z., & Wilkinson, A. J. (2022) · Journal of Service Theory and Practice · doi:10.1108/jstp-08-2021-0180 · Cited by 11
Does fear of COVID-19 undermine career optimism? A time-lagged quantitative inquiry of non-managerial employees
Ahmad, B., & Bilal, S. (2022) · Kybernetes · doi:10.1108/k-10-2021-1036 · Cited by 11
Prevalence and Factors Associated With the Risk of Delayed Sleep-Wake Phase Disorder in Japanese Youth
Tomishima, S., Komada, Y., Tanioka, K., Okajima, I., & Inoue, Y. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.878042 · Cited by 8
“In particular, research panels are thought to include a high proportion of people with high accessibility to the Internet (47).”
Background
Russian speakers’ media engagement and acculturation in Finland and Latvia
Khalimzoda, I., & Siitonen, M. (2022) · Comparative Migration Studies · doi:10.1186/s40878-022-00304-1 · Cited by 11
“Online surveys typically feature different types of selection bias (Bethlehem, 2010), and our study is no different in this regard.”
Background
“A Rollercoaster of Emotions”: Social Distancing, Anxiety, and Loneliness Among People With Disabilities and Chronic Health Conditions
Maroto, M. L., Pettinicchio, D., Chai, L., & Holmes, A. (2023) · Research in social science and disability · doi:10.1108/s1479-354720230000013004 · Cited by 13
“We did not employ post-strati ﬁ cation weights (Bethlehem, 2010).”
Assessing social perceptions of rewilding approaches in Spain using traditional domestic livestock
Pérez-Barbería, F. J., & Gordon, I. J. (2023) · Journal of Wildlife Management · doi:10.1002/jwmg.22425 · Cited by 11
LinkedIn Salary: A System for Secure Collection and Presentation of Structured Compensation Insights to Job Seekers
Kenthapadi, K., Chudhary, A., & Ambler, S. (2017) · IEEE Symposium on Privacy-Aware Computing (PAC) · doi:10.1109/pac.2017.9 · Cited by 11
“See [3] for a survey of non-response bias challenges, and [14] for an overview of selection bias.”
Background
Feasibility, Acceptability, and Influence of mHealth-Supported N-of-1 Trials for Enhanced Cognitive and Emotional Well-Being in US Volunteers
Kravitz, R. L., Aguilera, A., Chen, E. J., Choi, Y. K., Hekler, E. B., Karr, C. J., Kim, K. K., Phatak, S. S., Sarkar, S., Schueller, S. M., et al. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00260 · Cited by 11
“Second, the analytic sample was demographically narrow, limiting generalizability (42).”
Background
Predicting Empirical Patterns in Viewing Japanese TV Dramas Using Case-Based Decision Theory
Kinjo, K., & Sugawara, S. (2016) · The B E Journal of Theoretical Economics · doi:10.1515/bejte-2015-0014 · Cited by 11
“As a resource for econometric analysis, web surveys have a potential weakness in the form of sample selection, as summarized by Bethlehem (2010). Although our study is also exposed to the risk of sample selection, we have three responses to such criticism.”
People perceive themselves to adhere more strictly to COVID-19 guidelines than others
Mojzisch, A., Elster, C., & Germar, M. (2021) · Psychology Health & Medicine · doi:10.1080/13548506.2021.1906435 · Cited by 11
“Second, web surveys may suffer from methodological issues, like under-coverage and self-selection (Bethlehem, 2010).”
Background
An Integrated Approach to Surveying Emigrants Worldwide
Mieriņa, I. (2019) · IMISCOE research series · doi:10.1007/978-3-030-12092-4_2 · Cited by 11
“2 An Integrated Approach to Surveying Emigrants Worldwide 16 The greatest risks associated with web surveys are the potential bias caused by selfselection and the difficulties of reaching certain socio-demographic groups via the Internet (Askitas and Zimmermann 2015; Bethlehem 2010).”
Background
Prevalence Rates of Sexual Behaviors, Condom Use, and Contraception Among Australian Heterosexual Adolescents
Fisher, C., Kauer, S. D., Mikołajczak, G., Ezer, P., Kerr, L., Bellamy, R., Waling, A., & Lucke, J. C. (2020) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2020.08.009 · Cited by 11
A survey about the welfare problem "toe pecking" in Swiss laying hens
Gebhardt‐Henrich, S. G., Mueller, S., Zanini, L., & Toscano, M. J. (2023) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2023.105854 · Cited by 10
To vote or not to vote in the homeland elections? Insights into voting abstention in Italy's constituency abroad
Battiston, S., Luconi, S., & Valbruzzi, M. (2023) · Modern Italy · doi:10.1017/mit.2023.59 · Cited by 10
“Notably, this interview collection process, especially when carried out online, is not immune from the risk of selection bias (Bethlehem 2010).”
Background
Exploring the user experience, quality, and provision of urban greenspace: A mixed-method approach
Creed, C., & Carvalho, J. S. (2024) · Urban forestry & urban greening · doi:10.1016/j.ufug.2024.128470 · Cited by 13
Understanding sustainability perceptions: Insights from textual analysis and a survey experiment
Rotondi, V., Noris, A., & Carpanzano, E. (2024) · World Development Sustainability · doi:10.1016/j.wds.2024.100182 · Cited by 10
Examining consumer preferences for basic foodstuffs in a highly inflationary economic environment: The case of price-capped chicken breast fillet in Hungary
Maró, Z. M., Török, Á., & Czine, P. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e41279 · Cited by 10
Development of updated population norms for the SF-36 for Hungary and comparison with 1997–1998 norms
Bató, A., Brodszky, V., & Rencz, F. (2025) · Health and Quality of Life Outcomes · doi:10.1186/s12955-025-02343-5 · Cited by 12
“Selection bias might have occurred as online panel data collection methods may lead to self-selection; for instance, those who participate in health surveys often report better health status than non-partici-pants [59, 60].”
The Platform Matters: Selection and Measurement Bias in Online Reviews
Han, S., & Anderson, C. K. (2025) · Cornell Hospitality Quarterly · doi:10.1177/19389655251327536 · Cited by 10
“According to the Total Survey Error (TSE) framework (Bethlehem, 2010; Biemer, 2010; Groves et al., 2009), selection bias might arise at this stage unless the data collection is meticulously managed.”

Authors: Bethlehem, J.
Affiliations: Centraal Bureau voor de Statistiek (NL)
Corresponding author: Bethlehem, Jelke
Venue: International Statistical Review
Publisher: Wiley
Published: 2010-08
Volume: 78
Issue: 2
Pages: 161-188
ISSN: 0306-7734 (ISSN-L)
1751-5823
DOI: 10.1111/j.1751-5823.2010.00112.x
OpenAlex: W2129332419
Cited by: 1,315
References: 37
Citations by year: 2026: 228 · 2025: 173 · 2024: 151 · 2023: 149 · 2022: 122 · 2021: 144 · 2020: 109 · 2019: 47 · 2018: 38 · 2017: 54 · 2016: 20 · 2015: 32 · 2014: 19 · 2013: 14 · 2012: 6
FWCI: 29.95
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations: International Statistical Review (publishedVersion)
UvA-DARE (University of Amsterdam) (publishedVersion, other-oa)
RePEc: Research Papers in Economics (submittedVersion)
Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Topics: Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
Physical Sciences › Mathematics › Statistics and Probability › Census and Population Estimation
Keywords: online surveys, selection bias, undercoverage, self-selection, internet access, survey design, participant recruitment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Selection Bias in Web Surveys

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks