Cited by 1,315Survey Methodology and NonresponseSurvey Sampling and Estimation TechniquesCensus and Population Estimation

The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.

Selection Bias in Web Surveys

Open access

Bethlehem, Jelke·2010·International Statistical Review

DOI 10.1111/j.1751-5823.2010.00112.x

Open this paper in CoChat — free account Open PDF

Abstract

Summary: At first sight, web surveys seem to be an interesting and attractive means of data collection. They provide simple, cheap, and fast access to a large group of potential respondents. However, web surveys are not without methodological problems. Specific groups in the populations are under‐represented because they have less access to Internet. Furthermore, recruitment of respondents is often based on self‐selection. Both under‐coverage and self‐selection may lead to biased estimates. This paper describes these methodological problems. It also explores the effect of various correction techniques (adjustment weighting and use of reference surveys). This all leads to the question whether properly design web surveys can be used for data collection. The paper attempts to answer this question. It concludes that under‐coverage problems may solve itself in the future, but that self‐selection leads to unreliable survey outcomes.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1111/j.1751-5823.2010.00112.x

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,385
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008) · Cited by 5,345
A generalization of sampling without replacement from a finite universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483446 · Cited by 4,293
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
A Generalization of Sampling Without Replacement From a Finite Universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.2307/2280784 · Cited by 3,011
Web Surveys
Couper, M. P. (2000) · Public Opinion Quarterly · doi:10.1086/318641 · Cited by 2,086
On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection
Neyman, J. (1934) · Journal of the Royal Statistical Society · doi:10.2307/2342192 · Cited by 1,086
The Effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies
Cochran, W. G. (1968) · Biometrics · doi:10.2307/2528036 · Cited by 961

Show all 37 references

Research works printed but not linked: 20

Bethlehem J.G., 1988, Reduction of the nonresponse bias through regression estimation, J. Official Statist., 4, 251
Bethlehem J.G., 2002, Survey Nonresponse
Bethlehem J.G., 2009, The Rise of Survey Sampling
Börsch‐Supan A., 2004, Correcting the Participation Bias in an Online Survey
Bowley A.L., 1926, Measurement of the precision attained in sampling, Bull. Int. Statist. Inst., 6
Cobben F., 2004, Nonresponse Correction Techniques in Household Surveys at Statistics Netherlands: a CAPI‐CATI Comparison
Cochran W.G., 1953, Sampling Techniques
Cochran W.G., 1977, Sampling Techniques
De Haan J., 2006, Jaarboek ICT en Samenleving, De Digitale Generatie
Eurostat, 2007, More than 40% of Households have Broadband Internet Access
Hansen M.H., 1953, Survey Sampling Methods and Theory
Kendall M.G., 1960, Where shall the history of statistics begin, Biometrika, 47, 447
Kiaer A. N., 1895, Observations et Expériences Concernant des Dénombrements Représentatives, Bull. Internat. Statist. Inst., 176
Kiaer A. N., 1997, Den Repräsentative Undersökelsesmetode, Christiania Videnskabsselskabets Skrifter. II. Historiskfilosofiske klasse, 4, 37
Kish L., 1967, Survey Sampling
Kuusela V., 2003, Proceedings of the 4th ASC International Conference, 317
10.1111/j.2397-2335.1934.tb04184.x · DOI
Scherpenzeel A., 2010, Social Research and the Internet: Advances in applied Methods and New Research Strategies
Terhanian G., 2001, Exploiting analytical advances: Minimizing the biases associated with Internet‐based surveys of non‐random samples, ESOMAR Publication Services, 248, 247
Vonk T., 2006, The effects of panel recruitment and management on research results, a study among 19 online panels, ESOMAR Publication Services, 317, 79

Sorted by citations

Items 201–300 of 1,000

Mobile payment adoption process: a serial of multiple mediation and moderation analysis
Oentoro, W. (2021) · The Bottom Line Managing Library Finances · doi:10.1108/bl-09-2020-0059 · Cited by 37
Natural hazards compound COVID-19 impacts on small businesses disproportionately for historically underrepresented group operators
HELGESON, J. F., Aminpour, P., Fung, J. F., Roa-Henriquez, A. R., Zycherman, A., Butry, D. T., Nierenberg, C., & Zhang, Y. (2022) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2022.102845 · Cited by 28
“At the same time, not all potential respondents always have access to the Internet and their recruitment is characterized by self-selection; there can be “under-coverage bias” [66].”
Background
Willingness to pay for regional electricity generation – A question of green values and regional product beliefs?
Lehmann, N., Sloot, D., Ardone, A., & Fïchtner, W. (2022) · Energy Economics · doi:10.1016/j.eneco.2022.106003 · Cited by 28
Self-assessment instrument to measure the competencies of Namibian graduates: Testing of validity and reliability
Pop, C., & Khampirat, B. (2019) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2018.12.004 · Cited by 28
Conversational assessment using artificial intelligence is as clinically useful as depression scales and preferred by users
Weisenburger, R. L., Mullarkey, M. C., Labrada, J., Labrousse, D., Yang, M. Y., Huff, A. J., Hsu, K. J., Ugail, H., Shumake, J., & Beevers, C. G. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.01.212 · Cited by 31
Data gaps in prevalence rates of mental health conditions around the world: a retrospective analysis of nationally representative data
Casella, C. B., Kousoulis, A. A., Kohrt, B. A., Bantjes, J. R., Kieling, C., Cuijpers, P. J., Kline, S., Kotsis, K. T., Polanczyk, G. V., Stein, D. J., et al. (2025) · The Lancet Global Health · doi:10.1016/s2214-109x(24)00563-1 · Cited by 30
The Power of Good: A Leader's Personal Power as a Mediator of the Ethical Leadership-Follower Outcomes Link
Haller, D. J., Fischer, P., & Frey, D. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01094 · Cited by 33
The Relationship Between Volunteer Motivations and Variation in Frequency of Participation in Conservation Activities
Takase, Y., Hadi, A. A., & Furuya, K. (2018) · Environmental Management · doi:10.1007/s00267-018-1106-6 · Cited by 27
“On the other hand, online surveys may suffer from problems such as undercoverage and self-selection (Bethlehem 2010).”
Background
Food Safety Risk Information-Seeking Intention of WeChat Users in China
Zhao-hui, Y., Paudel, K. P., Wen, X., Sun, S., & Wang, Y. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17072376 · Cited by 27
“(3) Other issues that may limit the interpretation of the results are that an online survey may have under-coverage and self-selection bias problems [45].”
Background
Dog and cat owners’ use of online Facebook groups for pet health information
Kogan, L. R., Little, S. E., & Oxley, J. A. (2021) · Health Information & Libraries Journal · doi:10.1111/hir.12351 · Cited by 27
COVID-19 vaccine intention among young adults: Comparative results from a cross-sectional study in Canada and France
Coulaud, P.-J., Ablona, A., Bolduc, N., Fast, D., Bertrand, K., Ward, J. K., Greyson, D. L., Jauffret‐Roustide, M., & Knight, R. (2022) · Vaccine · doi:10.1016/j.vaccine.2022.02.085 · Cited by 27
Electronic cigarettes: a survey of perceived patient use and attitudes among members of the British thoracic oncology group
Sherratt, F. C., Newson, L., & Field, J. K. (2016) · Respiratory Research · doi:10.1186/s12931-016-0367-y · Cited by 26
“It should be noted that the study design entailed a self-selected sample and findings could therefore result in potential bias, however, this is a limitation of any self-selecting online survey [30].”
Background
Blogs as Elusive Ethnographic Texts
Kurtz, L. C., Trainer, S. S., Beresford, M., Wutich, A., & Brewis, A. (2017) · International Journal of Qualitative Methods · doi:10.1177/1609406917705796 · Cited by 35
Why do newcomers vote for a newcomer? Support for an immigrant party
Otjes, S., & Krouwel, A. P. M. (2018) · Journal of Ethnic and Migration Studies · doi:10.1080/1369183x.2018.1427053 · Cited by 27
“This approach allows us to deal with the problem that in VAAs, like many online surveys and panels, tend to under-cover specific groups because they are self-selected (Bethlehem 2010): young men with higher education are overrepresented in VAA data sets compared to the total voting population.”
Background
How to Combine Survey Media (Web, Telephone, Face-to-Face): Lyon and Rhône-alps Case Study
Bayart, C., & Bonnel, P. (2015) · Transportation research procedia · doi:10.1016/j.trpro.2015.12.011 · Cited by 27
“If the socioeconomic characteristics of households who answer on the internet differ from those of the target population, their mobility habits may not be the same (Bethlehem, 2010).”
Background
Public support for climate policies and its ideological predictors across countries of the Global North and Global South
Bretter, C., & Schulz, F. (2025) · Ecological Economics · doi:10.1016/j.ecolecon.2025.108603 · Cited by 30
Impact of Client Suicide on Psychologists in Australia
Finlayson, M., & Simmonds, J. G. (2016) · Australian Psychologist · doi:10.1111/ap.12240 · Cited by 26
“Technical issues such as slow modems and unreliable connections may have also deterred some participation (Bethlehem, 2010).”
Background
Prevalence and Determinants of Fasted Training in Endurance Athletes: A Survey Analysis
Rothschild, J. A., Kilding, A. E., & Plews, D. J. (2020) · International Journal of Sport Nutrition and Exercise Metabolism · doi:10.1123/ijsnem.2020-0109 · Cited by 26
Public opinions on seven different stray cat population management scenarios in Flanders, Belgium
Ruyver, C. D., Abatih, E., Villa, P. D., Peeters, E., Clements, J., Dufau, A., & Moons, C. P. H. (2021) · Research in Veterinary Science · doi:10.1016/j.rvsc.2021.02.025 · Cited by 30
“As with other surveys our sample is subject to self-selection bias (Bethlehem, 2010; Khazaal et al., 2014).”
Result
The status quo of digital transformation in insurance sales: an empirical analysis of the german insurance industry
Eckert, C., Eckert, J., & Zitzmann, A. (2021) · Zeitschrift für die gesamte Versicherungswissenschaft · doi:10.1007/s12297-021-00507-y · Cited by 26
“Hence, participants with higher levels of digital affinity are more likely to select themselves for the survey (Bethlehem 2010), which is why we rather overestimate the progress of the digital transformation in traditional insurance sales.”
BackgroundInfluential
Consumer intention to accept electric two-wheelers in India: a valence theory approach to unveil the role of identity and utility
Bhat, F. A., & Verma, A. (2023) · Transportation · doi:10.1007/s11116-023-10430-z · Cited by 27
“Online surveys are also more accessible to people who are technologically grown and have access to the internet which is not always the case in developing economies and subsequently leads to an underrepresentation of certain sections of the population (Bethlehem 2010).”
The potential use of social media and other internet-related data and communications for child maltreatment surveillance and epidemiological research: Scoping review and recommendations
Schwab‐Reese, L. M., Hovdestad, W., Tonmyr, L., & Fluke, J. (2018) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2018.01.014 · Cited by 26
“However, there may be bias in the results of studies using Amazon Mechanical Turk and Google Consumer Surveys related to varying access to the internet within a population so older adults, individuals with lower socioeconomic status, and others may be underrepresented (Bethlehem, 2010).”
Background
Presidential Address
Tourangeau, R. (2017) · Public Opinion Quarterly · doi:10.1093/poq/nfx031 · Cited by 26
Can computer game landscapes target new audiences for landscape quality assessment?
Swetnam, R. D., & Korenko, J. (2019) · Applied Geography · doi:10.1016/j.apgeog.2019.102102 · Cited by 27
Reinforcement Learning from Human Feedback in LLMs: Whose Culture, Whose Values, Whose Perspectives?
Barman, K. G., Lohse, S., & de Regt, H. W. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00861-0 · Cited by 43
“The issue here is that there may be certain people who are more inclined to provide this feedback than others, creating a possible self-selection bias (see Bethlehem, 2010 for self-selection effects in web surveys).”
Public Perceptions of Billboards: A Meta-Analysis
Franke, G. R., & Taylor, C. R. (2017) · Journal of Advertising · doi:10.1080/00913367.2017.1334248 · Cited by 25
Telerehabilitation to Address the Rehabilitation Gap in Anterior Cruciate Ligament Care: Survey of Patients
Dunphy, E., & Gardner, E. C. (2020) · JMIR Formative Research · doi:10.2196/19296 · Cited by 25
“All surveys are limited to the data gathered, and it is known that participation can be affected by socioeconomic condition and education or digital literacy levels, where some populations are known to be less likely to access internet surveys [43].”
Background
Physical Activity and Sedentary Behaviour in People with Myasthenia Gravis: A Cross-Sectional Study
Alsop, T., Williams, K., & Gomersall, S. R. (2021) · Journal of Neuromuscular Diseases · doi:10.3233/jnd-210722 · Cited by 25
Is There a Sampling Bias in Research on Work-Related Technostress? A Systematic Review of Occupational Exposure to Technostress and the Role of Socioeconomic Position
Borle, P., Reichel, K., & Voelter‐Mahlknecht, S. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18042071 · Cited by 26
“This issue has been reported in earlier methodological assessments of commercial online panels [54,55].”
Constructing meanings of luxury in Airbnb
Farmaki, A., Spanou, E., & Christou, P. (2021) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-04-2021-0429 · Cited by 25
Platform work in a Coordinated Market Economy
Funke, C., & Picot, G. (2021) · Industrial Relations Journal · doi:10.1111/irj.12339 · Cited by 25
Strengthening referees’ psychological well-being through engagement and authenticity
Kim, M., Kim, H. S., Simmond, A., & Warner, S. M. (2021) · Sport Management Review · doi:10.1080/14413523.2021.1930952 · Cited by 31
“Because self-selection relies on non-probability sampling methods (Bethlehem, 2010), the sample used in this study could not represent the entire population of high school basketball referees, thereby potentially affecting response levels or errors (Couper, 2000).”
Method
Longitudinal Study of Cognitive Decline before and after the COVID-19 Pandemic: Evidence from the PA-COVID Survey
Amieva, H., Retuerto, N., Hernández‐Ruiz, V., Meillon, C., Dartigues, J., & Pérès, K. (2022) · Dementia and Geriatric Cognitive Disorders · doi:10.1159/000521999 · Cited by 25
Student physiotherapists perceptions of online curriculum delivery during the COVID-19 pandemic
Chesterton, P., Richardson, M. C., & Tears, C. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03486-5 · Cited by 36
“It is possible that students who responded to the questionnaire are not representative of the entire target population [67].”
Background
Can Responsible Ownership Practices Influence Hunting Behavior of Owned Cats?: Results from a Survey of Cat Owners in Chile
Escobar, S., Alegría-Morán, R. A., Calderón-Amor, J., & Tadich, T. (2019) · Animals · doi:10.3390/ani9100745 · Cited by 25
“Thus, the researcher is not in control of the selection process [35].”
Background
Orthorexia nervosa: examining the Eating Habits Questionnaire’s reliability and validity, and its links to dietary adequacy among adult women
Halim, Z. M., Dickinson, K. M., Kemps, E., & Prichard, I. J. (2020) · Public Health Nutrition · doi:10.1017/s1368980019004282 · Cited by 25
The Opt‐In Internet Survey
Thielo, A. J., Graham, A., & Cullen, F. T. (2021) · The Encyclopedia of Research Methods in Criminology and Criminal Justice · doi:10.1002/9781119111931.ch55 · Cited by 25
Paternal Self-Efficacy: A Parenting Resilience Factor for Fathers with Depression
Trahan, M. H., & Shafer, K. M. (2019) · Social Work Research · doi:10.1093/swr/svz004 · Cited by 25
Impact of cause-affinity and CSR fit on consumer purchase intention
Gupta, S. S., & Wadera, D. (2020) · Society and Business Review · doi:10.1108/sbr-01-2020-0012 · Cited by 26
National Norms and Correlates of the PHQ-8 and GAD-7 in Parents of School-age Children
Sequeira, S. L., Morrow, K. E., Silk, J. S., Kolko, D. J., Pilkonis, P. A., & Lindhiem, O. J. (2021) · Journal of Child and Family Studies · doi:10.1007/s10826-021-02026-x · Cited by 27
“Relatedly, though the sample was recruited based on estimates of population characteristics, these estimates will always have some error (Bethlehem, 2010).”
Background
The relationship between COVID-related parenting stress, nonresponsive feeding behaviors, and parent mental health
Frankel, L. A., Kuno, C. B., & Sampige, R. R. (2021) · Current Psychology · doi:10.1007/s12144-021-02333-y · Cited by 24
“This is a common issue with online surveys (Bethlehem, 2010).”
Background
Motivators and barriers to the widespread adoption of electric four-wheelers in India – A discrete choice analysis of potential electric four-wheeler buyers
Bhat, F. A., Seth, Y., & Verma, A. (2024) · Travel Behaviour & Society · doi:10.1016/j.tbs.2024.100748 · Cited by 25
The war on antidepressants: What we can, and can't conclude, from the systematic review of antidepressant withdrawal effects by Davies and Read
Jauhar, S., & Hayes, J. (2019) · Addictive Behaviors · doi:10.1016/j.addbeh.2019.01.025 · Cited by 23
Examining Social Desirability Bias in Online and Offline Surveys
Koivula, A., Räsänen, P., & Sarpila, O. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-22646-6_11 · Cited by 23
Carbohydrate Intake Practices and Determinants of Food Choices During Training in Recreational, Amateur, and Professional Endurance Athletes: A Survey Analysis
Reinhard, C., & Galloway, S. D. R. (2022) · Frontiers in Nutrition · doi:10.3389/fnut.2022.862396 · Cited by 25
“Although we had prevented multiple votes per participant (the survey could be taken only once from the same device), online surveys generally pose the risk of a self-selection bias (43).”
Background
Pregnancy, fertility concerns and fertility preservation procedures in a national study of French breast cancer survivors
Mangiardi-Veltin, M., Sebbag, C., Rousset‐Jablonski, C., Ray-Coquard, I. L., Berkach, C., Laot, L., Wang, Y., Abdennebi, I., Labrosse, J., Sautter, C., et al. (2021) · Reproductive BioMedicine Online · doi:10.1016/j.rbmo.2021.12.019 · Cited by 23
Well-being of Canadian Veterans during the COVID-19 pandemic: cross-sectional results from the COVID-19 Veteran well-being study
Richardson, J. D., Cyr, K. S., Forchuk, C. A., Liu, J. J. W., Plouffe, R. A., Le, T., Gargala, D., Deda, E., Soares, V., Hosseiny, F., et al. (2022) · European Journal of Psychotraumatology · doi:10.1080/20008198.2021.2012374 · Cited by 23
“Thus, Veteran groups who do not have ready access to the internet, a computer, or smartphone/devices, or who are less familiar with the use of digital devices may be underrepresented in our sample (Bethlehem, 2010).”
Background
Real-world evidence on the economic implications of CGRP-mAbs as preventive treatment of migraine
Siersbæk, N., Kilsdal, L., Jervelund, C., Antić, S., & Bendtsen, L. (2023) · BMC Neurology · doi:10.1186/s12883-023-03302-7 · Cited by 24
“The sample was a non-random sample of patients who self-selected into participation in the research project, which may give rise to self-selection bias [35].”
BackgroundMethodInfluential
Provincial and territorial correctional service workers: A Canadian national and jurisdictional assessment of mental health
Ricciardelli, R., Carleton, R. N., Taillieu, T., Dorniani, S., Johnston, M. S., Carbonell, M., Coulling, R., Andrés, E., & Afifi, T. O. (2024) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2024.102168 · Cited by 23
Beyond conventional: Analysing the factors affecting the adoption of electric four-wheelers in an Indian metropolis
Bhat, F. A., Seth, Y., & Verma, A. (2024) · Transportation Research Part D: Transport and Environment · doi:10.1016/j.trd.2024.104200 · Cited by 20
The good, the bad, and the unseen: wild mammal encounters influence wildlife preferences of residents across socio-demographic gradients
Moesch, S., Straka, T. M., Jeschke, J. M., Haase, D., & Kramer‐Schadt, S. (2024) · Ecology and Society · doi:10.5751/es-15211-290306 · Cited by 27
“Because online surveys might attract only specific demographics (Bethlehem 2010, Andrade 2020), it is imperative to consider the potential influence of our dissemination strategy.”
Occupational stress and social support among nurses
Dziedzic, B., Łodziana, K., Marcysiak, M., & Kryczka, T. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1621312 · Cited by 28
“However, online surveys often yield more reliable data, likely because respondents feel less concerned about their privacy (63).”
Father-Adolescent Relationship Closeness: A Path Analysis of Family Factor Associates with Father-Adolescent Engagement and Relationship Quality
Trahan, M. H., Morley, R. H., & Shafer, K. (2020) · Child and Adolescent Social Work Journal · doi:10.1007/s10560-020-00677-1
“Additionally, respondents were paid for their participation in the panel survey, which may influence their responses and increase self-selection bias (Bethlehem, 2010), and quota sampling may not provide for identity of the respondent (Im & Chee, 2011).”
Background
Pre-Exercise Nutrition Habits and Beliefs of Endurance Athletes Vary by Sex, Competitive Level, and Diet
Rothschild, J. A., Kilding, A. E., & Plews, D. J. (2020) · Journal of the American College of Nutrition · doi:10.1080/07315724.2020.1795950 · Cited by 22
“Due to a selfselection bias (49), it is also possible that the respondents who chose to participate may not represent all endurance athletes.”
Background
Towards sustainable urban food systems: Analyzing contextual and intrapsychic drivers of growing food in small-scale urban agriculture
Alemu, M. H., & Grebitus, C. (2020) · PLoS ONE · doi:10.1371/journal.pone.0243949 · Cited by 22
The limited potential of regional electricity marketing – Results from two discrete choice experiments in Germany
Lehmann, N., Sloot, D., Ardone, A., & Fïchtner, W. (2021) · Energy Economics · doi:10.1016/j.eneco.2021.105351 · Cited by 23
“For example, self-selection bias may be present in both samples (see Bethlehem 2010).”
Background
A Cross-Sectional Examination of the Mental Wellbeing, Coping and Quality of Working Life in Health and Social Care Workers in the UK at Two Time Points of the COVID-19 Pandemic
McFadden, P., Neill, R. D., Moriarty, J., Gillen, P. A., Mallett, J., Manthorpe, J., Currie, D., Schroder, H. S., Ravalier, J. M., Nicholl, P., et al. (2021) · Epidemiologia · doi:10.3390/epidemiologia2030017 · Cited by 24
Disability, Discrimination, and Mental Health during the COVID-19 Pandemic: A Stress Process Model
Brown, R. L., & Ciciurkaite, G. (2022) · Society and Mental Health · doi:10.1177/21568693221115347 · Cited by 23
Turnover of Companies in OpenStack: Prevalence and Rationale
Zhang, Y., Liu, H., Tan, X. L., Zhou, M., Jin, Z. G., & Zhu, J. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3510849 · Cited by 22
“As pointed out by existing studies [3, 44, 65], survey results might be affected by a selection bias: companies that did not respond may have had different reasons for disengaging.”
Result
How Representative Are SMS Surveys in Africa? Experimental Evidence From Four Countries
Lau, C., Lombaard, A., Baker, M., Eyerman, J., & Thalji, L. (2018) · International Journal of Public Opinion Research · doi:10.1093/ijpor/edy008 · Cited by 22
Health and social behaviour through pandemic phases in Switzerland: Regional time-trends of the COVID-19 Social Monitor panel study
Moser, A., von Wyl, V., & Höglinger, M. (2021) · PLoS ONE · doi:10.1371/journal.pone.0256253 · Cited by 22
“First, because our survey is online-based, we probably include more individuals with a greater affinity to online processes and better educated individuals in the survey, which leads to some sampling selection bias [54].”
Background
Improving web survey quality
Steinmetz, S., Bianchi, A., Tijdens, K. G., & Biffignandi, S. (2014) · Wiley eBooks · doi:10.1002/9781118763520.ch12 · Cited by 25
The role of university switching costs, perceived service quality, perceived university image and student satisfaction in shaping student loyaltyPreprint
Ali, M., Amir, H., & Ahmed, M. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3261753 · Cited by 22
Can playing defense yield gains? Examining the relationships among regulatory focus, innovation, and SME performance
Barber, D., Peake, W. O., & Harris, M. L. (2022) · Journal of Small Business Management · doi:10.1080/00472778.2022.2141762 · Cited by 21
“As such self-selection bias remains a limitation of our methodological approach and survey data in general, whether internet based (Bethlehem, 2010) or mail in (Whitehead, 1991).”
Background
Evolution of burnout and psychological distress in healthcare workers during the COVID-19 pandemic: a 1-year observational study
Cyr, S., Marcil, M.-J., Houchi, C., Marin, M., Rosa, C., Tardif, J., Guay, S., Guertin, M., Genest, C., Forest, J., et al. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-04457-2 · Cited by 25
“The web-based aspect of the study might have caused a volunteer and selection bias [69].”
Background
Farmer and Veterinary Practices and Opinions Related to the Diagnosis of Mastitis and Metabolic Disease in UK Dairy Cows
Donadeu, F. X., Howes, N. L., Esteves, C. L., Howes, M. P., Byrne, T. J., & Macrae, A. (2020) · Frontiers in Veterinary Science · doi:10.3389/fvets.2020.00127 · Cited by 21
“Voluntary respondents in a survey typically tend to be members of the sample populations (UK farmer and veterinary communities in this case) that are more concerned about the topic under survey and have also stronger opinions about it (12).”
Background
Who Are the Internet Users, Mobile Internet Users, and Mobile-Mostly Internet Users?: Demographic Differences across Internet-Use Subgroups in the U.S.
(2015) · Ubiquity Press eBooks · doi:10.5334/bar.g · Cited by 21
Ethnic Restaurants: Bringing Uniqueness to the Table Through Handwriting
Chen, F. F., Liu, S. Q., & Mattila, A. S. (2021) · Cornell Hospitality Quarterly · doi:10.1177/19389655211025472 · Cited by 20
“Specifically, in the recruitment message we broadly described the research as a consumer behavior study and did not mention the ethnic restaurant context to minimize potential self-selection bias (Bethlehem, 2010).”
Background
Mental health emergencies and COVID-19: the impact of ‘lockdown’ in the East Midlands of the UK
Moore, H. E., Siriwardena, A. N., Gussy, M. G., Tanser, F. C., Hill, B., & Spaight, R. (2021) · BJPsych Open · doi:10.1192/bjo.2021.973 · Cited by 21
Migrants' Voter Turnout in the Home Country Elections: Non‐Integration or Political Anchor?
Gherghina, S., & Basarabă, A. (2023) · Politics and Governance · doi:10.17645/pag.7396 · Cited by 20
“However, two main disadvantages are the coverage bias rooted in unequal internet access (van Dijk, 2005) and self‐selection bias due to topic saliency (Bethlehem, 2010).”
Background
Mental health disorder symptoms among serving Royal Canadian Mounted Police
Carleton, R. N., Jamshidi, L., Nisbet, J., Shields, R. E., & Andrews, K. L. (2024) · Police Practice and Research · doi:10.1080/15614263.2024.2318209 · Cited by 20
“…data collection precludes causal claims because the higher prevalence may reflect other factors, including increased reporting due to RCMP and other efforts at increasing psychoeducation and reducing stigma (Carleton, 2021; Carleton et al., 2022) or self-selection bias (Bethlehem, 2010).”
Indirect effects of trait impulsivity on body mass
Meule, A., & Jens Blechert, A. (2017) · Eating Behaviors · doi:10.1016/j.eatbeh.2017.01.012 · Cited by 20
“Specifically, samples in web-based studies are biased due to under-coverage and self-selection (Bethlehem, 2010) and the majority of participants in the current study were female students who had normal weight.”
Background
App-Based Delivery of Clinical Emotional Freedom Techniques: Cross-Sectional Study of App User Self-Ratings
Church, D., Stapleton, P., & Sabot, D. (2020) · JMIR mhealth and uhealth · doi:10.2196/18545 · Cited by 20
“As with many web-based surveys, the current large convenience sample comprising app users was determined by self-selection rather than probability sampling, which can lead to biased estimates [82].”
Background
An evaluation of whether propensity score adjustment can remove the self-selection bias inherent to web panel surveys addressing sensitive health behaviours
Copas, A. J., Burkill, S. M., Conrad, F., Couper, M. P., & Erens, B. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01134-4 · Cited by 19
Effects of cooperative learning and situational simulation on nursing competence in clinical practice among nursing students: A quasi-experimental study
Wu, P. (2024) · Nurse Education Today · doi:10.1016/j.nedt.2024.106464 · Cited by 19
Beyond self-selection: the multilayered online review biases at the intersection of users, platforms and culture
Shen, X. S., Pan, B., Hu, T., Chen, K., Qiao, L., & Zhu, J. (2020) · Journal of Hospitality and Tourism Insights · doi:10.1108/jhti-02-2020-0012 · Cited by 20
Use of a media intervention to increase positive attitudes toward transgender and gender diverse individuals
Taracuk, M. D., & Koch, J. M. (2021) · International Journal of Transgender Health · doi:10.1080/26895269.2021.1878479 · Cited by 21
How personality affects reaction. A mental health behavioral insight review during the Pandemic
Mourelatos, E. (2021) · Current Psychology · doi:10.1007/s12144-021-02425-9 · Cited by 27
A Review of Nonprobability Sampling Using Mobile Apps for Fishing Effort and Catch Surveys
Brick, J. M., Andrews, W. R., & Foster, J. R. (2021) · Transactions of the American Fisheries Society · doi:10.1002/tafs.10342 · Cited by 19
Mental Health Disorders and Suicidal Behaviors Among Provincial Correctional Workers
Ricciardelli, R., McKendy, L., Jamshidi, L., & Carleton, R. N. (2022) · Journal of Occupational and Environmental Medicine · doi:10.1097/jom.0000000000002488 · Cited by 25
“The current results were still based on self-report, rather than clinical interviews, which may also limit generalizability.(46) Respondents may under (or over) report symptoms, even when anonymous, which could impact results.”
Result
Social influence and reduction of animal protein consumption among young adults: Insights from a socio-psychological model
Severijns, R., Streukens, S., Brouwer, J., & Lizin, S. (2023) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2023.102094 · Cited by 23
Never miss a beep: Using mobile sensing to investigate (non-)compliance in experience sampling studies
Reiter, T., & Schoedel, R. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02252-9 · Cited by 29
“Second, when interpreting our results, we should keep in mind that the lack of some effects, e.g., for the person features, might be related to one major challenge of many empiricalstudies:self-selectionorcollider-stratiﬁcation-bias (Bethlehem, 2010; Cinelli et al., 2022).”
Background
Enhancing Drought Resilience and Vulnerability Assessment in Small Farms: A Global Expert Survey on Multidimensional Indicators
Sass, K. S., Konak, A., de Macedo, M. B., Benso, M. R., Shrimpton, E. A., Balta‐Ozkan, N., Sarmah, T., Mendiondo, E. M., da Silva, G. J., da Silva, P. G. C., et al. (2023) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2024.104616 · Cited by 19
Échantillonner des populations rares. Une expérimentation du Respondent Driven Sampling en milieu musical
Bataille, P., Perrenoud, M., & Brändle, K. (2018) · Sociologie · doi:10.3917/socio.092.0197 · Cited by 19
“Si ce biais des enquêtes par internet est bien connu et documenté (Bethlehem, 2010), nos résultats montrent que l’échantillonnage par RDS peut être une alternative efficace et de qualité quand, dans les cas comme le nôtre, la population visée n’apparaît dans aucun registre préexistant.”
Background
Investigating the prevalence of anxiety and depression in people living with patellofemoral pain in the UK: the Dep-Pf Study
Wride, J., & Bannigan, K. (2019) · Scandinavian Journal of Pain · doi:10.1515/sjpain-2018-0347 · Cited by 19
“Since our respondents were able to choose whether to participate in the study, it is not possible to say that this was a representative sample of the population as a whole [51].”
Bridging the void: science communication videos for sustainable whale watching
Finkler, W., Higham, J. E. S., León, B., & Aitken, R. (2019) · International Journal of Science Education Part B · doi:10.1080/21548455.2019.1671636 · Cited by 19
“This reflects both the popularity of whales and whale watching (Chen, 2011) as well as the fact that online surveys can have some preselection bias towards topics of personal interests (Bethlehem, 2010; Lohmann & Schmücker, 2009).”
Background
Who Am I and What Do I Need? Identity Difficulties as a Mechanism of the Link Between Childhood Neglect and Adult Sexual Disturbances
Bigras, N., Bosisio, M., Daspe, M., & Godbout, N. (2020) · International Journal of Sexual Health · doi:10.1080/19317611.2020.1796881 · Cited by 22
“Also, individuals who select themselves for the survey (e.g., self-selection bias) may be of great importance (Bethlehem, 2010) regarding who would volunteer for online child maltreatment and sexuality research.”
Background
Comparative Optimism, Self-Superiority, Egocentric Impact Perception and Health Information Seeking: A COVID-19 Study
Hoorens, V., Scambler, S., Deschrijver, E., Coulson, N., Speed, E., & Asimakopoulou, K. G. (2022) · Psychologica Belgica · doi:10.5334/pb.1139 · Cited by 21
Diversity and ethics in trauma and acute care surgery teams: results from an international survey
Cobianchi, L., Mas, F. D., Massaro, M., Biffl, W. L., Catena, F., Coccolini, F., Dionigi, B., Dionigi, P., Saverio, S. D., Fugazzola, P., et al. (2022) · World Journal of Emergency Surgery · doi:10.1186/s13017-022-00446-8 · Cited by 18
Enhancing estimation methods for integrating probability and nonprobability survey samples with machine‐learning techniques. An application to a Survey on the impact of the COVID‐19 pandemic in Spain
Rueda, M. D. M., Pasadas‐del‐Amo, S., Cobo, B., Castro-Martín, L., & Ferri‐García, R. (2022) · Biometrical Journal · doi:10.1002/bimj.202200035 · Cited by 19
“…have large sample sizes, they present important selection and coverage problems since the sample generation process is unknown in most cases, so they can compromise the generalization of the results to the population under study (Bethlehem, 2010; Elliott & Haviland, 2007; Vehovar et al., 2016).”
Result
Beyond Racial Linked Fate: Inter-Minority Political Solidarity and Political Participation
Mejía, Y. D. M. (2023) · Political Behavior · doi:10.1007/s11109-023-09861-2 · Cited by 18
“2 There are concerns with web surveys due to sampling issues and internet accessibility (Bethlehem, 2010; Couper & Miller, 2008; Dillman, 2006).”
Prevalence of detransition in persons seeking gender-affirming hormonal treatments: a systematic review
Feigerlová, E. (2024) · The Journal of Sexual Medicine · doi:10.1093/jsxmed/qdae186 · Cited by 18
Modelling Nonlinearities and Reference Dependence in General Practitioners' Income Preferences
Holte, J. H., Sivey, P. M., Abelsen, B., & Olsen, J. A. (2015) · Health Economics · doi:10.1002/hec.3208 · Cited by 19
Size is in the eye of the beholder: How differences between neighbourhoods and individuals explain variation in estimations of the ethnic out-group size in the neighbourhood
Laméris, J., Kraaykamp, G., Ruiter, S., & Tolsma, J. (2018) · International Journal of Intercultural Relations · doi:10.1016/j.ijintrel.2018.01.002 · Cited by 18
Can a voluntary web survey be useful beyond explorative research?
Fabo, B., & Kahanec, M. (2018) · International Journal of Social Research Methodology · doi:10.1080/13645579.2018.1454639 · Cited by 18
“…of some sub-populations and selection into survey samples, relate to sources of survey error associated with coverage, non-response and measurement, as well as inference from non-probability samples (Bandilla, Bosnjak, & Altdorfer, 2003; Bethlehem, 2010; Couper, 2000; Dillman & Bowker, 2001).”
Background
Two shades of Green? The electorates of GreenLeft and the Party for the Animals
Otjes, S., & Krouwel, A. P. M. (2015) · Environmental Politics · doi:10.1080/09644016.2015.1067349 · Cited by 18
“Most online surveys are subject to problems of under-coverage and selfselection that can potentially bias estimates (Bethlehem 2010).”
Background
Distinguishing groups and exploring health differences among multiple job holders aged 45 years and older
Bouwhuis, S., Hoekstra, T., Bongers, P. M., Boot, C. R. L., Geuskens, G., & van der Beek, A. J. (2018) · International Archives of Occupational and Environmental Health · doi:10.1007/s00420-018-1351-2 · Cited by 18
“This may have biased our findings (Bethlehem 2010).”
Result
Alternate policing strategies: Cost-effectiveness of cautioning for cannabis offences
Shanahan, M., Hughes, C., McSweeney, T., & Griffin, B. A. (2017) · International Journal of Drug Policy · doi:10.1016/j.drugpo.2016.12.012 · Cited by 17
Profiles of Citizenship Orientations Among Youth
Tzankova, I., Prati, G., & Cicognani, E. (2021) · Young · doi:10.1177/11033088211008691 · Cited by 21
“As the population of interest were young people with greater Internet access (Bethlehem, 2010) and in Italy the rate of regular Internet usage is about 90% among those aged 15–24 (ISTAT, 2018b), under-coverage bias should be of less concern in the present study.”
BackgroundMethod
Potentially Psychologically Traumatic Event Exposure Histories of new Royal Canadian Mounted Police Cadets
Andrews, K. L., Jamshidi, L., Nisbet, J., Brunet, A., Afifi, T. O., Asmundson, G. J. G., Fletcher, A. J., Maguire, K. Q., Teckchandani, T. A., Lix, L. M., et al. (2023) · The Canadian Journal of Psychiatry · doi:10.1177/07067437221149467 · Cited by 19

Authors: Bethlehem, J.
Affiliations: Centraal Bureau voor de Statistiek (NL)
Corresponding author: Bethlehem, Jelke
Venue: International Statistical Review
Publisher: Wiley
Published: 2010-08
Volume: 78
Issue: 2
Pages: 161-188
ISSN: 0306-7734 (ISSN-L)
1751-5823
DOI: 10.1111/j.1751-5823.2010.00112.x
OpenAlex: W2129332419
Cited by: 1,315
References: 37
Citations by year: 2026: 228 · 2025: 173 · 2024: 151 · 2023: 149 · 2022: 122 · 2021: 144 · 2020: 109 · 2019: 47 · 2018: 38 · 2017: 54 · 2016: 20 · 2015: 32 · 2014: 19 · 2013: 14 · 2012: 6
FWCI: 29.95
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations: International Statistical Review (publishedVersion)
UvA-DARE (University of Amsterdam) (publishedVersion, other-oa)
RePEc: Research Papers in Economics (submittedVersion)
Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Topics: Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
Physical Sciences › Mathematics › Statistics and Probability › Census and Population Estimation
Keywords: online surveys, selection bias, undercoverage, self-selection, internet access, survey design, participant recruitment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Selection Bias in Web Surveys

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks