Cited by 1,315Survey Methodology and NonresponseSurvey Sampling and Estimation TechniquesCensus and Population Estimation

The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.

Selection Bias in Web Surveys

Open access

Bethlehem, Jelke·2010·International Statistical Review

DOI 10.1111/j.1751-5823.2010.00112.x

Open this paper in CoChat — free account Open PDF

Abstract

Summary: At first sight, web surveys seem to be an interesting and attractive means of data collection. They provide simple, cheap, and fast access to a large group of potential respondents. However, web surveys are not without methodological problems. Specific groups in the populations are under‐represented because they have less access to Internet. Furthermore, recruitment of respondents is often based on self‐selection. Both under‐coverage and self‐selection may lead to biased estimates. This paper describes these methodological problems. It also explores the effect of various correction techniques (adjustment weighting and use of reference surveys). This all leads to the question whether properly design web surveys can be used for data collection. The paper attempts to answer this question. It concludes that under‐coverage problems may solve itself in the future, but that self‐selection leads to unreliable survey outcomes.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1111/j.1751-5823.2010.00112.x

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,385
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008) · Cited by 5,345
A generalization of sampling without replacement from a finite universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483446 · Cited by 4,293
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
A Generalization of Sampling Without Replacement From a Finite Universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.2307/2280784 · Cited by 3,011
Web Surveys
Couper, M. P. (2000) · Public Opinion Quarterly · doi:10.1086/318641 · Cited by 2,086
On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection
Neyman, J. (1934) · Journal of the Royal Statistical Society · doi:10.2307/2342192 · Cited by 1,086
The Effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies
Cochran, W. G. (1968) · Biometrics · doi:10.2307/2528036 · Cited by 961

Show all 37 references

Research works printed but not linked: 20

Bethlehem J.G., 1988, Reduction of the nonresponse bias through regression estimation, J. Official Statist., 4, 251
Bethlehem J.G., 2002, Survey Nonresponse
Bethlehem J.G., 2009, The Rise of Survey Sampling
Börsch‐Supan A., 2004, Correcting the Participation Bias in an Online Survey
Bowley A.L., 1926, Measurement of the precision attained in sampling, Bull. Int. Statist. Inst., 6
Cobben F., 2004, Nonresponse Correction Techniques in Household Surveys at Statistics Netherlands: a CAPI‐CATI Comparison
Cochran W.G., 1953, Sampling Techniques
Cochran W.G., 1977, Sampling Techniques
De Haan J., 2006, Jaarboek ICT en Samenleving, De Digitale Generatie
Eurostat, 2007, More than 40% of Households have Broadband Internet Access
Hansen M.H., 1953, Survey Sampling Methods and Theory
Kendall M.G., 1960, Where shall the history of statistics begin, Biometrika, 47, 447
Kiaer A. N., 1895, Observations et Expériences Concernant des Dénombrements Représentatives, Bull. Internat. Statist. Inst., 176
Kiaer A. N., 1997, Den Repräsentative Undersökelsesmetode, Christiania Videnskabsselskabets Skrifter. II. Historiskfilosofiske klasse, 4, 37
Kish L., 1967, Survey Sampling
Kuusela V., 2003, Proceedings of the 4th ASC International Conference, 317
10.1111/j.2397-2335.1934.tb04184.x · DOI
Scherpenzeel A., 2010, Social Research and the Internet: Advances in applied Methods and New Research Strategies
Terhanian G., 2001, Exploiting analytical advances: Minimizing the biases associated with Internet‐based surveys of non‐random samples, ESOMAR Publication Services, 248, 247
Vonk T., 2006, The effects of panel recruitment and management on research results, a study among 19 online panels, ESOMAR Publication Services, 317, 79

Sorted by citations

Items 401–500 of 1,000

Digital Data Mining
Venturelli, P., Skov, C., Audzijonytė, A., Belmaker, J., Cardoso, A. S., Gordon, L., Gundelund, C., Hyder, K., Jarić, I., Lennox, R. J., et al. (2025) · Fish & fisheries series/Fish and fisheries series (Print) · doi:10.1007/978-3-031-99739-6_15 · Cited by 10
Managing Directors’ Perspectives on Digital Maturity in German Hospitals—A Multi-Point Online-Based Survey Study
Burmann, A., Fischer, B., Brinkkötter, N., & Meister, S. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19159709 · Cited by 10
“An under-representation of specific groups was not anticipated, while a self-selection bias was discussed [42].”
BackgroundMethod
Using Video Cameras as a Research Tool in Public Spaces: Addressing Ethical and Information Governance Challenges Under Data Protection Legislation
Benton, J. S., Evans, J., Mourby, M. J., Elliot, M. J., Anderson, J., Hipp, J. A., & French, D. (2023) · Journal for the Measurement of Physical Behaviour · doi:10.1123/jmpb.2022-0037 · Cited by 10
Adaptation and Validation of a French Version of the Vaccination Attitudes Examination (VAX) Scale
Eisenblaetter, M., Madiouni, C., Laraki, Y., Capdevielle, D., & Raffard, S. (2023) · Vaccines · doi:10.3390/vaccines11051001 · Cited by 12
“The most important disadvantages are selection biases: under-coverage and self-selection [52,53].”
Background
The mental health impact of the COVID‐19 pandemic and exposure to other potentially traumatic events up to old age
Havermans, D. C. D., Hoeboer, C. M., Sobczak, S., Primasari, I., Coimbra, B. M., Hovnanyan, A., Novaković, I. Z., Langevin, R., Aakvaag, H. F., Grace, E., et al. (2023) · Journal of Traumatic Stress · doi:10.1002/jts.22937 · Cited by 11
Psychometric properties and general population reference values for PROMIS Global Health in Hungary
Bató, A., Brodszky, V., Mitev, A. Z., Jenei, B., & Rencz, F. (2023) · The European Journal of Health Economics · doi:10.1007/s10198-023-01610-w · Cited by 9
“Selection bias might have occurred as online panel data collections may be subject to possible self-selection and underrepresentation of certain groups (e.g. those without internet access) [57].”
Background
Exploring the Big Data Paradox for various estimands using vaccination data from the global COVID-19 Trends and Impact Survey (CTIS)
Yang, Y., Dempsey, W., Han, P., Deshmukh, Y., Richardson, S. T., Tom, B. D. M., & Mukherjee, B. (2024) · Science Advances · doi:10.1126/sciadv.adj0266 · Cited by 9
“However, these non-probability surveys may suffer from selection bias, which can lead to invalid inference, especially with a very large sample (Bethlehem, 2010).”
Background
Pre-registration nursing students’ perceptions and experience of intentional rounding: A cross-sectional study
Ryan, L., Jackson, D. W., Woods, C., & Usher, K. J. (2019) · Nurse Education in Practice · doi:10.1016/j.nepr.2019.102691 · Cited by 10
Sample size and spatial configuration of volunteered geographic information affect effectiveness of spatial bias mitigation
Zhang, G., & Zhu, A. (2020) · Transactions in GIS · doi:10.1111/tgis.12679 · Cited by 10
“For mitigating bias in probability samples, other methods under the design-based statistical inference framework that make use of inclusion probability (e.g., Bethlehem, 2010; Stehman et al., 2018) may be more beneficial and efficient.”
Method
Effectiveness of risk minimisation measures for valproate: A cross‐sectional survey among physicians in Europe
Toussi, M., Bardoulat, I., Tcherny‐Lessenot, S., de Voogd, H., Dimos, V., & Kaplan, S. (2020) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5119 · Cited by 10
“The potential for selection bias due to voluntary participation of targeted physicians is another inherent bias in webbased surveys which affects the quality of the survey and leads to unreliable results and inferences.(22,23) Hence, to quantify any selection bias, the distribution of stratification criterion of physi…”
Background
Estimating General Parameters from Non-Probability Surveys Using Propensity Score Adjustment
Castro-Martín, L., Rueda, M. D. M., & Ferri‐García, R. (2020) · Mathematics · doi:10.3390/math8112096 · Cited by 10
“Calibration is able to remove selection bias in nonprobability samples if the selection mechanism is ignorable [4], and despite being originally developed for parametric estimation, further work [5–7] has extended calibration to distribution function, quantile and poverty measures estimation.”
BackgroundMethod
Not just a walk in the woods? Exploring the impact of individual characteristics and changing job roles on stress among conservation officers
Ledford, L. S., Osborne, D. L., Edwards, B. D., & Stickle, B. (2020) · Police Practice and Research · doi:10.1080/15614263.2020.1821682 · Cited by 10
“Self-selection bias is an issue inherent to online surveys, and as such, the results of the study should be viewed with some caution (Bethlehem, 2010).”
Background
Sources of Bias in Research on Gambling Among Older Adults: Considerations for a Growing Field
van der Maas, M., Nower, L., Matheson, F. I., Turner, N. E., & Mann, R. E. (2021) · Current Addiction Reports · doi:10.1007/s40429-021-00365-9 · Cited by 10
Consultant psychiatrists’ experience of the impact of the COVID-19 pandemic on mental health services
Kelleher, E., Geary, E., Tawfik, M., Mhuircheartaigh, E. N., Gavin, B. E., Wall, M., Lyne, J., Doherty, A. M., & McNicholas, F. (2021) · Irish Journal of Psychological Medicine · doi:10.1017/ipm.2021.41 · Cited by 10
“Those who responded to the survey were self-selecting and had access to the internet which may have introduced selection bias into the results (Bethlehem 2010).”
Background
Exploring mental illness attributions and treatment-seeking beliefs in a diverse Muslim-Australian sample
Bairami, K., Spivak, B., Burke, L. M., & Shepherd, S. M. (2021) · Clinical Psychologist · doi:10.1080/13284207.2021.1929143 · Cited by 10
Bringing Salary Transparency to the World
Kenthapadi, K., Ambler, S., Zhang, L., & Agarwal, D. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3132863 · Cited by 10
“See [1] for a survey of non-response bias challenges, and [10] for an overview of selection bias.”
Background
Lobbying in social media as a new source of survey bias
Taczanowska, K., Latosińska, B., Brandenburg, C., Leisch, F., Czachs, C., & Muhar, A. (2023) · Journal of Outdoor Recreation and Tourism · doi:10.1016/j.jort.2023.100689 · Cited by 10
Do justice and trust affect acceptability of Indonesian social health insurance policy?
Zein, R. A., Putri, N. K., & Ridlo, I. A. (2020) · International Journal of Health Governance · doi:10.1108/ijhg-05-2019-0028 · Cited by 10
Bringing Salary Transparency to the World: Computing Robust Compensation Insights via LinkedIn SalaryPreprint
Kenthapadi, K., Ambler, S., Zhang, L., & Agarwal, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.09845 · Cited by 10
“See [1] for a survey of non-response bias challenges, and [10] for an overview of selection bias.”
Background
The return of non-probability sample: the electoral polls at the time of internet and social media
Franco, G. D. (2024) · Quality & Quantity · doi:10.1007/s11135-024-01835-8 · Cited by 9
“…to correct the imbalances of the sample (ex-post weightings, use of complex statistical models, etc.; see, among others, Atkeson et al. 2014; Bethlehem 2010; Bethlehem et al. 2011; Biemer 2010; Biemer and Peytchev 2012; Blumberg and Luke 2007; Bosio 1996; Busse and Fuchs 2012; Callegaro and…”
Preferred level of vehicle automation: How technology adoption, knowledge, and personality affect automation preference in Türkiye and Sweden
Öztürk, İ., Warner, H. W., & Özkan, T. (2024) · Cogent Psychology · doi:10.1080/23311908.2024.2314840 · Cited by 9
“…to the survey if an individual attempted to access it again after completion (to prevent multiple responses), the data and findings of the study may be subject to certain limitations due to its methodology (see Bethlehem, 2010; Curran, 2016; Van Selm & Jankowski, 2006 for further discussion).”
Psychometric evaluation of the Parental Reflective Functioning Questionnaire in Polish mothers
Kamza, A., Luyten, P., & Piotrowski, K. (2024) · PLoS ONE · doi:10.1371/journal.pone.0299427 · Cited by 10
“Indeed, online recruitments are fraught with self-selection bias, e.g., participants tend to be more interested in the research problem and, thus, more motivated to participate [76].”
Background
Understanding the Relationship Between Social Identity and Self-Expression Through Animated Gifs on Social Media
Wang, M., Bhuiyan, M. M., Rho, E. H. R., Luther, K., & Lee, S. W. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641031 · Cited by 11
“Furthermore, self-selection bias exists in this study, as in most online survey studies [10].”
Development and validation of a US quality of life instrument for hereditary angioedema due to C1 inhibitor deficiency
Castaldo, A. J., Siersbæk, N., Corcoran, D., Selva, C. N., Jervelund, C., Christiansen, S. C., Riedl, M. A., Zuraw, B. L., & Busse, P. J. (2024) · Annals of Allergy Asthma & Immunology · doi:10.1016/j.anai.2024.07.018 · Cited by 9
Fainder: A Fast and Accurate Index for Distribution-Aware Dataset Search
Behme, L., Galhotra, S., Beedkar, K., & Markl, V. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3681954.3681999 · Cited by 9
“These challenges are particularly evident in industrial settings due to repurposing or reusing data [7, 17, 26, 63].”
Background
What drives the adoption of electric four-wheelers in India? An investigation of the reasons for and against
Bhat, F. A., & Verma, A. (2025) · Travel Behaviour & Society · doi:10.1016/j.tbs.2025.101016 · Cited by 11
Unraveling the nexus: Subjective well-being and left-behind places
Panori, A., Kalogeresis, A., Papastergiou, E., Ziogas, T., & Ballas, D. (2025) · Regional Science Policy & Practice · doi:10.1016/j.rspp.2025.100222 · Cited by 9
Care during exceptional times: results of the CUIDAR study on the COVID-19 pandemic in Chile
Navarro, S. R., Moller, F., Alarcón‐Arcos, S., Energici, M.-A., & Schöngut‐Grollmus, N. (2022) · Tapuya Latin American Science Technology and Society · doi:10.1080/25729861.2022.2038858 · Cited by 9
“Although this strategy ensures satisfactory response rates (Robson 2011), it also makes it challenging to identify and characterize a population of respondents (Bethlehem 2010).”
Background
The effect of My Health Record use in the emergency department on clinician-assessed patient care: results from a survey
Mullins, A. K., O’Donnell, R., Morris, H., Ben‐Meir, M., Hatzikiriakidis, K., Brichko, L., & Skouteris, H. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01920-8 · Cited by 9
“First, surveys are exposed to self-selection bias [30], therefore this survey may not be representative of the opinions of all ED clinicians—and may favour participation by MHR users versus MHR non users.”
Background
Quantitative Methods in Migration Research
Salamońska, J. (2022) · IMISCOE research series · doi:10.1007/978-3-030-92377-8_26 · Cited by 9
Financial Toxicities Persist for Cancer Survivors Irrespective of Current Cancer Status: An Analysis of Medical Expenditure Panel Survey
Karim, M. A., Talluri, R., Shastri, S. S., Kum, H., & Shete, S. S. (2022) · Cancer Research Communications · doi:10.1158/2767-9764.crc-22-0166 · Cited by 9
“The self-reported nature of MEPS carries a possibility of recall bias (47); however, possibilities of self-selection bias prevalent in web surveys (48) is potentially reduced through the implementation of personal interviews (49) with a populationrepresentative complex survey design in MEPS.”
Background
Marketing opportunities and challenges for locally raised meats: An online consumer survey in South Carolina
Richards, S. T., & Vassalos, M. (2023) · Journal of Agriculture Food Systems and Community Development · doi:10.5304/jafscd.2023.122.009 · Cited by 9
Understanding levels of online participation in the U.K. museum sector
Charlesworth, E., Beresford, A. M., Warwick, C., & Impett, L. (2023) · Museum Management and Curatorship · doi:10.1080/09647775.2023.2188478 · Cited by 9
“…they have limitations as a method to understand the sector as a whole; privacy and commercial considerations restrict what museums can share, while a self-selection bias in respondents skews our understanding of online audiences (Nuccio and Bertacchini 2021; Gran et al. 2019; Bethlehem 2010).”
Professional beliefs of physicians and allied health professionals and their willingness to promote health in primary care: a cross-sectional survey
Brandt, S. K., Essig, S., & Balthasar, A. (2024) · BMC Family Practice · doi:10.1186/s12875-024-02412-6 · Cited by 10
“Nevertheless, we cannot completely exclude the possibility of selection bias in our data [47].”
Result
Sustainability in street food: Elaboration likelihood model (ELM) and image theory perspective
Bayraktar, Y. (2024) · International Journal of Gastronomy and Food Science · doi:10.1016/j.ijgfs.2024.101029 · Cited by 9
Assessing mental health literacy among Danish adolescents - development and validation of a multifaceted assessment tool (the Danish MeHLA questionnaire)
Zenas, D. R., Nielsen, M. G., Fonager, K., Petersen, K. S., Szulevicz, T., & Overgaard, C. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2020.113373 · Cited by 9
What Do Victim/Survivors of Sexual Violence Think about Circles of Support and Accountability?
Richards, K., Death, J., & Ronken, C. A. (2020) · Victims & Offenders · doi:10.1080/15564886.2020.1850578 · Cited by 9
“A key limitation of this approach to sampling is that those with strongly-held views are most likely to opt to participate (Arnold et al., 2016; Bethlehem, 2010); the views expressed may therefore not be generalizable.”
Background
Students’ Confidence and Interest in Palliative and Bereavement Care: A European Study
Orkibi, H., Biancalani, G., Bucuță, M., Sassu, R., Wieser, M., Franchini, L., Raccichini, M., Azoulay, B., Ciepliński, K., Leitner, A., et al. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.616526 · Cited by 9
“Also, while self-report data reflect students’ perceptions of their competence and skills, this may introduce social desirability and selfenhancing biases, as well as self-selection bias (Bethlehem, 2010).”
Background
Does the Conservative Non-pharmacological Management of Knee Osteoarthritis in Switzerland Reflect the Clinical Guidelines? A Survey Among General Practitioners, Rheumatologists, and Orthopaedic Surgeons
Ettlin, L., Nast, I., Huber, E. O., & Niedermann, K. (2021) · Frontiers in Rehabilitation Sciences · doi:10.3389/fresc.2021.658831 · Cited by 9
“Furthermore, medical doctors are invited very often to participate in surveys and, due to lack of time or interest, cannot respond to all invitations (34).”
Background
Variable selection in Propensity Score Adjustment to mitigate selection bias in online surveys
Ferri‐García, R., & Rueda, M. D. M. (2022) · Statistical Papers · doi:10.1007/s00362-022-01296-x · Cited by 10
“Calibration has also been found to be effective for removing selection bias when the target variable is not related to the selection mechanism (Bethlehem 2010; Rueda 2019).”
Background
Nocturnal pain, is the pain different compared with pain during the day? An exploratory cross-sectional study in patients with hip and knee osteoarthritis
van Berkel, A. C., Ringelenberg, R., Bindels, P., Bierma‐Zeinstra, S., & Schiphof, D. (2022) · Family Practice · doi:10.1093/fampra/cmac074 · Cited by 9
Widening the gap: the differential impact of COVID-19 on tenants and homeowners
Dreesen, S., & Heylen, K. (2022) · Housing Studies · doi:10.1080/02673037.2022.2150149 · Cited by 9
“For the GCS, knowledge of the survey and personal interest are important determinants of participation, which may lead to the overrepresentation of specific groups (Bethlehem, 2010).”
Background
Essay: Sunday shopping - The case of three surveys
Bethlehem, J. G. (2015) · Survey research methods · doi:10.18148/srm/2015.v9i3.6202 · Cited by 9
“Bethlehem (2010) shows that the potential size of the bias due to nonresponse in a probability survey is much smaller than the self-selection bias in in a non-probability survey.”
BackgroundSelf-citation
Measuring diagnostic heterogeneity using text-mining of the lived experiences of patients
Ghosh, C. C., McVicar, D., Davidson, G., & Shannon, C. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03044-1 · Cited by 9
Authoritarianism2.0: Social Media and Political Discussion in ChinaPreprint
Stockmann, D., & Luo, T. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2650341 · Cited by 7
Psychometric Properties of the Parenting Sense of Competence Scale for a Sample of French-speaking Mothers
Delhalle, M., Monseur, C., Knüppel, I., & Blavier, A. (2024) · Journal of Child and Family Studies · doi:10.1007/s10826-023-02776-w · Cited by 9
“Secondly, the use of an online survey involves particular biases which reduce the generalizability of the results, such as the need for participants to have internet access, and a lack of control over self-selection bias (Bethlehem, 2010).”
Estimation of Multidimensional Poverty in Morocco: A Small Area Estimation Approach Using Meteorological and Socio-economic Covariates
Betti, G., Crescenzi, F., Molini, V., & Mori, L. (2024) · Social Indicators Research · doi:10.1007/s11205-024-03340-9 · Cited by 8
Perceived Risk and Trust in News Sources: A Multi-Group Analysis of Deepfakes and AI-Generated Misinformation
Zhang, X., Lou, N., & Li, M. (2025) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2025.2487050 · Cited by 9
“While our sample captures key demographic characteristics of each population, it slightly overrepresents younger, highly educated individuals, which is a common limitation in survey-based studies (Bethlehem, 2010).”
Developing and Validating the Perceived System Curiosity Scale (PSC): Measuring Users' Perceived Curiosity of Systems
Leusmann, J., Villa, S., Berberoglu, B., Wang, C., & Mayer, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713087 · Cited by 10
Exploring the Conditional ESG Payoff of AI Adoption: The Roles of Learning Capability, Digital TMT, and Operational Slack
Liu, L., Wang, X., Tang, L., Sun, Z., & Wang, X. (2025) · Systems · doi:10.3390/systems13060399 · Cited by 12
Monitoring and evaluation of travel behaviour change in mobility-as-a-service (MaaS) trials: Insights from a longitudinal study
Ali, N., Sucu, S., Ouelhadj, D., Dadashzadeh, N., Woods, L., & Fletcher, G. P. (2025) · Transport Policy · doi:10.1016/j.tranpol.2025.07.020 · Cited by 11
User-centred smart building technologies for climate-resilient urban development: evidence from Istanbul
Agboola, O. P., & Baby, A. (2026) · Frontiers in Sustainable Cities · doi:10.3389/frsc.2026.1772855 · Cited by 8
“An analysis of early and late respondents showed no statistically significant differences in key demographic variables or reported comfort perceptions ( p > 0.05), suggesting minimal non-response bias and supporting the represen-tativeness of the final sample (Bethlehem, 2010).”
Influential
Attitude of Midwives towards Fluoride Recommendations and Oral Prevention in Infants and Young Children
Geiken, A., Holtmann, L., Dörfer, C. E., Schwarz, C., & Graetz, C. (2022) · Children · doi:10.3390/children9081135 · Cited by 8
“Due to the recruitment of participants via the Internet, the current survey, as is the case with all online surveys, is subject to bias [44].”
Background
Impact of sampling and data collection methods on maternity survey response: a randomised controlled trial of paper and push-to-web surveys and a concurrent social media survey
Harrison, S., Alderdice, F. A., & QUIGLEY, M. A. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-01833-8 · Cited by 8
“These include under-coverage, with participation being limited to those who have access to the internet and who visit the relevant webpage or social media platform, and selfselection, with participants self-identifying and choosing to take part [13].”
To Look or Not to Look: Instagram’s Sensitive-Content Screens Do Not Deter Vulnerable People From Viewing Negative Content
Simister, E. T., Bridgland, V. M. E., & Takarangi, M. K. T. (2023) · Behavior Therapy · doi:10.1016/j.beth.2023.06.001 · Cited by 8
Validation of a new glucocorticoid-specific Patient-Reported Outcome Questionnaire (the Steroid PRO)
Bridgewater, S., Ndosi, M. E., Dawson, J., Richards, P., Silverthorne, C. A., Dures, E. K., Goodman, S. M., Hill, C. L., Mackie, S. L., & Robson, J. C. (2023) · Annals of the Rheumatic Diseases · doi:10.1136/ard-2023-224946 · Cited by 8
“18 This is an important limitation especially considering that patients with low health literacy are likely to experience a higher burden of disease 19–21 and (by implication) steroid impact.”
Background
Illness Perceptions, Fear of Cancer Recurrence, and Mental Health in Teenage and Young Adult Cancer Survivors
Horwood, M., Loades, M. E., Košir, U., & Davis, C. (2023) · Journal of Pediatric Hematology/Oncology Nursing · doi:10.1177/27527530231190378 · Cited by 11
“However, online surveys are at greater risk of self-selection bias (Bethlehem, 2010), and our sample predominantly identi ﬁ ed as British (91.1”
BackgroundInfluential
Bayesian Integration of Probability and Nonprobability Samples for Logistic Regression
Salvatore, C., Biffignandi, S., Sakshaug, J. W., Wiśniowski, A., & Struminskaya, B. (2023) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smad041 · Cited by 8
Students' perceptions towards the ethical considerations of using artificial intelligence algorithms in clinical decision-making
Lin, G. S. S., Tan, W. W., & Hashim, H. (2024) · BDJ · doi:10.1038/s41415-024-7184-3 · Cited by 10
Adaptação transcultural da Workplace Deviance Scale (WDS) para o contexto brasileiro
do Nascimento, E., Coutinho, A. C. A. M., de Andrade, J. E., & Mendonça, L. M. M. (2015) · Revista Psicologia Organizações e Trabalho · doi:10.17652/rpot/2015.4.649 · Cited by 8
Reliable Data Collection in Participatory Trials to Assess Digital Healthcare Applications
Park, J., Park, S., Kim, G., Kim, K., Jung, J., Yoo, S., Yi, G., & Lee, D. (2020) · IEEE Access · doi:10.1109/access.2020.2985122 · Cited by 8
“This is a difﬁcult problem to solve in the ﬁeld of Web-based data collection [107].”
Background
Barrieren und Förderfaktoren in der evidenzbasierten Praxis (EBP) der Physiotherapie – Eine Bestandsaufnahme und Eruierung von Barrieren und Förderfaktoren zur Anwendung von EBP mit Physiotherapeut*innen der deutschsprachigen Schweiz
Freitag, L., Hohenauer, E., Deflorin, C., Böni, A., & Clijsen, R. (2021) · physioscience · doi:10.1055/a-1347-0561 · Cited by 8
“Daraus kann ein weiteres Verzerrungsrisiko, das sogenannte Self-Selection Bias, resultieren, welches definiert wird als eine nicht gleichwertige Verteilung der Teilnehmenden zu Nichtteilnehmenden [24].”
Background
Self-reflection promotes learning in citizen science and serves as an effective assessment tool
Golumbic, Y. N., Motion, A., Chau, A., Choi, L., D'Silva, D., Ho, J., Nielsen, M., Shi, K. Y., Son, C. D., Wu, O., et al. (2022) · Computers and Education Open · doi:10.1016/j.caeo.2022.100104 · Cited by 8
Does the COVID-pandemic affect the educational and financial inequality in weekly sport participation in the Netherlands?
Grubben, M., Hoekman, R., & Kraaykamp, G. (2022) · Contemporary Social Science · doi:10.1080/21582041.2022.2155867 · Cited by 8
“Nevertheless, it should be noted that in the LISS panel, similar to other web panels and sport participation surveys specifically (Bethlehem, 2010; Haudenhuyse, 2018), vulnerable groups are underrepresented (Knoef & Vos, 2008).”
Background
Knowledge, behaviours, and training related to 2SLGBTQIA+ health education amongst entry-level physiotherapy students in Canada: results of a nationwide, cross-sectional survey
Primeau, C. A., Philpott, H. T., Vader, K., Unger, J., Le, C. Y., Birmingham, T. B., & MacDermid, J. C. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04499-4 · Cited by 8
Psychotic experiences, perceived stress, and suicidal ideation among the general population during the COVID-19 pandemic: Findings from Japan
Stickley, A., Shirama, A., & Sumiyoshi, T. (2023) · Schizophrenia Research · doi:10.1016/j.schres.2023.07.020 · Cited by 8
To Remind or Not to Remind During Recruitment? An Analysis of an Online Panel in Germany
Klee, B., Costa, D. S. A., Frese, T., Knoechelmann, A., Meyer, G., Meyer, T., Purschke, O., Schildmann, J., Steckelberg, A., & Mikolajczyk, R. (2024) · International Journal of Public Health · doi:10.3389/ijph.2024.1606770 · Cited by 8
“While online studies often suffer from an unknown sampling framework, based on volunteers (e.g., when participants are recruited through social media) [26], systematic population-based approaches to recruitment can also be used.”
Background
Permission-Based E-Mail Marketing Websites Success
Lin, H., Li, H.-T., & Wang, Y. (2015) · Journal of Global Information Management · doi:10.4018/jgim.2015040101 · Cited by 8
Customers’ relationships leading to brand tribalism and tribe behavioral intentions
Jeong, J. Y., Hwang, J., & Hyun, S. S. (2020) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2020.102529 · Cited by 9
What utility scores do mental health service users, healthcare professionals and members of the general public attribute to different health states? A co-produced mixed methods online survey
Flood, C., Barlow, S. H., Simpson, A., Burls, A., Price, A. I., Cartwright, M., & Brini, S. (2018) · PLoS ONE · doi:10.1371/journal.pone.0205223 · Cited by 8
A utilização do método de coleta de dados via internet na percepção dos executivos dos institutos de pesquisa de mercado atuantes no Brasil
Calliyeris, V. E., & Casas, A. L. L. (2016) · Interações (Campo Grande) · doi:10.20435/interacoes.v13i1.300 · Cited by 8
Using LASSO to Calibrate Non-probability Samples using Probability Samples.
Chen, K.-T. (2016) · Deep Blue (University of Michigan) · Cited by 8
Methodological flaws in web surveys: Commentary to “Abuse and disrespect in childbirth assistance in Italy: A community based survey”
Lauria, L., Lega, I., Maraschini, A., D’Aloja, P., Ferraro, C., & Donati, S. (2018) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2018.05.023 · Cited by 8
An Ecological Approach to Data GovernancePreprint
McNealy, J. E. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.4164112 · Cited by 11
Data collection in pandemic times: the case of the Belgian COVID-19 health surveys
Braekman, E., Charafeddine, R., Berete, F., Bruggeman, H., Drieskens, S., Gisle, L., Hermans, L., der Heyden, J. V., & Demarest, S. S. (2023) · Archives of Public Health · doi:10.1186/s13690-023-01135-x · Cited by 8
“Web surveys exclude people without internet access or skills to use the internet by default to complete the survey [25].”
Background
Acceptance of online customer channels for damage claims in Germany
Lang, F., & Riegel, L. (2023) · Information Technology and Management · doi:10.1007/s10799-023-00404-z · Cited by 7
“Moreover, biases such as under-coverage and self-selection happen often in web surveys [57].”
Variation in gaze following across the life span: A process‐level perspective
Prein, J. C., Maurits, L., Werwach, A., Haun, D. B. M., & Bohn, M. (2024) · Developmental Science · doi:10.1111/desc.13546 · Cited by 11
“Regarding our sample of elderly adults, we expect a sampling bias (Bethlehem, 2010; Gosling et al., 2004; Remillard et al., 2014).”
Borrower discouragement: Explaining under-represented racial minority entrepreneurs’ relationship with debt financing
Mercado, M. (2024) · International Small Business Journal Researching Entrepreneurship · doi:10.1177/02662426241288174 · Cited by 7
“Third, the interpretation of the results may be limited by two issues that underlie web survey research, under coverage and self-selection (Bethlehem, 2010).”
Environmental Attitudes, Concerns, and Behaviors Across Survey Modes. Assessing Selection and Measurement Biases in ISSP 2020 ‘Push-to-Web’ Surveys
Lindholm, A., Steinmetz, S., & Sapin, M. (2024) · International Journal of Sociology · doi:10.1080/00207659.2024.2420295 · Cited by 7
“In case of web surveys, non-response error can also occur when sample members discontinue their survey participation due to technical hurdles they may face when completing the survey (Bethlehem 2010).”
Cooperate or Compete? Attributional Perspective for Interorganizational Conflicts in Construction Projects With Trust as a Moderator
Xu, Z., Lu, W., Zhang, L., & Guo, W. (2024) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2024.3506790 · Cited by 7
Health professionals’ willingness to share responsibility and strengthen interprofessional collaboration: a cross-sectional survey
Brandt, S. K., Essig, S., & Balthasar, A. (2025) · BMC Medical Education · doi:10.1186/s12909-024-06351-9 · Cited by 7
Exploring variation in the six-month review for stroke survivors: a national survey of current practice in England
Holmes, R., Ackerley, S. J., Fisher, R. J., & Connell, L. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-12323-6 · Cited by 7
“Firstly, there may have been a degree of self-selection bias in which those providers with a particular interest in 6MRs or with a clear service provision were more likely to respond [30].”
The BostonWalks study: a longitudinal travel survey using smartphone tracking
Meister, A., Bashan, N. F., Basu, R., Shen, X., Wang, R. Q., & Sevtsuk, A. (2025) · Transportation · doi:10.1007/s11116-025-10637-2 · Cited by 7
“Paid panels (as typically used in market research) for recruitment can introduce well-known biases (e.g. self-selection) (Bethlehem 2010).”
Non-response bias in gambling surveys
Heirene, R. M., Cobb‐Clark, D. A., Tymula, A. A., Santos, T., & Gainsbury, S. (2025) · International Gambling Studies · doi:10.1080/14459795.2025.2530106 · Cited by 7
“There are methodological ways to reduce the impact of selection bias, including obtaining a high response rate and demographic weighting (Sturgis & Kuha, 2022); however, neither can ensure the representativeness of a sample (Bethlehem, 2010; Mazor et al., 2002).”
Driving the change: How do personal factors and socio-economic context influence electric vehicles adoption across Europe?
Schiaroli, V., & Fraccascia, L. (2025) · Environmental Innovation and Societal Transitions · doi:10.1016/j.eist.2025.101044 · Cited by 7
Digital religion and Generation Z: an empirical study in the context of China
Liu, Z., Ghouri, A. M., Wang, J., & Lin, C. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1536644 · Cited by 40
“Self-selection bias is a concern in online surveys, as individuals who choose to participate may differ systematically from those who do not, potentially skewing results (Bethlehem, 2010).”
Exploring Prospective Teachers’ Intentions for Artificial Intelligence Integration in Education: The Role of Motivation
Yurt, E., & Kaşarcı, İ. (2025) · AERA Open · doi:10.1177/23328584251403990 · Cited by 9
“The online survey format, while ensuring broad reach, also may have attracted respondents with higher digital engagement, potentially skewing the sample toward those already predisposed to technology adoption (Bethlehem, 2010; Daikeler et al., 2020).”
The Role of Attachment Style, Adverse Childhood Experiences and Dissociation in Migraine
Mays, I., Flynn, J., McGuire, B. E., & Egan, J. (2021) · Journal of Trauma & Dissociation · doi:10.1080/15299732.2021.1989114 · Cited by 8
“…and self-selection bias are other important limitations to consider with the use of an online survey, given that they only capture data of those with internet access or adequate computer literacy and may not meet the principles of probability sampling (Bethlehem, 2010; Couper, 2000; Wright, 2005).”
Background
Psychometric properties of the Caring Efficacy Scale among personal care attendants working in residential aged care settings
Shrestha, S., Wells, Y., While, C., & Rahman, M. A. (2023) · Australian Journal on Ageing · doi:10.1111/ajag.13211 · Cited by 7
Engaging Senior Citizens and Flood Risk Reduction: Innovations in Community Engagement and the Resulting Spillover Effects
Cook, B. R., Kamstra, P., Winterton, R., Willis, R., & Kammoora, R. (2022) · Journal of Extreme Events · doi:10.1142/s2345737623410014 · Cited by 8
A blended learning approach for teaching thoracic radiology to medical students: a proof-of-concept study
Stoehr, F., Yang, Y., Müller, L., Gerstenmeier, P., dos Santos, D. P., Dietz, P., Weimer, A., Ludwig, M., Kloeckner, R., & Weimer, J. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1272893 · Cited by 7
“We had to consider selection bias, meaning that interested participants were more likely to complete the questionnaire (46).”
Background
Exploring the relationships between psychological variables and loot box engagement, part 1: pre-registered hypotheses
Close, J., Spicer, S. G., Nicklin, L. L., Uther, M., Whalley, B., Fullwood, C., Stiff, C., Parke, J., Lloyd, J., & Lloyd, H. M. (2023) · Royal Society Open Science · doi:10.1098/rsos.231045 · Cited by 6
“…sample was limited to a convenience sample of UK adults from the Prolific platform, which may be liable to similar selection biases to other online surveys [67], although there is evidence that it surpasses both MTurk and undergraduate student samples on a number of data quality measures [68,69].”
Exploring the relationships between psychological variables and loot box engagement, part 2: exploratory analyses of complex relationships
Spicer, S. G., Close, J., Nicklin, L. L., Uther, M., Whalley, B., Fullwood, C., Parke, J., Lloyd, J., & Lloyd, H. M. (2024) · Royal Society Open Science · doi:10.1098/rsos.231046 · Cited by 7
“…sample was limited to a convenience sample of UK adults from the Prolific platform, which may be liable to similar selection biases as other online surveys [56], although there is evidence that it surpasses both MTurk and undergraduate student samples on a number of data quality measures [57,58].”
Exploring consumer perception of uncoated wooden cladding: a study across three European countries
Gustavsen, G. W., van Niekerk, P. B., Niklewski, J., Brischke, C., & Alfredsen, G. (2025) · European Journal of Wood and Wood Products · doi:10.1007/s00107-025-02235-x · Cited by 7
“…et al. 2008, 2010; Sandak et al. 2015; Sandak and Sandak 2017; Rüther and Time 2015; Forsthuber and Grüll 2018) and even slight variations in meso-and microclimatic conditions will influence the aesthetic appearance of uncoated wooden clad-ding (Gobakken et al. 2008, 2010; Rüther and Time 2015).”
Influential
Understanding grower demographics, motivations and management practices to improve engagement, extension and industry resilience: a case study of the macadamia industry in the Northern Rivers, Australia
Stimpson, K., Luke, H., & Lloyd, D. J. (2018) · Australian Geographer · doi:10.1080/00049182.2018.1463832 · Cited by 7
“The survey sample was not randomly selected as the study relied on growers taking an interest and filling out the survey, therefore respondents were self-selected and hence it is possible that the results could reflect non-response bias (Bethlehem 2010).”
Background
Relationships among COVID-19 Prevention Practices, Risk Perception and Individual Characteristics: A Temporal Analysis
Wang, L., Yu, J., Chen, D., & Yang, L. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182010901 · Cited by 7
“For example, the online survey tended to exclude those who did not have access to technology and was more appealing to people with higher education levels [47,48]; we also received a large number of autogenerated invalid responses for the purpose of receiving incentives.”
Background
The Effect of Extralegal Factors in Decision-Making About Juvenile Offenders in Chile: A Quasi-Experimental Study
Carvacho, P., Droppelmann, C., & Piñones, M. M. (2022) · International Journal of Offender Therapy and Comparative Criminology · doi:10.1177/0306624x211066839 · Cited by 7
“This means, as in most self-selection web surveys (Bethlehem, 2010), that the sample is not statistically representative, because it may have a selection bias.”
Background
Changes in sleep behavior, sleep problems, and psychological distress/health-related quality of life of young Japanese individuals before and during the COVID-19 pandemic
Tanioka, K., Kayaba, M., Tomishima, S., Komada, Y., & Inoue, Y. (2022) · Chronobiology International · doi:10.1080/07420528.2022.2034839 · Cited by 8
“However, most of these limitations have been also observed previously (Bethlehem 2010).”
Background
Perceived behavioral traits as key determinants of entrepreneurial intention in higher education institutions
Sandoval-Álvarez, C. (2023) · TEC Empresarial · doi:10.18845/te.v18i1.6935 · Cited by 15
“In this respect, Bethlehem (2010) points out that volunteer bias is detrimental for validity and quality of survey results because we cannot apply probability sampling theory and the estimates obtained will be biased.”
Background
Prevalence of Immunity Presumed Using Rabies Vaccination History and Household Factors Associated with Vaccination Status among Domestic Dogs in Japan
Hidano, A., Hayama, Y., & Tsutsui, T. (2012) · Japanese Journal of Infectious Diseases · doi:10.7883/yoken.65.396 · Cited by 7

Authors: Bethlehem, J.
Affiliations: Centraal Bureau voor de Statistiek (NL)
Corresponding author: Bethlehem, Jelke
Venue: International Statistical Review
Publisher: Wiley
Published: 2010-08
Volume: 78
Issue: 2
Pages: 161-188
ISSN: 0306-7734 (ISSN-L)
1751-5823
DOI: 10.1111/j.1751-5823.2010.00112.x
OpenAlex: W2129332419
Cited by: 1,315
References: 37
Citations by year: 2026: 228 · 2025: 173 · 2024: 151 · 2023: 149 · 2022: 122 · 2021: 144 · 2020: 109 · 2019: 47 · 2018: 38 · 2017: 54 · 2016: 20 · 2015: 32 · 2014: 19 · 2013: 14 · 2012: 6
FWCI: 29.95
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations: International Statistical Review (publishedVersion)
UvA-DARE (University of Amsterdam) (publishedVersion, other-oa)
RePEc: Research Papers in Economics (submittedVersion)
Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Topics: Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
Physical Sciences › Mathematics › Statistics and Probability › Census and Population Estimation
Keywords: online surveys, selection bias, undercoverage, self-selection, internet access, survey design, participant recruitment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Selection Bias in Web Surveys

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks