Methods for Evaluating Respondent Attrition in Web-Based Surveys
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Electronic surveys are convenient, cost effective, and increasingly popular tools for collecting information. While the online platform allows researchers to recruit and enroll more participants, there is an increased risk of participant dropout in Web-based research. Often, these dropout trends are simply reported, adjusted for, or ignored altogether. OBJECTIVE: To propose a conceptual framework that analyzes respondent attrition and demonstrates the utility of these methods with existing survey data. METHODS: First, we suggest visualization of attrition trends using bar charts and survival curves. Next, we propose a generalized linear mixed model (GLMM) to detect or confirm significant attrition points. Finally, we suggest applications of existing statistical methods to investigate the effect of internal survey characteristics and patient characteristics on dropout. In order to apply this framework, we conducted a case study; a seventeen-item Informed Decision-Making (IDM) module addressing how and why patients make decisions about cancer screening. RESULTS: Using the framework, we were able to find significant attrition points at Questions 4, 6, 7, and 9, and were also able to identify participant responses and characteristics associated with dropout at these points and overall. CONCLUSIONS: When these methods were applied to survey data, significant attrition trends were revealed, both visually and empirically, that can inspire researchers to investigate the factors associated with survey dropout, address whether survey completion is associated with health outcomes, and compare attrition patterns between groups. The framework can be used to extract information beyond simple responses, can be useful during survey development, and can help determine the external validity of survey results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524 · cited 8×
“Although subject attrition is an issue in all types of health services research, dropout in Web-based health research can exceed expectations, reducing statistical power and potentially introducing bias [4].”
InfluentialMethods to increase response to postal and electronic questionnaires
Edwards, P. J., Roberts, I., Clarke, M. J., DiGuiseppi, C., Wentz, R., Kwan, I., Cooper, R., Felix, L. M., & Pratap, S. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000008.pub4 · Cited by 1,379
Persuasive System Design Does Matter: a Systematic Review of Adherence to Web-based Interventions
Kelders, S. M., Kok, R. N., Ossebaard, H. C., & Van Gemert-Pijnen, J. E. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2104 · Cited by 1,308 · cited 2×
“Additionally, dropout is often ill reported or presented in a way that prevents readers from being able to fully understand attrition [5].”
Systematic review identifies number of strategies important for retaining study participants
Robinson, K. A., Dennison, C. R., Wayman, D. M., Pronovost, P. J., & Needham, D. M. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.11.023 · Cited by 222
Methodological Challenges in Online Trials
Murray, E., Khadjesari, Z., White, I. R., Kalaitzaki, E., Godfrey, C., McCambridge, J., Thompson, S. G., & Wallace, P. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1052 · Cited by 195 · cited 1×
“Prior research in this area has investigated both the degree to which attrition from clinical trials occurs and methods for retaining participants [8-12].”
The Law of Attrition Revisited
Christensen, H., & Mackinnon, A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.3.e20 · Cited by 161
Guertler, D., Vandelanotte, C., Kirwan, M., & Duncan, M. J. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4339 · Cited by 147
Differential attrition in health behaviour change trials: a systematic review and meta-analysis
Crutzen, R., Viechtbauer, W., Spigt, M., & Kotz, D. (2015) · Psychology & Health · doi:10.1080/08870446.2014.953526 · Cited by 132 · cited 1×
“Prior research in this area has investigated both the degree to which attrition from clinical trials occurs and methods for retaining participants [8-12].”
Cited by
A randomized controlled trial of three smartphone apps for enhancing public mental health
Bakker, D., Kazantzis, N., Rickwood, D., & Rickard, N. (2018) · Behaviour Research and Therapy · doi:10.1016/j.brat.2018.08.003 · Cited by 176
Marshall, J. M., Dunstan, D. A., & Bartik, W. (2020) · JMIR Research Protocols · doi:10.2196/17159 · Cited by 24
The AI Double Standard: Humans Judge All AIs for the Actions of One
Manoli, A., Pauketat, J. V. T., & Anthis, J. R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711083 · Cited by 12
Metadata
- Authors
- Hochheimer, C. J., Sabo, R. T., Krist, A. H., Day, T., Cyrus, J., & Woolf, S. H.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2016-11-22
- Volume
- 18
- Issue
- 11
- Pages
- e301
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2554658613
- Cited by
- 97
- References
- 8
- FWCI
- 18.10
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.