Impact and Costs of Incentives to Reduce Attrition in Online Trials: Two Randomized Controlled Trials

Khadjesari, Zarnie; Murray, Elizabeth; Kalaitzaki, Eleftheria; White, Ian R; McCambridge, Jim; Thompson, Simon G; Wallace, Paul; Godfrey, Christine

doi:10.2196/jmir.1523

Cited by 114Survey Methodology and NonresponseEthics in Clinical ResearchStatistical Methods in Clinical Trials

Impact and Costs of Incentives to Reduce Attrition in Online Trials: Two Randomized Controlled Trials

Open access

Khadjesari, Zarnie, Murray, Elizabeth, Kalaitzaki, Eleftheria, White, Ian R, McCambridge, Jim, Thompson, Simon G, Wallace, Paul, Godfrey, Christine·2011·Journal of Medical Internet Research

DOI 10.2196/jmir.1523PMC PMC3221348

Open this paper in CoChat — free account Open PDF

Abstract

BACKGROUND: Attrition from follow-up is a major methodological challenge in randomized trials. Incentives are known to improve response rates in cross-sectional postal and online surveys, yet few studies have investigated whether they can reduce attrition from follow-up in online trials, which are particularly vulnerable to low follow-up rates. OBJECTIVES: Our objective was to determine the impact of incentives on follow-up rates in an online trial. METHODS: Two randomized controlled trials were embedded in a large online trial of a Web-based intervention to reduce alcohol consumption (the Down Your Drink randomized controlled trial, DYD-RCT). Participants were those in the DYD pilot trial eligible for 3-month follow-up (study 1) and those eligible for 12-month follow-up in the DYD main trial (study 2). Participants in both studies were randomly allocated to receive an offer of an incentive or to receive no offer of an incentive. In study 1, participants in the incentive arm were randomly offered a £5 Amazon.co.uk gift voucher, a £5 charity donation to Cancer Research UK, or entry in a prize draw for £250. In study 2, participants in the incentive arm were offered a £10 Amazon.co.uk gift voucher. The primary outcome was the proportion of participants who completed follow-up questionnaires in the incentive arm(s) compared with the no incentive arm. RESULTS: In study 1 (n = 1226), there was no significant difference in response rates between those participants offered an incentive (175/615, 29%) and those with no offer (162/611, 27%) (difference = 2%, 95% confidence interval [CI] -3% to 7%). There was no significant difference in response rates among the three different incentives offered. In study 2 (n = 2591), response rates were 9% higher in the group offered an incentive (476/1296, 37%) than in the group not offered an incentive (364/1295, 28%) (difference = 9%, 95% CI 5% to 12%, P < .001). The incremental cost per extra successful follow-up in the incentive arm was £110 in study 1 and £52 in study 2. CONCLUSION: Whereas an offer of a £10 Amazon.co.uk gift voucher can increase follow-up rates in online trials, an offer of a lower incentive may not. The marginal costs involved require careful consideration. TRIAL REGISTRATION: ISRCTN31070347; http://www.controlled-trials.com/ISRCTN31070347 (Archived by WebCite at http://www.webcitation.org/5wgr5pl3s).

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.2196/jmir.1523

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Monetary incentives have been found to increase enrolment and assessment completion in online interventions.
“Monetary incentives have been found to increase enrolment and assessment completion in online interventions [64, 65].”
From Recruitment, adherence and attrition challenges in internet-based indicated prevention programs for eating disorders: lessons learned from a randomised controlled trial of ProYouth OZ (2022) · cites this paper as [64]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

EuroQol: a new facility for the measurement of health-related quality of life
(1990) · Health Policy · doi:10.1016/0168-8510(90)90421-9 · Cited by 15,736 · see citing works
The AUDIT Alcohol Consumption Questions (AUDIT-C)<subtitle>An Effective Brief Screening Test for Problem Drinking</subtitle>
Bush, K. (1998) · Archives of Internal Medicine · doi:10.1001/archinte.158.16.1789 · Cited by 6,196 · see citing works
Untitled work
(2001) · Cited by 5,455 · see citing works
The Law of Attrition
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524 · see citing works
Methods to increase response to postal and electronic questionnaires
Edwards, P. J., Roberts, I., Clarke, M. J., DiGuiseppi, C., Wentz, R., Kwan, I., Cooper, R., Felix, L. M., & Pratap, S. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000008.pub4 · Cited by 1,379 · see citing works
Barriers to Participation in Randomised Controlled Trials
Ross, S., Grant, A., Counsell, C., Gillespie, W., Russell, I., & Prescott, R. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00141-9 · Cited by 907 · see citing works
Estimating the Effect of Incentives on Mail Survey Response Rates: A Meta-Analysis
Church, A. H. (1993) · Public Opinion Quarterly · doi:10.1086/269355 · Cited by 705 · see citing works
Alcohol Use Disorders Identification Test
Rebecca, B. (2010) · doi:10.4135/9781412964500.n20 · Cited by 481 · see citing works
Factors that limit the quality, number and progress of randomised controlled trials
Prescott, Counsell, Gillespie, Grant, Russell, Kiauka, Colthart, Ross, Shepherd, & Russell (1999) · Health Technology Assessment · doi:10.3310/hta3200 · Cited by 390 · see citing works
Reasons for participating in randomised controlled trials: conditional altruism and considerations for self
McCann, S., Campbell, M., & Entwistle, V. A. (2010) · Trials · doi:10.1186/1745-6215-11-31 · Cited by 309 · see citing works
Development of the Leeds Dependence Questionnaire (LDQ): a questionnaire to measure alcohol and opiate dependence in the context of a treatment evaluation package
Raistrick, D., Bradshaw, J., Tober, G., Weiner, J., ALLISON, J., & Healey, C. (1994) · Addiction · doi:10.1111/j.1360-0443.1994.tb03332.x · Cited by 295 · see citing works
Web-based screening and brief intervention for hazardous drinking: a double blind randomized controlled trial
Kypri, K., Saunders, J. B., Williams, S. M., McGee, R. O., Langley, J. D., Cashell‐Smith, M. L., & Gallagher, S. J. (2004) · Addiction · doi:10.1111/j.1360-0443.2004.00847.x · Cited by 287 · see citing works
Rates and Determinants of Repeated Participation in a Web-Based Behavior Change Program for Healthy Body Weight and Healthy Lifestyle
Verheijden, M. W., Jans, M. P., Hildebrandt, V. H., & Hopman-Rock, M. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.1.e1 · Cited by 195 · see citing works
Methodological Challenges in Online Trials
Murray, E., Khadjesari, Z., White, I. R., Kalaitzaki, E., Godfrey, C., McCambridge, J., Thompson, S. G., & Wallace, P. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1052 · Cited by 195 · see citing works
Self-citation
The Internet in Britain 2009
Gerber, W. H. D. A. E. J. H. A. M. M. (2009) · Cited by 180 · see citing works
Ethical Issues in the Design and Conduct of Randomised Controlled Trials
Lilford, R. J., Edwards, S., Braunholtz, D., et al. (2001) · SAGE Publications Ltd eBooks · doi:10.4135/9781848608344.n1 · Cited by 154 · see citing works
Recruitment and retention of an online sample for an HIV prevention intervention targeting men who have sex with men: the smart sex quest project
Bull, S. S., Lloyd, L. V., RIETMEIJER, C. A. M., & McFarlane, M. (2004) · AIDS Care · doi:10.1080/09540120412331292507 · Cited by 147 · see citing works
Incentives for Survey Participation
Singer, E., & Bossarte, R. M. (2006) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2006.07.013 · Cited by 130 · see citing works
Ethical issues in the design and conduct of randomised controlled trials.
Edwards, Lilford, Braunholtz, et al. (1998) · Health Technology Assessment · doi:10.3310/hta2150 · Cited by 124 · see citing works
Duty, desire or indifference? A qualitative study of patient decisions about recruitment to an epilepsy treatment trial
Canvin, K., & Jacoby, A. (2006) · Trials · doi:10.1186/1745-6215-7-32 · Cited by 103 · see citing works
Why Patients Continue to Participate in Clinical Research
Wendler, D. S. (2008) · Archives of Internal Medicine · doi:10.1001/archinte.168.12.1294 · Cited by 81 · see citing works
Effect of Incentives and Mailing Features on Online Health Program Enrollment
Alexander, G. L., Divine, G. W., Couper, M. P., McClure, J. B., Stopponi, M. A., Fortman, K. K., Tolsma, D. D., Strecher, V. J., & Johnson, C. C. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.01.028 · Cited by 80 · see citing works
Development of a Psychologically Enhanced Interactive Online Intervention for Hazardous Drinking
Linke, S., McCambridge, J., Khadjesari, Z., Wallace, P., & Murray, E. (2008) · Alcohol and Alcoholism · doi:10.1093/alcalc/agn066 · Cited by 68 · see citing works
Self-citation
The Alcohol Problems Questionnaire: reliability and validity
Williams, B. T. R., & Drummond, C. (1994) · Drug and Alcohol Dependence · doi:10.1016/0376-8716(94)90080-9 · Cited by 64 · see citing works
Following Up Nonrespondents to an Online Weight Management Intervention: Randomized Trial Comparing Mail versus Telephone
Couper, M. P., Peytchev, A., Strecher, V. J., Rothert, K., & Anderson, J. P. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.2.e16 · Cited by 63 · see citing works
Response and non‐response to postal questionnaire follow‐up in a clinical trial – a qualitative study of the patient’s perspective
Nakash, R. A., Hutton, J. L., Lamb, S. E., Gates, S., & Fisher, J. D. (2007) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2007.00838.x · Cited by 52 · see citing works
The DYD-RCT protocol: an on-line randomised controlled trial of an interactive computer-based intervention compared with a standard information website to reduce alcohol consumption among hazardous drinkers
Murray, E., McCambridge, J., Khadjesari, Z., White, I. R., Thompson, S. G., Godfrey, C., Linke, S., & Wallace, P. (2007) · BMC Public Health · doi:10.1186/1471-2458-7-306 · Cited by 38 · see citing works
Self-citation
A Systematic Review Examining the Literature on Attitudes of Patients with Advanced Cancer Toward Research
Todd, A., Laird, B. J. A., Boyle, D., et al. (2009) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2008.07.009 · Cited by 38 · see citing works
Test–retest reliability of an online measure of past week alcohol consumption (the TOT-AL), and comparison with face-to-face interview
Khadjesari, Z., Murray, E., Kalaitzaki, E., White, I. R., McCambridge, J., Godfrey, C., & Wallace, P. (2008) · Addictive Behaviors · doi:10.1016/j.addbeh.2008.11.010 · Cited by 35 · see citing works
Self-citation

Flagged first, then by citations

Research works printed but not linked: 2

[8]Singer E Van Hoewyk J Gebler N Raghunathan T McGonagle K The Effect of Incentives on Response Rates in Interviewer-Mediated Surveys Journal of Official Statistics -Stockholm 1999 15 2 217 30
[9]Simmons E Wilmot A Incentive Payments on Social Surveys: a Literature Review Social Survey Methodology Bulletin 2004 53 1 11

Sorted by citations

Conceptualising engagement with digital behaviour change interventions: A systematic review using principles from critical interpretive synthesis
Perski, O., Blandford, A., West, R., & Michie, S. (2017) · Translational Behavioral Medicine · doi:10.1007/s13142-016-0453-1 · Cited by 1,193 · see citing works
Behavioral Intervention Technologies: Evidence review and recommendations for future research in mental health
Mohr, D. C., Burns, M. N., Schueller, S. M., Clarke, G., & Klinkman, M. (2013) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2013.03.008 · Cited by 693 · see citing works
“) [24,69]; adding human support and adherence promotion via phone, e-mail or other media [4]; using incentives [70,71] and making resources more engaging through gaming elements [72].”
Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis
Brueton, V. C., Tierney, J. F., Stenning, S., Meredith, S., Harding, S., Nazareth, I., & Rait, G. (2014) · BMJ Open · doi:10.1136/bmjopen-2013-003821 · Cited by 253 · see citing works
Gamification and Adherence to Web-Based Mental Health Interventions: A Systematic Review
Brown, M., O'Neill, N., van Woerden, H., Eslambolchilar, P., Jones, M., & John, A. (2016) · JMIR Mental Health · doi:10.2196/mental.5710 · Cited by 178 · see citing works
Effectiveness of a self-guided web-based cannabis treatment program: randomized controlled trial
Rooke, S., Copeland, J., Norberg, M., Hine, D., & McCambridge, J. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2256 · Cited by 141 · see citing works
Self-citation
Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study
Perski, O., Crane, D., Beard, E., & Brown, J. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619880676 · Cited by 135 · see citing works
Web intervention for OEF/OIF veterans with problem drinking and PTSD symptoms: a randomized clinical trial
Brief, D. J., Rubin, A., Keane, T. M., Enggasser, J. L., Roy, M., Helmuth, E., Hermos, J., Lachowicz, M., Rybin, D., & Rosenbloom, D. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0033697 · Cited by 125 · see citing works
Optimizing engagement with <scp>I</scp> nternet‐based health behaviour change interventions: Comparison of self‐assessment with and without tailored feedback using a mixed methods approach
Morrison, L., Moss‐Morris, R., Michie, S., & Yardley, L. (2014) · British Journal of Health Psychology · doi:10.1111/bjhp.12083 · Cited by 110 · see citing works
Effectiveness of a Web-Based Tailored Interactive Health Communication Application for Patients With Type 2 Diabetes or Chronic Low Back Pain: Randomized Controlled Trial
Weymann, N., Dirmaier, J., von Wolff, A., Kriston, L., & Härter, M. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3904 · Cited by 73 · see citing works
Participating in Online Mental Health Interventions: Who Is Most Likely to Sign Up and Why?
Crisp, D. A., & Griffiths, K. M. (2014) · Depression Research and Treatment · doi:10.1155/2014/790457 · Cited by 76 · see citing works
Puzzling With Online Games (BAM-COG): Reliability, Validity, and Feasibility of an Online Self-Monitor for Cognitive Performance in Aging Adults
Aalbers, T., Baars, M. A. E., Olde Rikkert, M. G. M., & Kessels, R. P. C. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2860 · Cited by 59 · see citing works
Recruitment, adherence and attrition challenges in internet-based indicated prevention programs for eating disorders: lessons learned from a randomised controlled trial of ProYouth OZ
Ali, K., Fassnacht, D. B., Farrer, L. M., Rieger, E., Moessner, M., Bauer, S., & Griffiths, K. M. (2022) · Journal of Eating Disorders · doi:10.1186/s40337-021-00520-7 · Cited by 44 · see citing works
A Web-Based Intervention for Relatives of People Experiencing Psychosis or Bipolar Disorder: Design Study Using a User-Centered Approach
Honary, M., Fisher, N. R., McNaney, R., & Lobban, F. (2018) · JMIR Mental Health · doi:10.2196/11473 · Cited by 28 · see citing works
Which Bundles of Features in a Web-Based Personally Controlled Health Management System Are Associated With Consumer Help-Seeking Behaviors for Physical and Emotional Well-Being?
Lau, A. Y., Proudfoot, J., Andrews, A., Liaw, S.-T., Crimmins, J., Arguel, A., & Coiera, E. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2414 · Cited by 28 · see citing works
Smoking cessation avatar-led Acceptance and Commitment Therapy digital intervention: feasibility and acceptability in young adults
Karekla, M., & Savvides, S. N. (2021) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz128 · Cited by 25 · see citing works
Protocol for an online randomised controlled trial to evaluate the clinical and cost-effectiveness of a peer-supported self-management intervention for relatives of people with psychosis or bipolar disorder: relatives education and coping toolkit (REACT)
Lobban, F., Robinson, H., Appelbe, D., Barraclough, J., Bedson, E., Collinge, L., Dodd, S., Flowers, S., Honary, M., Johnson, S., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-016965 · Cited by 18 · see citing works
Self-citation

Authors: Khadjesari, Z., Murray, E., Kalaitzaki, E., White, I. R., McCambridge, J., Thompson, S. G., Wallace, P., & Godfrey, C.
Affiliations: University College London (GB)
Medical Research Council (GB)
MRC Biostatistics Unit (GB)
London School of Hygiene & Tropical Medicine (GB)
University of York (GB)
Corresponding author: Khadjesari, Zarnie
ORCID: Khadjesari, Zarnie
Murray, Elizabeth
White, Ian R
McCambridge, Jim
Thompson, Simon G
Wallace, Paul
Godfrey, Christine
Venue: Journal of Medical Internet Research
Publisher: JMIR Publications Inc.
Published: 2011-03-02
Volume: 13
Issue: 1
Pages: e26
ISSN: 1438-8871 (ISSN-L)
DOI: 10.2196/jmir.1523
PMC: PMC3221348
OpenAlex: W2069193924
Cited by: 114
References: 29
Citations by year: 2026: 3 · 2025: 7 · 2024: 7 · 2023: 5 · 2022: 6 · 2021: 11 · 2020: 9 · 2019: 5 · 2018: 4 · 2017: 10 · 2016: 10 · 2015: 6 · 2014: 9 · 2013: 14 · 2012: 6
FWCI: 33.10
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
Locations: Journal of Medical Internet Research (publishedVersion, cc-by)
Europe PMC (PubMed Central) (publishedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
Research Portal (King's College London) (publishedVersion, other-oa)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
UCL Discovery (University College London) (submittedVersion)
LSHTM Research Online (London School of Hygiene and Tropical Medicine) (submittedVersion)
UEA Digital Repository (University of East Anglia) (submittedVersion)
Funders: Cancer Research UK
National Institute for Health and Care Research
British Heart Foundation (RG/08/014/24067)
Diabetes UK
Medical Research Council
Economic and Social Research Council
World Cancer Research Fund
Topics: Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: attrition, follow-up rates, incentives, online trials, response rate, gift vouchers, alcohol consumption reduction, online intervention, participant retention, follow-up survey, prize draw
MeSH terms: Adult, Alcohol Drinking, Costs and Cost Analysis, Female, Follow-Up Studies, Humans, Male, Middle Aged, Motivation, Pilot Projects, Time Factors, Internet, Electronic Mail

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Impact and Costs of Incentives to Reduce Attrition in Online Trials: Two Randomized Controlled Trials

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks