Participant Recruitment and Retention in Remote eHealth Intervention Trials: Methods and Lessons Learned From a Large Randomized Controlled Trial of Two Web-Based Smoking Interventions
Open accessPublished version of a preprint
The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Despite having many advantages, online eHealth trials are not without challenges-notably, participant recruitment, and outcome data retention. Moreover, publications from these trials rarely provide detailed information on the methods used for recruitment and retention or discuss implications of the methods for future studies. OBJECTIVE: To address this need for empirical guidance regarding recruitment and outcome data retention planning, we aim to describe the methods and lessons learned from the recruitment and retention procedures used in a large randomized trial of 2 Web-based smoking cessation interventions. METHODS: To ensure a demographically and geographically diverse participant sample, we used the recruitment strategies (1) traditional, (2) Web-based, and (3) online survey panel methods and adaptively modified each in response to recruitment success. At baseline, participants indicated how they heard about the study and answered demographic questions. To maximize trial retention at each of the 3-, 6-, and 12-month assessment points, 4 survey modalities (first Web, followed by phone, mail, and postcard) were sequentially timed over a 30-day period. Participants received US $25 for submitting their responses, regardless of modality, and received an additional US $10 bonus for completing the Web survey within 24h of electronic notification. RESULTS: We randomized 2637 smokers in 16 months and achieved 88% retention at 12-months. Participants (79.26% female, 72.60% Caucasian) were recruited from all 50 states. The majority of participants were recruited through Facebook (49.43%), followed by the survey panel (20.85%), free internet sources (14.54%), traditional media (11.34%), and Google ads (3.84%). Descriptively, participant demographics varied by recruitment source. Of the completed follow-up surveys, most were completed by Web (92%). Retention rates did not vary by recruitment source. CONCLUSIONS: Continuous monitoring and refinement of multiple recruitment methods, particularly of online advertising campaigns, allowed us to maximize the effectiveness of recruitment strategies in recruiting a large, diverse sample of smokers. Likewise, offering multiple follow-up survey modalities in sequential order along with time-dependent bonus incentives enabled us to obtain outcome data from a very high level of enrolled participants for the duration of the trial protocol. These strategies may be similarly useful in other trials. TRIAL REGISTRATION: ClinicalTrials.gov NCT01812278; https://clinicaltrials.gov/ct2/show/NCT01812278 (Archived by WebCite at http://www.webcitation.org/71gy5GLvO).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“An established protocol that has yielded high outcome data retention rates (85-90%) in our previous trials,60 and proven feasible in the Quit2Heal pilot trial28 is currently being implemented.”
From Efficacy of smartphone applications to help cancer patients quit smoking: Protocol of the Quit2Heal randomized controlled trial (2023) · cites this paper as [60]
The cited source says what the claim attributes to it.
In the citing paper
“we are employing social media recruitment strategies that yielded nationwide recruitment in previous digital smoking cessation trials”
From Efficacy of smartphone applications to help cancer patients quit smoking: Protocol of the Quit2Heal randomized controlled trial (2023) · cites this paper as [60]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
The CES-D Scale: A Self-Report Depression Scale for Research in the General Population
Radloff, L. S. (1977) · Applied Psychological Measurement · doi:10.1177/014662167700100306 · Cited by 53,267
A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.10.1092 · Cited by 31,765
Morbidity and Mortality Weekly Report
(2010) · Wilderness & Environmental Medicine · doi:10.1016/j.wem.2010.09.015 · Cited by 2,950
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524 · cited 2×
“Another major methodological challenge faced by online eHealth trials is keeping participants engaged in the study protocol and achieving adequate rates of outcome data retention [2,9,16-19].”
BackgroundKontos, E., Blake, K. D., Chou, W.-Y. S., & Prestin, A. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3117 · Cited by 917 · cited 1×
“This is not surprising as women are more likely to utilize eHealth programs [59].”
BackgroundDillman, D. A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., & Messer, B. L. (2009) · Social Science Research · doi:10.1016/j.ssresearch.2008.03.007 · Cited by 850
The Use of Facebook in Recruiting Participants for Health Research Purposes: A Systematic Review
Whitaker, C., Stevelink, S., & Fear, N. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7071 · Cited by 724
How much loss to follow-up is acceptable in long-term randomised trials and prospective studies?
Fewtrell, M. S., Kennedy, K., Singhal, A., Martin, R. M., Ness, A., Hadders-Algra, M., Koletzko, B., & Lucas, A. (2008) · Archives of Disease in Childhood · doi:10.1136/adc.2007.127316 · Cited by 601
Dropout from Internet‐based treatment for psychological disorders
Melville, K. M., Casey, L. M., & Kavanagh, D. J. (2010) · British Journal of Clinical Psychology · doi:10.1348/014466509x472138 · Cited by 597 · cited 1×
“Another major methodological challenge faced by online eHealth trials is keeping participants engaged in the study protocol and achieving adequate rates of outcome data retention [2,9,16-19].”
Background
Research works printed but not linked: 1
- Jahangiry, L, 2014, Iran J Public Health, 43, 1248
Cited by
Efficacy of Smartphone Applications for Smoking Cessation: A Randomized Clinical Trial
Bricker, J. B., Watson, N. L., Mull, K. E., Sullivan, B. M., & Heffner, J. L. (2020) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2020.4055 · Cited by 207
Self-citationSocial Media Use for Research Participant Recruitment: Integrative Literature Review
Darko, E. M., Kleib, M., & Olson, J. (2022) · Journal of Medical Internet Research · doi:10.2196/38015 · Cited by 202
Online Patient Recruitment in Clinical Trials: Systematic Review and Meta-Analysis
Brøgger-Mikkelsen, M., Ali, Z., Zibert, J. R., Andersen, A. D., & Thomsen, S. F. (2020) · Journal of Medical Internet Research · doi:10.2196/22179 · Cited by 165
“As pointed out previously, a large proportion of the studies that compared online and offline methods utilized online recruitment strategies only after realizing that offline strategies did not provide enough participants [6,7,28,36,46,62].”
InfluentialCohen, S. A., & Greaney, M. L. (2022) · Current Epidemiology Reports · doi:10.1007/s40471-022-00313-9 · Cited by 198
The Role of Social Media in Enhancing Clinical Trial Recruitment: Scoping Review
Darmawan, I., Bakker, C., Brockman, T. A., Patten, C. A., & Eder, M. (2020) · Journal of Medical Internet Research · doi:10.2196/22810 · Cited by 164
“1299/2637 (49) — Watson et al (2018) [48]”
BackgroundInfluentialGoodson, N., Wicks, P. J., Morgan, J., Hashem, L., Callinan, S., & Reites, J. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00603-y · Cited by 164
Perski, O., Crane, D., Beard, E., & Brown, J. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619880676 · Cited by 135
Using digital technologies in clinical trials: Current and future applications
Rosa, C., Marsch, L. A., Winstanley, E. L., Brunner, M., & Campbell, A. N. C. (2020) · Contemporary Clinical Trials · doi:10.1016/j.cct.2020.106219 · Cited by 130
“Since then, researchers are increasingly using social media in clinical trials to improve outreach, and education (Watson, et al 2018, Poblete & Nieto, 2020; Maggio et al., 2019) as well as delivery of interventions (Park, 2019 & 2020, Radovic et al. (2017).”
BackgroundSocial media recruitment for mental health research: A systematic review
Sanchez, C., Grzenda, A. L., Varias, A. D., Widge, A. S., Carpenter, L. L., McDonald, W. M., Nemeroff, C. B., Kalin, N. H., Martin, G., Tohen, M. F., et al. (2020) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2020.152197 · Cited by 142
Bots and nots: safeguarding online survey research with underrepresented and diverse populations
Bybee, S., Cloyes, K. G., Baucom, B., Supiano, K. P., Mooney, K. H., & Ellington, L. (2021) · Psychology and Sexuality · doi:10.1080/19419899.2021.1936617 · Cited by 104
Heim, E., Ramia, J. A., Hana, R. A., Burchert, S., Carswell, K., Cornelisz, I., Cuijpers, P., El Chammay, R., Noun, P., van Klaveren, C., et al. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100380 · Cited by 61
Daniore, P., Nittas, V., & von Wyl, V. (2022) · Journal of Medical Internet Research · doi:10.2196/39910 · Cited by 68
Bricker, J. B., Watson, N. L., Heffner, J. L., Sullivan, B., Mull, K., Kwon, D., Westmaas, J. L., & Ostroff, J. (2020) · JMIR Formative Research · doi:10.2196/16652 · Cited by 41
“The procedures for follow-up data collection were modeled after the procedures that have been successful in our previous trials at maximizing data retention [52,61,62].”
MethodResultSelf-citationHall, C. L., Sanderson, C., Brown, B. J., Andrén, P., Bennett, S. D., Chamberlain, L. R., Davies, E. B., Khan, K., Kouzoupi, N., Mataix‐Cols, D., et al. (2020) · Trials · doi:10.1186/s13063-020-04902-1 · Cited by 37
“The following inclusion criteria were developed to ensure we identified all appropriate comparator studies: (1) recruited participants with a mental health/ behavioural condition, as classified by ICD-10 [33]; (2) used a RCT design (feasibility and pilot RCTs were included); (3) reported a psychological or behavioural…”
BackgroundInfluentialRecommendations to promote equity, diversity and inclusion in decentralized clinical trials
Aiyegbusi, O. L., Rivera, S. C., Kamudoni, P., Anderson, N. E., Collis, P., Denniston, A. K., Harding, R., Hughes, S., Khunti, K., Kotecha, D., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03323-w · Cited by 41
“Sponsors, trial delivery teams and researchers need to be aware of issues relating to intersectionality of participant attributes and the possibility that success addressing one aspect of inclusion may result in meaningful biases in other areas 12,59 .”
InfluentialBricker, J. B., Sullivan, B. M., Mull, K. E., Santiago‐Torres, M., & Ferres, J. L. (2024) · JMIR mhealth and uhealth · doi:10.2196/57318 · Cited by 49
“To maximize outcome data completion, we followed our team’s successful protocol [87]: 4 sequential survey modalities (first web, followed by phone, mail, and postcard).”
Self-citationSantiago‐Torres, M., Mull, K. E., Sullivan, B. M., Kwon, D. M., Henderson, P. N., Nelson, L. A., Patten, C. A., & Bricker, J. B. (2021) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntab213 · Cited by 32
Self-citationGarnett, C. V., Oldham, M., Angus, C., Beard, E., Burton, R., Field, M., Greaves, F., Hickman, M., Kaner, E., Loebenberg, G., et al. (2020) · Addiction · doi:10.1111/add.15287 · Cited by 31
“The trial cost per participant and methods of follow-up contact are comparable with similar studies that achieved high retention rates [70,74].”
MethodResultSehrawat, O., Noseworthy, P. A., Siontis, K. C., Haddad, T. C., Halamka, J. D., & Liu, H. (2023) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2023.02.003 · Cited by 34
Côté, J., Rouleau, G., Ramirez-Garcìa, P., Auger, P., Thomas, R., & Leblanc, J. (2020) · JMIR Public Health and Surveillance · doi:10.2196/17733 · Cited by 28
“Despite their success, Watson et al [35] recognized that obtaining an adequate sample size, keeping participants engaged in the study, and achieving adequate rates of outcome data retention were extremely challenging tasks.”
InfluentialGraham, A. L., Papandonatos, G. D., Cha, S., Amato, M. S., Jacobs, M. A., Cohn, A. M., Abroms, L. C., & Whittaker, R. (2021) · Addiction · doi:10.1111/add.15677 · Cited by 28
“The first reminder included $2 to encourage response by engaging the reciprocity norm [41].”
BackgroundStephens, D., Peterson, R., Singer, M., Johnson, J., Rushing, S. C., & Kelley, A. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17249437 · Cited by 23
“Previous research indicates there are no standards for how to recruit or retain participants in mHealth intervention trials [21].”
BackgroundThe many faces of recruitment in a randomized controlled trial
Pinto, B. M., & DUNSIGER, S. I. (2021) · Contemporary Clinical Trials · doi:10.1016/j.cct.2021.106285 · Cited by 23
Santiago‐Torres, M., Mull, K. E., Sullivan, B. M., Kwon, D., Nollen, N. L., Zvolensky, M. J., & Bricker, J. B. (2021) · Addiction · doi:10.1111/add.15721 · Cited by 26
Self-citationVillanti, A. C., Vallencourt, C. P., West, J. C., Peasley‐Miklus, C., LePine, S. E., McCluskey, C., Klemperer, E. M., Priest, J. S., Logan, A., Patton, B., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/18446 · Cited by 24
“Similar to other web-based studies [28,38], Facebook and Instagram advertisements provided the greatest reach and lowest cost per survey start when considering parent, youth, and young adult screener surveys; these were followed by Google advertisements via display and Gmail.”
BackgroundResultLeveraging Digital Health to Improve the Cardiovascular Health of Women
Azizi, Z., Adedinsewo, D. A., Rodríguez, F., Lewey, J., Merchant, R. M., & Brewer, L. C. (2023) · Current Cardiovascular Risk Reports · doi:10.1007/s12170-023-00728-z · Cited by 26
“For instance, a randomized trial for smoking cessation by Watson et al. [74] compared traditional (non-digital) versus web-based survey panel methods for recruitment (11.35% traditional vs. 88.65% digital).”
BackgroundInfluentialOetzmann, C., White, K. M., Ivan, A., Julie, J., Leightley, D., Lavelle, G., Lamers, F., Siddi, S., Annas, P., Arranz, S., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00680-z · Cited by 22
Naughton, F., Hope, A., Brown, C., Grant, K., Barton, G., Notley, C. J., Mascolo, C., Coleman, T. J., Shepstone, L., Sutton, S. R., et al. (2023) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntad032 · Cited by 26
Byaruhanga, J., Tzelepis, F., Paul, C. L., Wiggers, J., Byrnes, E., & Lecathelinais, C. (2019) · Journal of Medical Internet Research · doi:10.2196/14911 · Cited by 20
“For instance, in a US study of a Web-based smoking intervention [49], the total advertisement cost per randomized participant was US $40.51 (£33.33 and €36.32) for F acebook, US $34.71 (£28.56 and €31.12) for Google, and US $20.30 (£16.7 and €18.2) for traditional sources but excluded the costs for…”
InfluentialDiego, G. A., González‐Roz, A., Loredo, V. M., Krotter, A., & Secades‐Villa, R. (2020) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2020.108259 · Cited by 19
Haddad, E. N., & Comstock, S. S. (2021) · Methods and Protocols · doi:10.3390/mps4030052 · Cited by 22
“Attrition may be reduced in the future by incorporating web-based questionnaires, which has been shown in a randomized intervention follow-up to be effective in garnering a high level of data retention (~89%) alongside time-dependent bonus incentives [61].”
BackgroundSimmons, L. A., Phipps, J. E., Whipps, M. D. M., Smith, P. M., Carbajal, K. A., Overstreet, C., McLaughlin, J., Lombaert, K. D., & Noonan, D. (2022) · Digital Health · doi:10.1177/20552076221129065 · Cited by 24
Social Media and Clinical Research in Dermatology
Geist, R., Militello, M., Albrecht, J., Presley, C. L., Anderson, J. B., Laughter, M. R., & Rundle, C. W. (2021) · Current Dermatology Reports · doi:10.1007/s13671-021-00350-5 · Cited by 19
Baggett, K. M., Davis, B., Sheeber, L. B., Ammerman, R. T., Mosley, E. A., Miller, K., & Feil, E. G. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17238978 · Cited by 15
“Another study compared non-paid and paid recruitment approaches to a mobile health smoking cessation intervention and found that online paid advertising and survey panel approaches were best for increasing racial/ethnic diversity in their sample to reach at minimum 25% [34].”
BackgroundNoonan, D., Silva, S. G., Fish, L. J., Peter, K., Conley, C., Simmons, L. A., Severson, H. H., & Pollak, K. I. (2020) · Addictive Behaviors · doi:10.1016/j.addbeh.2020.106434 · Cited by 16
Spahrkäs, S. S., Looijmans, A., Sanderman, R., & Hagedoorn, M. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100362 · Cited by 19
Increasing Rigor in Online Health Surveys Through the Reduction of Fraudulent Data
Ng, W. Z., Sundarimaa, E., Liu, J. C., Tucker, J. D., & Tan, R. K. J. (2025) · Journal of Medical Internet Research · doi:10.2196/68092 · Cited by 24
“Furthermore, researchers can easily reach underrepresented populations, as well as create samples that ignore geographic boundaries [9,10].”
Notenbomer, A., Roelen, C. A. M., Groothoff, J. W., van Rhenen, W., & Bültmann, U. (2018) · Journal of Medical Internet Research · doi:10.2196/10821 · Cited by 16
“Several EHI studies found that recruitment using social media such as Facebook is more successful than more traditional recruitment strategies such as advertising [53,54].”
BackgroundReuter, K., Liu, C., Le, N., Angyan, P., & Finley, J. M. (2021) · Journal of Medical Internet Research · doi:10.2196/28923 · Cited by 20
Fletcher, S. L., Clarke, J., Sanatkar, S., Baldwin, P. A., Gunn, J. M., Zwar, N. A., Campbell, L. V., Wilhelm, K. A., Harris, M., Lapsley, H., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/12793 · Cited by 16
“It is worth noting that much of our understanding of this relationship relates to cohort studies rather than RCTs and to studies of smoking cessation more so than other health behaviors [30,31].”
Blakeslee, J. E., Kothari, B. H., & Miller, R. A. (2023) · Children and Youth Services Review · doi:10.1016/j.childyouth.2022.106753 · Cited by 12
“, 2012) and with remote delivery as well (Hanna et al., 2014; Murray et al., 2009; Watson et al., 2018).”
BackgroundAbi Ramia, J., Abi Hana, R., Noun, P., Cuijpers, P., Carswell, K., van't Hof, E., Heim, E., Zoghbi, E., Sijbrandij, M., & El Chammay, R. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1293187 · Cited by 15
Bricker, J. B., Sullivan, B. M., Mull, K. E., Lavista-Ferres, J., & Santiago‐Torres, M. (2024) · Contemporary Clinical Trials · doi:10.1016/j.cct.2024.107727 · Cited by 14
Self-citationCavallo, D. N., Lim, R., Ishler, K. J., Pagano, M. E., Perovsek, R., Albert, E. V., Gonzalez, S. J. K., Trapl, E., & Flocke, S. A. (2019) · Journal of Medical Internet Research · doi:10.2196/12619 · Cited by 14
“73 US (Twitter) [12], and a cost per enrolled smoker for randomized controlled trials ranging from $41 to $62 US (Facebook) [25,26].”
BackgroundSantiago‐Torres, M., Mull, K. E., Sullivan, B. M., Kendzor, D. E., & Bricker, J. B. (2021) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2021.109258 · Cited by 16
Self-citationOnline surveys: lessons learned in detecting and protecting against insincerity and bots
Thompson, A. D., & Utz, R. (2024) · Quality & Quantity · doi:10.1007/s11135-024-01973-z · Cited by 17
“Furthermore, recruiting survey respondents via postings on social media (e.g., Facebook) can be highly efficient and effective (Boas et al. 2020; Christensen et al. 2017; Schneider and Harknett 2022; Watson et al. 2018).”
Faro, J. M., Nagawa, C. S., Orvek, E., Smith, B., Blok, A. C., Houston, T. K., Kamberi, A., Allison, J. J., Person, S. D., & Sadasivam, R. S. (2021) · Contemporary Clinical Trials · doi:10.1016/j.cct.2021.106314 · Cited by 15
“Despite the importance, only a few studies have evaluated the success of recruitment strategies in digital smoking cessation trials.(3)”
BackgroundInfluentialGarbett, K. M., Craddock, N., Haywood, S., Nasution, K., White, P., Saraswati, L. A., Medise, B. E., Effect, G., Galactic, P., Diedrichs, P. C., et al. (2021) · JMIR Research Protocols · doi:10.2196/33596 · Cited by 12
Heffner, J. L., Mull, K. E., Watson, N. L., McClure, J. B., & Bricker, J. B. (2019) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntz112 · Cited by 12
Self-citationMcCarthy, E., & Mazza, D. (2019) · Journal of Medical Internet Research · doi:10.2196/15869 · Cited by 12
“Facebook has also been successfully used to recruit participants for intervention studies on depression [7], posttraumatic stress disorder [8], physical activity [9], and smoking [10].”
BackgroundBricker, J. B., Mull, K. E., Sullivan, B. M., & Forman, E. M. (2021) · Translational Behavioral Medicine · doi:10.1093/tbm/ibab012 · Cited by 11
Self-citationSridharan, V., Shoda, Y., Heffner, J. L., & Bricker, J. B. (2019) · JMIR mhealth and uhealth · doi:10.2196/14602 · Cited by 16
“A minimum inclusion level for men was added to recruitment criteria because studies using similar recruitment methods tend to overrecruit women [24].”
BackgroundResultSelf-citationRobinson, H., Appelbe, D., Dodd, S. R., Flowers, S., Johnson, S., Jones, S. H., Mateus, C., Mezes, B., Murray, E., Rainford, N., et al. (2020) · JMIR Mental Health · doi:10.2196/15878 · Cited by 13
“Some studies have found that older participants are more likely to be recruited through web-based strategies [19,40], whereas others have found that younger people are more likely to be recruited on the web [11,39].”
BackgroundResultInfluentialPérez-Muñoz, A., Horn, T. L., Graber, J., Chowdhury, S. M. R., Bursac, Z., & Krukowski, R. A. (2022) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2022.101037 · Cited by 10
“[8] explored traditional, web-based, and online survey panel recruitment methods.”
BackgroundHorvath, A., Haller, R., Feldbacher, N., Habisch, H., Žukauskaitė, K., Madl, T., & Stadlbauer, V. (2024) · Nutrients · doi:10.3390/nu16223970 · Cited by 9
MacDonnell, K., Cowen, E. A., Cunningham, D. J., Ritterband, L. M., & Ingersoll, K. S. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100240 · Cited by 10
Recruitment of Participants for a 3D Virtual Supermarket: Cross-sectional Observational Study
Hoenink, J. C., Mackenbach, J. D., van der Laan, L. N., Lakerveld, J., Waterlander, W. E., & Beulens, J. W. J. (2021) · JMIR Formative Research · doi:10.2196/19234 · Cited by 10
“Despite the advantages of 3D virtual supermarkets, it may be more difficult to recruit participants for online studies compared to interventions in real-world settings, as participants need to be adept at using technology and need to have access to a smartphone or computer with internet access [12,13].”
BackgroundWeisblum, M., Trussell, E., Schwinn, T. M., Pacheco, A. R., & Nurkin, P. (2023) · JMIR Pediatrics and Parenting · doi:10.2196/47984 · Cited by 8
Dolsen, E. A., Nishimi, K. M., LeWinn, K. Z., Byers, A. L., Tripp, P., Woodward, E., Khan, A. J., Marx, B. P., Borsari, B. E., Jiha, A., et al. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.10.009 · Cited by 11
“Craigslist and Reddit are online communities that have been previously utilized to recruit participants for online survey research (Watson et al., 2018).”
Recruiting Young People for Digital Mental Health Research: Lessons From an AI-Driven Adaptive Trial
Zheng, W. Y., Shvetcov, A., Slade, A., Jenkins, Z., Hoon, L., Whitton, A. E., Logothetis, R., Ravindra, S., Kurniawan, S., Gupta, S., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/60413 · Cited by 9
“In addition, marketing collaterals such as images, headlines, and copy can be changed in order to find the most effective study advertisement [7].”
InfluentialStrategies for recruiting participants underrepresented in clinical research: A scoping review
Helm, A. F., Kaufman, M., Glassy, N., Rodríguez, C. P., Shaffer, P. M., Nelson, B. B., & Smelson, D. A. (2025) · Social Science & Medicine · doi:10.1016/j.socscimed.2025.118603 · Cited by 8
Cardenas‐Rojas, A., Pacheco‐Barrios, K., Castelo-Branco, L. E. C., Giannoni‐Luza, S., Balbuena-Pareja, A., Luna‐Cuadros, M. A., Felippe, L. V., Uygur-Kucukseymen, E., Gonzalez‐Mego, P., Gunduz, M. E., et al. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e07475 · Cited by 7
Boudreau, F., Dagenais, G. R., de Vries, H., Walthouwer, M. J. L., Côté, J., Turbide, G., Bourlaud, A.-S., & Poirier, P. (2020) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2020.1850287 · Cited by 7
“It is possible that a more optimal recruitment and retention strategies would have resulted in a larger sample size at 3 month follow up (Watson, Mull, He ff ner, McClure, & Bricker, 2018).”
Hanson, K. L., Marshall, G., Graham, M., Villarreal, D. L., Volpe, L. C., & Seguin-Fowler, R. (2024) · Methods and Protocols · doi:10.3390/mps7060093 · Cited by 7
Predicting accrual success for better clinical trial resource allocation
Ma, S., Wang, Y., Wagner, J. E., Johnson, S., Pakhomov, S., & Aliferis, C. (2025) · Scientific Reports · doi:10.1038/s41598-025-88400-x · Cited by 7
Naughton, F., Hope, A., Brown, C., Grant, K., Notley, C. J., Colles, A., West, C., Mascolo, C., Coleman, T. J., Barton, G., et al. (2024) · Public Health Research · doi:10.3310/kqyt5412 · Cited by 8
Cook, S. K., Jerome, R. N., Dunagan, J., Kennedy, N., Edwards, T. L., Minnix, J. A., Witmer, L., Ferguson, J. A., Cinciripini, P. M., Wilkins, C. H., et al. (2021) · Contemporary Clinical Trials · doi:10.1016/j.cct.2021.106461 · Cited by 7
Oldham, M., Dinu, L. M., Loebenberg, G., Field, M., Hickman, M., Michie, S., Brown, J., & Garnett, C. V. (2023) · JMIR Formative Research · doi:10.2196/51839 · Cited by 9
“The multistage follow-up strategy and stepped approach to incentives (eg, an additional £12 if completed within 24 hours at 6 months) undertaken throughout the iDEAS trial were also informed by previous research [11].”
BackgroundMethodResultInfluentialWatson, N. L., Heffner, J. L., Mull, K. E., McClure, J. B., & Bricker, J. B. (2019) · Journal of Medical Internet Research · doi:10.2196/13500 · Cited by 6
“Further details on the recruitment and retention methods can be found elsewhere [22].”
Self-citationOetzmann, C., White, K. M., Ivan, A., Julie, J., Leightley, D., Lavelle, G., Lamers, F., Siddi, S., Annas, P., Garcia, S. A., et al. (2021) · doi:10.31219/osf.io/ns7dc · Cited by 5
Outcomes of a Comprehensive Mobile Vaping Cessation Program in Adults Who Vape Daily: Cohort Study
Marler, J. D., Fujii, C. A., Utley, M. T., Balbierz, D. J., Galanko, J. A., & Utley, D. S. (2024) · JMIR Formative Research · doi:10.2196/57376 · Cited by 8
“…of participants identifying as White individuals was similar to goals implemented in other studies assessing digital interventions in tobacco users [20,21] and in acknowledgment that individuals identifying as belonging to racial and ethnic minority groups are underrepresented in research studies…”
ResultNelson, L. A., Williamson, S., LeStourgeon, L. M., & Mayberry, L. S. (2021) · Contemporary Clinical Trials · doi:10.1016/j.cct.2021.106388 · Cited by 5
Fakolade, A. O., Awadia, Z., Cardwell, K. L., McKenna, O. J., Venasse, M., Hume, T. A., Ludgate, J., Freedman, M. S., Finlayson, M. L., Latimer‐Cheung, A. E., et al. (2023) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2023.101222 · Cited by 5
“Such panels have been successfully used in previous behavioural intervention research [54].”
MethodImplementing online interventions in ICare: A biostatistical perspective
Görlich, D., & Faldum, A. (2019) · Internet Interventions · doi:10.1016/j.invent.2018.12.004 · Cited by 5
“A main source of missing data in online interventions is attrition (cf. Eysenbach, 2005; Watson et al., 2018).”
BackgroundWatson, N. L., Mull, K. E., & Bricker, J. B. (2020) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2020.108394 · Cited by 9
“Procedures A full description of the study methodology can be found elsewhere (Bricker et al., 2018; Watson et al., 2018).”
MethodSelf-citationBricker, J. B., Mull, K. E., Sullivan, B. M., Forman, E. M., Lillis, J., McTiernan, A., & Santiago‐Torres, M. (2023) · Contemporary Clinical Trials · doi:10.1016/j.cct.2023.107091 · Cited by 5
Self-citationBrock, D. P., Ritterband, L. M., You, W., Reid, A., Porter, K., Markwalter, T. H., & Zoellner, J. (2025) · The Journal of Rural Health · doi:10.1111/jrh.70042 · Cited by 6
McDonald, H., Valmaggia, L., Antonova, E., & Chadwick, P. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1380077 · Cited by 5
Collette, K., Feeley, N., Galipeau, R., & Lebel, V. (2023) · Early Human Development · doi:10.1016/j.earlhumdev.2023.105717 · Cited by 7
Maiwald, P., Bischoff, M., Lindinger, P., Tinsel, I., Sehlbrede, M., Fichtner, U. A., Metzner, G., Schlett, C., & Farin‐Glattacker, E. (2023) · Journal of Medical Internet Research · doi:10.2196/47463 · Cited by 5
“…the participants; the considerable dropout rate (albeit this is common for internet-based interventions [16]; the retention rate could probably have been increased with higher incentives [43]); and the possibility of cointervention bias, which we were unable to prevent in the context of our study.”
Kolls, B. J., Muir, K. W., Savitz, S. I., Wechsler, L. R., Pilitsis, J. G., Rahimi, S. Y., Beckman, R. L., Holmes, V., Chen, P. R., Albers, D. S., et al. (2024) · Trials · doi:10.1186/s13063-024-07988-z · Cited by 4
“Behavioral intervention and survey-based studies were early adopters, leveraging web-based recruitment with social media advertising, and were simple and efficient [13–15].”
MethodResearch participation effects and where to find them: a systematic review of studies on alcohol
Gunnarsson, K. U., Collier, E. S., & Bendtsen, M. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111668 · Cited by 4
Sibley, A. L., Noar, S. M., Muessig, K. E., Gottfredson, N. C., Paquette, C. E., Spears, A. G., Miller, W. C., & Go, V. F.-L. (2024) · JMIR Research Protocols · doi:10.2196/59224 · Cited by 3
Parbhoo-Ebrahim, N., & Fourie, I. (2020) · ISIC: The Information Behaviour Conference (ISIC) · doi:10.47989/irisic2026 · Cited by 3
Fennell, B. S., Jones, S. R., Sutton, S. K., Hoogland, C. E., Cottrell‐Daniels, C., Wetter, D. W., Shih, Y. T., Simmons, V. N., Stephens, Y. P., Vidrine, D. J., et al. (2024) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntae049 · Cited by 3
Pauls, A., Koppelin, F., & Zeeb, H. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1334180 · Cited by 5
Manne, S., Day, A. K., Coups, E., & Kashy, D. (2018) · Preventive medicine reports · doi:10.1016/j.pmedr.2018.10.010 · Cited by 3
“…a study information page, following up with email reminders, including information about the importance of sun protection in skin cancer risk reduction, and using multiple social media recruitment methods (Google, Instagram) may increase uptake (Treweek et al., 2018; Watson et al., 2018).”
Morales-Lara, A. C., Garzon-Siatoya, W. T., Campos, B. A. F., & Adedinsewo, D. A. (2023) · JACC Advances · doi:10.1016/j.jacadv.2023.100272 · Cited by 4
Bosma, R. L., Mustafa, N., Burke, E., Alsius, A., Bisson, E. J., Cooper, L., Salomons, T. V., Galica, J., Poulin, P. A., Wiggin, M., et al. (2024) · Pain Management Nursing · doi:10.1016/j.pmn.2024.10.020 · Cited by 2
Zwiers, L. C., Veen, D., Mitratza, M., Brakenhoff, T. B., Goodale, B. M., Klaver, P., Hage, K. Y., van Willigen, M., Downward, G. S., Lugtig, P. J., et al. (2025) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2025.100264 · Cited by 2
Lessons Learned Identifying and Controlling Fraudulent Participation in Online Randomized Trials
Siebers, R., Magane, K. M., Slayton, H., Karzhevsky, S., Palfai, T. P., Abrantes, A. M., Quintiliani, L. M., & Stein, M. D. (2025) · Journal of Medical Internet Research · doi:10.2196/77512 · Cited by 1
“Using IP address checks to track when a single actor tries to submit multiple forms to enter a research study may be helpful [6-8].”
Improving participant representation in the era of digital clinical studies
Ramos, E., Baca-Motes, K., Pandit, J. A., & Ajayi, T. A. (2022) · Trends in Molecular Medicine · doi:10.1016/j.molmed.2022.07.004 · Cited by 2
High, J., Grant, K., Hope, A., Shepstone, L., West, C., Colles, A., & Naughton, F. (2024) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntae068 · Cited by 2
“Retention of participants can be more challenging when trials are conducted remotely or wholly online 5 and when there is minimal contact with participants due to non-interpersonal treatment delivery.”
BackgroundRodriguez, C. R., Campbell, M., Bonar, E. E., Goldstick, J. E., Walton, M. A., Lin, L. A., & Coughlin, L. N. (2025) · JMIR Research Protocols · doi:10.2196/69956 · Cited by 3
“…a range of options, including phone-or SMS-based contacts, social media, or web-based advertising, either driven by the research team or through contracting recruitment companies that manage online advertisement campaigns [3], remote recruitment has been shown to be reliable and effective [4-7].”
Reach and Recruitment of Microentrepreneurs: Lessons from a Finnish Health Promotion Intervention
Mäkiniemi, J.-P., Ahola, S., Korkiakangas, E. E., Kaksonen, T., Heikkilä-Tammi, K., Laitinen, J. H., Kekkonen, M., Muhos, M., Oksanen, T., B, H. O.-K. A., et al. (2022) · Nordic Journal of Working Life Studies · doi:10.18291/njwls.132982 · Cited by 2
“89 per participant) for the recruitment campaign (Watson et al. 2018).”
BackgroundMethodInfluentialLo, B., Shi, J., Wong, H. W., Abi-Jaoudé, A., Johnson, A., Hollenberg, E., Chaim, G., Cleverley, K., Henderson, J. L., LEVINSON, A. J., et al. (2021) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2021.02.010 · Cited by 2
Santiago‐Torres, M., Mull, K. E., Sullivan, B. M., Zvolensky, M. J., Kahler, C. W., & Bricker, J. B. (2022) · Addictive Behaviors · doi:10.1016/j.addbeh.2022.107377 · Cited by 2
Self-citationGarzon-Siatoya, W. T., Morales-Lara, A. C., & Adedinsewo, D. A. (2023) · Cardiovascular Innovations and Applications · doi:10.15212/cvia.2023.0024 · Cited by 6
Bricker, J. B., Westmaas, J. L., Ostroff, J. S., Mull, K. E., Sullivan, B. M., & Santiago-Torres, M. (2023) · Contemporary Clinical Trials · doi:10.1016/j.cct.2023.107180 · Cited by 2
Self-citationRecruitment, retention, and adherence of family caregivers: Lessons from a multisite trial
Díaz, L. J. R., Guedes, É. D. S., & da Cruz, D. D. A. L. M. (2023) · Investigación y Educación en Enfermería · doi:10.17533/udea.iee.v41n2e04 · Cited by 1
“(10) Although the threats against participant recruitment and retention are significant when evaluating a remote intervention, (11,12) studies that have testing interventions delivered exclusively by telephone for family caregivers do not detail the recruitment process (13) or the strategies used…”
Background
Metadata
- Authors
- Watson, N. L., Mull, K. E., Heffner, J. L., McClure, J. B., & Bricker, J. B.
- Affiliations
- Fred Hutch Cancer Center (US)
- Kaiser Permanente Washington Health Research Institute (US)
- University of Washington (US)
- Corresponding author
- Watson, Noreen L
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2018-08-24
- Volume
- 20
- Issue
- 8
- Pages
- e10351
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6128955
- OpenAlex
- W2886267432
- Cited by
- 118
- References
- 59
- Citations by year
- 2026: 5 · 2025: 13 · 2024: 21 · 2023: 14 · 2022: 13 · 2021: 21 · 2020: 18 · 2019: 11 · 2018: 2
- FWCI
- 8.92
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- National Institute on Drug Abuse (F32DA041100, R01DA038411)
- National Cancer Institute (R01CA192849, R01CA166646, P30CA006516)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Smoking Behavior and Cessation
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- participant recruitment, participant retention, digital health interventions, online advertising, Facebook recruitment
- MeSH terms
- Tobacco Smoking, Female, Humans, Male, Middle Aged, Smoking Cessation, Telemedicine, Patient Selection, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.


