Cited by 195Digital Mental Health InterventionsMobile Health and mHealth ApplicationsStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.

Methodological Challenges in Online Trials

Open access

Murray, Elizabeth, Khadjesari, Zarnie, White, Ian R, Kalaitzaki, Eleftheria, Godfrey, Christine, McCambridge, Jim, Thompson, Simon G, Wallace, Paul·2009·Journal of Medical Internet Research

DOI 10.2196/jmir.1052PMC PMC2762798

Open this paper in CoChat — free account Open PDF

Abstract

Health care and health care services are increasingly being delivered over the Internet. There is a strong argument that interventions delivered online should also be evaluated online to maximize the trial's external validity. Conducting a trial online can help reduce research costs and improve some aspects of internal validity. To date, there are relatively few trials of health interventions that have been conducted entirely online. In this paper we describe the major methodological issues that arise in trials (recruitment, randomization, fidelity of the intervention, retention, and data quality), consider how the online context affects these issues, and use our experience of one online trial evaluating an intervention to help hazardous drinkers drink less (DownYourDrink) to illustrate potential solutions. Further work is needed to develop online trial methodology.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.2196/jmir.1052

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Untitled work
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Clinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,918
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
The Law of Attrition
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524
Developing and Evaluating Complex Interventions: New Guidance
Craig, P., Dieppe, P. A., MacIntyre, S., Michie, S., Nazareth, I., & Petticrew, M. P. (2006) · UCL Discovery (University College London) · Cited by 1,878
Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: A meta-analysis
SPEK, V., CUIJPERS, P., NYKLÍČEK, I., RIPER, H., KEYZER, J., & POP, V. (2007) · Psychological Medicine · doi:10.1017/s0033291706008944 · Cited by 1,586
Designing and evaluating complex interventions to improve health care
Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H., Wilson, P., & Kinmonth, A. L. (2007) · BMJ · doi:10.1136/bmj.39108.379965.be · Cited by 1,331
Self-citation
Health related virtual communities and electronic support groups: systematic review of the effects of online peer to peer interactions
Eysenbach, G., Powell, J., Englesakis, M., Rizo, C., & Stern, A. (2004) · BMJ · doi:10.1136/bmj.328.7449.1166 · Cited by 1,268
Clinical trials: A practical approach
Daly, M. J. (1984) · Trends in Pharmacological Sciences · doi:10.1016/0165-6147(84)90515-7 · Cited by 1,057
The Effectiveness of Web-Based vs. Non-Web-Based Interventions: A Meta-Analysis of Behavioral Change Outcomes
Wantland, D. J., Portillo, C. J., Holzemer, W. L., Slaughter, R., & McGhee, E. M. (2004) · Journal of Medical Internet Research · doi:10.2196/jmir.6.4.e40 · Cited by 966 · cited 1×
“These range from simple health information sites to complex self-management programs incorporating interactive components such as decision support, social support, behavior change support [6-8], and computerized cognitive behavioral therapy [9,10].”
Background

Show all 48 references

Research works printed but not linked: 6

Fox, S, 2002, The online health care revolution: how the Web helps Americans take better care of themselves
Murray, E, 2005, Cochrane Database Syst Rev, CD004274
MRC Health Services and Public Health Research Board, 2000, A Framework for Development and Evaluation of RCTs for Complex Interventions to Improve Health, 1
Mapstone, J, 2007, Cochrane Database Syst Rev, MR000013
10.1001/jama.273.5.408 · DOI
Konstan, J, 2005, J Comput Mediated Commun, 10

Sorted by citations

Using the Internet to Promote Health Behavior Change: A Systematic Review and Meta-analysis of the Impact of Theoretical Basis, Use of Behavior Change Techniques, and Mode of Delivery on Efficacy
Webb, T. L., Joseph, J., Yardley, L., & Michie, S. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1376 · Cited by 2,517
Systematic Literature Review on the Spread of Health-related Misinformation on Social Media
Wang, Y., McKee, M., Torbica, A., & Stuckler, D. (2019) · Social Science & Medicine · doi:10.1016/j.socscimed.2019.112552 · Cited by 1,806
Evaluating Digital Health Interventions
Murray, E., Hekler, E. B., Andersson, G., Collins, L. M., Doherty, A., Hollis, C., Rivera, D. E., West, R., & Wyatt, J. C. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.008 · Cited by 847
Self-citation
Internet-based interventions for smoking cessation
Taylor, G. M. J., Dalili, M. N., Semwal, M., Čivljak, M., Sheikh, A., & Car, J. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007078.pub5 · Cited by 676
Got Bots? Practical Recommendations to Protect Online Survey Data from Bot Attacks
Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.16.5.p472 · Cited by 329
Strategies to improve retention in randomised trials
Brueton, V. C., Tierney, J., Stenning, S., Harding, S., Meredith, S., Nazareth, I., & Rait, G. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000032.pub2 · Cited by 279
Engagement and retention: measuring breadth and depth of participant use of an online intervention
Couper, M. P., Alexander, G. L., Zhang, N., Little, R. J., Maddy, N., Nowak, M. A., McClure, J. B., Calvi, J. J., Rolnick, S. J., Stopponi, M. A., et al. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1430 · Cited by 244
Internet-based interventions for smoking cessation
Civljak, M., Stead, L. F., Hartmann-Boyce, J., Sheikh, A., & Car, J. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007078.pub4 · Cited by 249
Social Cognitive Determinants of Nutrition and Physical Activity Among Web-Health Users Enrolling in an Online Intervention: The Influence of Social Support, Self-Efficacy, Outcome Expectations, and Self-Regulation
Anderson-Bill, E. S., Winett, R. A., & Wojcik, J. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1551 · Cited by 247
Using Smartphone Technology to Monitor Physical Activity in the 10,000 Steps Program: A Matched Case–Control Trial
Kirwan, M., Duncan, M. J., Vandelanotte, C., & Mummery, W. K. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1950 · Cited by 189
Interactive computer-based interventions for sexual health promotion
Bailey, J. V., Murray, E., Rait, G., Mercer, C. H., Morris, R. W., Peacock, R., Cassell, J., & Nazareth, I. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006483.pub2 · Cited by 183
Self-citation
Effectiveness of a Web-based Intervention Aimed at Healthy Dietary and Physical Activity Behavior: A Randomized Controlled Trial About Users and Usage
Kelders, S. M., Van Gemert-Pijnen, J. E. W. C., Werkman, A., Nijland, N., & Seydel, E. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1624 · Cited by 156
Conducting a fully mobile and randomised clinical trial for depression: access, engagement and expense
Anguera, J. A., Jordan, J. T., Castaneda, D., Gazzaley, A., & Areán, P. A. (2016) · BMJ Innovations · doi:10.1136/bmjinnov-2015-000098 · Cited by 172
“…with very little contact between the participant and research team, and our retention rates were higher than in the typical internet-based RCT. 34 Internet-based studies have shown that when there is more direct contact between the research team and participant through technologies such as…”
Result
Participant recruitment and retention in remote eHealth intervention trials: methods and lessons learned from a large randomized controlled trial of two web-based smoking interventions
Watson, N. L., Mull, K. E., Heffner, J. L., McClure, J. B., & Bricker, J. B. (2018) · Journal of Medical Internet Research · doi:10.2196/10351 · Cited by 118
Stages of use: consideration, initiation, utilization, and outcomes of an internet-mediated intervention
Chiu, T., & Eysenbach, G. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-73 · Cited by 142
Sexual health promotion for young people delivered via digital media: a scoping review
Bailey, J., Mann, S., Wayal, S., Hunter, R. M., Free, C., Abraham, C., & Murray, E. (2015) · Public Health Research · doi:10.3310/phr03130 · Cited by 144
Self-citation
Strategies to improve retention in randomised trials
Gillies, K., Kearney, A., Keenan, C., Treweek, S. P., Hudson, J., Brueton, V. C., Conway, T., Hunter, A., Murphy, L., Carr, P. J., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000032.pub3 · Cited by 142
Attrition Revisited: Adherence and Retention in a Web-Based Alcohol Trial
Murray, E., White, I. R., Varagunam, M., Godfrey, C., Khadjesari, Z., & McCambridge, J. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2336 · Cited by 134
Self-citation
Effects of eHealth-Based Multiple Health Behavior Change Interventions on Physical Activity, Healthy Diet, and Weight in People With Noncommunicable Diseases: Systematic Review and Meta-analysis
Duan, Y., Shang, B., Liang, W., Du, G., Yang, M., & Rhodes, R. E. (2021) · Journal of Medical Internet Research · doi:10.2196/23786 · Cited by 129
Systematic review and meta-analysis of internet interventions for smoking cessation among adults
Graham, A., Carpenter, K., Cha, S., Cole, S., Jacobs, M., Raskob, M., & Cole-Lewis, H. (2016) · Substance Abuse and Rehabilitation · doi:10.2147/sar.s101660 · Cited by 114
“Attrition bias: Attrition is a particular challenge in webbased studies.(49) We focused on two elements when rating attrition bias: reporting outcomes and differential attrition by study arm.”
Strategies to address participant misrepresentation for eligibility in Web‐based research
Kramer, J. M., Rubin, A., Coster, W. J., Helmuth, E., Hermos, J. A., Rosenbloom, D., Moed, R., Dooley, M., Kao, Y., Liljenquist, K. S., et al. (2014) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1415 · Cited by 118
An Internet-Based Intervention to Promote Mental Fitness for Mildly Depressed Adults: Randomized Controlled Trial
Bolier, L., Haverman, M., Kramer, J., Westerhof, G. J., Riper, H., Walburg, J. A., Boon, B., & Bohlmeijer, E. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2603 · Cited by 116
Impact and Costs of Incentives to Reduce Attrition in Online Trials: Two Randomized Controlled Trials
Khadjesari, Z., Murray, E., Kalaitzaki, E., White, I. R., McCambridge, J., Thompson, S. G., Wallace, P., & Godfrey, C. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1523 · Cited by 114
Self-citation
Using Facebook for health-related research study recruitment and program delivery
Pedersen, E. R., & Kurz, J. (2015) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2015.09.011 · Cited by 101
Internet-based randomized controlled trials: a systematic review
Mathieu, E., McGeechan, K., Barratt, A., & Herbert, R. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001175 · Cited by 98
Methods for Evaluating Respondent Attrition in Web-Based Surveys
Hochheimer, C. J., Sabo, R. T., Krist, A. H., Day, T., Cyrus, J. W. W., & Woolf, S. H. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.6342 · Cited by 94
Attrition in web-based treatment for problem drinkers
Postel, M. G., de Haan, H. A., ter Huurne, E. D., van der Palen, J., Becker, E. S., & de Jong, C. A. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1811 · Cited by 87
Using Facebook to Recruit Young Adult Veterans: Online Mental Health Research
Pedersen, E. R., Helmuth, E., Marshall, G. N., Schell, T. L., PunKay, M., & Kurz, J. (2015) · JMIR Research Protocols · doi:10.2196/resprot.3996 · Cited by 93
Using Information Technology and Social Networking for Recruitment of Research Participants: Experience From an Exploratory Study of Pediatric Klinefelter Syndrome
Close, S., Smaldone, A. M., Fennoy, I., Reame, N. K., & Grey, M. J. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2286 · Cited by 89
Impact of Length or Relevance of Questionnaires on Attrition in Online Trials: Randomized Controlled Trial
McCambridge, J., Kalaitzaki, E., White, I. R., Khadjesari, Z., Murray, E., Linke, S., Thompson, S. G., Godfrey, C., & Wallace, P. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1733 · Cited by 74
Self-citation
Allowing for missing outcome data and incomplete uptake of randomised interventions, with application to an Internet‐based alcohol trial
White, I. R., Kalaitzaki, E., & Thompson, S. G. (2011) · Statistics in Medicine · doi:10.1002/sim.4360 · Cited by 63
Self-citation
Review of Extracting Information From the Social Web for Health Personalization
Fernandez-Luque, L., Karlsen, R., & Bonander, J. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1432 · Cited by 61
Superusers in social networks for smoking cessation: Analysis of demographic characteristics and posting behavior from the canadian cancer society’s smokers’ helpline online and stopsmokingcenter.net
van Mierlo, T., Voci, S., Lee, S., Fournier, R., & Selby, P. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1854 · Cited by 58
“A particular strength is that both programs have relatively few barriers to enrollment in comparison with entry barriers that are typical of Internet-based clinical trials [39], as we collected data from WATI operating in their naturalistic Internet environments.”
Background
The use of emerging technologies in alcohol treatment.
Cunningham, J. A., Kypri, K., & McCambridge, J. (2011) · Europe PMC (PubMed Central) · Cited by 57
Self-citation
The Sexunzipped Trial: Optimizing the Design of Online Randomized Controlled Trials
Bailey, J., Pavlou, M., Copas, A. J., McCarthy, O., Carswell, K., Rait, G., Hart, G. J., Nazareth, I., Free, C., French, R., et al. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2668 · Cited by 58
Self-citation
Effectiveness of a Proactive Mail-Based Alcohol Internet Intervention for University Students: Dismantling the Assessment and Feedback Components in a Randomized Controlled Trial
Bendtsen, P., McCambridge, J., Bendtsen, M., Karlsson, N. E., & Nilsén, P. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2062 · Cited by 57
Self-citation
Evaluation of a Web-Based Intervention to Promote Hand Hygiene: Exploratory Randomized Controlled Trial
Yardley, L., Miller, S., Schlotz, W., & Little, P. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1963 · Cited by 55
Effectiveness of a Web-Based Screening and Fully Automated Brief Motivational Intervention for Adolescent Substance Use: A Randomized Controlled Trial
Arnaud, N., Baldus, C., Elgán, T. H., Paepe, N. D., Tønnesen, H., Csémy, L., & Thomasius, R. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.4643 · Cited by 50
Effectiveness of a web-based, individually tailored decision aid for depression or acute low back pain: A randomized controlled trial
Simón, D., Kriston, L., von Wolff, A., Buchholz, A., Vietor, C., Hecke, T., Loh, A., Zenker, M., Weiss, M., & Härter, M. C. (2011) · Patient Education and Counseling · doi:10.1016/j.pec.2011.10.009 · Cited by 50
Promoting healthy home environments and lifestyles in families with preschool children: HomeStyles, a randomized controlled trial
Byrd‐Bredbenner, C., Martin‐Biggers, J. T., Alleman, G. P., Worobey, J., Hongu, N. K., & Quick, V. (2017) · Contemporary Clinical Trials · doi:10.1016/j.cct.2017.10.012 · Cited by 50
Online Alcohol Assessment and Feedback for Hazardous and Harmful Drinkers: Findings From the AMADEUS-2 Randomized Controlled Trial of Routine Practice in Swedish Universities
Bendtsen, P., Bendtsen, M., Karlsson, N. E., White, I. R., & McCambridge, J. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4020 · Cited by 50
Self-citation
Recruitment into a guided internet based CBT (iCBT) intervention for depression: Lesson learnt from the failure of a prevalence recruitment strategy
Woodford, J., Farrand, P. A., Bessant, M., & Williams, C. M. (2011) · Contemporary Clinical Trials · doi:10.1016/j.cct.2011.04.013 · Cited by 48
Negotiating the ‘grey area between normal social drinking and being a smelly tramp’: a qualitative study of people searching for help online to reduce their drinking
Khadjesari, Z., Stevenson, F., Godfrey, C., & Murray, E. (2015) · Health Expectations · doi:10.1111/hex.12351 · Cited by 46
Self-citation
Predictors of Engagement, Response to Follow Up, and Extent of Alcohol Reduction in Users of a Smartphone App (Drink Less): Secondary Analysis of a Factorial Randomized Controlled Trial
Garnett, C. V., Perski, O., Tombor, I., West, R., Michie, S., & Brown, J. (2018) · JMIR mhealth and uhealth · doi:10.2196/11175 · Cited by 49
Impact of a U.S. antismoking national media campaign on beliefs, cognitions and quit intentions
Duke, J. C., Davis, K., Alexander, R. L., MacMonegle, A. J., Fraze, J. L., Rodes, R., & Beistle, D. M. (2015) · Health Education Research · doi:10.1093/her/cyv017 · Cited by 42
Effects of a Guided Internet-Delivered Self-Help Intervention for Adolescents With Chronic Pain
Voerman, J. S., Remerie, S. C., Westendorp, T., Timman, R., Busschbach, J. J. V., Passchier, J., & de Klerk, C. (2015) · Journal of Pain · doi:10.1016/j.jpain.2015.07.011 · Cited by 40
Methodological and Ethical Challenges in a Web-Based Randomized Controlled Trial of a Domestic Violence Intervention
Tarzia, L., Valpied, J., Koziol‐McLain, J., Glass, N. E., & Hegarty, K. L. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7039 · Cited by 40
Engagement and attrition in Internet smoking cessation interventions: insights from a cross-sectional survey of “one-hit-wonders”
Saul, J. E., Amato, M. S., Cha, S., & Graham, A. L. (2016) · Internet Interventions · doi:10.1016/j.invent.2016.07.001 · Cited by 39
Improving contraceptive use among Latina adolescents: A cluster-randomized controlled trial evaluating an mHealth application, Health-E You/Salud iTu
Tebb, K., Rodriguez, F. J., Pollack, L. M., Adams, S. H., Rico, R., Renteria, R., Trieu, S. L., Hwang, L. Y., Brindis, C. D., Ozer, E. M., et al. (2021) · Contraception · doi:10.1016/j.contraception.2021.03.004 · Cited by 39
Opportunities and challenges of delivering digital clinical trials: lessons learned from a randomised controlled trial of an online behavioural intervention for children and young people
Hall, C. L., Sanderson, C., Brown, B. J., Andrén, P., Bennett, S. D., Chamberlain, L. R., Davies, E. B., Khan, K., Kouzoupi, N., Mataix‐Cols, D., et al. (2020) · Trials · doi:10.1186/s13063-020-04902-1 · Cited by 36
Self-citation
A Pilot Randomized Controlled Trial of an Internet-Based Alcohol Intervention in a Workplace Setting
Brendryen, H., Johansen, A. B., Duckert, F., & Nesvåg, S. M. (2017) · International Journal of Behavioral Medicine · doi:10.1007/s12529-017-9665-0 · Cited by 37
Quality of life among hazardous and harmful drinkers: EQ-5D over a 1-year follow-up period
Essex, H. N., White, I. R., Khadjesari, Z., Linke, S., McCambridge, J., Murray, E., Parrott, S., & Godfrey, C. (2013) · Quality of Life Research · doi:10.1007/s11136-013-0521-7 · Cited by 34
Self-citation
Internet trials: participant experiences and perspectives
Mathieu, E., Barratt, A. L., Carter, S. M., & Jamtvedt, G. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-162 · Cited by 35
An Observational Study of Social and Emotional Support in Smoking Cessation Twitter Accounts: Content Analysis of Tweets
Rocheleau, M., Sadasivam, R. S., Baquis, K., Stahl, H., Kinney, R. L., Pagoto, S., & Houston, T. K. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3768 · Cited by 33
USABILITY OF ONLINE PSYCHOEDUCATION FOR SIBLINGS OF PEOPLE WITH PSYCHOSIS
Sin, J., Henderson, C., & Norman, I. J. (2014) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462314000488 · Cited by 33
GamblingLess: A Randomised Trial Comparing Guided and Unguided Internet-Based Gambling Interventions
Dowling, N. A., Merkouris, S., Rodda, S. N., Smith, D., Aarsman, S. R., Lavis, T., Lubman, D. I., Austin, D., Cunningham, J. A., Battersby, M., et al. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10112224 · Cited by 33
User Retention and Engagement With a Mobile App Intervention to Support Self-Management in Australians With Type 1 or Type 2 Diabetes (My Care Hub): Mixed Methods Study
Adu, M. D., Malabu, U. H., Malau‐Aduli, A. E. O., Drovandi, A. D., & Malau‐Aduli, B. S. (2020) · JMIR mhealth and uhealth · doi:10.2196/17802 · Cited by 32
A web-based self-management programme for people with type 2 diabetes: the HeLP-Diabetes research programme including RCT
Murray, E., Ross, J., Pal, K., Li, J., Dack, C., Stevenson, F. A., Sweeting, M. J., Parrott, S., Barnard, M., Yardley, L., et al. (2018) · Programme Grants for Applied Research · doi:10.3310/pgfar06050 · Cited by 33
Self-citation
Facilitating mental health help-seeking by young adults with a dedicated online program: a feasibility study of Link
Kauer, S. D., Buhagiar, K., Blake, V., Cotton, S., & Sanci, L. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-015303 · Cited by 30
The Sexunzipped Trial: Young People’s Views of Participating in an Online Randomized Controlled Trial
Nicholas, A., Bailey, J., Stevenson, F. A., & Murray, E. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2647 · Cited by 30
Self-citation
Overcoming Addictions, a Web-Based Application, and SMART Recovery, an Online and In-Person Mutual Help Group for Problem Drinkers, Part 2: Six-Month Outcomes of a Randomized Controlled Trial and Qualitative Feedback From Participants
Campbell, W. P., Hester, R. K., Lenberg, K. L., & Delaney, H. D. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5508 · Cited by 30
Understanding adherence to web-based interventions
Kelders, S. M. (2012) · doi:10.3990/1.9789036534178 · Cited by 29
eMoms: Electronically-mediated weight interventions for pregnant and postpartum women. Study design and baseline characteristics
Fernandez, I. D., Groth, S. W., Reschke, J. E., Graham, M. L., Strawderman, M. S., & Olson, C. M. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.04.013 · Cited by 30
Competitive Memory Training (COMET) for OCD: A Self-treatment Approach to Obsessions
Schneider, B. C., Wittekind, C. E., Talhof, A., Korrelboom, K., & Moritz, S. (2014) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2014.981758 · Cited by 29
A Web-Based Intervention for Relatives of People Experiencing Psychosis or Bipolar Disorder: Design Study Using a User-Centered Approach
Honary, M., Fisher, N. R., McNaney, R., & Lobban, F. (2018) · JMIR Mental Health · doi:10.2196/11473 · Cited by 28
IT Infrastructure Components for Biobanking
Beck, A., Ganslandt, T., Hummel, M., Kiehntopf, M., Sax, U., Ückert, F., Semler, S. C., & Prokosch, H. (2010) · Applied Clinical Informatics · doi:10.4338/aci-2010-05-ra-0034 · Cited by 28
Effectiveness of eHealth Interventions on Moderate-to-Vigorous Intensity Physical Activity Among Patients in Cardiac Rehabilitation: Systematic Review and Meta-analysis
Yu, T., Xu, H., Sui, X., Zhang, X., Pang, Y., Yu, T., Lian, X., Zeng, T., Wu, Y., Leng, X., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/42845 · Cited by 26
Sugarsquare, a Web-Based Patient Portal for Parents of a Child With Type 1 Diabetes: Multicenter Randomized Controlled Feasibility Trial
Boogerd, E. A., Schaaijk, N. M.-V., Sas, T., Boers, A. C.-D., Smallenbroek, M., Nuboer, R., Noordam, C., & Verhaak, C. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6639 · Cited by 29
Research Into Digital Health Intervention for Mental Health: 25-Year Retrospective on the Ethical and Legal Challenges
Hall, C. L., Bergin, A., & Rennick‐Egglestone, S. (2024) · Journal of Medical Internet Research · doi:10.2196/58939 · Cited by 35
A Web-Based Versus Paper Questionnaire on Alcohol and Tobacco in Adolescents
Lygidakis, C., Rigon, S., Cambiaso, S., Bottoli, E., Cuozzo, F., Bonetti, S., Bella, C. D., & Marzo, C. (2010) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2010.0062 · Cited by 26
Baseline Characteristics and Generalizability of Participants in an Internet Smoking Cessation Randomized Trial
Cha, S., Erar, B., NIAURA, R. S., & Graham, A. L. (2016) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9804-x · Cited by 25
The Men's Safer Sex (MenSS) trial: protocol for a pilot randomised controlled trial of an interactive digital intervention to increase condom use in men
Bailey, J., Webster, R., Hunter, R. M., Freemantle, N., Rait, G., Michie, S., Estcourt, C. S., Anderson, J., Gerressu, M., Stephenson, J., et al. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-007552 · Cited by 25
Self-citation
A web-based, peer-supported self-management intervention to reduce distress in relatives of people with psychosis or bipolar disorder: the REACT RCT
Lobban, F. A., Akers, N., Appelbe, D., Capuccinello, R. I., Chapman, L. J., Collinge, L., Dodd, S. R., Flowers, S., Hollingsworth, B. P., Honary, M., et al. (2020) · Health Technology Assessment · doi:10.3310/hta24320 · Cited by 26
Self-citation
Protocol for a randomised controlled trial investigating self-help email messages for sub-threshold depression: the Mood Memos study
Morgan, A., Jorm, A. F., & Mackinnon, A. (2011) · Trials · doi:10.1186/1745-6215-12-11 · Cited by 25
Alcohol email assessment and feedback study dismantling effectiveness for university students (AMADEUS-1): study protocol for a randomized controlled trial
McCambridge, J., Bendtsen, P., Bendtsen, M., & Nilsén, P. (2012) · Trials · doi:10.1186/1745-6215-13-49 · Cited by 21
Self-citation
The Men’s Safer Sex project: intervention development and feasibility randomised controlled trial of an interactive digital intervention to increase condom use in men
Bailey, J., Webster, R., Hunter, R. M., Griffin, M., Freemantle, N., Rait, G., Estcourt, C. S., Michie, S., Anderson, J., Stephenson, J., et al. (2016) · Health Technology Assessment · doi:10.3310/hta20910 · Cited by 22
Self-citation
Web-Based Survey Methodology
Wright, K. B. (2019) · Springer eBooks · doi:10.1007/978-981-10-5251-4_18 · Cited by 23
GAMBLINGLESS: for LIFE study protocol: a pragmatic randomised trial of an online cognitive-behavioural programme for disordered gambling
Merkouris, S. S., Rodda, S. N., Austin, D., Lubman, D. I., Harvey, P., Battersby, M., Cunningham, J., Lavis, T., Smith, D., & Dowling, N. A. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014226 · Cited by 21
An Internet-Based Intervention Condom-Him to Increase Condom Use Among Human Immunodeficiency Virus-Positive Men Who Have Sex With Men: Protocol for a Randomized Controlled Trial
Miranda, J., Côté, J., Godin, G., Blais, M., Otis, J., Guéhéneuc, Y., Fadel, G., Barton, L., & Fowler, S. (2013) · JMIR Research Protocols · doi:10.2196/resprot.2723 · Cited by 21
Clinical trials: can technology solve the problem of low recruitment?
Reynolds, T. (2011) · BMJ · doi:10.1136/bmj.d3662 · Cited by 22
The Impact of Cessation Media Messages on Cessation-Related Outcomes: Results from a National Experiment of Smokers
Duke, J. C., Nonnemaker, J. M., Davis, K., Watson, K. A., & Farrelly, M. C. (2013) · American Journal of Health Promotion · doi:10.4278/ajhp.120920-quan-452 · Cited by 21
Who Participates in Web-Assisted Tobacco Interventions? The Quit-Primo and National Dental Practice-Based Research Network Hi-Quit Studies
Sadasivam, R. S., Kinney, R. L., DeLaughter, K. L., Rao, S. R., Williams, J. H., Coley, H. L., Ray, M. N., Gilbert, G. H., Allison, J. J., Ford, D. E., et al. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2385 · Cited by 20
Improving Women's Self‐Compassion Through an Online Program: A Randomized Controlled Trial
Nadeau, M. M., Caporale-Berkowitz, N. A., & Rochlen, A. B. (2020) · Journal of Counseling & Development · doi:10.1002/jcad.12353 · Cited by 24
Lessons Learned Recruiting and Retaining Pregnant and Postpartum Individuals in Digital Trials: Viewpoint
Parks, A. M., Duffecy, J. L., McCabe, J. E., Breman, R. B., Milgrom, J., Hirshler, Y., Gemmill, A. W., Segre, L. S., Felder, J. N., & Uscher‐Pines, L. (2022) · JMIR Pediatrics and Parenting · doi:10.2196/35320 · Cited by 21
Integrating Psychological Theory Into the Design of an Online Intervention for Sexual Health: The Sexunzipped Website
Carswell, K., McCarthy, O., Murray, E., & Bailey, J. (2012) · JMIR Research Protocols · doi:10.2196/resprot.2114 · Cited by 20
Self-citation
The acceptability of an online intervention using positive psychology for depression: A qualitative study
Walsh, S., Szymczyńska, P., Taylor, S. J., & Priebe, S. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.07.003 · Cited by 19
Beating Cancer-Related Fatigue With the Untire Mobile App: Protocol for a Waiting List Randomized Controlled Trial
Spahrkäs, S. S., Looijmans, A., Sanderman, R., & Hagedoorn, M. (2020) · JMIR Research Protocols · doi:10.2196/15969 · Cited by 20
The belonging model of trust
Knapp, J. R., Sprinkle, T. A., Urick, M. J., & Delaney‐Klinger, K. (2019) · Nonprofit Management and Leadership · doi:10.1002/nml.21370 · Cited by 19
The MOSEXY trial: mobile phone intervention for sexual health in youth—a pragmatic randomised controlled trial to evaluate the effect of a smartphone application on sexual health in youth in Stockholm, Sweden
Nielsen, A. M., Costa, A. D., Gemzell‐Danielsson, K., Marrone, G., Boman, J., Salazar, M., & Diwan, V. K. (2019) · Sexually Transmitted Infections · doi:10.1136/sextrans-2019-054027 · Cited by 19
Smoking-Cessation e-Referrals
Ray, M. N., Funkhouser, E. M., Williams, J. H., Sadasivam, R. S., Gilbert, G. H., Coley, H. L., Rindal, D. B., & Houston, T. K. (2014) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.10.018 · Cited by 19
Protocol for an online randomised controlled trial to evaluate the clinical and cost-effectiveness of a peer-supported self-management intervention for relatives of people with psychosis or bipolar disorder: relatives education and coping toolkit (REACT)
Lobban, F., Robinson, H., Appelbe, D., Barraclough, J., Bedson, E., Collinge, L., Dodd, S., Flowers, S., Honary, M., Johnson, S., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-016965 · Cited by 18
Self-citation
Online Nonprobability Samples
Jeremy Freese, A., & Jin, O. S. (2025) · Annual Review of Sociology · doi:10.1146/annurev-soc-090524-043117 · Cited by 21
Costs and difficulties of recruiting patients to provide e-health support: pilot study in one primary care trust
Jones, R. B., O’Connor, A., Brelsford, J., Parsons, N., & Skirton, H. (2012) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-12-25 · Cited by 19
The usefulness of a mobile device-based system for patient-reported outcomes in a spine outpatient clinic
Kim, C. H., Chung, C. K., Choi, Y., Shin, H.-J., Woo, J. W., Kim, S.-M., & Lee, H. (2016) · The Spine Journal · doi:10.1016/j.spinee.2016.02.048 · Cited by 18
A Virtual Home for the Virtual Clinical Trial
Persky, S. (2020) · Journal of Medical Internet Research · doi:10.2196/15582 · Cited by 18
The effects of environmental information provision on plastic bag use and marine environment status in the context of the environmental levy in Greece
Mentis, C., Maroulis, G., Latinopoulos, D., & Bithas, K. (2022) · Environment Development and Sustainability · doi:10.1007/s10668-022-02465-6 · Cited by 18
The KIND Challenge community intervention to reduce loneliness and social isolation, improve mental health, and neighbourhood relationships: an international randomized controlled trial
Lim, M. H., Hennessey, A., Qualter, P., Smith, B. J., Thurston, L., Eres, R., & Holt‐Lunstad, J. (2024) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-024-02740-z · Cited by 24
Web-Based Intervention for Women With Type 1 Diabetes in Pregnancy and Early Motherhood: Critical Analysis of Adherence to Technological Elements and Study Design
Berg, M., Lindén, K., Adolfsson, A., Sparud‐Lundin, C., & Ranerup, A. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9665 · Cited by 17
A Direct-to-Public Peer Support Program (Big White Wall) Versus Web-Based Information to Aid the Self-management of Depression and Anxiety: Results and Challenges of an Automated Randomized Controlled Trial
Morriss, R. K., Kaylor‐Hughes, C., Rawsthorne, M., Coulson, N., Simpson, S., Guo, B., James, M., Lathe, J., Moran, P., Tata, L. J., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/23487 · Cited by 17
COPe-support - a multi-component digital intervention for family carers for people affected by psychosis: study protocol for a randomized controlled trial
Sin, J., Henderson, C., Cornelius, V. R., Chen, T., Elkes, J., Woodham, L. A., Hernández, A. S., Spence-Polin, D., Batchelor, R., & Gillard, S. (2020) · BMC Psychiatry · doi:10.1186/s12888-020-02528-w · Cited by 16

Authors: Murray, E., Khadjesari, Z., White, I. R., Kalaitzaki, E., Godfrey, C., McCambridge, J., Thompson, S. G., & Wallace, P.
Affiliations: University College London (GB)
MRC Biostatistics Unit (GB)
Medical Research Council (GB)
University of York (GB)
London School of Hygiene & Tropical Medicine (GB)
Corresponding author: Murray, Elizabeth
ORCID: Murray, Elizabeth
Khadjesari, Zarnie
White, Ian R
Godfrey, Christine
McCambridge, Jim
Thompson, Simon G
Wallace, Paul
Venue: Journal of Medical Internet Research
Publisher: JMIR Publications Inc.
Published: 2009-04-03
Volume: 11
Issue: 1
Pages: e9
ISSN: 1438-8871 (ISSN-L)
DOI: 10.2196/jmir.1052
PMC: PMC2762798
OpenAlex: W2165002752
Cited by: 195
References: 48
Citations by year: 2026: 1 · 2025: 7 · 2024: 9 · 2023: 13 · 2022: 8 · 2021: 13 · 2020: 11 · 2019: 8 · 2018: 12 · 2017: 17 · 2016: 16 · 2015: 16 · 2014: 6 · 2013: 19 · 2012: 11
FWCI: 15.30
Type: journal article
Language: en
Open access: gold
doi.org
Licence: other-oa
Locations: Journal of Medical Internet Research (publishedVersion, other-oa)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
Research Portal (King's College London) (publishedVersion, other-oa)
LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
UCL Discovery (University College London) (submittedVersion)
UEA Digital Repository (University of East Anglia) (submittedVersion)
Funders: Cancer Research UK
National Institute for Health and Care Research
British Heart Foundation (RG/08/014/24067)
Diabetes UK
Alcohol Change UK
Medical Research Council (U.1052.00.006, U.1052.00.001)
Economic and Social Research Council
Topics: Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: online trials, online intervention, clinical trial design, participant recruitment, randomization, intervention fidelity, participant retention, data quality, generalizability, internal validity, hazardous drinking
MeSH terms: Clinical Trials as Topic, Delivery of Health Care, Humans, Online Systems, Reproducibility of Results, Randomized Controlled Trials as Topic, Internet

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Methodological Challenges in Online Trials

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks