The Effectiveness of Prompts to Promote Engagement With Digital Interventions: A Systematic Review
Open accessAbstract
BACKGROUND: Digital interventions have been effective in improving numerous health outcomes and health behaviors; furthermore, they are increasingly being used in different health care areas, including self-management of long-term conditions, mental health, and health promotion. The full potential of digital interventions is hindered by a lack of user engagement. There is an urgent need to develop effective strategies that can promote users' engagement with digital interventions. One potential method is the use of technology-based reminders or prompts. OBJECTIVE: To evaluate the effectiveness of technology-based strategies for promoting engagement with digital interventions. METHODS: Cochrane Collaboration guidelines on systematic review methodology were followed. The search strategy was executed across 7 electronic databases: the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase, Web of Science, the Education Resources Information Center (ERIC), PsycINFO, and the Cumulative Index to Nursing and Allied Health Literature (CINAHL). Databases were searched from inception to September 13, 2013, with no language or publication type restrictions, using three concepts: randomized controlled trials, digital interventions, and engagement. Gray literature and reference lists of included studies were also searched. Titles and abstracts were independently screened by 2 authors, then the full texts of potentially eligible papers were obtained and double-screened. Data from eligible papers were extracted by one author and checked for accuracy by another author. Bias was assessed using the Cochrane risk of bias assessment tool. Narrative synthesis was performed on all included studies and, where appropriate, data were pooled using meta-analysis. All findings were reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. RESULTS: A total of 14 studies were included in the review with 8774 participants. Of the 14 studies, 9 had sufficient data to be included in the meta-analyses. The meta-analyses suggested that technology-based strategies can potentially promote engagement compared to no strategy for dichotomous outcomes (relative risk [RR] 1.27, 95% CI 1.01-1.60, I(2)=71%), but due to considerable heterogeneity and the small sample sizes in most studies, this result should be treated with caution. No studies reported adverse or economic outcomes. Only one study with a small sample size compared different characteristics; the study found that strategies promoting new digital intervention content and those sent to users shortly after they started using the digital intervention were more likely to engage users. CONCLUSIONS: Overall, studies reported borderline positive effects of technology-based strategies on engagement compared to no strategy. However, the results have to be interpreted with caution. More research is needed to replicate findings and understand which characteristics of the strategies are effective in promoting engagement and how cost-effective they are.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Findings among people of color are consistent with other studies showing that receiving prompts (eg, text messages, app notifications) leads to higher treatment engagement in digital interventions.
“These findings among POC are consistent with other studies showing that receiving prompts (eg, text messages, app notifications) leads to higher treatment engagement in digital interventions [57].”
From App-Based Mindfulness Meditation for People of Color Who Experience Race-Related Stress: Protocol for a Randomized Controlled Trial (2022) · cites this paper as [57]
The Discussion’s Conclusions (page unavailable) say technology-based strategies may promote engagement compared with no strategy, but caution that the evidence came from a small, heterogeneous set of studies. The citing sentence presents the effect as established rather than tentative; the paper’s listed delivery modes include text messages but do not establish an effect for app notifications specifically. The reference to findings among people of color describes the citing study’s findings, not a result attributed to this review. (LLM, conf=0.87)
Experimental — accuracy not yet measuredReminders, coaching, and tailored feedback delivered via telephone or email have modest to moderate effects toward increasing engagement with digital interventions targeting physical and mental health outcomes, compared to no strategy.
“Reminders, coaching, and tailored feedback delivered via telephone or email have been found to have modest to moderate effects toward increasing engagement with digital interventions targeting physical and mental health outcomes, compared to if no strategy was used (16).”
From Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis (2021) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · see citing works
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,577 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578 · see citing works
Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J., & Wood, C. E. (2013) · Annals of Behavioral Medicine · doi:10.1007/s12160-013-9486-6 · Cited by 7,629 · see citing works
Self-citationEysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524 · see citing works
Webb, T. L., Joseph, J., Yardley, L., & Michie, S. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1376 · Cited by 2,517 · see citing works
Self-citationSynthesising qualitative and quantitative evidence: A review of possible methods
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005) · Journal of Health Services Research & Policy · doi:10.1177/135581960501000110 · Cited by 2,132 · see citing works
Eysenbach, G., & CONSORT-EHEALTH Group (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1923 · Cited by 1,863 · see citing works
Persuasive System Design Does Matter: a Systematic Review of Adherence to Web-based Interventions
Kelders, S. M., Kok, R. N., Ossebaard, H. C., & Van Gemert-Pijnen, J. E. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2104 · Cited by 1,308 · see citing works
Adherence in Internet Interventions for Anxiety and Depression
Christensen, H., Griffiths, K. M., & Farrer, L. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1194 · Cited by 1,133 · see citing works
Wantland, D. J., Portillo, C. J., Holzemer, W. L., Slaughter, R., & McGhee, E. M. (2004) · Journal of Medical Internet Research · doi:10.2196/jmir.6.4.e40 · Cited by 966 · see citing works
A Systematic Review of the Impact of Adherence on the Effectiveness of e-Therapies
Donkin, L., Christensen, H., Naismith, S. L., Neal, B., Hickie, I. B., & Glozier, N. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1772 · Cited by 876 · see citing works
Interactive Health Communication Applications for people with chronic disease
Murray, E., Burns, J., See Tai, S., Lai, R., & Nazareth, I. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004274.pub4 · Cited by 722 · see citing works
Self-citationWebsite-Delivered Physical Activity Interventions
Vandelanotte, C., Spathonis, K. M., Eakin, E. G., & Owen, N. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.02.041 · Cited by 475 · see citing works
Brouwer, W., Kroeze, W., Crutzen, R., de Nooijer, J., de Vries, N. K., Brug, J., & Oenema, A. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1639 · Cited by 455 · see citing works
Fry, J. P., & Neff, R. A. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1138 · Cited by 447 · see citing works
Online Prevention Aimed at Lifestyle Behaviors: A Systematic Review of Reviews
Kohl, L. F., Crutzen, R., & de Vries, N. K. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2665 · Cited by 388 · see citing works
Berger, T., Hämmerli, K., Gubser, N., Andersson, G., & Caspar, F. (2011) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2011.616531 · Cited by 383 · see citing works
Clarke, G., Eubanks, D., Reid, E., Kelleher, C., O'Connor, E., DeBar, L. L., Lynch, F., Nunley, S., & Gullion, C. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.2.e16 · Cited by 372 · cited 10× · see citing works
“Six studies used the strategy at least once per week [36,37,39,40,45,47], one used it for 2 weeks [41], one used it three times [35], and one used it to encourage users to complete sessions with up to 4 email prompts for each session [11].”
InfluentialComputer-based diabetes self-management interventions for adults with type 2 diabetes mellitus
Pal, K., Eastwood, S. V., Michie, S., Farmer, A. J., Barnard, M. L., Peacock, R., Wood, B., Inniss, J. D., & Murray, E. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008776.pub2 · Cited by 344 · see citing works
Self-citation(2008) · doi:10.1016/b978-0-12-373558-4.x0001-5 · Cited by 287 · see citing works
Glasgow, R. E., Boles, S. M., McKay, H. G., Feil, E. G., & Barrera, M. (2003) · Preventive Medicine · doi:10.1016/s0091-7435(02)00056-7 · Cited by 287 · see citing works
Robinson, E., Titov, N., Andrews, G., McIntyre, K., Schwencke, G., & Solley, K. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010942 · Cited by 277 · cited 1× · see citing works
“…likely to influence effectiveness include timing (ie, when should a prompt be used), duration (ie, for how long should it be used) [18,25-27], frequency [22], mode of delivery (eg, email, text message, or telephone call [23]), sender [28,29], content [30], and theoretical underpinning [23].”
Berger, T., Caspar, F., Richardson, R., Kneubühler, B., Sutter, D., & Andersson, G. (2011) · Behaviour Research and Therapy · doi:10.1016/j.brat.2010.12.007 · Cited by 265 · cited 9× · see citing works
“Six studies used the strategy at least once per week [36,37,39,40,45,47], one used it for 2 weeks [41], one used it three times [35], and one used it to encourage users to complete sessions with up to 4 email prompts for each session [11].”
InfluentialStrecher, V., McClure, J., Alexander, G., Chakraborty, B., Nair, V., Konkel, J., Greene, S., Couper, M., Carlier, C., Wiese, C., et al. (2008) · Journal of Medical Internet Research · doi:10.2196/jmir.1002 · Cited by 264 · see citing works
Noar, S. M. (2007) · AIDS and Behavior · doi:10.1007/s10461-007-9313-9 · Cited by 251 · see citing works
Engagement and retention: measuring breadth and depth of participant use of an online intervention
Couper, M. P., Alexander, G. L., Zhang, N., Little, R. J., Maddy, N., Nowak, M. A., McClure, J. B., Calvi, J. J., Rolnick, S. J., Stopponi, M. A., et al. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1430 · Cited by 244 · see citing works
Adherence in Internet-based interventions.
(2008) · PubMed Central · Cited by 215 · see citing works
Can stand‐alone computer‐based interventions reduce alcohol consumption? A systematic review
Khadjesari, Z., Murray, E., Hewitt, C., Hartley, S., & Godfrey, C. (2011) · Addiction · doi:10.1111/j.1360-0443.2010.03214.x · Cited by 208 · see citing works
Self-citationComputerised cognitive-behavioural therapy for depression: systematic review
Kaltenthaler, E., Parry, G., Beverley, C., & Ferriter, M. (2008) · British Journal of Psychiatry · doi:10.1192/bjp.bp.106.025981 · Cited by 206 · cited 1× · see citing works
“There has been substantial investment in DIs in developed countries, and they have been used in different health domains, including self-management of long-term conditions [2-4], promotion of healthy behaviors [1,5-7], and mental health [8].”
Lehto, T., & Oinas-Kukkonen, H. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1559 · Cited by 203 · see citing works
Lalmas, M., O’Brien, H., & Yom-Tov, E. (2015) · Synthesis Lectures on Information Concepts, Retrieval, and Services · doi:10.1007/978-3-031-02289-0 · Cited by 192 · see citing works
Remote and web 2.0 interventions for promoting physical activity
Foster, C. E., Richards, J. A., Thorogood, M., & Hillsdon, M. M. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010395.pub2 · Cited by 186 · see citing works
Interactive computer-based interventions for sexual health promotion
Bailey, J. V., Murray, E., Rait, G., Mercer, C. H., Morris, R. W., Peacock, R., Cassell, J., & Nazareth, I. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006483.pub2 · Cited by 183 · see citing works
Self-citationDigital Asthma Self-Management Interventions: A Systematic Review
Morrison, D., Wyke, S., Agur, K., Cameron, E. J., Docking, R. I., MacKenzie, A., McConnachie, A., Raghuvir, V., Thomson, N. C., & Mair, F. S. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2814 · Cited by 177 · see citing works
Chronic Health Conditions and Internet Behavioral Interventions
Schubart, J. R., Stuckey, H. L., GANESHAMOORTHY, A., & Sciamanna, C. (2011) · CIN Computers Informatics Nursing · doi:10.1097/ncn.0b013e3182155274 · Cited by 154 · see citing works
Johnston, L., Titov, N., Andrews, G., Spence, J., & Dear, B. F. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028079 · Cited by 150 · see citing works
Proudfoot, J., Parker, G., Manicavasagar, V., Hadzi-Pavlovic, D., Whitton, A., Nicholas, J., Smith, M., & Burckhardt, R. (2012) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.04.007 · Cited by 146 · see citing works
Farrer, L., Christensen, H., Griffiths, K. M., & Mackinnon, A. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028099 · Cited by 140 · see citing works
SCHUBART, J. R., STUCKEY, H. L., GANESHAMOORTHY, A., & SCIAMANNA, C. N. (2011) · CIN: Computers, Informatics, Nursing · doi:10.1097/ncn.0b013e3182065eed · Cited by 135 · see citing works
Hutton, H. E., Wilson, L. M., Apelberg, B. J., Avila Tang, E., Odelola, O., Bass, E. B., & Chander, G. (2011) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntq252 · Cited by 132 · see citing works
Attrition Revisited: Adherence and Retention in a Web-Based Alcohol Trial
Murray, E., White, I. R., Varagunam, M., Godfrey, C., Khadjesari, Z., & McCambridge, J. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2336 · Cited by 131 · see citing works
Self-citationInternational Spanish/English Internet smoking cessation trial yields 20% abstinence rates at 1 year
Muñoz, R. F., Barrera, A. Z., Delucchi, K., Penilla, C., Torres, L. D., & Pérez-Stable, E. J. (2009) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntp090 · Cited by 127 · see citing works
Randomized Controlled Trial of Web-Based Treatment of Social Phobia Without Clinician Guidance
Titov, N., Andrews, G., Choi, I., Schwencke, G., & Johnston, L. (2009) · Australian & New Zealand Journal of Psychiatry · doi:10.1080/00048670903179160 · Cited by 122 · see citing works
Brouwer, W., Oenema, A., Crutzen, R., de Nooijer, J., de Vries, N. K., & Brug, J. (2008) · Journal of Medical Internet Research · doi:10.2196/jmir.956 · Cited by 115 · see citing works
WHAT WORKS AND HOW? DESIGNING MORE EFFECTIVE INTERVENTIONS NEEDS ANSWERS TO BOTH QUESTIONS
Michie, S. (2008) · Addiction · doi:10.1111/j.1360-0443.2007.02112.x · Cited by 98 · see citing works
Self-citationFunk, K. L., Stevens, V. J., Appel, L. J., Bauck, A., Brantley, P. J., Champagne, C. M., Coughlin, J., Dalcin, A. T., Harvey-Berino, J., Hollis, J. F., et al. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1504 · Cited by 94 · see citing works
McClure, J. B., Shortreed, S. M., Bogart, A., Derry, H., Riggs, K., St John, J., Nair, V., & An, L. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2508 · Cited by 83 · see citing works
Characterizing Periodic Messaging Interventions Across Health Behaviors and Media: Systematic Review
León, E. D., Fuentes, L. W., & Cohen, J. E. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2837 · Cited by 81 · see citing works
Identifying, documenting, and examining heterogeneity in systematic reviews of complex interventions
Pigott, T., & Shepperd, S. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.06.013 · Cited by 76 · see citing works
Zbikowski, S. M., Jack, L. M., McClure, J. B., Deprey, M., Javitz, H. S., McAfee, T. A., Catz, S. L., Richards, J. A., Bush, T. M., & Swan, G. E. (2011) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntq257 · Cited by 66 · see citing works
An Online Recovery Plan Program: Can Peer Coaching Increase Participation?
Simon, G. E., Ludman, E. J., Goodale, L. C., Dykstra, D. M., Stone, E., Cutsogeorge, D., Operskalski, B., Savarino, J., & Pabiniak, C. (2011) · Psychiatric Services · doi:10.1176/ps.62.6.pss6206_0666 · Cited by 62 · see citing works
Schneider, F., van Osch, L., Schulz, D. N., Kremers, S., & de Vries, H. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1939 · Cited by 59 · see citing works
Schneider, F., de Vries, H., Candel, M. J. J. M., van de Kar, A., & van Osch, L. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2151 · Cited by 58 · see citing works
Alkhaldi, G., Hamilton, F. L., Lau, R., Webster, R., Michie, S., & Murray, E. (2015) · JMIR Research Protocols · doi:10.2196/resprot.3990 · Cited by 46 · see citing works
Self-citationGreaney, M. L., Sprunck‐Harrild, K., Bennett, G. G., Puleo, E., Haines, J., Viswanath, K., & Emmons, K. M. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1981 · Cited by 37 · see citing works
Pilot Implementation of Computerized Cognitive Behavioral Therapy in a University Health Setting
Santucci, L. C., McHugh, R. K., Elkins, R. M., Schechter, B., Ross, M. S., Landa, C. E., Eisen, S. V., & Barlow, D. H. (2013) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0488-2 · Cited by 23 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Carey, R., Bohlen, L. C., Johnston, M., Rothman, A. J., de Bruin, M., Kelly, M. P., & Michie, S. (2018) · Annals of Behavioral Medicine · doi:10.1093/abm/kay078 · Cited by 692 · see citing works
Self-citationGan, D. Z. Q., McGillivray, L., Han, J., Christensen, H., & Torok, M. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.764079 · Cited by 305 · see citing works
Roberts, A., Fisher, A., Smith, L., Heinrich, M., & POTTS, H. W. W. (2017) · Journal of Cancer Survivorship · doi:10.1007/s11764-017-0632-1 · Cited by 270 · see citing works
Bidargaddi, N., Almirall, D., Murphy, S., Nahum-Shani, I., Kovalcik, M., Pituch, T., Maaieh, H., & Strecher, V. (2018) · JMIR mHealth and uHealth · doi:10.2196/10123 · Cited by 189 · see citing works
Versluis, A., Verkuil, B., Spinhoven, P., van der Ploeg, M. M., & Brosschot, J. F. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5642 · Cited by 186 · see citing works
Rauschenberg, C., Schick, A., Hirjak, D., Seidler, A., Paetzold, I., Apfelbacher, C., Riedel-Heller, S. G., & Reininghaus, U. (2021) · Journal of Medical Internet Research · doi:10.2196/23365 · Cited by 205 · see citing works
Digital Interventions on Healthy Lifestyle Management: Systematic Review
Chatterjee, A., Prinz, A., Gerdes, M. W., & Martínez, S. (2021) · Journal of Medical Internet Research · doi:10.2196/26931 · Cited by 214 · see citing works
Warrington, L., Absolom, K. L., Conner, M. T., Kellar, I., Clayton, B., Ayres, M., & Velikova, G. (2018) · Journal of Medical Internet Research · doi:10.2196/10875 · Cited by 200 · see citing works
Engagement and retention in digital mental health interventions: a narrative review
Boucher, E. M., & Raiker, J. S. (2024) · BMC Digital Health · doi:10.1186/s44247-024-00105-9 · Cited by 201 · see citing works
Batra, S., Baker, R., Wang, T., Forma, F., DiBiasi, F., & Peters-Strickland, T. (2017) · Medical Devices: Evidence and Research · doi:10.2147/mder.s144158 · Cited by 169 · see citing works
Moberg, C., Niles, A., & Beermann, D. (2019) · Journal of Medical Internet Research · doi:10.2196/12556 · Cited by 157 · see citing works
Digital Health Interventions for Adolescent and Young Adult Cancer Survivors
Devine, K. A., Viola, A., Coups, E. J., & Wu, Y. P. (2018) · JCO Clinical Cancer Informatics · doi:10.1200/cci.17.00138 · Cited by 174 · see citing works
Digital health intervention during cardiac rehabilitation: A randomized controlled trial
Widmer, J. R., Allison, T., Lennon, R., Lopez-Jimenez, F., Lerman, L., & Lerman, A. (2017) · American Heart Journal · doi:10.1016/j.ahj.2017.02.016 · Cited by 166 · see citing works
Pal, K., Dack, C., Ross, J., Michie, S., May, C., Stevenson, F., Farmer, A., Yardley, L., Barnard, M., & Murray, E. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8439 · Cited by 139 · see citing works
Self-citationTang, M. Y., Smith, D. M., Mc Sharry, J., Hann, M., & French, D. P. (2019) · Annals of Behavioral Medicine · doi:10.1093/abm/kay090 · Cited by 141 · see citing works
Muench, F. J., & Baumel, A. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7463 · Cited by 138 · see citing works
Atema, V., van Leeuwen, M., Kieffer, J. M., Oldenburg, H. S. A., van Beurden, M., Gerritsma, M. A., Kuenen, M. A., Plaisier, P. W., Lopes Cardozo, A. M. F., van Riet, Y. E. A., et al. (2019) · Journal of Clinical Oncology · doi:10.1200/jco.18.00655 · Cited by 105 · see citing works
Mclaughlin, M., Delaney, T., Hall, A., Byaruhanga, J., Mackie, P., Grady, A., Reilly, K., Campbell, E., Sutherland, R., Wiggers, J., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/23180 · Cited by 105 · see citing works
Virtual Laboratories- A historical review and bibliometric analysis of the past three decades
Raghu Raman, R., Achuthan, K., Nair, V. K., & Nedungadi, P. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11058-9 · Cited by 98 · see citing works
Effectiveness of virtual laboratory in engineering education: A meta-analysis
Li, J., & Liang, W. (2024) · PLoS ONE · doi:10.1371/journal.pone.0316269 · Cited by 95 · see citing works
Rhodes, A., Smith, A. D., Chadwick, P., Croker, H. A., & Llewellyn, C. H. (2020) · JMIR mhealth and uhealth · doi:10.2196/18255 · Cited by 89 · see citing works
Jain, S. R., Sui, Y., Ng, C. H., Chen, Z. X., Goh, L. H., & Shorey, S. (2020) · PLoS ONE · doi:10.1371/journal.pone.0237647 · Cited by 87 · see citing works
Lau, Y., Chee, D. G. H., Chow, X. P., Cheng, L. J., & Wong, S. N. (2020) · Preventive Medicine · doi:10.1016/j.ypmed.2020.106001 · Cited by 78 · see citing works
Ogunjobi, T. T., Ohaeri, P. N., Akintola, O. T., Atanda, D. O., Orji, F. P., Adebayo, J. O., Abdul, S. O., Eji, C. A., Asebebe, A. B., Shodipe, O. O., et al. (2024) · Medinformatics · doi:10.47852/bonviewmedin42022335 · Cited by 85 · see citing works
Gan, D. Z. Q., McGillivray, L., Larsen, M. E., Christensen, H. M., & Torok, M. H. (2022) · Digital Health · doi:10.1177/20552076221098268 · Cited by 81 · see citing works
Sezgin, E., Militello, L. K., Huang, Y., & Lin, S. (2020) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz141 · Cited by 83 · see citing works
Bidargaddi, N., Pituch, T., Maaieh, H., Short, C., & Strecher, V. (2018) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2018.07.004 · Cited by 77 · see citing works
Barakat, S., Maguire, S., Smith, K. E., Mason, T. B., Crosby, R. D., & Touyz, S. (2019) · International Journal of Eating Disorders · doi:10.1002/eat.23131 · Cited by 77 · see citing works
Smartphone-Based Psychotherapeutic Micro-Interventions to Improve Mood in a Real-World Setting
Meinlschmidt, G., Lee, J.-H., Stalujanis, E., Belardi, A., Oh, M., Jung, E. K., Kim, H.-C., Alfano, J., Yoo, S.-S., & Tegethoff, M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01112 · Cited by 68 · see citing works
Spaulding, E. M., Marvel, F. A., Piasecki, R. J., Martin, S. S., & Allen, J. K. (2021) · JMIR Cardio · doi:10.2196/18834 · Cited by 70 · see citing works
Nour, M. M., Rouf, A. S., & Allman-Farinelli, M. A. (2017) · Appetite · doi:10.1016/j.appet.2017.10.016 · Cited by 70 · see citing works
Martin, A., Caon, M., Adorni, F., Andreoni, G., Ascolese, A., Atkinson, S., Bul, K., Carrión, C., Castell, C., Ciociola, V., et al. (2019) · JMIR mhealth and uhealth · doi:10.2196/14118 · Cited by 66 · see citing works
Trinh, L., Arbour‐Nicitopoulos, K. P., Sabiston, C. M., Berry, S. R., Loblaw, A., Alibhai, S. M. H., Jones, J. M., & Faulkner, G. (2018) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-018-0686-0 · Cited by 67 · see citing works
Smartphone apps for improving medication adherence in hypertension: patients’ perspectives
Morrissey, E. C., Casey, M., Glynn, L. G., Walsh, J. C., & Molloy, G. J. (2018) · Patient Preference and Adherence · doi:10.2147/ppa.s145647 · Cited by 68 · see citing works
Thornton, L. K., Gardner, L. A., Osman, B., Green, O., Champion, K. E., Bryant, Z., Teesson, M., KAY‐LAMBKIN, F. J., Chapman, C., & The Health4Life Team (2021) · JMIR Formative Research · doi:10.2196/25513 · Cited by 66 · see citing works
Turchioe, M. R., & Merrill, J. A. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy006 · Cited by 65 · see citing works
Hightow‐Weidman, L., & Bauermeister, J. A. (2020) · mHealth · doi:10.21037/mhealth.2019.10.01 · Cited by 64 · see citing works
Dang, T. H., Nguyen, T. A., Minh, H. V., Santin, O., Oanh, T. T. M., & Schofield, P. E. (2021) · Journal of Medical Internet Research · doi:10.2196/24601 · Cited by 62 · see citing works
Bricker, J. B., Mull, K. E., Santiago-Torres, M., Miao, Z., Perski, O., & Di, C. (2022) · Journal of Medical Internet Research · doi:10.2196/39208 · Cited by 44 · see citing works
Kim, M., Kim, Y., Go, Y., Lee, S., Na, M., Lee, Y., Choi, S., & Choi, H. J. (2020) · JMIR mHealth and uHealth · doi:10.2196/14817 · Cited by 56 · see citing works
Dunphy, E., Hamilton, F. L., Spasić, ., & Button, K. (2017) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-017-1846-0 · Cited by 56 · see citing works
Self-citationWhat makes online substance-use interventions engaging? A systematic review and narrative synthesis
Milward, J., Drummond, C., Fincham-Campbell, S., & Deluca, P. (2018) · Digital Health · doi:10.1177/2055207617743354 · Cited by 55 · see citing works
Multiple moderator meta-analysis using the R-package Meta-CART
Li, X., Dusseldorp, E., Su, X., & Meulman, J. J. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01360-0 · Cited by 55 · see citing works
Lin, P., Grambow, S., Intille, S., Gallis, J., Lazenka, T., Bosworth, H., Voils, C., Bennett, G., Batch, B., Allen, J., et al. (2018) · JMIR mHealth and uHealth · doi:10.2196/10471 · Cited by 43 · see citing works
Grant, S., Hodgkinson, J., Schwartz, C., Bradburn, P., Franssen, M., Hobbs, F. R., Jowett, S., McManus, R. J., & Greenfield, S. (2019) · British Journal of General Practice · doi:10.3399/bjgp19x704585 · Cited by 45 · see citing works
Whitehead, L., Robinson, S., Arabiat, D. H., Jenkins, M., & Mörelius, E. (2023) · JMIR Pediatrics and Parenting · doi:10.2196/44199 · Cited by 58 · see citing works
Graham, A. L., Papandonatos, G. D., Cha, S., Erar, B., Amato, M. S., Cobb, N. K., NIAURA, R. S., & Abrams, D. B. (2016) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntw282 · Cited by 52 · see citing works
Renfrew, M. E., Morton, D. P., Morton, J. K., Hinze, J. S., Beamish, P., Przybylko, G., & Craig, B. A. (2020) · Journal of Medical Internet Research · doi:10.2196/15592 · Cited by 53 · see citing works
Schofield, P. E., Shaw, T. J., & Pascoe, M. C. (2019) · Journal of Medical Internet Research · doi:10.2196/12382 · Cited by 49 · see citing works
Pfob, A., Mehrara, B. J., Nelson, J. A., Wilkins, E. G., Pusic, A. L., & Sidey-Gibbons, C. J. (2021) · The Breast · doi:10.1016/j.breast.2021.09.009 · Cited by 48 · see citing works
Graham, S. A., Stein, N., Shemaj, F., Branch, O. H., Paruthi, J., & Kanick, S. C. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.642818 · Cited by 49 · see citing works
McLaughlin, M. J., Delaney, T. M., Hall, A. E., Byaruhanga, J., Mackie, P., Grady, A., Reilly, K., Campbell, E. M., Sutherland, R., Wiggers, J. H., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/29094 · Cited by 46 · see citing works
Nelson, L. A., Spieker, A. J., Greevy, R. A., LeStourgeon, L. M., Wallston, K. A., & Mayberry, L. S. (2020) · JMIR mhealth and uhealth · doi:10.2196/17534 · Cited by 44 · see citing works
Leonard, N. R., Casarjian, B., Fletcher, R., Prata, C., Sherpa, D., Kelemen, A., Rajan, S., Salaam, R., Cleland, C. M., & Gwadz, M. V. (2018) · JMIR Pediatrics and Parenting · doi:10.2196/pediatrics.9037 · Cited by 43 · see citing works
Engaging youth in mHealth: what works and how can we be sure?
Hightow‐Weidman, L., Horvath, K. J., Scott, H. M., Hill-Rorie, J., & Bauermeister, J. A. (2021) · mHealth · doi:10.21037/mhealth-20-48 · Cited by 42 · see citing works
Usability Testing of the BRANCH Smartphone App Designed to Reduce Harmful Drinking in Young Adults
Milward, J., Deluca, P., Drummond, C., Watson, R., Dunne, J., & Kimergård, A. (2017) · JMIR mhealth and uhealth · doi:10.2196/mhealth.7836 · Cited by 42 · see citing works
Alkhaldi, G., Modrow, K., Hamilton, F. L., Pal, K., Ross, J., & Murray, E. (2017) · Interactive Journal of Medical Research · doi:10.2196/ijmr.6952 · Cited by 39 · see citing works
Self-citationTarzia, L., Valpied, J., Koziol‐McLain, J., Glass, N. E., & Hegarty, K. L. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7039 · Cited by 40 · see citing works
Using Digital Health Technology to Prevent and Treat Diabetes
Kaufman, N., & Dadashi, M. (2018) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2018.2506 · Cited by 39 · see citing works
Martin, F., Wright, H., Moody, L., Whiteman, B., McGillion, M., Clyne, W., Pearce, G., & Turner, A. (2020) · Journal of Medical Internet Research · doi:10.2196/17824 · Cited by 33 · see citing works
Holzmann, S. L., & Holzapfel, C. (2019) · Journal of Personalized Medicine · doi:10.3390/jpm9020031 · Cited by 38 · see citing works
Militello, L. K., Sobolev, M., Okeke, F., Adler, D. A., & Nahum-Shani, I. B. (2021) · JMIR Formative Research · doi:10.2196/30606 · Cited by 36 · see citing works
Toward the Science of Engagement With Digital Interventions
Nahum-Shani, I. B., & Yoon, C. (2024) · Current Directions in Psychological Science · doi:10.1177/09637214241254328 · Cited by 37 · see citing works
Gamifying Breastfeeding for Fathers: Process Evaluation of the Milk Man Mobile App
White, B. K., Giglia, R. C., White, J. A., Dhaliwal, S. S., Burns, S. K., & Scott, J. A. (2019) · JMIR Pediatrics and Parenting · doi:10.2196/12157 · Cited by 38 · see citing works
Bell, L., Garnett, C., Bao, Y., Cheng, Z., Qian, T., Perski, O., Potts, H. W. W., & Williamson, E. (2023) · JMIR mHealth and uHealth · doi:10.2196/38342 · Cited by 35 · see citing works
Lokman, S., Leone, S. S., Sommers‐Spijkerman, M., van der Poel, A., Smit, F., & Boon, B. J. F. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6581 · Cited by 37 · see citing works
Lin, Z. J., Althoff, T., & Leskovec, J. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3178876.3186062 · Cited by 35 · see citing works
E-Health Interventions for Adult and Aging Population With Intellectual Disability: A Review
Vázquez‐Martínez, A., Río, C. J., Flores, N., Bagnato, M. J., Rodríguez, M. D. C. P., & Cruz, M. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02323 · Cited by 36 · see citing works
Whelan, M., Orme, M. W., Kingsnorth, A. P., Sherar, L. B., Denton, F., & Esliger, D. (2019) · JMIR mhealth and uhealth · doi:10.2196/14195 · Cited by 37 · see citing works
Pfob, A., Sidey-Gibbons, C. J., Schuessler, M., Lu, S.-C., Xu, C., Dubsky, P. C., Golatta, M., & Heil, J. (2021) · JCO Clinical Cancer Informatics · doi:10.1200/cci.21.00032 · Cited by 37 · see citing works
Jones, S. H., Jovanoska, J., Calam, R. M., Wainwright, L. D., Vincent, H., Asar, Ö., Diggle, P. J., Parker, R., Long, R., Sanders, M. R., et al. (2017) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12745 · Cited by 34 · see citing works
Grady, A., Pearson, N., Lamont, H., Leigh, L., Wolfenden, L., Barnes, C., Wyse, R., Finch, M., McLaughlin, M. J., Delaney, T. M., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/47987 · Cited by 38 · see citing works
Bell, L., Garnett, C. V., Qian, T., Perski, O., Williamson, E., & POTTS, H. W. W. (2020) · Journal of Medical Internet Research · doi:10.2196/23369 · Cited by 33 · see citing works
Wu, N., Gong, E., Wang, B., Gu, W., Nan, D., Zhang, Z., Chen, M., Yan, L. L., Oldenburg, B. F., & Xu, L.-Q. (2019) · JMIR mhealth and uhealth · doi:10.2196/13503 · Cited by 33 · see citing works
Renfrew, M. E., Morton, D. P., Morton, J. K., & Przybylko, G. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.716106 · Cited by 32 · see citing works
Akdemir, A., Smith, A. B., Wu, V. S., Rincones, O., Russell, H., Lyhne, J. D., Kemp, E., David, M., & Bamgboje‐Ayodele, A. (2024) · Psycho-Oncology · doi:10.1002/pon.6290 · Cited by 33 · see citing works
Murray, E., Ross, J., Pal, K., Li, J., Dack, C., Stevenson, F. A., Sweeting, M. J., Parrott, S., Barnard, M., Yardley, L., et al. (2018) · Programme Grants for Applied Research · doi:10.3310/pgfar06050 · Cited by 30 · see citing works
Self-citationCôté, J., Fortin, M., Auger, P., Rouleau, G., Dubois, S., Boudreau, N., Vaillant, I., & Gélinas-Lemay, É. (2018) · JMIR Formative Research · doi:10.2196/formative.9707 · Cited by 30 · see citing works
Turchioe, M. R., Heitkemper, E., Lor, M., Burgermaster, M., & Mamykina, L. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.08.001 · Cited by 30 · see citing works
Graham, A. L., Papandonatos, G. D., Jacobs, M. A., Amato, M. S., Cha, S., Cohn, A. M., Abroms, L. C., & Whittaker, R. (2020) · Journal of Medical Internet Research · doi:10.2196/17734 · Cited by 29 · see citing works
Effectiveness of a Dyadic Buddy App for Smoking Cessation: Randomized Controlled Trial
Schwaninger, P., Berli, C., Scholz, U., & Lüscher, J. (2021) · Journal of Medical Internet Research · doi:10.2196/27162 · Cited by 27 · see citing works
Bell, L., Garnett, C. V., Qian, T., Perski, O., POTTS, H. W. W., & Williamson, E. (2020) · JMIR Research Protocols · doi:10.2196/18690 · Cited by 28 · see citing works
User engagement in mobile apps for people with schizophrenia: A scoping review
de Almeida, R. S., & Marques, A. J. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.1023592 · Cited by 29 · see citing works
Associations between Digital Health Intervention Engagement and Dietary Intake: A Systematic Review
Delaney, T. M., McLaughlin, M. J., Hall, A. E., Yoong, S. L., Brown, A., O’Brien, K. M., Dray, J., Barnes, C., Hollis, J., Wyse, R., et al. (2021) · Nutrients · doi:10.3390/nu13093281 · Cited by 27 · see citing works
Jenkins, C. L., Imran, S., Mahmood, A., Bradbury, K. J., Murray, E., Stevenson, F. A., & Hamilton, F. L. (2021) · JMIR Research Protocols · doi:10.2196/32538 · Cited by 27 · see citing works
Self-citationKodish, T., Schueller, S. M., & Lau, A. S. (2023) · Journal of Behavioral and Cognitive Therapy · doi:10.1016/j.jbct.2022.12.002 · Cited by 29 · see citing works
Kunkel, T., Hayduk, T., & Lock, D. (2023) · European Journal of Marketing · doi:10.1108/ejm-06-2021-0388 · Cited by 26 · see citing works
Singh, G., MacGillivray, M. K., Mills, P. B., Adams, J. R., Sawatzky, B., & Mortenson, W. B. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1487-y · Cited by 26 · see citing works
Caudwell, K. M., Mullan, B. A., & Hagger, M. S. (2018) · International Journal of Behavioral Medicine · doi:10.1007/s12529-018-9736-x · Cited by 25 · see citing works
Nkyekyer, J., Clifford, S. A., Mensah, F. K., Wang, Y., Chiu, L., & Wake, M. A. (2021) · Journal of Medical Internet Research · doi:10.2196/23499 · Cited by 25 · see citing works
Mobile health use predicts self-efficacy and self-management in adolescents with sickle cell disease
Hood, A., Nwankwo, C. N., Walton, A. E., McTate, E., Joffe, N. E., Quinn, C. T., Britto, M. T., Peugh, J., Mara, C. A., & Crosby, L. E. (2021) · Translational Behavioral Medicine · doi:10.1093/tbm/ibab041 · Cited by 26 · see citing works
Graham, A. L., Papandonatos, G. D., Cha, S., Amato, M. S., Jacobs, M. A., Cohn, A. M., Abroms, L. C., & Whittaker, R. (2021) · Addiction · doi:10.1111/add.15677 · Cited by 25 · see citing works
Digital encounters: Human interactions in mHealth behavior change interventions
Müssener, U. (2021) · Digital Health · doi:10.1177/20552076211029776 · Cited by 24 · see citing works
Lau, S. C. L., Judycki, S., Mix, M., DePaul, O., Tomazin, R., Hardi, A., Wong, A., & Baum, C. M. (2022) · American Journal of Occupational Therapy · doi:10.5014/ajot.2022.049117 · Cited by 27 · see citing works
Müssener, U., Thomas, K., Linderoth, C., Löf, M., Åsberg, K., Henriksson, P., & Bendtsen, M. (2020) · JMIR mhealth and uhealth · doi:10.2196/17999 · Cited by 22 · see citing works
Thomas, P. C., Curtis, K., POTTS, H. W. W., Bark, P., Perowne, R., Rookes, T., & Rowe, S. L. (2024) · JMIR Mental Health · doi:10.2196/57577 · Cited by 24 · see citing works
Ali, E. E., Chan, S. S. L., Leow, J. L., Chew, L. S. T., & Yap, K. Y. (2018) · Journal of Oncology Pharmacy Practice · doi:10.1177/1078155218778106 · Cited by 23 · see citing works
Nelson, L. A., Spieker, A. J., Mayberry, L. S., McNaughton, C., & Greevy, R. A. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab254 · Cited by 23 · see citing works
Bailey, J., Webster, R., Hunter, R. M., Griffin, M., Freemantle, N., Rait, G., Estcourt, C. S., Michie, S., Anderson, J., Stephenson, J., et al. (2016) · Health Technology Assessment · doi:10.3310/hta20910 · Cited by 22 · see citing works
Self-citationPhenotypes of engagement with mobile health technology for heart rhythm monitoring
Lee, J., Turchioe, M. R., Creber, R. M., BIVIANO, A. B., Hickey, K. T., & Henry, S. B. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab043 · Cited by 20 · see citing works
- Authors
- Alkhaldi, G., Hamilton, F. L., Lau, R., Webster, R., Michie, S., & Murray, E.
- Affiliations
- University College London (GB)
- Corresponding author
- Alkhaldi, Ghadah
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2016-01-08
- Volume
- 18
- Issue
- 1
- Pages
- e6
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC4723726
- OpenAlex
- W2231658978
- Cited by
- 273
- References
- 58
- Citations by year
- 2026: 15 · 2025: 28 · 2024: 24 · 2023: 22 · 2022: 26 · 2021: 43 · 2020: 27 · 2019: 23 · 2018: 39 · 2017: 17 · 2016: 9
- FWCI
- 5.60
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- University College London
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- digital interventions, user engagement, prompting, reminders, self-management, chronic disease, mental health, health promotion, mobile health, user-generated content, user-generated content sharing
- MeSH terms
- Cost-Benefit Analysis, Health Promotion, Humans, Medical Informatics, Self Care, Health Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
