Digital health interventions for all? Examining inclusivity across all stages of the digital health intervention research process
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Digital interventions offer many possibilities for improving health, as remote interventions can enhance reach and access to underserved groups of society. However, research evaluating digital health interventions demonstrates that such technologies do not equally benefit all and that some in fact seem to reinforce a “digital health divide.” By better understanding these potential pitfalls, we may contribute to narrowing the digital divide in health promotion. The aim of this article is to highlight and reflect upon study design decisions that might unintentionally enhance inequities across key research stages—recruitment, enrollment, engagement, efficacy/effectiveness, and retention. To address the concerns highlighted, we propose strategies including (1) the standard definition of “effectiveness” should be revised to include a measure of inclusivity; (2) studies should report a broad range of potential inequity indicators of participants recruited, randomized, and retained and should conduct sensitivity analyses examining potential sociodemographic differences for both the effect and engagement of the digital interventions; (3) participants from historically marginalized groups should be involved in the design of study procedures, including those related to recruitment, consent, intervention implementation and engagement, assessment, and retention; (4) eligibility criteria should be minimized and carefully selected and the screening process should be streamlined; (5) preregistration of trials should include recruitment benchmarks for sample diversity and comprehensive lists of sociodemographic characteristics assessed; and (6) studies within trials should be embedded to systematically test recruitment and retention strategies to improve inclusivity. The implementation of these strategies would enhance the ability of digital health trials to recruit, randomize, engage, and retain a broader and more representative population in trials, ultimately minimizing the digital divide and broadly improving population health.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In-trial dropout is itself differential.Lower-income participants and racial and ethnic minorities leave earlier and more frequently than the trial sample average 36 .”
From The digital native fallacy: how age-based assumptions exclude young adults with chronic disease from digital health innovations (2026) · cites this paper as [36]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524 · cited 1×
“Digital health studies often suffer from high attrition rates [33], greatly affecting external validity and ability to generalize study results to the broader target population.”
BackgroundBarriers to recruiting underrepresented populations to cancer clinical trials: a systematic review
Ford, J. G., Howerton, M. W., Lai, G. Y., Gary, T. L., Bolen, S., Gibbons, M. C., Tilburt, J., Baffi, C., Tanpitukpongse, T. P., Wilson, R. F., et al. (2008) · Cancer · doi:10.1002/cncr.23157 · Cited by 1,083
Lehtimaki, S., Martic, J., Wahl, B., Foster, K. T., & Schwalbe, N. (2021) · JMIR Mental Health · doi:10.2196/25847 · Cited by 488
Pratap, A., Neto, E. C., Snyder, P., Stepnowsky, C., Elhadad, N., Grant, D., Mohebbi, M. H., Mooney, S., Suver, C., Wilbanks, J., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0224-8 · Cited by 410 · cited 2×
“…of racial and ethnic minor-itized participants recruited was substantially lower than would be expected given the states’ population averages [19]; on average, 9.2% fewer participants who identified as African American/Black and 8.1% fewer participants who identified as Hispanic/Latino were…”
InfluentialMcGill, R., Anwar, E., Orton, L., Bromley, H., Lloyd-Williams, F., O’Flaherty, M., Taylor-Robinson, D., Guzman-Castillo, M., Gillespie, D., Moreira, P., et al. (2015) · BMC Public Health · doi:10.1186/s12889-015-1781-7 · Cited by 407 · cited 1×
“…interventions such as dietary counseling can widen social inequalities by being less effective in minoritized populations, potentially because they require more individual agency (which minoritized populations tend to have less of due to a host of factors, including structural barriers) [7].”
BackgroundBack to the Future: Achieving Health Equity Through Health Informatics and Digital Health
Brewer, L. C., Fortuna, K. L., Jones, C., Walker, R., Hayes, S. N., Patten, C. A., & Cooper, L. A. (2020) · JMIR mHealth and uHealth · doi:10.2196/14512 · Cited by 367
Saleem, M., Kühne, L., De Santis, K. K., Christianson, L., Brand, T., & Busse, H. (2021) · JMIR Mental Health · doi:10.2196/30000 · Cited by 199 · cited 1×
“Several strategies such as personalization, gamification, and reminders have been linked to increased engagement [26].”
BackgroundSelf-citationComparing Two Web-Based Smoking Cessation Programs: Randomized Controlled Trial
McKay, H. G., Danaher, B. G., Seeley, J. R., Lichtenstein, E., & Gau, J. M. (2008) · Journal of Medical Internet Research · doi:10.2196/jmir.993 · Cited by 99 · cited 1×
“Predictors of dropout include younger age [19, 34–36], lower education levels [34, 36], lower SES [37], lower confidence in the ability to make target health behavior changes [36], and poorer health status at baseline [34].”
Background
Research works printed but not linked: 44
- Whitehead M, Dahlgren G. What can be done about inequalities in health? Lancet. 1991;338(8774):1059–63. · DOI · Paper page
- Bell JAH, Balneaves LG. Cancer patient decision making related to clinical trial participation: an integrative review with implications for patients’ relational autonomy. Support Care Cancer. 2015;23(4):1169–96. · DOI
- Houghton C, Dowling M, Meskell P, Hunter A, Gardner H, Conway A, et al. Factors that impact on recruitment to randomised trials in health care: a qualitative evidence synthesis. Cochrane Database Syst Rev. 2020(10):MR000045. · DOI · Paper page
- Wallace N, O’Keeffe S, Gardner H, Shiely F. Underrecording and underreporting of participant ethnicity in clinical trials is persistent and is a threat to inclusivity and generalizability. J Clin Epidemiol. 2023;162:81–9. · DOI
- Gong G, Phillips SG, Hudson C, Curti D, Philips BU. Higher US rural mortality rates linked to socioeconomic status, physician shortages, and lack of health insurance. Health Aff. 2019;38(12):2003–10. · DOI · Paper page
- Steptoe A, Zaninotto P. Lower socioeconomic status and the acceleration of aging: an outcome-wide analysis. Proc Natl Acad Sci. 2020;117(26):14911–7. · DOI · Paper page
- Barbosa W, Zhou K, Waddell E, Myers T, Dorsey ER. Improving access to care: telemedicine across medical domains. Annu Rev Public Health. 2021;42:463–81. · DOI · Paper page
- Rush KL, Singh S, Seaton CL, Burton L, Li E, Jones C, et al. Telehealth use for enhancing the health of rural older adults: a systematic mixed studies review. Gerontologist. 2022;62(10):e564–77. · DOI
- Western MJ, Armstrong MEG, Islam I, Morgan K, Jones UF, Kelson MJ. The effectiveness of digital interventions for increasing physical activity in individuals of low socioeconomic status: a systematic review and meta-analysis. Int J Behav Nutr Phys Act. 2021;18(1):148. · DOI · Paper page
- Szinay D, Forbes CC, Busse H, DeSmet A, Smit ES, König LM. Is the uptake, engagement, and effectiveness of exclusively mobile interventions for the promotion of weight-related behaviors equal for all? A systematic review. Obes Rev. 2023;24(3):e13542. · DOI · Paper page
- Jahnel T, Dassow H-H, Gerhardus A, Schüz B. The digital rainbow: digital determinants of health inequities. Digital Health. 2022;8:20552076221129092. · DOI · Paper page
- König LM, Krukowski RA, Kuntsche E, Busse H, Gumbert L, Gemesi K, Neter E, Firdous Mohamed N, Ross KM, John-Akinola YO, Cooper R, Allmeta A, Macedo Silva A, Forbes CC, Western MJ. Reducing intervention- and research-induced inequalities to tackle the digital divide in health promotion. Int J Equity Health. in press.
- Pérez-Muñoz A, Horn TL, Graber J, Chowdhury SMR, Bursac Z, Krukowski RA. Recruitment strategies for a post cessation weight management trial: a comparison of strategy cost-effectiveness and sample diversity. Contemp Clin Trials Commun. 2022;30:101037. · DOI
- Rodríguez-Torres E, González-Pérez MM, Díaz-Pérez C. Barriers and facilitators to the participation of subjects in clinical trials: an overview of reviews. Contemp Clin Trials Commun. 2021;23:100829. · DOI
- Scharff DP, Mathews KJ, Jackson P, Hoffsuemmer J, Martin E, Edwards D. More than Tuskegee: understanding mistrust about research participation. J Health Care Poor Underserved. 2010;21(3):879. · DOI · Paper page
- Adedinsewo D, Eberly L, Sokumbi O, Rodriguez JA, Patten CA, Brewer LC. Health disparities, clinical trials, and the digital divide thematic reviews on forward thinking on clinical trials in clinical practice. Mayo Clin Proc. 2023;98(12):1875–87. · DOI
- Anderson M. Mobile technology and home broadband. Pew Research Center. 2019.
- Roessler P. The mobile phone revolution and digital inequality: scope, determinants and consequences. Prosperity Comm Backgr Pap Ser. 2018(15):1–39.
- Venter W, Coleman J, Chan VL, Shubber Z, Phatsoane M, Gorgens M, et al. Improving linkage to HIV care through mobile phone apps: randomized controlled trial. JMIR Mhealth Uhealth. 2018;6(7):e8376. · DOI
- McVay MA, Lavoie HA, Rajoria M, Leong MC, Lou X, McMahon LN, et al. Pre-enrollment steps and run-ins in weight loss trials: a meta-regression. Am J Prev Med. 2023;64(6):910–7. · DOI
- Krukowski RA, Harvey-Berino J, Bursac Z, Ashikaga T, West DS. Patterns of success: online self-monitoring in a web-based behavioral weight control program. Health Psychol. 2013;32(2):164. · DOI · Paper page
- Harvey J, Krukowski R, Priest J, West D. Log often, lose more: Electronic dietary self-monitoring for weight loss. Obesity. 2019;27(3):380–4. · DOI · Paper page
- Devine S, Leeds C, Shlay JC, Leytem A, Beum R, Bull S. Methods to assess youth engagement in a text messaging supplement to an effective teen pregnancy program. J Biomed Inform. 2015;56:379–86. · DOI
- Patel MS, Bachireddy C, Small DS, Harrison JD, Harrington TO, Oon AL, et al. Effect of goal-setting approaches within a gamification intervention to increase physical activity among economically disadvantaged adults at elevated risk for major adverse cardiovascular events: the ENGAGE randomized clinical trial. JAMA cardiology. 2021;6(12):1387–96. · DOI
- König LM, Attig C, Franke T, Renner B. Barriers to and facilitators for using nutrition apps: systematic review and conceptual framework. JMIR Mhealth Uhealth. 2021;9(6):e20037. · DOI · Paper page
- Kalmbach DA, Cheng P, Reffi AN, Seymour GM, Ruprich MK, Bazan LF, et al. Racial disparities in treatment engagement and outcomes in digital cognitive behavioral therapy for insomnia among pregnant women. Sleep Health. 2023;9(1):18–25. · DOI
- Schüz B, Li AS-W, Hardinge A, McEachan RR, Conner M. Socioeconomic status as a moderator between social cognitions and physical activity: systematic review and meta-analysis based on the Theory of Planned Behavior. Psychol Sport Exerc. 2017;30:186–95. · DOI
- Geraghty AWA, Torres LD, Leykin Y, Pérez-Stable EJ, Muñoz RF. Understanding attrition from international internet health interventions: a step towards global eHealth. Health Promot Int. 2012;28(3):442–52. · DOI · Paper page
- Van der Mispel C, Poppe L, Crombez G, Verloigne M, De Bourdeaudhuij I. A self-regulation-based eHealth intervention to promote a healthy lifestyle: investigating user and website characteristics related to attrition. J Med Internet Res. 2017;19(7):e241. · DOI · Paper page
- Cheng P, Luik AI, Fellman-Couture C, Peterson E, Joseph CL, Tallent G, et al. Efficacy of digital CBT for insomnia to reduce depression across demographic groups: a randomized trial. Psychol Med. 2019;49(3):491–500. · DOI · Paper page
- Griffith Fillipo IR, Pullmann MD, Hull TD, Zech J, Wu J, Litvin B, et al. Participant retention in a fully remote trial of digital psychotherapy: Comparison of incentive types. Front Digit Health. 2022;4:963741. · DOI
- Khadjesari Z, Murray E, Kalaitzaki E, White IR, McCambridge J, Thompson SG, et al. Impact and costs of incentives to reduce attrition in online trials: two randomized controlled trials. J Med Internet Res. 2011;13(1):e26. · DOI · Paper page
- Kolovson S, Pratap A, Duffy J, Allred R, Munson SA, Areán PA. Understanding participant needs for engagement and attitudes towards passive sensing in remote digital health studies. Int Conf Pervasive Comput Technol Healthc. 2020;2020:347–62. · DOI
- Withall J, Greaves CJ, Thompson JL, de Koning JL, Bollen JC, Moorlock SJ, et al. The tribulations of trials: lessons learnt recruiting 777 older adults into REtirement in ACTion (REACT), a trial of a community, group-based active aging intervention targeting mobility disability. J Gerontol A. 2020;75(12):2387–95. · DOI
- Rick J, Graffy J, Knapp P, Small N, Collier DJ, Eldridge S, et al. Systematic techniques for assisting recruitment to trials (START): study protocol for embedded, randomized controlled trials. Trials. 2014;15(1):1–11. · DOI · Paper page
- Kim ES, Bruinooge SS, Roberts S, Ison G, Lin NU, Gore L, et al. Broadening eligibility criteria to make clinical trials more representative: American Society of Clinical Oncology and Friends of Cancer Research Joint Research Statement. J Clin Oncol. 2017;35(33):3737. · DOI · Paper page
- Miller SJ, Sly JR, Alcaraz KI, Ashing K, Christy SM, Gonzalez B, et al. Equity and behavioral digital health interventions: strategies to improve benefit and reach. Transl Behav Med. 2023;13(6):400–5. · DOI
- Skovlund PC, Nielsen BK, Thaysen HV, Schmidt H, Finset A, Hansen KA, et al. The impact of patient involvement in research: a case study of the planning, conduct and dissemination of a clinical, controlled trial. Res Involvement Engagement. 2020;6(1):1–16. · DOI
- Fahey MC, Talcott GW, Bauer CMC, Bursac Z, Gladney L, Hare ME, et al. Moms fit 2 fight: Rationale, design, and analysis plan of a behavioral weight management intervention for pregnant and postpartum women in the US military. Contemp Clin Trials. 2018;74:46–54. · DOI
- Estevez Burns R, Hare ME, Andres A, Klesges RC, Talcott GW, LeRoy K, et al. An interim analysis of a gestational weight gain intervention in military personnel and other TRICARE beneficiaries. Obesity. 2022;30(10):1951–62. · DOI
- Kristensen FB, Mäkelä M, Neikter SA, Rehnqvist N, Håheim LL, Mørland B, et al. European network for Health Technology Assessment, EUnetHTA: planning, development, and implementation of a sustainable European network for Health Technology Assessment. Int J Technol Assess Health Care. 2009;25(S2):107–16. · DOI
- Doherty L, Parker A, Arundel C, Clark L, Coleman E, Hewitt C, et al. PROMoting the use of studies within a trial (PROMETHEUS): results and experiences from a large programme to evaluate the routine embedding of recruitment and retention strategies within randomised controlled trials routinely. Res Methods Med Health Sci. 2023;4(3):113–22.
- Gültzow T, Neter E, Zimmermann H. Making Research Look Like the World Looks: Introducing the’Inclusivity & Diversity Add-On for Preregistration Forms’ Developed During an EHPS2022 Pre-Conference Workshop. 2023.
- Pebley K, Bursac Z, Klesges RC, Ebbert JO, Womack CR, Graber J, et al. A randomized controlled trial to reduce post-cessation weight gain. Int J Obes. 2023;47:1–8. · DOI
Cited by
Western, M. J., Smit, E. S., Gültzow, T., Neter, E., Sniehotta, F. F., Malkowski, O. S., Wright, C., Busse, H., Peuters, C., Rehackova, L., et al. (2025) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2025.2493139 · Cited by 116
“Research teams would also do well to consider solutions to common barriers to engagement across different sections of the research process, such as ensuring study adverts, procedures, travel, and intervention demands are accommodating of vulnerable sub populations (Krukowski et al., 2024).”
Self-citationEkstrom, V. S. M. (2026) · npj Digital Public Health · doi:10.1038/s44482-026-00033-7
Metadata
- Authors
- Krukowski, R. A., Ross, K. M., Western, M. J., Cooper, R., Busse, H., Forbes, C., Kuntsche, E., Allmeta, A., Silva, A. M., John-Akinola, Y. O., & König, L. M.
- Venue
- Trials
- Published
- 2024-01-30
- Volume
- 25
- Issue
- 1
- ISSN
- 1745-6215
- Cited by
- 83
- References
- 8
- Language
- en
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Funders
- Universität Bayreuth (Centre of International Excellence "Alexander von Humboldt")
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Digital health interventions for all? Examining inclusivity across all stages of the digital health intervention research process. https://app.cochat.ai/papers/10.1186/s13063-024-07937-w
- The paper
- Krukowski, R. A., Ross, K. M., Western, M. J., Cooper, R., Busse, H., Forbes, C., Kuntsche, E., Allmeta, A., Silva, A. M., John-Akinola, Y. O., & König, L. M. (2024). Digital health interventions for all? Examining inclusivity across all stages of the digital health intervention research process. Trials. https://doi.org/10.1186/s13063-024-07937-w
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.