CoChat

Digital health interventions for all? Examining inclusivity across all stages of the digital health intervention research process

Krukowski, Rebecca A., Ross, Kathryn M., Western, Max J., Cooper, Rosie, Busse, Heide, Forbes, Cynthia, Kuntsche, Emmanuel, Allmeta, Anila, Silva, Anabelle Macedo, John-Akinola, Yetunde O., König, Laura M. · 2024 · Trials
Cited by 83

The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.

Abstract

Digital interventions offer many possibilities for improving health, as remote interventions can enhance reach and access to underserved groups of society. However, research evaluating digital health interventions demonstrates that such technologies do not equally benefit all and that some in fact seem to reinforce a “digital health divide.” By better understanding these potential pitfalls, we may contribute to narrowing the digital divide in health promotion. The aim of this article is to highlight and reflect upon study design decisions that might unintentionally enhance inequities across key research stages—recruitment, enrollment, engagement, efficacy/effectiveness, and retention. To address the concerns highlighted, we propose strategies including (1) the standard definition of “effectiveness” should be revised to include a measure of inclusivity; (2) studies should report a broad range of potential inequity indicators of participants recruited, randomized, and retained and should conduct sensitivity analyses examining potential sociodemographic differences for both the effect and engagement of the digital interventions; (3) participants from historically marginalized groups should be involved in the design of study procedures, including those related to recruitment, consent, intervention implementation and engagement, assessment, and retention; (4) eligibility criteria should be minimized and carefully selected and the screening process should be streamlined; (5) preregistration of trials should include recruitment benchmarks for sample diversity and comprehensive lists of sociodemographic characteristics assessed; and (6) studies within trials should be embedded to systematically test recruitment and retention strategies to improve inclusivity. The implementation of these strategies would enhance the ability of digital health trials to recruit, randomize, engage, and retain a broader and more representative population in trials, ultimately minimizing the digital divide and broadly improving population health.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13063-024-07937-w

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 8 references

Research works printed but not linked: 44

  1. Whitehead M, Dahlgren G. What can be done about inequalities in health? Lancet. 1991;338(8774):1059–63. · DOI · Paper page
  2. Bell JAH, Balneaves LG. Cancer patient decision making related to clinical trial participation: an integrative review with implications for patients’ relational autonomy. Support Care Cancer. 2015;23(4):1169–96. · DOI
  3. Houghton C, Dowling M, Meskell P, Hunter A, Gardner H, Conway A, et al. Factors that impact on recruitment to randomised trials in health care: a qualitative evidence synthesis. Cochrane Database Syst Rev. 2020(10):MR000045. · DOI · Paper page
  4. Wallace N, O’Keeffe S, Gardner H, Shiely F. Underrecording and underreporting of participant ethnicity in clinical trials is persistent and is a threat to inclusivity and generalizability. J Clin Epidemiol. 2023;162:81–9. · DOI
  5. Gong G, Phillips SG, Hudson C, Curti D, Philips BU. Higher US rural mortality rates linked to socioeconomic status, physician shortages, and lack of health insurance. Health Aff. 2019;38(12):2003–10. · DOI · Paper page
  6. Steptoe A, Zaninotto P. Lower socioeconomic status and the acceleration of aging: an outcome-wide analysis. Proc Natl Acad Sci. 2020;117(26):14911–7. · DOI · Paper page
  7. Barbosa W, Zhou K, Waddell E, Myers T, Dorsey ER. Improving access to care: telemedicine across medical domains. Annu Rev Public Health. 2021;42:463–81. · DOI · Paper page
  8. Rush KL, Singh S, Seaton CL, Burton L, Li E, Jones C, et al. Telehealth use for enhancing the health of rural older adults: a systematic mixed studies review. Gerontologist. 2022;62(10):e564–77. · DOI
  9. Western MJ, Armstrong MEG, Islam I, Morgan K, Jones UF, Kelson MJ. The effectiveness of digital interventions for increasing physical activity in individuals of low socioeconomic status: a systematic review and meta-analysis. Int J Behav Nutr Phys Act. 2021;18(1):148. · DOI · Paper page
  10. Szinay D, Forbes CC, Busse H, DeSmet A, Smit ES, König LM. Is the uptake, engagement, and effectiveness of exclusively mobile interventions for the promotion of weight-related behaviors equal for all? A systematic review. Obes Rev. 2023;24(3):e13542. · DOI · Paper page
  11. Jahnel T, Dassow H-H, Gerhardus A, Schüz B. The digital rainbow: digital determinants of health inequities. Digital Health. 2022;8:20552076221129092. · DOI · Paper page
  12. König LM, Krukowski RA, Kuntsche E, Busse H, Gumbert L, Gemesi K, Neter E, Firdous Mohamed N, Ross KM, John-Akinola YO, Cooper R, Allmeta A, Macedo Silva A, Forbes CC, Western MJ. Reducing intervention- and research-induced inequalities to tackle the digital divide in health promotion. Int J Equity Health. in press.
  13. Pérez-Muñoz A, Horn TL, Graber J, Chowdhury SMR, Bursac Z, Krukowski RA. Recruitment strategies for a post cessation weight management trial: a comparison of strategy cost-effectiveness and sample diversity. Contemp Clin Trials Commun. 2022;30:101037. · DOI
  14. Rodríguez-Torres E, González-Pérez MM, Díaz-Pérez C. Barriers and facilitators to the participation of subjects in clinical trials: an overview of reviews. Contemp Clin Trials Commun. 2021;23:100829. · DOI
  15. Scharff DP, Mathews KJ, Jackson P, Hoffsuemmer J, Martin E, Edwards D. More than Tuskegee: understanding mistrust about research participation. J Health Care Poor Underserved. 2010;21(3):879. · DOI · Paper page
  16. Adedinsewo D, Eberly L, Sokumbi O, Rodriguez JA, Patten CA, Brewer LC. Health disparities, clinical trials, and the digital divide thematic reviews on forward thinking on clinical trials in clinical practice. Mayo Clin Proc. 2023;98(12):1875–87. · DOI
  17. Anderson M. Mobile technology and home broadband. Pew Research Center. 2019.
  18. Roessler P. The mobile phone revolution and digital inequality: scope, determinants and consequences. Prosperity Comm Backgr Pap Ser. 2018(15):1–39.
  19. Venter W, Coleman J, Chan VL, Shubber Z, Phatsoane M, Gorgens M, et al. Improving linkage to HIV care through mobile phone apps: randomized controlled trial. JMIR Mhealth Uhealth. 2018;6(7):e8376. · DOI
  20. McVay MA, Lavoie HA, Rajoria M, Leong MC, Lou X, McMahon LN, et al. Pre-enrollment steps and run-ins in weight loss trials: a meta-regression. Am J Prev Med. 2023;64(6):910–7. · DOI
  21. Krukowski RA, Harvey-Berino J, Bursac Z, Ashikaga T, West DS. Patterns of success: online self-monitoring in a web-based behavioral weight control program. Health Psychol. 2013;32(2):164. · DOI · Paper page
  22. Harvey J, Krukowski R, Priest J, West D. Log often, lose more: Electronic dietary self-monitoring for weight loss. Obesity. 2019;27(3):380–4. · DOI · Paper page
  23. Devine S, Leeds C, Shlay JC, Leytem A, Beum R, Bull S. Methods to assess youth engagement in a text messaging supplement to an effective teen pregnancy program. J Biomed Inform. 2015;56:379–86. · DOI
  24. Patel MS, Bachireddy C, Small DS, Harrison JD, Harrington TO, Oon AL, et al. Effect of goal-setting approaches within a gamification intervention to increase physical activity among economically disadvantaged adults at elevated risk for major adverse cardiovascular events: the ENGAGE randomized clinical trial. JAMA cardiology. 2021;6(12):1387–96. · DOI
  25. König LM, Attig C, Franke T, Renner B. Barriers to and facilitators for using nutrition apps: systematic review and conceptual framework. JMIR Mhealth Uhealth. 2021;9(6):e20037. · DOI · Paper page
  26. Kalmbach DA, Cheng P, Reffi AN, Seymour GM, Ruprich MK, Bazan LF, et al. Racial disparities in treatment engagement and outcomes in digital cognitive behavioral therapy for insomnia among pregnant women. Sleep Health. 2023;9(1):18–25. · DOI
  27. Schüz B, Li AS-W, Hardinge A, McEachan RR, Conner M. Socioeconomic status as a moderator between social cognitions and physical activity: systematic review and meta-analysis based on the Theory of Planned Behavior. Psychol Sport Exerc. 2017;30:186–95. · DOI
  28. Geraghty AWA, Torres LD, Leykin Y, Pérez-Stable EJ, Muñoz RF. Understanding attrition from international internet health interventions: a step towards global eHealth. Health Promot Int. 2012;28(3):442–52. · DOI · Paper page
  29. Van der Mispel C, Poppe L, Crombez G, Verloigne M, De Bourdeaudhuij I. A self-regulation-based eHealth intervention to promote a healthy lifestyle: investigating user and website characteristics related to attrition. J Med Internet Res. 2017;19(7):e241. · DOI · Paper page
  30. Cheng P, Luik AI, Fellman-Couture C, Peterson E, Joseph CL, Tallent G, et al. Efficacy of digital CBT for insomnia to reduce depression across demographic groups: a randomized trial. Psychol Med. 2019;49(3):491–500. · DOI · Paper page
  31. Griffith Fillipo IR, Pullmann MD, Hull TD, Zech J, Wu J, Litvin B, et al. Participant retention in a fully remote trial of digital psychotherapy: Comparison of incentive types. Front Digit Health. 2022;4:963741. · DOI
  32. Khadjesari Z, Murray E, Kalaitzaki E, White IR, McCambridge J, Thompson SG, et al. Impact and costs of incentives to reduce attrition in online trials: two randomized controlled trials. J Med Internet Res. 2011;13(1):e26. · DOI · Paper page
  33. Kolovson S, Pratap A, Duffy J, Allred R, Munson SA, Areán PA. Understanding participant needs for engagement and attitudes towards passive sensing in remote digital health studies. Int Conf Pervasive Comput Technol Healthc. 2020;2020:347–62. · DOI
  34. Withall J, Greaves CJ, Thompson JL, de Koning JL, Bollen JC, Moorlock SJ, et al. The tribulations of trials: lessons learnt recruiting 777 older adults into REtirement in ACTion (REACT), a trial of a community, group-based active aging intervention targeting mobility disability. J Gerontol A. 2020;75(12):2387–95. · DOI
  35. Rick J, Graffy J, Knapp P, Small N, Collier DJ, Eldridge S, et al. Systematic techniques for assisting recruitment to trials (START): study protocol for embedded, randomized controlled trials. Trials. 2014;15(1):1–11. · DOI · Paper page
  36. Kim ES, Bruinooge SS, Roberts S, Ison G, Lin NU, Gore L, et al. Broadening eligibility criteria to make clinical trials more representative: American Society of Clinical Oncology and Friends of Cancer Research Joint Research Statement. J Clin Oncol. 2017;35(33):3737. · DOI · Paper page
  37. Miller SJ, Sly JR, Alcaraz KI, Ashing K, Christy SM, Gonzalez B, et al. Equity and behavioral digital health interventions: strategies to improve benefit and reach. Transl Behav Med. 2023;13(6):400–5. · DOI
  38. Skovlund PC, Nielsen BK, Thaysen HV, Schmidt H, Finset A, Hansen KA, et al. The impact of patient involvement in research: a case study of the planning, conduct and dissemination of a clinical, controlled trial. Res Involvement Engagement. 2020;6(1):1–16. · DOI
  39. Fahey MC, Talcott GW, Bauer CMC, Bursac Z, Gladney L, Hare ME, et al. Moms fit 2 fight: Rationale, design, and analysis plan of a behavioral weight management intervention for pregnant and postpartum women in the US military. Contemp Clin Trials. 2018;74:46–54. · DOI
  40. Estevez Burns R, Hare ME, Andres A, Klesges RC, Talcott GW, LeRoy K, et al. An interim analysis of a gestational weight gain intervention in military personnel and other TRICARE beneficiaries. Obesity. 2022;30(10):1951–62. · DOI
  41. Kristensen FB, Mäkelä M, Neikter SA, Rehnqvist N, Håheim LL, Mørland B, et al. European network for Health Technology Assessment, EUnetHTA: planning, development, and implementation of a sustainable European network for Health Technology Assessment. Int J Technol Assess Health Care. 2009;25(S2):107–16. · DOI
  42. Doherty L, Parker A, Arundel C, Clark L, Coleman E, Hewitt C, et al. PROMoting the use of studies within a trial (PROMETHEUS): results and experiences from a large programme to evaluate the routine embedding of recruitment and retention strategies within randomised controlled trials routinely. Res Methods Med Health Sci. 2023;4(3):113–22.
  43. Gültzow T, Neter E, Zimmermann H. Making Research Look Like the World Looks: Introducing the’Inclusivity & Diversity Add-On for Preregistration Forms’ Developed During an EHPS2022 Pre-Conference Workshop. 2023.
  44. Pebley K, Bursac Z, Klesges RC, Ebbert JO, Womack CR, Graber J, et al. A randomized controlled trial to reduce post-cessation weight gain. Int J Obes. 2023;47:1–8. · DOI

Cited by

See all 83

Metadata

Authors
Krukowski, R. A., Ross, K. M., Western, M. J., Cooper, R., Busse, H., Forbes, C., Kuntsche, E., Allmeta, A., Silva, A. M., John-Akinola, Y. O., & König, L. M.
Venue
Trials
Published
2024-01-30
Volume
25
Issue
1
ISSN
1745-6215
Cited by
83
References
8
Language
en
Licence
Funders
Universität Bayreuth (Centre of International Excellence "Alexander von Humboldt")

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Digital health interventions for all? Examining inclusivity across all stages of the digital health intervention research process. https://app.cochat.ai/papers/10.1186/s13063-024-07937-w
The paper
Krukowski, R. A., Ross, K. M., Western, M. J., Cooper, R., Busse, H., Forbes, C., Kuntsche, E., Allmeta, A., Silva, A. M., John-Akinola, Y. O., & König, L. M. (2024). Digital health interventions for all? Examining inclusivity across all stages of the digital health intervention research process. Trials. https://doi.org/10.1186/s13063-024-07937-w

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.