Assessing the Internal and External Validity of Mobile Health Physical Activity Promotion Interventions: A Systematic Literature Review Using the RE-AIM Framework
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Mobile health (mHealth) interventions are effective in promoting physical activity (PA); however, the degree to which external validity indicators are reported is unclear. OBJECTIVE: The purpose of this systematic review was to use the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to determine the extent to which mHealth intervention research for promoting PA reports on factors that inform generalizability across settings and populations and to provide recommendations for investigators planning to conduct this type of research. METHODS: Twenty articles reflecting 15 trials published between 2000 and 2012 were identified through a systematic review process (ie, queries of three online databases and reference lists of eligible articles) and met inclusion criteria (ie, implementation of mobile technologies, target physical activity, and provide original data). Two researchers coded each article using a validated RE-AIM data extraction tool (reach, efficacy/effectiveness, adoption, implementation, maintenance). Two members of the study team independently abstracted information from each article (inter-rater reliability >90%) and group meetings were used to gain consensus on discrepancies. RESULTS: The majority of studies were randomized controlled trials (n=14). The average reporting across RE-AIM indicators varied by dimension (reach=53.3%, 2.67/5; effectiveness/efficacy=60.0%, 2.4/4; adoption=11.1%, 0.7/6; implementation=24.4%, 0.7/3; maintenance=0%, 0/3). While most studies described changes in the primary outcome (effectiveness), few addressed the representativeness of participants (reach) or settings (adoption) and few reported on issues related to maintenance and degree of implementation fidelity. CONCLUSIONS: This review suggests that more focus is needed on research designs that highlight and report on both internal and external validity indicators. Specific recommendations are provided to encourage future mHealth interventionists and investigators to report on representativeness, settings, delivery agents for planned interventions, the extent to which protocol is delivered as intended, and maintenance of effects at the individual or organizational level.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Physical Activity in the United States Measured by Accelerometer
TROIANO, R. P., BERRIGAN, D., DODD, K. W., MÂSSE, L. C., TILERT, T., & MCDOWELL, M. (2008) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e31815a51b3 · Cited by 7,638 · cited 1×
“Likewise, it is a concern in the United States where only 6-11% of children [2] and 8.”
BackgroundEvaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · cited 10×
“Useful for replication and comparison of resources needed to resources available in a practice setting [26].”
BackgroundInfluentialHeron, K. E., & Smyth, J. M. (2010) · British Journal of Health Psychology · doi:10.1348/135910709x466063 · Cited by 1,438
Behavior change interventions delivered by mobile telephone short message service
Fjeldsoe, B. S., Marshall, A. L., & Miller, Y. D. (2009) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.09.040 · Cited by 1,213 · cited 2×
“Specifically, related to PA, mHealth interventions that deliver information and behavioral strategies through short message service (SMS) via mobile phones have been developed to increase PA [13-17].”
BackgroundPhysical Activity in U.S. Adults
Tucker, J. M., Welk, G. J., & Beyler, N. K. (2011) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2010.12.016 · Cited by 977 · cited 1×
“2% of adults meet the national PA guidelines based on objective PA assessments [3].”
BackgroundHealthcare via Cell Phones: A Systematic Review
Krishna, S., Boren, S. A., & Balas, E. A. (2009) · Telemedicine and e-Health · doi:10.1089/tmj.2008.0099 · Cited by 882
Glasgow, R. E., Klesges, L. M., Dzewaltowski, D. A., Bull, S. S., & Estabrooks, P. (2004) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2701_2 · Cited by 635
Self-citationMartínez-Pérez, B., de la Torre-Díez, I., & López-Coronado, M. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2600 · Cited by 550 · cited 1×
“This growth in mobile technology ownership has led to the development of a number of mobile health (mHealth) intervention reviews [9-12].”
BackgroundHurling, R., Catt, M., De Boni, M., Fairley, B. W., Hurst, T., Murray, P., Richardson, A., & Sodhi, J. S. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.2.e7 · Cited by 536
Meta-analysis of internet-delivered interventions to increase physical activity levels
Davies, C. A., Spence, J. C., Vandelanotte, C., Caperchione, C. M., & Mummery, W. (2012) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-9-52 · Cited by 488
Increasing Physical Activity With Mobile Devices: A Meta-Analysis
Fanning, J., Mullen, S. P., & McAuley, E. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2171 · Cited by 457
Self-citationDynamic adaptation process to implement an evidence-based child maltreatment intervention
Aarons, G. A., Green, A. E., Palinkas, L. A., Self-Brown, S., Whitaker, D. J., Lutzker, J. R., Silovsky, J. F., Hecht, D. B., & Chaffin, M. J. (2012) · Implementation Science · doi:10.1186/1748-5908-7-32 · Cited by 449 · cited 2×
“Organizational or delivery level facets of RE-AIM have consistently been underreported across behavior change intervention studies; yet, studies on mHealth PA interventions appear to be even less likely to report on organizational adoption, implementation, and maintenance [29,30,37-39].”
BackgroundFlowers or a robot army?: encouraging awareness & activity with personal, mobile displays
Consolvo, S., Klasnja, P., McDonald, D. W., Avrahami, D., Froehlich, J., LeGrand, L., Libby, R., Mosher, K., & Landay, J. A. (2008) · Proceedings of the 10th international conference on Ubiquitous computing · doi:10.1145/1409635.1409644 · Cited by 397 · cited 11×
“Other studies’ recruitment methods comprised obtaining names and contact information from pulmonary rehabilitation coordinators [51], contacting individuals on a weight research registry via letter or email [43-45], targeting previous trial participants [43-45], and using a market research recruitment agency [46,53].”
BackgroundMethodInfluentialInternet-based physical activity interventions: a systematic review of the literature.
van den Berg, M. H., Schoones, J. W., & Vliet Vlieland, T. P. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.3.e26 · Cited by 377 · cited 1×
“Similarly, a review of Internet-based PA interventions concluded that interactive technology interventions were effective for PA promotion [18].”
BackgroundBurke, L. E., Conroy, M. B., Sereika, S. M., Elci, O. U., Styn, M. A., Acharya, S. D., Sevick, M. A., Ewing, L. J., & Glanz, K. (2011) · Obesity · doi:10.1038/oby.2010.208 · Cited by 329 · cited 8×
“Nearly all studies (n=13) used mobile phones while two used personal digital assistants [43-45,53].”
BackgroundMethodInfluentialRapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise
Riley, W. T., Glasgow, R. E., Etheredge, L., & Abernethy, A. P. (2013) · Clinical and Translational Medicine · doi:10.1186/2001-1326-2-10 · Cited by 267 · cited 2×
“Some potential reasons for the lack of translation of these interventions into more widespread use are that the scientific approach typically emphasizes high internal validity at the expense of external validity [20] and that the traditional research pace impedes the flow of disseminating relevant findings [21].”
BackgroundBehavior change intervention research in healthcare settings
Glasgow, R. E., Bull, S. S., Gillette, C., Klesges, L. M., & Dzewaltowski, D. A. (2002) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(02)00437-3 · Cited by 181 · cited 4×
“Recommendations to improve the likelihood of broad dissemination of effective mHealth interventions are also provided based on the literature [24,27-38].”
BackgroundMethodResultInfluentialEfficacy of a cell phone-based exercise programme for COPD
Liu, W.-T., Wang, C.-H., Lin, H.-C., Lin, S.-M., Lee, K.-Y., Lo, Y.-L., Hung, S.-H., Chang, Y.-M., Chung, K. F., & Kuo, H.-P. (2008) · European Respiratory Journal · doi:10.1183/09031936.00104407 · Cited by 176 · cited 8×
“The least reported component was representativeness, with only four studies reporting [43-45,47-49,56,62].”
BackgroundMethodInfluentialPilot study of a cell phone-based exercise persistence intervention post-rehabilitation for COPD
Nguyen, H. (2009) · International Journal of Chronic Obstructive Pulmonary Disease · doi:10.2147/copd.s6643 · Cited by 165
Promoting physical activity through hand-held computer technology
King, A. C., Ahn, D. K., Oliveira, B. M., Atienza, A. A., Castro, C. M., & Gardner, C. D. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.09.025 · Cited by 160 · cited 12×
“In addition to English speaking, the most common inclusion criteria were PA requirement [46,50,53,54,56,58,59], weight requirement [43-45,53], or required possession/access to a mobile phone [46-49,54,56].”
BackgroundMethodInfluentialCan implementation intentions and text messages promote brisk walking? A randomized trial.
Prestwich, A., Perugini, M., & Hurling, R. (2010) · Health Psychology · doi:10.1037/a0016993 · Cited by 158
Pedometers and Text Messaging to Increase Physical Activity
Newton, K. H., Wiltshire, E. J., & Elley, C. R. (2009) · Diabetes Care · doi:10.2337/dc08-1974 · Cited by 140 · cited 9×
“None of the studies were categorized as high reporting quality, six studies were moderate (range 8-11) [43-45,51,52,56,58,59,62], and nine studies were of low reporting quality [46-50,53-55,57,60,61].”
BackgroundMethodInfluentialFukuoka, Y., Lindgren, T., & Jong, S. (2012) · Public Health Nursing · doi:10.1111/j.1525-1446.2011.00997.x · Cited by 49
Cited by
Vandelanotte, C., Müller, A. M., Short, C. E., Hingle, M., Nathan, N., Williams, S. L., Lopez, M. L., Parekh, S., & Maher, C. A. (2016) · Journal of Nutrition Education and Behavior · doi:10.1016/j.jneb.2015.12.006 · Cited by 396
A systematic review of just-in-time adaptive interventions (JITAIs) to promote physical activity
Hardeman, W., Houghton, J., Lane, K., Jones, A., & Naughton, F. (2019) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-019-0792-7 · Cited by 396
Methods for knowledge synthesis: An overview
Whittemore, R., Chao, A., Jang, M., Minges, K. E., & Park, C. (2014) · Heart & Lung · doi:10.1016/j.hrtlng.2014.05.014 · Cited by 349
Integrating Physical Activity in Primary Care Practice
AuYoung, M., Linke, S. E., Pagoto, S., Buman, M. P., Craft, L. L., Richardson, C. R., Hutber, A., Marcus, B. H., Estabrooks, P., & Sheinfeld Gorin, S. (2016) · The American Journal of Medicine · doi:10.1016/j.amjmed.2016.02.008 · Cited by 124
Self-citationvan Drongelen, A., Boot, C. R., Hlobil, H., Twisk, J. W., Smid, T., & van der Beek, A. J. (2014) · Scandinavian Journal of Work, Environment & Health · doi:10.5271/sjweh.3447 · Cited by 101
Metadata
- Authors
- Blackman, K. C., Zoellner, J., Berrey, L. M., Alexander, R., Fanning, J., Hill, J. L., & Estabrooks, P. A.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2013-10-04
- Volume
- 15
- Issue
- 10
- Pages
- e224
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2037935612
- Cited by
- 98
- References
- 23
- FWCI
- 1.55
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.