Behavior change intervention research in community settings: how generalizable are the results?
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
This review examines the extent to which recent behavioral intervention studies conducted in community settings reported on elements of internal and external validity, with an emphasis on whether research has been conducted in representative settings with representative populations. A targeted review was conducted on community-based intervention studies that promoted good nutrition, physical activity or smoking cessation/prevention, and were published in 11 leading health behavior journals between 1996 and 2000. The RE-AIM framework (reach, efficacy, adoption, implementation and maintenance) was used to evaluate the extent to which each paper reported on elements of reach, efficacy/effectiveness, adoption, implementation and maintenance. A total of 27 publications were reviewed. Although most studies (88%) reported participation rates among eligible members of the target audience ('reach'), only 11% of studies reported the participation rate ('adoption') among eligible community-based organizations or settings. Few studies reported if participating individuals or settings were representative of those found in the broader population. Although a majority of studies (59%) reported whether the intervention was delivered ('implementation'), few reported whether individuals maintained behavior change (30%) or whether organizations maintained or institutionalized interventions (0%). To increase the potential to translate community research findings to practice, studies should place a greater emphasis on obtaining and reporting external validity information, such as representativeness. The lack of external validity information limits researchers' and practitioners' ability to judge the generalizability of effects and the comparative utility of interventions. Improved reporting will facilitate implementation of proven and broadly applicable intervention strategies in communities. To make significant progress, all parties, including researchers, reviewers, editors and funders, need to take responsibility for increased emphasis on external validity information and ask what role they can best play to facilitate this process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · cited 2×
“The Kappa statistic (Cohen, 1968), which adjusts for chance agreement across all coding categories, was in the excellent range at 0.87.”
BackgroundInfluentialEvaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · cited 4×
“The setting level definition of maintenance refers to the institutionalization of a program and is assessed according to the percent of settings that continue the intervention program, in part or in whole, beyond the study duration (Glasgow et al., 1999; Glasgow et al., 2001).”
BackgroundMethodInfluentialMoher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Flay, B. R. (1986) · Preventive Medicine · doi:10.1016/0091-7435(86)90024-1 · Cited by 723
Efficacy of an individualized, motivationally-tailored physical activity intervention
Marcus, B. H., Bock, B. C., Pinto, B. M., Forsyth, L. A. H., Roberts, M. B., & Traficante, R. M. (1998) · Annals of Behavioral Medicine · doi:10.1007/bf02884958 · Cited by 407
Reduction in Cardiovascular Disease Risk Factors: 6-Month Results from ProjectActive
Dunn, A. L., Marcus, B. H., Kampert, J. B., Garcia, M. E., Kohl, H. W., & Blair, S. N. (1997) · Preventive Medicine · doi:10.1006/pmed.1997.0218 · Cited by 262
Behavior change intervention research in healthcare settings
Glasgow, R. E., Bull, S. S., Gillette, C., Klesges, L. M., & Dzewaltowski, D. A. (2002) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(02)00437-3 · Cited by 181 · cited 2×
“We excluded interventions that were delivered through schools, worksites or health care settings, as this work has been reported elsewhere (Bull et al., 2003; Estabrooks et al., 2002; Glasgow et al., 2002).”
BackgroundMethodSelf-citationSettings as an Important Dimension in Health Education/Promotion Policy, Programs, and Research
Mullen, P. D., Evans, D., Forster, J., Gottlieb, N. H., Kreuter, M., Moon, R., O'Rourke, T., & Strecher, V. J. (1995) · Health Education Quarterly · doi:10.1177/109019819402200306 · Cited by 115 · cited 1×
“This definition suggests that community-based strategies and initiatives are implemented in the major social structures that provide channels to reach and influence defined populations (Mullen et al., 1995).”
BackgroundThe Effects of a Telephone-Based Intervention on Weight Loss
Hellerstedt, W. L., & Jeffery, R. W. (1997) · American Journal of Health Promotion · doi:10.4278/0890-1171-11.3.177 · Cited by 44
Cited by
(2009) · Cited by 2,013
Reis, R. S., Salvo, D., Ogilvie, D., Lambert, E. V., Goenka, S., & Brownson, R. C. (2016) · The Lancet · doi:10.1016/s0140-6736(16)30728-0 · Cited by 719
Glasgow, R. E., Klesges, L. M., Dzewaltowski, D. A., Bull, S. S., & Estabrooks, P. (2004) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2701_2 · Cited by 635
Self-citationImplementation of school based physical activity interventions: A systematic review
Naylor, P.-J., Nettlefold, L., Race, D., Hoy, C., Ashe, M. C., Wharf Higgins, J., & McKay, H. A. (2015) · Preventive Medicine · doi:10.1016/j.ypmed.2014.12.034 · Cited by 550
Determinants of participation in worksite health promotion programmes: a systematic review
Robroek, S. J., van Lenthe, F. J., van Empelen, P., & Burdorf, A. (2009) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-6-26 · Cited by 465
Klesges, L. M., Estabrooks, P. A., Dzewaltowski, D. A., Bull, S. S., & Glasgow, R. E. (2005) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2902s_10 · Cited by 324
Self-citationA sports setting matrix for understanding the implementation context for community sport
Finch, C. F., & Donaldson, A. (2010) · British Journal of Sports Medicine · doi:10.1136/bjsm.2008.056069 · Cited by 307
Reporting Guidelines for Music-based Interventions
Robb, S. L., Carpenter, J. S., & Burns, D. S. (2011) · Journal of Health Psychology · doi:10.1177/1359105310374781 · Cited by 261
Internal and external validity of cluster randomised trials: systematic review of recent trials
Eldridge, S., Ashby, D., Bennett, C., Wakelin, M., & Feder, G. (2008) · BMJ · doi:10.1136/bmj.39517.495764.25 · Cited by 224
Applying the RE-AIM framework to assess the public health impact of policy change
Jilcott, S., Ammerman, A., Sommers, J., & Glasgow, R. E. (2007) · Annals of Behavioral Medicine · doi:10.1007/bf02872666 · Cited by 162
“However, its application to date has been largely to individually-based interventions (42,43) or to organizational-level programs such as schools (44), worksites (45), or communities ( 46 ).”
BackgroundMcGoey, T., Root, Z., Bruner, M. W., & Law, B. (2016) · Preventive Medicine · doi:10.1016/j.ypmed.2015.11.004 · Cited by 93
Personalizing behavioral interventions through single-patient (N-of-1) trials
Davidson, K. W., Peacock, J., Kronish, I. M., & Edmondson, D. (2014) · Social and Personality Psychology Compass · doi:10.1111/spc3.12121 · Cited by 86
Horodyska, K., Luszczynska, A., Hayes, C. B., O’Shea, M. P., Langøien, L. J., Roos, G., van den Berg, M., Hendriksen, M., De Bourdeaudhuij, I., & Brug, J. (2015) · BMC Public Health · doi:10.1186/s12889-015-2585-5 · Cited by 80
“Extent to which protocol was delivered as intended/protocol adherence Systematic reviews [45, 47, 99, 125]; Position reviews [52, 128].”
BackgroundMethodInfluentialLe Strat, Y., Rehm, J., & Le Foll, B. (2011) · Tobacco Control · doi:10.1136/tc.2010.038703 · Cited by 67
McGoey, T., Root, Z., Bruner, M. W., & Law, B. (2015) · Preventive Medicine · doi:10.1016/j.ypmed.2015.04.006 · Cited by 61
Hoertel, N., Le Strat, Y., Blanco, C., Lavaud, P., & Dubertret, C. (2012) · Depression and Anxiety · doi:10.1002/da.21937 · Cited by 61
Evans, K., Morrell, C. J., & Spiby, H. (2018) · Journal of Advanced Nursing · doi:10.1111/jan.13456 · Cited by 33
Generalizability of Clinical Trial Results for Adolescent Major Depressive Disorder
Blanco, C., Hoertel, N., Franco, S., Olfson, M., He, J.-P., López, S., González-Pinto, A., Limosin, F., & Merikangas, K. R. (2017) · Pediatrics · doi:10.1542/peds.2016-1701 · Cited by 29
Metadata
- Authors
- Dzewaltowski, D. A.
- Venue
- Health Promotion International
- Publisher
- Oxford University Press (OUP)
- Published
- 2004-06-01
- Volume
- 19
- Issue
- 2
- Pages
- 235-245
- ISSN
- 0957-4824 (ISSN-L)
- 1460-2245
- OpenAlex
- W2099104857
- Cited by
- 164
- References
- 9
- FWCI
- 18.47
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.