The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Implementation and scale-up of physical activity and behavioural nutrition interventions: an evaluation roadmap
Open accessAbstract
Background: Interventions that work must be effectively delivered at scale to achieve population level benefits. Researchers must choose among a vast array of implementation frameworks (> 60) that guide design and evaluation of implementation and scale-up processes. Therefore, we sought to recommend conceptual frameworks that can be used to design, inform, and evaluate implementation of physical activity (PA) and nutrition interventions at different stages of the program life cycle. We also sought to recommend a minimum data set of implementation outcome and determinant variables (indicators) as well as measures and tools deemed most relevant for PA and nutrition researchers. Methods: We adopted a five-round modified Delphi methodology. For rounds 1, 2, and 3 we administered online surveys to PA and nutrition implementation scientists to generate a rank order list of most commonly used; i) implementation and scale-up frameworks, ii) implementation indicators, and iii) implementation and scale-up measures and tools. Measures and tools were excluded after round 2 as input from participants was very limited. For rounds 4 and 5, we conducted two in-person meetings with an expert group to create a shortlist of implementation and scale-up frameworks, identify a minimum data set of indicators and to discuss application and relevance of frameworks and indicators to the field of PA and nutrition. Results: The two most commonly referenced implementation frameworks were the Framework for Effective Implementation and the Consolidated Framework for Implementation Research. We provide the 25 most highly ranked implementation indicators reported by those who participated in rounds 1–3 of the survey. From these, the expert group created a recommended minimum data set of implementation determinants (n = 10) and implementation outcomes (n = 5) and reconciled differences in commonly used terms and definitions. Conclusions: Researchers are confronted with myriad options when conducting implementation and scale-up evaluations. Thus, we identified and prioritized a list of frameworks and a minimum data set of indicators that have potential to improve the quality and consistency of evaluating implementation and scale-up of PA and nutrition interventions. Advancing our science is predicated upon increased efforts to develop a common ‘language’ and adaptable measures and tools.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“A range of potential outcomes that could be used to assess implementation strategy effects has been identified, including measures of adoption, reach, fidelity and sustainability [46].”
From Guidance for conducting feasibility and pilot studies for implementation trials (2020) · cites this paper as [46]
The cited source says what the claim attributes to it.
In the citing paper
“McKay and colleagues have also identified a priority set of determinants for implementation trials that could be considered for use in implementation feasibility and pilot studies, including measures of context, acceptability, adaptability, feasibility, compatibility, cost, culture, dose, complexity and self-efficacy [46].”
From Guidance for conducting feasibility and pilot studies for implementation trials (2020) · cites this paper as [46]
The cited source says what the claim attributes to it.
In the citing paper
“Recent work by McKay and colleagues suggests a minimum data set of implementation outcomes that includes measures of adoption, reach, dose, fidelity and sustainability [46].”
From Guidance for conducting feasibility and pilot studies for implementation trials (2020) · cites this paper as [46]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We focus on randomised trials as such designs are the most common trial design in implementation research, and randomised designs may provide the most robust estimates of the potential effect of implementation strategies [46].”
From Guidance for conducting feasibility and pilot studies for implementation trials (2020) · cites this paper as [46]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We focus on randomised trials as such designs are the most common trial design in implementation research, and randomised designs may provide the most robust estimates of the potential effect of implementation strategies [46].”
From Guidance for conducting feasibility and pilot studies for implementation trials (2020) · cites this paper as [46]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Self-efficacy: The exercise of control
Bandura, A., Freeman, W. H., & Lightsey, R. (1999) · Journal of Cognitive Psychotherapy · doi:10.1891/0889-8391.13.2.158 · Cited by 46,835
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · An Integrated Approach to Communication Theory and Research · doi:10.4324/9780203710753-35 · Cited by 37,322
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496
Michie, S., van Stralen, M. M., & West, R. (2011) · Implementation Science · doi:10.1186/1748-5908-6-42 · Cited by 13,697
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721
Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217
Stanton, B. F., & Rogers, E. M. (1963) · Journal of Farm Economics · doi:10.2307/1235767 · Cited by 6,675
the delphi method: techniques and applications
Linstone, H. A., & Turoff, M. (1975) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 6,591
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951
Durlak, J. A., & DuPre, E. P. (2008) · American Journal of Community Psychology · doi:10.1007/s10464-008-9165-0 · Cited by 5,304
Research works printed but not linked: 4
- Centre for Disease Control and Prevention. Developing Evaluation Indicators. Program Performance and Evaluation Office; 2016.
- World Health Organization, ExpandNet. Nine steps for developing a scaling-up strategy. France Research DoRHa; 2010.
- Rogers EM. Diffusion of innovations. 5th edition ed. New York: The Free Press, a Division of Macmillan Publishing Co., Inc.; 2003.
- Fixsen DL, Blase KA, Van Dyke MK. Chapter 2: Science and Implementation. In: Implementation Practice & Science. Chapel Hill: Active Implementation Research Network; 2019. p. 14–6.
Cited by
Physical activity behaviours in adolescence: Current evidence and opportunities for intervention
van Sluijs, E. M. F., Ekelund, U., Crochemore-Silva, I., Guthold, R., Ha, A., Lubans, D., Oyeyemi, A. L., Ding, D., & Katzmarzyk, P. T. (2021) · The Lancet · doi:10.1016/s0140-6736(21)01259-9 · Cited by 919
Guidance for conducting feasibility and pilot studies for implementation trials
Pearson, N., Naylor, P.-J., Ashe, M. C., Fernandez, M., Yoong, S. L., & Wolfenden, L. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00634-w · Cited by 475
Self-citationDesigning and undertaking randomised implementation trials: guide for researchers
Wolfenden, L., Foy, R., Presseau, J., Grimshaw, J. M., Ivers, N. M., Powell, B. J., Taljaard, M., Wiggers, J., Sutherland, R., Nathan, N., et al. (2021) · BMJ · doi:10.1136/bmj.m3721 · Cited by 270
Self-citationCooper, J., Murphy, J., Woods, C., Van Nassau, F., McGrath, A., Callaghan, D., Carroll, P., Kelly, P., Murphy, N., Murphy, M., et al. (2021) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-021-01177-w · Cited by 114
Self-citationGray, S. M., McKay, H. A., Nettlefold, L., Race, D., Macdonald, H. M., Naylor, P.-J., & Sims-Gould, J. (2021) · British Journal of Sports Medicine · doi:10.1136/bjsports-2020-102465 · Cited by 35
Self-citation
Metadata
- Authors
- McKay, H., Naylor, P.-J., Lau, E., Gray, S. M., Wolfenden, L., Milat, A., Bauman, A., Race, D., Nettlefold, L., & Sims-Gould, J.
- Affiliations
- Vancouver Coastal Health (CA)
- University of Victoria (CA)
- Hunter New England Local Health District (AU)
- The University of Sydney (AU)
- Corresponding author
- McKay, Heather
- ORCID
- Venue
- International Journal of Behavioral Nutrition and Physical Activity
- Published
- 2019-11-07
- Volume
- 16
- Issue
- 1
- Pages
- 102-102
- ISSN
- 1479-5868 (ISSN-L)
- PMC
- PMC6839114
- OpenAlex
- W2983079051
- Cited by
- 169
- References
- 56
- Citations by year
- 2026: 14 · 2025: 22 · 2024: 28 · 2023: 42 · 2022: 21 · 2021: 20 · 2020: 22
- FWCI
- 21.44
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Locations
- International Journal of Behavioral Nutrition and Physical Activity (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- cIRcle (University of British Columbia) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- UVic’s Research and Learning Repository (University of Victoria) (submittedVersion, cc-by)
- Figshare (submittedVersion, cc-by)
- NOVA (University of Newcastle Australia) (submittedVersion, cc-by)
- Funders
- Canadian Institutes of Health Research (392349, New Investigator Award)
- Michael Smith Foundation for Health Research (Scholar Award)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Physical Activity and Health
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- implementation research, scale-up, physical activity interventions, nutrition interventions, implementation framework, implementation outcomes, implementation factors, Minimum Data Set, Consolidated Framework for Implementation Research, evaluation metrics
- MeSH terms
- Health Promotion, Humans, Surveys and Questionnaires, Research Design, Health Behavior, Exercise
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Implementation and scale-up of physical activity and behavioural nutrition interventions: an evaluation roadmap. https://app.cochat.ai/papers/10.1186/s12966-019-0868-4
- The paper
- McKay, H., Naylor, P.-J., Lau, E., Gray, S. M., Wolfenden, L., Milat, A., Bauman, A., Race, D., Nettlefold, L., & Sims-Gould, J. (2019). Implementation and scale-up of physical activity and behavioural nutrition interventions: an evaluation roadmap. International Journal of Behavioral Nutrition and Physical Activity. https://doi.org/10.1186/s12966-019-0868-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
