Generalizing about Public Health Interventions: A Mixed-Methods Approach to External Validity
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Public health researchers and practitioners are calling for greater focus on external validity, the ability to generalize findings of evidence-based interventions (EBIs) beyond the limited number of studies testing effectiveness. For public health, the goal is applicability: to translate, disseminate, and implement EBIs for an impact on population health. This article is a review of methods and how they might be combined to better assess external validity. The methods include (a) better description of EBIs and their contexts; (b) combining of statistical tools and logic to draw inferences about study samples; (c) sharper definition of the theory behind the intervention and core intervention components; and (d) more systematic consultation of practitioners. For population impact, studies should focus on context features that are likely to be both important (based on program theory) and frequently encountered by practitioners. Mixed-method programs of research will allow public health to expand causal generalizations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548
Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · cited 1×
“These are the areas in which expanding on generalizations will have the largest impact on public health (36, 45, 54).”
BackgroundNewman (2005) · doi:10.1080/00107510500052444 · Cited by 5,749
Durlak, J. A., & DuPre, E. P. (2008) · American Journal of Community Psychology · doi:10.1007/s10464-008-9165-0 · Cited by 5,304
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544 · cited 1×
“In medicine, the details of EBIs and their contexts are often so poor that clinicians cannot possibly judge whether the results apply to particular kinds of patients (37, 72).”
BackgroundGlanz, K., & Bishop, D. B. (2010) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.012809.103604 · Cited by 2,336
The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change
Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-117 · Cited by 1,724
Dane, A. V., & Schneider, B. H. (1998) · Clinical Psychology Review · doi:10.1016/s0272-7358(97)00043-3 · Cited by 1,557 · cited 1×
“Unfortunately, program theory appears to be badly underdeveloped in many outcome studies of prevention interventions for physical and mental health, as seen in two systematic reviews (27, 30), as well as a scan of EBIs for violence prevention (67).”
BackgroundCoproduction of healthcare service
Batalden, M., Batalden, P., Margolis, P., Seid, M., Armstrong, G., Opipari-Arrigan, L., & Hartung, H. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004315 · Cited by 1,078 · cited 1×
“(5) cite a long tradition of research in business and public administration to support their distinction between health care as a uniform product (e.”
BackgroundGreen, L. W., & Glasgow, R. E. (2006) · Evaluation & the Health Professions · doi:10.1177/0163278705284445 · Cited by 922 · cited 5×
“Tracy Costigan, Randall Brown, Frank Davidoff, and Sharon Wilson gave valuable suggestions and comments; Thomas Cook, John Ovretveit, Gareth Parry, and Matt Trujillo contributed to related papers on external validity; and Robert Wood Johnson Foundation grantees are implementing research on local adaptation that contri…”
BackgroundInfluential
Cited by
Diffusion Of Innovations Theory, Principles, And Practice
Dearing, J. W., & Cox, J. G. (2018) · Health Affairs · doi:10.1377/hlthaff.2017.1104 · Cited by 757
Selecting and Improving Quasi-Experimental Designs in Effectiveness and Implementation Research
Handley, M. A., Lyles, C. R., McCulloch, C., & Cattamanchi, A. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014128 · Cited by 428
Brownson, R. C., Fielding, J. E., & Green, L. W. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014746 · Cited by 396
von Thiele Schwarz, U., Förberg, U., Sundell, K., & Hasson, H. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0496-8 · Cited by 20
Roura, S., Alvarez, G., Hohenschurz-Schmidt, D., Solà, I., Núñez-Cortés, R., Bracchiglione, J., Fernández-Jané, C., Phalip, J., Gich, I., Sitjà-Rabert, M., et al. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02393-1 · Cited by 2
Metadata
- Authors
- Leviton, L. C.
- Venue
- Annual Review of Public Health
- Publisher
- Annual Reviews
- Published
- 2017-03-20
- Volume
- 38
- Issue
- 1
- Pages
- 371-391
- ISSN
- 0163-7525 (ISSN-L)
- 1545-2093
- OpenAlex
- W2576394042
- Cited by
- 55
- References
- 25
- FWCI
- 1.41
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generalizing about Public Health Interventions: A Mixed-Methods Approach to External Validity. https://app.cochat.ai/papers/10.1146/annurev-publhealth-031816-044509
- The paper
- Leviton, L. C. (2017). Generalizing about Public Health Interventions: A Mixed-Methods Approach to External Validity. Annual Review of Public Health. https://doi.org/10.1146/annurev-publhealth-031816-044509
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.