CoChat
Sign Up
Cited by 1,042Evaluation and Performance AssessmentHealth Policy Implementation ScienceMeta-analysis and systematic reviews

Criteria for evaluating evidence on public health interventions

Open access
Rychetnik, L, Frommer, M, Hawe, P, Shiell, A·2002·Journal of Epidemiology and Community Health
DOI 10.1136/jech.56.2.119PMC PMC1732065
Open this paper in CoChat — free accountOpen PDF

Abstract

Public health interventions tend to be complex, programmatic, and context dependent. The evidence for their effectiveness must be sufficiently comprehensive to encompass that complexity. This paper asks whether and to what extent evaluative research on public health interventions can be adequately appraised by applying well established criteria for judging the quality of evidence in clinical practice. It is adduced that these criteria are useful in evaluating some aspects of evidence. However, there are other important aspects of evidence on public health interventions that are not covered by the established criteria. The evaluation of evidence must distinguish between the fidelity of the evaluation process in detecting the success or failure of an intervention, and the success or failure of the intervention itself. Moreover, if an intervention is unsuccessful, the evidence should help to determine whether the intervention was inherently faulty (that is, failure of intervention concept or theory), or just badly delivered (failure of implementation). Furthermore, proper interpretation of the evidence depends upon the availability of descriptive information on the intervention and its context, so that the transferability of the evidence can be determined. Study design alone is an inadequate marker of evidence quality in public health intervention evaluation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/jech.56.2.119

Cites

122 of 122 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

2 of the 124 references couldn’t be matched to a known paper.

Research works printed but not linked: 10

  1. Hawe P. How much trial and error should we tolerate in community trials? [Letter]. BMJ 2000;320:119.
  2. Thomas H, Siracusa L, Ross G, et al. Effectiveness of school-based interventions in reducing adolescent risk behaviour: a systematic review of reviews. Prepared by the Effective Public Health Practice Project for the Public Health Branch. Ontario Ministry of Health, Canada, 1999.
  3. Mulrow CD, Oxman AD, eds. Cochrane Collaboration Reviewers Handbook. In: The Cochrane Library. The Cochrane Collaboration. Oxford: Update Software; Issue 4, 1999 or online: http:// www.update-software.com/ccweb/cochrane/hbook.htm
  4. The Cochrane Collaboration Methods Groups Newsletter 2001;5:16. (http://www.cochrane.de/newslett/mgnews5.pdf)
  5. Deeks JJ, D'Amico, Sakarovitch C, et al. How big are the biases associated with non-randomised designs used in evaluations of healthcare interventions? An empirical investigation. Abstract from paper oral presentation at the 3rd Oxford Symposium on Systematic Reviews, July 2000. (http://www.ihs.ox.ac.uk/csm/Oralabstract2K.htm#Deeks1) 59 Reference withdrawn.
  6. Commonwealth of Australia. Guidelines for pharmaceutical industry on preparation of submission to the Pharmaceutical Benefits Advisory Rychetnik, Frommer, Hawe, et al www.jech.com on August 20, 2023 by guest. Protected by copyright. http://jech.bmj.com/ J Epidemiol Community Health: first published as 10.1136/jech.56.2.119 on 1 February 2002. Downloaded from Committee: including submissions involving economic analyses. Canberra: Australian Government Printing Office, 1992.
  7. Department of Health guidelines for the economic evaluation of pharmaceuticals [press release 94/251]. London: Department of Health, 1994.
  8. Task Force on Principles for Economic Analysis of Health Care Technology. Economic analysis of health care technology. A report on principles. Ann Intern Med 1995;122:61-70.
  9. Woodward A, Blakely T, Gander P, et al. Regulating public health in New Zealand. Healthcare Review online 1999;3 or www.enigma.co.nz.hcro 113
  10. Wooldridge M. The importance of evidence-based medicine. Address by Honourable Dr Michael Wooldridge (Federal Health Minister). Opening of The Royal Australasian College of Physicians Annual Scientific Conference, Canberra. Fellowship Affairs 1996;September:29. 115

Cited by

Sorted by citations

Related

Authors
Rychetnik, L., Frommer, M., Hawe, P., & Shiell, A.
Affiliations
  • The University of Sydney (AU)
  • University of Calgary (CA)
Corresponding author
Rychetnik, L
ORCID
Venue
Journal of Epidemiology and Community Health
Publisher
BMJ
Published
2002-02
Volume
56
Issue
2
Pages
119-127
ISSN
  • 0141-7681 (ISSN-L)
  • 0143-005X
  • 1470-2738
DOI
10.1136/jech.56.2.119
PMC
PMC1732065
OpenAlex
W2125451304
Cited by
1,042
References
124
Citations by year
2026: 17 · 2025: 30 · 2024: 36 · 2023: 35 · 2022: 31 · 2021: 45 · 2020: 66 · 2019: 31 · 2018: 36 · 2017: 61 · 2016: 61 · 2015: 55 · 2014: 81 · 2013: 55 · 2012: 51
FWCI
10.13
Type
journal article
Language
en
Open access
Locations
  • Journal of Epidemiology & Community Health (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
public health interventions, intervention evaluation, evidence appraisal, implementation failure, research design
MeSH terms
Health Services Research, Humans, Program Evaluation, Public Health Practice, Treatment Outcome, Evidence-Based Medicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review