Real world reviews: a beginner's guide to undertaking systematic reviews of public health policy interventions
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Background: The systematic review is becoming an increasingly popular and established research method in public health. Obtaining systematic review skills are therefore becoming a common requirement for most public health researchers and practitioners. However, most researchers still remain apprehensive about conducting their first systematic review. This is often because an ‘ideal’ type of systematic review is promoted in the methods literature. Methods: This brief guide is intended to help dispel these concerns by providing an accessible overview of a ‘real’ approach to conducting systematic reviews. The guide draws upon an extensive practical experience of conducting various types of systematic reviews of complex social interventions. Results: The paper discusses what a systematic review is and how definitions vary. It describes the stages of a review in simple terms. It then draws on case study reviews to reflect on five key practical aspects of the conduct of the method, outlining debates and potential ways to make the method shorter and smarter—enhancing the speed of production of systematic reviews and reducing labour intensity while still maintaining high methodological standards. Conclusion: There are clear advantages in conducting the high quality pragmatic reviews that this guide has described: (1) time and labour resources are saved; (2) it enables reviewers to inform or respond to developments in policy and practice in a timelier manner; and (3) it encourages researchers to conduct systematic reviews before embarking on primary research. Well-conducted systematic reviews remain a valuable part of the public health methodological tool box.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Systematic Reviews in the Social Sciences: A Practical Guide
Petticrew, M., & Roberts, H. (2006) · doi:10.1002/9780470754887 · Cited by 3,586
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907 · cited 1×
“qualitative evidence.[39, 40] Another approach would be to examine the tools used by existing SRs in a similar subject area and adapt them.”
BackgroundSystematic Reviews: Rationale for systematic reviews
Mulrow, C. D. (1994) · BMJ · doi:10.1136/bmj.309.6954.597 · Cited by 1,677
Egger, M., Jüni, P., Bartlett, C., Holenstein, F., & Sterne, J. (2003) · Health Technology Assessment · doi:10.3310/hta7010 · Cited by 1,181
Popay, J., Rogers, A., & Williams, G. (1998) · Qualitative Health Research · doi:10.1177/104973239800800305 · Cited by 987
Systematic Reviews from Astronomy to Zoology: Myths and Misconceptions
Petticrew, M. (2001) · BMJ · doi:10.1136/bmj.322.7278.98 · Cited by 550
Evidence, hierarchies, and typologies: horses for courses: Table 1
Petticrew, M., & Roberts, H. (2003) · Journal of Epidemiology and Community Health · doi:10.1136/jech.57.7.527 · Cited by 539
Ogilvie, D., Egan, M., Hamilton, V., & Petticrew, M. (2005) · Journal of Epidemiology and Community Health · doi:10.1136/jech.2005.034199 · Cited by 167
Research works printed but not linked: 24
- Solesbury, 2002, The ascendancy of evidence, Plann Theory Practice, 3, 90, 10.1080/14649350220117834 · DOI
- Klein, 2000, From evidence based medicine to evidence based policy?, J Health Serv Res Policy, 5, 65, 10.1177/135581960000500201 · DOI
- Davies H Nutley S Smith P . What works? Evidence-based policy and practice in public services. Bristol: Policy Press, 2000. · DOI · Paper page
- Higgins JP Green S . Cochrane handbook for systematic reviews of interventions. Cambridge: Wiley Blackwell, 2008. · DOI · Paper page
- Egger M Davey Smith G Altman D , eds. Systematic reviews in health care: meta-analysis in context. Cambridge: Wiley Blackwell, 2001. · DOI · Paper page
- 10.1016/j.socscimed.2004.09.002 · DOI · Paper page
- 10.1136/jech.2006.054965 · DOI · Paper page
- 10.1136/jech.2006.054999 · DOI · Paper page
- 10.1016/j.amepre.2007.12.023 · DOI
- 10.1136/jech.2007.067249 · DOI · Paper page
- Smith KE Bambra C Joyce KE . Partners in health? A systematic review of the effects on health and health inequalities of partnership working. J Public Health 2009;210–21. · DOI · Paper page
- Bambra, 2005, Reviewing the evidence: reflections from experience, Evidence Policy – Journal, 1, 241
- Egan, 2009, Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of the health effects of organisational-level workplace interventions, J Epidemiol Community Health, 61, 4, 10.1136/jech.2007.071233 · DOI · Paper page
- Oakley A Fullerton D . ‘The lamp-post of research: support or illumination’? In: Oakley A Roberts H , eds. Evaluating social interventions: a report of two workshops funded by the Economic and Social Research Council. Ilford: Barnardos, 1998:4–38.
- 10.1016/j.ypmed.2008.12.018 · DOI
- 10.1136/jech.57.5.327 · DOI · Paper page
- 10.1186/1471-2458-8-178 · DOI · Paper page
- 10.1016/S0749-3797(00)00137-9 · DOI
- 10.1136/bmj.322.7280.204 · DOI · Paper page
- 10.1136/ip.9.3.200 · DOI · Paper page
- 10.1136/bmj.38216.714560.55 · DOI · Paper page
- Egan, 2003, New roads and human health: a systematic review, Am J Public Health, 93, 1463, 10.2105/AJPH.93.9.1463 · DOI · Paper page
- 10.1136/jech.2005.034181 · DOI · Paper page
- Petticrew, 2003, Why certain systematic reviews reach uncertain conclusions, BMJ, 7392, 756, 10.1136/bmj.326.7392.756 · DOI · Paper page
Cited by
Cairns, G., Angus, K., Hastings, G., & Caraher, M. (2013) · Appetite · doi:10.1016/j.appet.2012.04.017 · Cited by 680
Effects of physical exercise on Autism Spectrum Disorders: A meta-analysis
Sowa, M., & Meulenbroek, R. (2012) · Research in Autism Spectrum Disorders · doi:10.1016/j.rasd.2011.09.001 · Cited by 386
Flexible working conditions and their effects on employee health and wellbeing
Joyce, K., Pabayo, R., Critchley, J. A., & Bambra, C. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008009.pub2 · Cited by 325
Self-citationNoyes, J., Booth, A., Lewin, S., Carlsen, B., Glenton, C., Colvin, C. J., Garside, R., Bohren, M. A., Rashidian, A., Wainwright, M., et al. (2018) · Implementation Science · doi:10.1186/s13012-017-0693-6 · Cited by 266
Katikireddi, S. V., Egan, M., & Petticrew, M. (2015) · Journal of Epidemiology and Community Health · doi:10.1136/jech-2014-204711 · Cited by 126
Metadata
- Authors
- Bambra, C.
- Venue
- Journal of Epidemiology and Community Health
- Publisher
- BMJ
- Published
- 2009-08-25
- Volume
- 65
- Issue
- 1
- Pages
- 14-19
- ISSN
- 0141-7681 (ISSN-L)
- 0143-005X
- 1470-2738
- OpenAlex
- W2142019074
- Cited by
- 50
- References
- 8
- FWCI
- 0.23
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.