The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary
Open accessAbstract
BACKGROUND: Overviews of reviews (overviews) compile data from multiple systematic reviews to provide a single synthesis of relevant evidence for decision-making. Despite their increasing popularity, there is limited methodological guidance available for researchers wishing to conduct overviews. The objective of this scoping review is to identify and collate all published and unpublished documents containing guidance for conducting overviews examining the efficacy, effectiveness, and/or safety of healthcare interventions. Our aims were to provide a map of existing guidance documents; identify similarities, differences, and gaps in the guidance contained within these documents; and identify common challenges involved in conducting overviews. METHODS: We conducted an iterative and extensive search to ensure breadth and comprehensiveness of coverage. The search involved reference tracking, database and web searches (MEDLINE, EMBASE, DARE, Scopus, Cochrane Methods Studies Database, Google Scholar), handsearching of websites and conference proceedings, and contacting overview producers. Relevant guidance statements and challenges encountered were extracted, edited, grouped, abstracted, and presented using a qualitative metasummary approach. RESULTS: We identified 52 guidance documents produced by 19 research groups. Relatively consistent guidance was available for the first stages of the overview process (deciding when and why to conduct an overview, specifying the scope, and searching for and including systematic reviews). In contrast, there was limited or conflicting guidance for the latter stages of the overview process (quality assessment of systematic reviews and their primary studies, collecting and analyzing data, and assessing quality of evidence), and many of the challenges identified were also related to these stages. An additional, overarching challenge identified was that overviews are limited by the methods, reporting, and coverage of their included systematic reviews. CONCLUSIONS: This compilation of methodological guidance for conducting overviews of healthcare interventions will facilitate the production of future overviews and can help authors address key challenges they are likely to encounter. The results of this project have been used to identify areas where future methodological research is required to generate empirical evidence for overview methods. Additionally, these results have been used to update the chapter on overviews in the next edition of the Cochrane Handbook for Systematic Reviews of Interventions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“There is limited guidance on how to assess quality of evidence of outcome data in umbrella reviews (Gates et al., 2020, Pollock et al., 2016).”
From Impact of organized activities on mental health in children and adolescents: An umbrella review (2022) · cites this paper as Pollock et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Scoping studies: advancing the methodology
Levac, D., Colquhoun, H., & O'Brien, K. K. (2010) · Implementation Science · doi:10.1186/1748-5908-5-69 · Cited by 15,268
Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1. The Cochrane Collaboration
Higgins, J., Green, S. R., & Higgins, J. (2008) · Cited by 5,544
Greenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045
Shea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188
Handbook for Synthesizing Qualitative Research
Sandelowski, M., & Barroso, J. V. (2006) · Medical Entomology and Zoology · Cited by 1,096
Epidemiology and reporting characteristics of systematic reviews.
Moher, D., Tetzlaff, J., Tricco, A. C., Sampson, M., & Altman, D. G. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040078 · Cited by 891 · cited 1×
“Lastly, including SRs published outside of the CDSR can increase the complexity of the latter stages of the overview process due to greater variation in the methods and reporting of non-Cochrane SRs [22, 23] and the potential for topic overlap across multiple similar SRs [24].”
BackgroundBiologics for rheumatoid arthritis: an overview of Cochrane reviews
Singh, J. A., Christensen, R., Wells, G. A., Suarez-Almazor, M. E., Buchbinder, R., Lopez-Olivo, M. A., Ghogomu, E. T., & Tugwell, P. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007848.pub2 · Cited by 482
Research works printed but not linked: 8
- Grimshaw J. A knowledge synthesis chapter. In: A guide to knowledge synthesis. Canadian Institutes of Health Research; 2010. http://www.cihr-irsc.gc.ca/e/41382.html . Accessed 30 Nov 2015
- Foisy M, Dryden DM, Fernandes RM, Hartling L, Thomson D. Advancing methods for overviews of reviews: a discussion of challenges and potential solutions. Abstracts of the 22nd Cochrane Colloquium, 21-26 Sept 2014. Hyderabad: Wiley; 2014.
- Foisy M, Dryden DM, Fernandes RM, Hartling L. Multiple systematic reviews published on the same topic area: an analysis of systematic reviews that overlap in content. Abstracts of the 22nd Cochrane Colloquium, 21-26 Sep 2014. Hyderabad: Wiley; 2014.
- Green S, Higgins JPT. Chapter 2: preparing a Cochrane review. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions (version 5.1.0). The Cochrane Collaboration; 2011. www.handbook.cochrane.org .
- Foisy M, Hartling L. Challenges and considerations involved in using AMSTAR in overviews of reviews. Abstracts of the 22nd Cochrane Colloquium, 21-26 Sep 2014. Hyderabad: Wiley; 2014.
- Jia P, Chen J, Zhang L, Zhao P, Zhang M. Challenges and considerations in assessing the reporting quality of systematic reviews in overviews using PRISMA. Abstracts of the 23rd Cochrane Colloquium, 03-07 Oct 2015. Vienna: Wiley; 2015.
- Foisy M, Fernandes RM, Dryden DM, Hartling L. Grading the quality of evidence in existing systematic reviews: challenges and considerations. Abstracts of the 22nd Cochrane Colloquium, 21-26 Sep 2014. Hyderabad: Wiley; 2014.
- Cochrane: Special collection: Cochrane overviews. http://www.cochranelibrary.com/app/content/special-collections/article/?doi=10.1002/(ISSN)14651858(CAT)na(VI)SC000036 (2014). Accessed 30 Nov 2015. · DOI
Cited by
Sorted by citations
Adverse Outcomes of Polypharmacy in Older People: Systematic Review of Reviews
Davies, L. E., Spiers, G., Kingston, A., Todd, A., Adamson, J., & Hanratty, B. (2020) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2019.10.022 · Cited by 597
Montazeri, A., Mohammadi, S., M.Hesari, P., Ghaemi, M., Riazi, H., & Sheikhi-Mobarakeh, Z. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02410-2 · Cited by 433
Panel, S. G. P. C., Hoekstra, F., Mrklas, K., Khan, M., McKay, R. C., Vis‐Dunbar, M., Sibley, K. M., Nguyen, T., Graham, I. D., & Gainforth, H. L. (2020) · Health Research Policy and Systems · doi:10.1186/s12961-020-0544-9 · Cited by 365
An Overview of Reviews on Interprofessional Collaboration in Primary Care: Barriers and Facilitators
Rawlinson, C., Carron, T., Cohidon, C., Arditi, C., Hong, Q. N., Pluye, P., Peytremann‐Bridevaux, I., & Gilles, I. (2021) · International Journal of Integrated Care · doi:10.5334/ijic.5589 · Cited by 335
Hunt, H., Pollock, A., Campbell, P., Estcourt, L., & Brunton, G. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0695-8 · Cited by 314
Metadata
- Authors
- Pollock, M., Fernandes, R. M., Becker, L. A., Featherstone, R., & Hartling, L.
- Affiliations
- University of Alberta (CA)
- University of Lisbon (PT)
- SUNY Upstate Medical University (US)
- Corresponding author
- Hartling, Lisa
- Venue
- Systematic Reviews
- Published
- 2016-11-14
- Volume
- 5
- Issue
- 1
- Pages
- 190-190
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC5109841
- OpenAlex
- W2556359523
- Cited by
- 195
- References
- 29
- Citations by year
- 2026: 9 · 2025: 13 · 2024: 21 · 2023: 22 · 2022: 26 · 2021: 29 · 2020: 28 · 2019: 21 · 2018: 14 · 2017: 11
- FWCI
- 5.26
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Canadian Institutes of Health Research (119373)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- overviews of reviews, methodological recommendations, health interventions, evidence synthesis, quality of evidence, research methodology challenges
- MeSH terms
- Humans, Publications, Research Design, Research Personnel, Review Literature as Topic, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
