Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.

Toward a comprehensive evidence map of overview of systematic review methods: paper 1—purpose, eligibility, search and data extraction

Open access

Abstract

BACKGROUND: Overviews of systematic reviews attempt to systematically retrieve and summarise the results of multiple systematic reviews. Methods for conducting, interpreting and reporting overviews are in their infancy. To date, there has been no evidence map of the methods used in overviews, thus making it difficult to determine the gaps and priorities for methods research. Our objectives were to develop and populate a comprehensive framework of methods for conducting, interpreting and reporting overviews (stage I) and to create an evidence map by mapping studies that have evaluated overview methods to the framework (stage II). METHODS: We searched methods collections (e.g. Cochrane Methodology Register, Meth4ReSyn library, AHRQ Effective Health Care Program) to identify eligible studies for both stages of this research. In stage I, cross-sectional studies, guidance documents and commentaries that described methods proposed for, or used in, overviews were used to develop and populate the framework of methods. Drafts and multiple iterations of the framework were discussed and refined by all authors. In stage II, we identified and described studies evaluating overview methods and mapped these evaluations to the framework. RESULTS: In this paper, we present results for the four initial steps of conducting an overview: (a) specification of the purpose, objectives and scope, (b) specification of the eligibility criteria, (c) search methods and (d) data extraction. Twenty-nine studies mentioned or described methods relevant to one or more of these steps. In the developed framework, identified methods and approaches were grouped according to the steps an overview author would need to undertake. Fifteen studies evaluated identified methods, all of which mapped to the search methods step. These studies either reported the development and evaluation of a new search filter to retrieve systematic reviews or compared the performance of multiple filters. CONCLUSION: Gaps in the evaluation of methods were found for the majority of steps in the framework. More empirical studies are needed to evaluate the methods outlined and provide a comprehensive evidence map. The framework is useful for planning these evaluations and for planning methods required to deal with challenges that arise when conducting an overview.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13643-017-0617-1

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Summary of the research reported in each paper
Fig. 1 Summary of the research reported in each paperCC BY · PubMed Central
Fig. 2 Stages in the development of an evidence map of overview methods
Fig. 2 Stages in the development of an evidence map of overview methodsCC BY · PubMed Central
Fig. 3 Flowchart of studies retrieved for both stages I and II. *The 42 stage I studies contributed to multiple steps
Fig. 3 Flowchart of studies retrieved for both stages I and II. *The 42 stage I studies contributed to multiple stepsCC BY · PubMed Central
Fig. 4 Flowchart of stage II studies of search filter evaluations
Fig. 4 Flowchart of stage II studies of search filter evaluationsCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 52 references

Research works printed but not linked: 8

  1. CMIMG C: Review Type & Methodological Considerations --background paper for the first part of the Paris CMIMG discussion. 2012.
  2. Hartling L, Dryden D, Vandermeer B, Fernandes R. Generating empirical evidence to support methods for overviews of reviews. In: Cochrane Colloquium: 2013; Quebec City, Canada; 2013.
  3. Kramer S, Langendam M, Elbers R, Scholten R, Hooft L. Preparing an overview of reviews: lessons learned. Poster. In: Cochrane colloquium: 2009 Oct 11–14 2009; Singapore; 2009.
  4. Buchter R, Pieper D, Jerinic P. Overviews of systematic reviews often do not assess methodological quality of included reviews. Poster. In: 19th Cochrane colloquium. Madrid: Cochrane Database Syst Rev; 2011. p. 105–6.
  5. Robinson KA, Chou R, Berkman ND, Newberry SJ, Fu R, Hartling L, Dryden D, Butler M, Foisy M, Anderson J, et al. Integrating bodies of evidence: existing systematic reviews and primary studies. In: Methods guide for effectiveness and comparative effectiveness reviews. Rockville: Agency for Healthcare Research and Quality (US); 2008.
  6. White CM, Ip S, McPheeters MC, Tim S, Chou R, Lohr KN, Robinson K, McDonald K, Whitlock EP. Using existing systematic reviews to replace de novo processes in conducting comparative effectiveness reviews. In: Methods guide for comparative effectiveness reviews. Rockville: Agency for Healthcare Research and Quality; 2009.
  7. Ryan R, Hill S. Developing an overview of systematic reviews: a framework for synthesising the evidence on interventions to improve communication. In: Cochrane Colloquium: 2005; Melbourne, Australia; 2005.
  8. Büchter R, Pieper D. How do authors of Cochrane overviews deal with conflicts of interest relating to their own systematic reviews? In: Cochrane Colloquium: 2015; Vienna, Austria; 2015.

Cited by

See all 184

Metadata

Authors
Lunny, C., Brennan, S. E., McDonald, S., & McKenzie, J. E.
Affiliations
Monash University (AU)
Corresponding author
McKenzie, Joanne E.
Venue
Systematic Reviews
Published
2017-11-21
Volume
6
Issue
1
Pages
231-231
ISSN
2046-4053 (ISSN-L)
OpenAlex
W2768872497
Cited by
184
References
52
Citations by year
2026: 11 · 2025: 14 · 2024: 18 · 2023: 32 · 2022: 32 · 2021: 35 · 2020: 20 · 2019: 14 · 2018: 7
FWCI
2.77
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Systematic Reviews (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Monash University Research Portal (Monash University) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Monash University
  • Medical Research Council
  • National Health and Medical Research Council (1072366)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
overviews of systematic reviews, evidence map, search filters, eligibility criteria, data extraction, literature search strategies
MeSH terms
Humans, Research Design, Review Literature as Topic, Evidence-Based Medicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Toward a comprehensive evidence map of overview of systematic review methods: paper 1—purpose, eligibility, search and data extraction. https://app.cochat.ai/papers/10.1186/s13643-017-0617-1
The paper
Lunny, C., Brennan, S. E., McDonald, S., & McKenzie, J. E. (2017). Toward a comprehensive evidence map of overview of systematic review methods: paper 1—purpose, eligibility, search and data extraction. Systematic Reviews. https://doi.org/10.1186/s13643-017-0617-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.