Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index; 15 of the 68 sources its claims cite were checked for support.

Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews

Open access
Lunny, Carole, Pieper, Dawid, Thabet, Pierre, Kanji, Salmaan · 2021 · BMC Medical Research Methodology

Published version of a preprint

Abstract

Background: Overviews often identify and synthesise a large number of systematic reviews on the same topic, which is likely to lead to overlap (i.e. duplication) in primary studies across the reviews. Using a primary study result multiple times in the same analysis overstates its sample size and number of events, falsely leading to greater precision in the analysis. This paper aims to: (a) describe types of overlapping data that arise from the same primary studies reported across multiple reviews, (b) describe methods to identify and explain overlap of primary study data, and (c) present six case studies illustrating different approaches to manage overlap. Methods: We first updated the search in PubMed for methods from the MOoR framework relating to overlap of primary studies. One author screened the studies titles and abstracts, and any full-text articles retrieved, extracted methods data relating to overlap of primary studies and mapped it to the overlap methods from the MOoR framework. We also describe six case studies as examples of overviews that use specific overlap methods across the steps in the conduct of an overview. For each case study, we discuss potential methodological implications in terms of limitations, efficiency, usability, and resource use. Results: Nine methods studies were found and mapped to the methods identified by the MOoR framework to address overlap. Overlap methods were mapped across four steps in the conduct of an overview – the eligibility criteria step, the data extraction step, the assessment of risk of bias step, and the synthesis step. Our overview case studies used multiple methods to reduce overlap at different steps in the conduct of an overview. Conclusions: Our study underlines that there is currently no standard methodological approach to deal with overlap in primary studies across reviews. The level of complexity when dealing with overlap can vary depending on the yield, trends and patterns of the included literature and the scope of the overview question. Choosing a method might be dependent on the number of included reviews and their primary studies. Gaps in evaluation of methods to address overlap were found and further investigation in this area is needed.

Overview

CoChat Summary

TL;DR

This paper maps methods for managing overlap of primary studies across systematic reviews in overviews and illustrates them with six case studies. It concludes that no standard methodological approach exists and that evaluation of overlap methods remains limited.

Key findings

  • Nine methods studies were identified and mapped to the MOoR framework; six were articles describing methods, two were guidance documents, and one was an empirical study.
  • Overlap methods mapped across four steps in an overview: eligibility criteria, data extraction, risk of bias assessment, and synthesis/presentation/summary of findings; seven of nine studies addressed the eligibility criteria step.
  • Case studies used multiple methods: Bidonde counted 29 (48%) overlapping RCTs across 9 reviews and 31 (52%) unique RCTs; Murphy assessed 165 unique primary studies across 16 reviews and selected higher-quality reviews when overlap exceeded 70%; Prousali estimated overlap at 6.2% using CCA across 18 reviews and 44 unique primary studies; Thabet found moderate-to-high overlap across 21 reviews and 71 primary studies, with unique references ranging from 38% to 78%.
  • The paper reports no standardised approach to visualise, quantify, and resolve overlap, and notes that only one identified study empirically evaluated inclusion decisions, finding that selecting a Cochrane review maximised outcome data included.

Methods

  • The authors updated a PubMed search for methods from the MOoR framework relating to overlap of primary studies, using a search algorithm combining method/meta-epidemiology terms with an overview search filter; search dates were January 2016 to March 2020, plus forward citation searching on a key 2014 study via Google Scholar.
  • One author screened titles/abstracts and full texts, extracted methods data, and mapped it to overlap methods from the MOoR framework; results are presented descriptively and in tables.
  • Six case studies were selectively chosen to illustrate different approaches to managing overlap across overview steps, with commentary on methodological limitations, efficiency, usability, and resource use.

Limitations

  • The paper states that, due to resource limitations, one author screened studies and extracted data, so relevant citations could have been missed and data errors could have been introduced; it also states that only one database was searched, though expert knowledge and forward citation searching were used to mitigate this.
  • The paper states that there is a lack of empirical evidence testing overlap methods and that significant gaps exist in evaluation of methods used in most steps of overview conduct.
  • Evident from the text: the six case studies were selectively chosen based on variety of approaches rather than systematically sampled, and the review of methods studies included only English-language articles and excluded network meta-analysis, protocols, and overviews of non-intervention review types.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12874-021-01269-y

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Three reviews included in a fictional overview with overlapping randomised control trials (RCTs)
Fig. 1 Three reviews included in a fictional overview with overlapping randomised control trials (RCTs)CC BY · PubMed Central
Fig. 2 Flowchart of included studies
Fig. 2 Flowchart of included studiesCC BY · PubMed Central
Fig. 3 CCA calculations for pairs of reviews. Overlap categorization: 0–5% - slight (white), 6–10% - moderate (green), 11–15% - high (yellow), > 15% - very high (red)
Fig. 3 CCA calculations for pairs of reviews. Overlap categorization: 0–5% - slight (white), 6–10% - moderate (green), 11–15% - high (yellow), > 15% - very high (red)CC BY · PubMed Central
Fig. 4 Citation matrix. Green indicates a trial included in a review, red indicates a trial excluded or omitted from a review and black indicates that the dates of publication make a trial ineligible for inclusion in a review
Fig. 4 Citation matrix. Green indicates a trial included in a review, red indicates a trial excluded or omitted from a review and black indicates that the dates of publication make a trial ineligible for inclusion in a reviewCC BY · PubMed Central

Verification

Claims the paper makes

10 Valid · 0 Not valid · 33 Not checked · 57 Own results and interpretations

33 more claims

Create a free account to see all 100 claims and to verify the 53 references still open.

See all claims — free account
  • Probably not a substantial claim. 41
  • It may state two separate claims as one. 39
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 52 references

Research works printed but not linked: 3

  1. Australian Government Department of Health. Review of the Australian Government Rebate on Natural Therapies for Private Health Insurance. Canberra: Communication Branch, Department of Health; 2015.
  2. Pollock, M., Fernandez, R.M., Becker, L., Pieper, D., Hartling, L., Chapter V: Overviews of reviews, in Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019), T.J. Higgins JPT, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors), Editor. 2019, Cochrane. Available from www.training.cochrane.org/handbook.
  3. Database of Abstracts of Reviews of Effects (DARE). Quality-assessed Reviews. 1995. Available from: https://www.ncbi.nlm.nih.gov/books/NBK285222/.

Cited by

See all 252

Metadata

Authors
Lunny, C., Pieper, D., Thabet, P., & Kanji, S.
Affiliations
  • University of British Columbia (CA)
  • Witten/Herdecke University (DE)
  • Montfort Hospital (CA)
  • Ottawa Hospital (CA)
Corresponding author
Pieper, Dawid
Venue
BMC Medical Research Methodology
Published
2021-07-07
Volume
21
Issue
1
Pages
140-140
ISSN
1471-2288 (ISSN-L)
OpenAlex
W3121736026
Cited by
252
References
52
Citations by year
2026: 55 · 2025: 55 · 2024: 59 · 2023: 39 · 2022: 43 · 2021: 1
FWCI
1.48
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • uO Research (University of Ottawa) (submittedVersion, public-domain)
  • cIRcle (University of British Columbia) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords
overviews of reviews, data extraction, risk of bias assessment, eligibility criteria
MeSH terms
Systematic Reviews as Topic, Delivery of Health Care, Humans, Publications, Research Design, Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews. https://app.cochat.ai/papers/10.1186/s12874-021-01269-y
The paper
Lunny, C., Pieper, D., Thabet, P., & Kanji, S. (2021). Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-021-01269-y

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.