The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index; 15 of the 68 sources its claims cite were checked for support.
Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews
Open accessPublished version of a preprint
Abstract
Background: Overviews often identify and synthesise a large number of systematic reviews on the same topic, which is likely to lead to overlap (i.e. duplication) in primary studies across the reviews. Using a primary study result multiple times in the same analysis overstates its sample size and number of events, falsely leading to greater precision in the analysis. This paper aims to: (a) describe types of overlapping data that arise from the same primary studies reported across multiple reviews, (b) describe methods to identify and explain overlap of primary study data, and (c) present six case studies illustrating different approaches to manage overlap. Methods: We first updated the search in PubMed for methods from the MOoR framework relating to overlap of primary studies. One author screened the studies titles and abstracts, and any full-text articles retrieved, extracted methods data relating to overlap of primary studies and mapped it to the overlap methods from the MOoR framework. We also describe six case studies as examples of overviews that use specific overlap methods across the steps in the conduct of an overview. For each case study, we discuss potential methodological implications in terms of limitations, efficiency, usability, and resource use. Results: Nine methods studies were found and mapped to the methods identified by the MOoR framework to address overlap. Overlap methods were mapped across four steps in the conduct of an overview – the eligibility criteria step, the data extraction step, the assessment of risk of bias step, and the synthesis step. Our overview case studies used multiple methods to reduce overlap at different steps in the conduct of an overview. Conclusions: Our study underlines that there is currently no standard methodological approach to deal with overlap in primary studies across reviews. The level of complexity when dealing with overlap can vary depending on the yield, trends and patterns of the included literature and the scope of the overview question. Choosing a method might be dependent on the number of included reviews and their primary studies. Gaps in evaluation of methods to address overlap were found and further investigation in this area is needed.
Overview
CoChat Summary
TL;DR
This paper maps methods for managing overlap of primary studies across systematic reviews in overviews and illustrates them with six case studies. It concludes that no standard methodological approach exists and that evaluation of overlap methods remains limited.
Key findings
- Nine methods studies were identified and mapped to the MOoR framework; six were articles describing methods, two were guidance documents, and one was an empirical study.
- Overlap methods mapped across four steps in an overview: eligibility criteria, data extraction, risk of bias assessment, and synthesis/presentation/summary of findings; seven of nine studies addressed the eligibility criteria step.
- Case studies used multiple methods: Bidonde counted 29 (48%) overlapping RCTs across 9 reviews and 31 (52%) unique RCTs; Murphy assessed 165 unique primary studies across 16 reviews and selected higher-quality reviews when overlap exceeded 70%; Prousali estimated overlap at 6.2% using CCA across 18 reviews and 44 unique primary studies; Thabet found moderate-to-high overlap across 21 reviews and 71 primary studies, with unique references ranging from 38% to 78%.
- The paper reports no standardised approach to visualise, quantify, and resolve overlap, and notes that only one identified study empirically evaluated inclusion decisions, finding that selecting a Cochrane review maximised outcome data included.
Methods
- The authors updated a PubMed search for methods from the MOoR framework relating to overlap of primary studies, using a search algorithm combining method/meta-epidemiology terms with an overview search filter; search dates were January 2016 to March 2020, plus forward citation searching on a key 2014 study via Google Scholar.
- One author screened titles/abstracts and full texts, extracted methods data, and mapped it to overlap methods from the MOoR framework; results are presented descriptively and in tables.
- Six case studies were selectively chosen to illustrate different approaches to managing overlap across overview steps, with commentary on methodological limitations, efficiency, usability, and resource use.
Limitations
- The paper states that, due to resource limitations, one author screened studies and extracted data, so relevant citations could have been missed and data errors could have been introduced; it also states that only one database was searched, though expert knowledge and forward citation searching were used to mitigate this.
- The paper states that there is a lack of empirical evidence testing overlap methods and that significant gaps exist in evaluation of methods used in most steps of overview conduct.
- Evident from the text: the six case studies were selectively chosen based on variety of approaches rather than systematically sampled, and the review of methods studies included only English-language articles and excluded network meta-analysis, protocols, and overviews of non-intervention review types.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
10 Valid · 0 Not valid · 33 Not checked · 57 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The data extraction approaches outlined in the MOoR framework to manage discrepant data across systematic reviews involve retrieving either published or registry reports of the primary studies, or contacting systematic review or primary study authors, or both, for clarification regarding discrepancies [7, 16].”
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Results > Methods studies identified and mapped to the MOoR framework
What the sources say
- Partly supportedcites [7] Lunny C, Brennan SE, McDonald S, McKenzie JE. Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction. Syst Rev. 2017;6(1):231 https://doi.org/10.1186/s13643-017-0617-1. sourceExperimental — accuracy not yet measured
In Results, Stage I (Tables 6 and 7; page not provided), the framework lists primary studies and registry entries as data sources and identifies discrepant data across systematic reviews as an issue for data extraction. The passages available do not establish that contacting systematic-review or primary-study authors for clarification is one of the approaches to those discrepancies; the author contact described in Methods concerns study eligibility instead. (LLM, conf=0.78)
- Partly supportedcites [16] Lunny C, Brennan SE, McDonald S, McKenzie JE. Toward a comprehensive evidence map of overview of systematic review methods: paper 2-risk of bias assessment; synthesis, presentation and summary of the findings; and assessment of the certainty of the evidence. Syst Rev. 2018;7(1):159 https://doi.org/10.1186/s13643-018-0784-8. sourceExperimental — accuracy not yet measured
In Results, Stage I (page unavailable), the paper lists retrieving primary studies to reconcile discrepancies as an option, but discusses it specifically for discrepant risk-of-bias assessments. It does not specify published or registry reports or describe contacting systematic-review or primary-study authors for clarification; it directs readers to its first companion paper for data-extraction methods. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“when low overlap was found between reviews with inconsistent and discordant findings, discrepancies were most often attributed to differences in scope or differences in dates of review publication.”
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Results > Presentation of case studies
What the sources say
- Partly supportedcites [26] Thabet P, Joshi A, MacDonald E, Hutton B, Cheng W, Stevens A, et al. Clinical and pharmacokinetic/dynamic outcomes of prolonged infusions of beta-lactam antimicrobials: An overview of systematic reviews. PLoS One. 2021;16(1):e0244966 https://doi.org/10.1371/journal.pone.0244966. sourceExperimental — accuracy not yet measured
In Results > Findings, clinical cure and treatment failure (page unavailable), Thabet et al. say that several pairs of reviews with differing findings had slight or no overlap, explainable by differences in scope. They also note that four of six reviews finding no clinical-cure benefit were published in 2013 or earlier, but do not say publication dates most often explained the discrepancies. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“an empirical study found that selecting a Cochrane systematic review, as opposed to the most recent or highest quality non-Cochrane systematic review, maximized the amount of outcome data included in an overview [19]”
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Results > Presentation of case studies
What the sources say
- Supportedcites [19] Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. The impact of different inclusion decisions on the comprehensiveness and complexity of overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):18 https://doi.org/10.1186/s13643-018-0914-3. source
The Discussion states that, for groups of overlapping systematic reviews, selecting the Cochrane review rather than the most recent or highest-quality review maximized the outcome data included in an overview. It adds that this can hold even when a more recent or higher-quality non-Cochrane review is available. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Murphy et al. used five methods, across the four steps in the conduct of an overview, to manage the overlap in 165 unique primary studies across the 16 included systematic reviews (Table 2).”
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Results > Presentation of case studies
What the sources say
- Partly supportedcites [23] Murphy LA, Harrington P, Taylor SJC, Teljeur C, Smith SM, Pinnock H, et al. Clinical-effectiveness of self-management interventions in chronic obstructive pulmonary disease: an overview of reviews. Chron Respir Dis. 2017;14(3):276–88 https://doi.org/10.1177/1479972316687208. sourceExperimental — accuracy not yet measured
The Results (Study detail; page unavailable) report 16 systematic reviews containing 165 unique RCTs. The Methods (Data extraction and quality assurance; page unavailable) describe assessing RCT crossover, selecting the better-quality review when crossover exceeded 70%, and using narrative synthesis because of crossover. Our copy does not identify these as five methods across four steps, so that specific count and framework are not verified. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overlap methods were mapped across four steps in the conduct of an overview – the eligibility criteria step, the data extraction step, the assessment of risk of bias step, and the synthesis step.”
- Also stated in: § Methods studies identified and mapped to the MOoR framework · plain
“Several methods map across four steps in the conduct of an overview – the eligibility criteria step, the data extraction step, the assessment of risk of bias step, and the synthesis step.”
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Methods studies identified and mapped to the MOoR framework · plain
33 more claims
Create a free account to see all 100 claims and to verify the 53 references still open.
- Probably not a substantial claim. 41
- It may state two separate claims as one. 39
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“including only 22 (ref. 25 ), 19 (ref. 24 ), 36 (ref. 26 ), 5 (ref. 27 ), 12 (ref. 28 ) and 5 (ref. 29 ) studies”
From Suicide risk and mortality among patients with cancer (2022) · cites this paper as [28]
Experimental — accuracy not yet measuredIn the citing paper
“only few meta-analyses examined suicide risk among patients with cancer and those studies lacked a comprehensive literature search, generated incomplete findings regarding certain risk groups and/or analyzed only specific cancer entities, and/or presented only incidence rates without comparisons with the general population 24-29 .”
From Suicide risk and mortality among patients with cancer (2022) · cites this paper as [28]
In the citing paper
“An increased overall risk for suicide mortality in patients with cancer compared with the general population was consistently observed in those studies.”
From Suicide risk and mortality among patients with cancer (2022) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Results > Methods studies identified and mapped to the MOoR framework
“Note: aCochrane reviews with approximately 50% or more of their studies already captured by Cochrane reviews were excluded”
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Results > Presentation of case studies
“All Cochrane reviews meeting the overviews inclusions criteria were included, and non-Cochrane reviews were excluded if they had substantial duplication of content with Cochrane reviews and were of low quality.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Results > Presentation of case studies
“In the case of substantial overlap (over 70%), the higher quality review (using R-AMSTAR [45]) was selected if it was published the same year or more recently than the comparison reviews.”
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Results > Presentation of case studies
“In selecting the highest quality review with under 70% overlap, Murphy et al. minimize the impact of duplicate primary studies in their overview.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Results > Presentation of case studies
“There are two major advantages to including only relevant Cochrane reviews in an overview.”
§ Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews > Results > Presentation of case studies
“Third, an empirical study found that selecting a Cochrane systematic review, as opposed to the most recent or highest quality non-Cochrane systematic review, maximized the amount of outcome data included in an overview [19], making that overview more relevant and useful for decision makers.”
Citations
Cites
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
ROBIS: A new tool to assess risk of bias in systematic reviews was developed
Whiting, P., Savović, J., Higgins, J. P. T., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., & Churchill, R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.06.005 · Cited by 2,272
Bornmann, L., & Mutz, R. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23329 · Cited by 1,576
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483
Systematic review finds overlapping reviews were not mentioned in every other overview
Pieper, D., Antoine, S.-L., Mathes, T., Neugebauer, E. A. M., & Eikermann, M. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.11.007 · Cited by 1,318
Self-citationMethodology in conducting a systematic review of systematic reviews of healthcare interventions
Smith, V., Devane, D., Begley, C. M., & Clarke, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-15 · Cited by 1,227
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188
Journal of the Association for Information Science and Technology
(2018) · Journal of the Association for Information Science and Technology · doi:10.1002/(issn)2330-1643 · Cited by 911
A guide to interpreting discordant systematic reviews.
Jadad, A., Cook, D. J., & Browman, G. P. (1997) · Europe PMC (PubMed Central) · Cited by 399
Research works printed but not linked: 3
- Australian Government Department of Health. Review of the Australian Government Rebate on Natural Therapies for Private Health Insurance. Canberra: Communication Branch, Department of Health; 2015.
- Pollock, M., Fernandez, R.M., Becker, L., Pieper, D., Hartling, L., Chapter V: Overviews of reviews, in Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019), T.J. Higgins JPT, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors), Editor. 2019, Cochrane. Available from www.training.cochrane.org/handbook.
- Database of Abstracts of Reviews of Effects (DARE). Quality-assessed Reviews. 1995. Available from: https://www.ncbi.nlm.nih.gov/books/NBK285222/.
Cited by
Recommendations for the Extraction, Analysis, and Presentation of Results in Scoping Reviews
Pollock, D., Peters, M. D. J., Khalil, H., McInerney, P., Alexander, L., Tricco, A. C., Evans, C., de Moraes, É. B., Godfrey, C. M., Pieper, D., et al. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-22-00123 · Cited by 1,975
Self-citationGraphical Representation of Overlap for OVErviews: GROOVE tool
Bracchiglione, J., Meza, N., Bangdiwala, S. I., Niño de Guzmán, E., Urrútia, G., Bonfill, X., & Madrid, E. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1557 · Cited by 299
Barriers and Facilitators to Substance Use Disorder Treatment: An Overview of Systematic Reviews
Farhoudian, A., Razaghi, E., Hooshyari, Z., Noroozi, A., Pilevari, A., Mokri, A., Mohammadi, M. R., & Malekinejad, M. (2022) · Substance Abuse: Research and Treatment · doi:10.1177/11782218221118462 · Cited by 158
Suicide risk and mortality among patients with cancer
Heinrich, M., Hofmann, L., Baurecht, H., Kreuzer, P. M., Knüttel, H., Leitzmann, M. F., & Seliger, C. (2022) · Nature Medicine · doi:10.1038/s41591-022-01745-y · Cited by 136
Hult, M., Terkamo‐Moisio, A., Kaakinen, P., Karki, S., Nurmeksela, A., Palonen, M., Peltonen, L., & Häggman‐Laitila, A. (2023) · Nursing Open · doi:10.1002/nop2.1876 · Cited by 221
“We assessed the overlap of the included reviews to avoid the risk of including overlapping data from multiple reviews (Lunny et al., 2021; Pollock et al., 2017).”
Method
Metadata
- Authors
- Lunny, C., Pieper, D., Thabet, P., & Kanji, S.
- Affiliations
- University of British Columbia (CA)
- Witten/Herdecke University (DE)
- Montfort Hospital (CA)
- Ottawa Hospital (CA)
- Corresponding author
- Pieper, Dawid
- Venue
- BMC Medical Research Methodology
- Published
- 2021-07-07
- Volume
- 21
- Issue
- 1
- Pages
- 140-140
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC8265144
- OpenAlex
- W3121736026
- Cited by
- 252
- References
- 52
- Citations by year
- 2026: 55 · 2025: 55 · 2024: 59 · 2023: 39 · 2022: 43 · 2021: 1
- FWCI
- 1.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- uO Research (University of Ottawa) (submittedVersion, public-domain)
- cIRcle (University of British Columbia) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- overviews of reviews, data extraction, risk of bias assessment, eligibility criteria
- MeSH terms
- Systematic Reviews as Topic, Delivery of Health Care, Humans, Publications, Research Design, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews. https://app.cochat.ai/papers/10.1186/s12874-021-01269-y
- The paper
- Lunny, C., Pieper, D., Thabet, P., & Kanji, S. (2021). Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-021-01269-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.