The DOI matches its Crossref record; 59 of the 62 references it cites were found in a registry or an index.
Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness
Abstract
Systematic review aims to systematically identify, critically appraise, and summarize all relevant studies that match predefined criteria and answer predefined questions. The most common type of systematic review is that assessing the effectiveness of an intervention or therapy. In this article, we discuss some of the common methodological issues that arise when conducting systematic reviews and meta-analyses of effectiveness data, including issues related to study designs, meta-analysis, and the use and interpretation of effect sizes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Additionally, it was recommended that the protocol should define the methods to be used for risk of bias assessment, meta-analysis and exploration of heterogeneity [41, 42, 44].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [44]
In the citing paper
“The articles recommended that the protocol should state the objectives, hypotheses to be tested and the rationale of the review, [29] and that it should describe eligibility criteria [29, 33, 35, 36, 38, 39, 41], define the type of studies to be included [35, 37, 42, 44], and give the reason when including observational studies in the review [35, 37, 40].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [44]
In the citing paper
“Twenty-five (27%) articles gave recommendations on item “considering different study designs” [8, 29, 34, 35, 37–39, 42, 44, 49, 51, 56, 61, 65, 70, 73, 75, 76, 77–83], with publication dates between 1995 and 2016 (median year of publication 2007).”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [44]
In the citing paper
“A variation of this recommendation was to exclude studies that are at high risk of bias [42, 44, 54, 73, 82].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [44]
Citations
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
(2000) · Cited by 9,198
A basic introduction to fixed-effect and random-effects models for meta-analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.12 · Cited by 6,793
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Morris, S. B., & DeShon, R. P. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.105 · Cited by 2,502
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,312
Meta-analyses involving cross-over trials: methodological issues
Elbourne, D. R., Altman, D. G., Higgins, J. P., Curtin, F., Worthington, H. V., & Vail, A. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.140 · Cited by 2,305
Hayes, R., & Moulton, L. H. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781584888178 · Cited by 881
15 Design and Analysis of Cross-Over Trials
Kenward, M. G., & Jones, B. C. (2007) · Handbook of statistics · doi:10.1016/s0169-7161(07)27015-4 · Cited by 873
Research works printed but not linked: 3
- Walter, 1998, Odds ratios revisited., Evidence-Based Med, 3, 71
- Olkin, 1998, Odds ratios revisited., Evidence-Based Med, 3, 71
- Altman, 1998, Odds ratios revisited., Evidence-Based Med, 3, 71
Cited by
Sorted by citations
Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0468-4 · Cited by 1,164
Self-citationQuek, J., Chan, K. E., Wong, Z. Y., Tan, C., Tan, B., Lim, W. H., Tan, D. J. H., Tang, A. S. P., Tay, P., Xiao, J., et al. (2023) · The Lancet Gastroenterology & Hepatology · doi:10.1016/s2468-1253(22)00317-x · Cited by 749
Barker, T. H., Migliavaca, C. B., Stein, C., Colpani, V., Falavigna, M., Aromataris, E., & Munn, Z. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01381-z · Cited by 738
“As with traditional comparative methods of metaanalysis, there are different options in terms of model choice when performing a proportional meta-analysis (discussed above) [39].”
Self-citationMueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0495-9 · Cited by 400
“proach takes clinical as well as statistical considerations into account when deciding on pooling and the choice of the model [31, 38, 41, 44, 47, 61, 116], because “statistical”
BackgroundMethodInfluentialEn Li Cho, E., Ang, C. Z., Quek, J., Fu, C. E., Lim, L. K. E., Heng, Z. E. Q., Tan, D. J. H., Lim, W. H., Yong, J. N., Zeng, R., et al. (2023) · Gut · doi:10.1136/gutjnl-2023-330110 · Cited by 380
Metadata
- Authors
- Tufanaru, C., Munn, Z., Stephenson, M., & Aromataris, E.
- Affiliations
- The University of Adelaide (AU)
- Venue
- International Journal of Evidence-Based Healthcare
- Published
- 2015-09
- Volume
- 13
- Issue
- 3
- Pages
- 196-207
- ISSN
- 1744-1595 (ISSN-L)
- 1744-1609
- OpenAlex
- W2180006680
- Cited by
- 1,057
- References
- 62
- Citations by year
- 2026: 77 · 2025: 151 · 2024: 204 · 2023: 167 · 2022: 157 · 2021: 101 · 2020: 65 · 2019: 57 · 2018: 48 · 2017: 22 · 2016: 8
- FWCI
- 4.28
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Evidence-Based Healthcare (publishedVersion)
- PubMed (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- fixed-effects meta-analysis, random-effects meta-analysis, systematic review methodology, intervention effectiveness, effect size, research design
- MeSH terms
- Humans, Research Design, Review Literature as Topic, Meta-Analysis as Topic, Odds Ratio, Guidelines as Topic, Cross-Over Studies, Evidence-Based Medicine, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
