Meta-analysis and the science of research synthesis
Open accessThe DOI matches its Crossref record; 102 of the 102 references it cites were found in a registry or an index; 9 of the 69 sources its claims cite were checked for support.
Abstract
Meta-analysis is the quantitative, scientific synthesis of research results. Since the term and modern approaches to research synthesis were first introduced in the 1970s, meta-analysis has had a revolutionary effect in many scientific fields, helping to establish evidence-based practice and to resolve seemingly contradictory research outcomes. At the same time, its implementation has engendered criticism and controversy, in some cases general and others specific to particular disciplines. Here we take the opportunity provided by the recent fortieth anniversary of meta-analysis to reflect on the accomplishments, limitations, recent advances and directions for future developments in the field of research synthesis.
Overview
CoChat Summary
TL;DR
This review reflects on meta-analysis as quantitative research synthesis since the 1970s, its contributions to evidence-based practice, and its methodological development across medicine, social sciences, and ecology/evolution/conservation. It argues that meta-analysis has resolved contradictory evidence and enabled broad generalizations, but faces a “midlife crisis” from poor-quality applications, publication/research bias, and misuse of the term.
Key findings
- Meta-analysis was first introduced in the 1970s; the term was coined by Gene Glass in 1976, and the paper marks its 40th anniversary. It has helped establish evidence-based practice, resolve seemingly contradictory results, and spread across many scientific fields.
- In medicine, a meta-analysis of 20 clinical studies found maternal obesity associated with increased neural tube defect risk: a 1.7-fold rise for moderately obese women and over a 3-fold risk for severely obese women. In social science, a meta-analysis of 35 studies found no significant differences between multisystemic therapy and conventional social services for serious juvenile offenders.
- In ecology, evolution and conservation, meta-analysis was adopted in the early 1990s. Parmesan and Yohe used data on over 1,700 species to show significant range shifts and advancement of spring events; Benayas and colleagues found ecological restoration increased biodiversity and ecosystem services, though values remained lower in restored versus intact reference ecosystems.
- Methodological advances include fixed- and random-effects models, meta-regression, heterogeneity quantification (τ and I²), publication-bias tools, network meta-analysis, phylogenetic meta-analysis, and software such as MetaWin and OpenMEE. PRISMA includes a 27-item checklist for systematic review reporting.
Methods
- Design: narrative review/commentary on meta-analysis and research synthesis, not a new meta-analysis. The authors state they highlight principles, limitations, utility, and future developments.
- Data or sample: draws on published literature and illustrative examples across medicine, social sciences, and ecology/evolution/conservation; includes Box 1 on meta-analytic models and Table 1 on EEC milestones.
- Main analysis: conceptual and methodological synthesis; no primary data collection, systematic search protocol, or quantitative pooling is reported in the paper.
Limitations
- The paper states that meta-analysis and systematic reviews can highlight evidence deficiencies but cannot overcome them; publication bias and research bias may be suspected and sometimes estimated but cannot truly be corrected; selective or incomplete reporting can constrain synthesis.
- The paper states that too many published meta-analyses are of low quality, the term “meta-analysis” is frequently misapplied, and vote-counting persists despite being discredited; it also notes concerns about academic inequality and the “research parasites” controversy.
- The text makes evident that, as a narrative review, it does not provide a systematic search strategy, formal inclusion criteria, or a quantitative synthesis; its examples are illustrative rather than exhaustive.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 4 Not valid · 37 Not checked · 33 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Similarly, a synthesis may be constrained by either selective or incomplete reporting 66,67 .”
What the sources say
- Not checked yetcites [66] Stewart, L. A . et al. Preferred reporting items for a systematic review and meta-analysis of individual participant data: the PRISMA-IPD statement. J. Am. Med. Assoc. 313, 1657–1665 (2015) source
- Partly supportedcites [67] Saldanha, I. J . et al. Evaluating Data Abstraction Assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial. Syst. Rev. 5, 196 (2016) sourceExperimental — accuracy not yet measured
The Background (page unavailable) reports that omissions and errors during data abstraction changed summary meta-analytic results, supporting the general point that a synthesis can be affected by missing information. It describes information present in articles but omitted by abstractors, not selective or incomplete reporting by the studies themselves; that specific mechanism is unsupported. (LLM, conf=0.78)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“When the number of studies is an order of magnitude greater than that, the inability of narrative processes to adequately summarize the outcomes increases exponentially 2,3 .”
What the sources say
- Not checked yetcites [2] Roberts, P. D ., Stewart, G. B. & Pullin, A. S. Are review articles a reliable source of evidence to support conservation and environmental management? A comparison with medicine. Biol. Conserv. 132, 409–423 (2006) source
- Partly supportedcites [3] Bastian, H ., Glasziou, P . & Chalmers, I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 7, e1000326 (2010) sourceExperimental — accuracy not yet measured
Our copy supports the qualitative half of the claim: the 'How Close Are We to Archie Cochrane's Goal?' section cites Goldschmidt and Mulrow for 'how great the potential is for error in reviews of health literature that were not conducted systematically' and states that 'the staple of medical literature synthesis remains the non-systematic narrative review', while the same section (Figure 4 discussion) and 'Where to Now?' document the growth from ~14 trials a day in 1979 to 75 trials and 11 systematic reviews a day, with 'the task keeps increasing in size and complexity' ('Where Are We Now?'). What the source does NOT support is the quantitative assertion attributed to it: nowhere does our copy say the inability of narrative processes to summarise outcomes 'increases exponentially', nor tie it to study counts an order of magnitude beyond a dozen — the only dozen-scale figure in the paper is the median number of trials per systematic review (six to 16, Cochrane reviews averaging over 12), which is offered as a measure of reviewing workload and coverage, not of narrative-review failure. The exponential scaling is therefore the citing authors' characterisation, not this paper's. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, as the number of publications and research results has mushroomed across scientific fields, the difficulties encountered in carrying out narrative reviews to identify and summarize evidence in a transparent manner has become increasingly apparent 1 .”
What the sources say
- Not checked yetcites [1] Jennions, M. D ., Lortie, C. J. & Koricheva, J. in The Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J . et al.) Ch. 23, 364–380 (Princeton Univ. Press, 2013) source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, in many cases of the resolution of scientific problems, when the results of an individual study are combined with those of other studies, the resulting synthesis provides a more general and complete picture of the questions being addressed.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A statement the paper makes itself — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Sometimes, the striking results of initial studies are not confirmed by those of subsequent studies or by syntheses of a body of research.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
15 more claims
Create a free account to see all 78 claims and to verify the 60 references still open.
- It may state two separate claims as one. 32
- Probably not a substantial claim. 48
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The trade-off with our method is that it does not allow for determining the magnitude of the effect of the practices33–36 but only the proportion of impacts33,37 as positive, negative or neutral.”
From Farming practices to enhance biodiversity across biomes: a systematic review (2024) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Besides, publication biases that have been documented in several scientific fields90,91 can also affect ML-based science.92,93”
From Leakage and the reproducibility crisis in machine-learning-based science (2023) · cites this paper as [91]
Experimental — accuracy not yet measuredIn the citing paper
“Moreover, since a meta-analysis can be conducted only when sufficient and appropriate quantitative data from original studies are available, the number of published meta-analyses about a certain topic is an indicator of the overall understanding of the subject, allowing us to reveal research gaps (25, 26).”
From Agricultural diversification promotes multiple ecosystem services without compromising yield (2020) · cites this paper as [25]
In the citing paper
“Meta-analysis optimises statistical power for detecting treatment effects by combining data from multiple similar studies [22, 23].”
From Does the growing of Bt maize change abundance or ecological function of non-target animals compared to the growing of non-GM maize? A systematic review (2022) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Box 1: Meta-analytic models and heterogeneity
“𝜀 is the within-study sampling error (solid black lines), the deviation from the study-specific mean.”
§ Box 1: Meta-analytic models and heterogeneity
“the solid black lines are deviations from the overall mean (panels a,b) or slope (panels c,d)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Box 1: Meta-analytic models and heterogeneity
“The assumption of the fixed-effect meta-regression is that a moderator(s) can explain all between-study heterogeneity”
§ Box 1: Meta-analytic models and heterogeneity
“the blue distributions the between-study variance (panels a-d).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Box 1: Meta-analytic models and heterogeneity
“Two basic models of meta-analysis are the fixed-effect model (Fig 1a) and the random-effects model (Fig 1b) 6 .”
§ Box 1: Meta-analytic models and heterogeneity
“Figure 1. A variety of charts and plots common in meta-analysis. a. PRISMA diagram 20 , b. a forest plot showing overall means based on a fixed-effect model and random-effects model”
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 1×
“Procedural protocols for systematic reviews are well established especially in the medical and social sciences (Preferred Reporting Items for Systematic Reviews, or PRISMA; www.prisma-statement.org) 20 .”
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · cited 1×
“…the use of machine learning to screen studies 82 for inclusion in systematic reviews and meta-analyses for large reviews, increasingly sophisticated software and models for complex meta-regression 53,83 , and integration of meta-analysis with decision support in medicine and other domains 84 .”
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Research works printed but not linked: 3
- Hedges, L. & Olkin, I. Statistical Methods for Meta-analysis (Academic Press, 1985)
- Higgins, J. P. T . & Green, S. (eds) Cochrane Handbook for Systematic Reviews of Interventions: Version 5.1.0 (Wiley, 2011).This large collaborative work provides definitive guidance for the production of systematic reviews in medicine and is of broad interest for methods development outside the medical field.
- Cooper, H. M ., Hedges, L. V . & Valentine, J. C. (eds) The Handbook of Research Synthesis and Meta-analysis 2nd edn (Russell Sage Foundation, 2009).This book is an important compilation that builds on the ground-breaking first edition to set the standard for best practice in meta-analysis, primarily in the social sciences but with applications to medicine and other fields.
Cited by
Items 601–700 of 1,000
Zhang, L., Wu, Z., Zhou, J., Zhou, L., Lu, Y., Xiang, Y., Zhang, R., Deng, Q., & Wu, W. (2022) · Agronomy · doi:10.3390/agronomy12081857 · Cited by 33
Kaur, J., Bhardwaj, N., Fernandes, R. L., Vidya, V., & Farooqui, N. A. (2022) · Journal of Islamic marketing · doi:10.1108/jima-09-2021-0304 · Cited by 45
A Procedure and Guidelines for Analyzing Groups of Software Engineering Replications
Santos, A., Vegas, S., Oivo, M., & Juristo, N. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2935720 · Cited by 32
Faridah, D. N., Damaiyanti, S., Indrasti, D., Jayanegara, A., & Afandi, F. A. (2021) · International Journal of Food Science & Technology · doi:10.1111/ijfs.15276 · Cited by 25
Buhos: A web-based systematic literature review management software
Bustos, C., Malverde, M. ́. M., Salcedo, P., & Mújica, A. D. (2018) · SoftwareX · doi:10.1016/j.softx.2018.10.004 · Cited by 26
Meta-Analysis of Empirical Estimates of Loss-AversionPreprint
Brown, A. L., Imai, T., Vieider, F. M., & Camerer, C. (2020) · Social Science Research Network · doi:10.31222/osf.io/hnefr · Cited by 161
Jeon, J., Park, S. H., Choi, J., Han, S. M., Kim, H., Shim, S. R., & Hyun, J. K. (2024) · Acta Biomaterialia · doi:10.1016/j.actbio.2024.06.011 · Cited by 24
Cadmium exposure and health outcomes:An umbrella review of meta-analyses
Xu, W., Wang, S., Ruan, W., Hao, M., Jiang, K., Guo, H., Geng, A., Man, M., Hu, Z., Yun, L., et al. (2025) · Environmental Research · doi:10.1016/j.envres.2025.121547 · Cited by 23
Rynkowski, L., Ellis, J. I., Needham, H. R., & Pilditch, C. A. (2025) · Marine Biology · doi:10.1007/s00227-025-04671-y · Cited by 24
“In a meta-analysis, the effect size found in each individual experiment is considered an estimate of the ‘true’ effect size and by gathering and com - bining multiple estimates from individual studies, an overall effect size can be calculated (Gates 2002; Gurevitch et al. 2018 ).”
Spotted owls and forest fire: Comment
Jones, G. M., Gutiérrez, R. J., Block, W. M., Carlson, P. C., Comfort, E. J., Cushman, S. A., Davis, R. J., Eyes, S. A., Franklin, A. B., Ganey, J. L., et al. (2020) · Ecosphere · doi:10.1002/ecs2.3312 · Cited by 21
“While meta-analysis has helped to advance scientific understanding across a broad range of disciplines by offering tools for science synthesis (Gurevitch et al. 2018), its ability to lead to novel understanding is limited by the input data and decisions made by the meta-analyst.”
BackgroundLi, J., Wang, Y., Jiang, R., & Li, J. (2022) · International Journal of Occupational Safety and Ergonomics · doi:10.1080/10803548.2021.2024020 · Cited by 26
“The following thresholds for the qualitative interpretation of the I2 range from 0 to 100%, indicating decreasing to increasing heterogeneity, and are sub-classified as low (< 25%), medium (≥ 25 to < 50%) and high (≥ 75%), respectively [30].”
BackgroundZhang, L., Zhang, M., Li, Y., Li, J., Jing, Y., Xiang, Y., Yao, B. W., & Deng, Q. (2022) · Agronomy · doi:10.3390/agronomy12020247 · Cited by 24
A critique of narrative reviews of the evidence-base for ECT in depression
Meechan, C. F., Laws, K. R., Young, A. H., McLoughlin, D. M., & Jauhar, S. (2022) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796021000731 · Cited by 22
“Such vote-counting is a flawed method of research synthesis, and acknowledged as misleading (see Hedges and Olkin, 1980; Gurevitch et al., 2018) – not least of all because it does not quantify the magnitude of group differences or consider sample sizes.”
Qin, M., Zheng, E., Hou, D., Meng, X., Meng, F., Gao, Y., Chen, P., Qi, Z., & Xu, T. (2023) · Plants · doi:10.3390/plants12142709 · Cited by 40
“Meta-analysis is a method of quantitative investigation that offers an organized and thorough study of the literature on a certain issue and permits a summary of the results of previous research [19].”
MethodUnveiling the Effects of Carbon-Based Nanomaterials on Crop Growth: From Benefits to Detriments
Li, K., Tan, H., Li, J., Li, Z., Qin, F., Luo, H., Qin, D., Weng, H., & Zhang, C. (2023) · Journal of Agricultural and Food Chemistry · doi:10.1021/acs.jafc.3c02768 · Cited by 20
Liu, E., Liu, Z., Sun, Z., Li, J., Gong, H., Peng, J., & Ouyang, Z. (2023) · Journal of Environmental Management · doi:10.1016/j.jenvman.2023.119070 · Cited by 21
New horizons for comparative studies and meta-analyses
Pottier, P., Noble, D. W. A., Seebacher, F., Wu, N. C., Burke, S., Lagisz, M., Schwanz, L. E., Drobniak, S. M., & Nakagawa, S. (2024) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2023.12.004 · Cited by 27
“Meta-analysis was developed in medicine and social sciences in parallel with evolutionary comparative analyses [16,17].”
Self-citationKawsar, M. A., Munny, F. J., Saif, U. M., Harun‐Al‐Rashid, A., Rahman, M., Barman, S. K., Adikari, D., Alam, M. T., Kunda, M., & Pandıt, D. (2024) · Journal of Hazardous Materials Advances · doi:10.1016/j.hazadv.2024.100430 · Cited by 27
Budgen, D., Brereton, P., Williams, N., & Drummond, S. (2020) · e-Informatica Software Engineering Journal · doi:10.37190/e-inf200101 · Cited by 27
Shi, L., Chu, H., & Lin, L. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1415 · Cited by 23
Factors that impact fragility index and their visualizations
Lin, L. (2020) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13428 · Cited by 22
On the Rationality of Institutional Investors: The Case of Major Industrial Accidents
Carpentier, C., & Suret, J. (2020) · Journal of Behavioral Finance · doi:10.1080/15427560.2020.1774593 · Cited by 21
“This standardization is essential for the effect-size metric to be comparable and interpretable (Gurevitch et al. 2018).”
BackgroundShi, Q., Wang, Z., Zhou, Q. X., Hou, R., Gao, X., He, S.-E., Zhao, S., Ma, Y., Zhang, X., Guan, Q., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.06.020 · Cited by 20
McLean, R. K. D., & Kunal Sen, A. (2018) · Research Evaluation · doi:10.1093/reseval/rvy026 · Cited by 21
Thomsen, M. S. (2020) · Nature Ecology & Evolution · doi:10.1038/s41559-020-1117-6 · Cited by 23
Infodemiological Study on COVID-19 Epidemic and COVID-19 InfodemicPreprint
Zhi, H., Yang, Z., Li, Q., Zhang, A., & Huang, Y. (2020) · Preprints.org · doi:10.20944/preprints202002.0380.v2 · Cited by 19
“Under the umbrella of infodemiological scenarios, meta-analysis on diachronic discourses of pertinent keywords and phrases promises to articulate the unfolding chronological picture since their debutants on a historical time scale [10].”
BackgroundZinati, Z., & Nazari, L. (2023) · Scientific Reports · doi:10.1038/s41598-023-40189-3 · Cited by 22
Effect of rail transit on travel behavior: A systematic review and meta-analysis
Wang, J., Cheng, W., Lü, Y., & Wang, D. (2023) · Transportation Research Part D Transport and Environment · doi:10.1016/j.trd.2023.103882 · Cited by 24
Zhu, R., He, S., Ling, H., Liang, Y., Wei, B., Yuan, X., Cheng, W., Peng, B., Xiao, J., Wei, J., et al. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1500544 · Cited by 19
Wen, S., Cui, N., Wang, Y., Daozhi, G., Wang, Z., Xing, L., Wu, Z., & Zhang, Y. (2025) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2025.110571 · Cited by 24
Tan, L., & Schultz, D. M. (2021) · Weather Climate and Society · doi:10.1175/wcas-d-21-0132.1 · Cited by 20
Emergence of Nano-Dentistry as a Reality of Contemporary Dentistry
Кочан, О. В., Boitsaniuk, S. I., Levkiv, M., Przystupa, К., Манащук, Н. В., Pohoretska, K., Чорній, Н. В., Tsvyntarna, I., & Patskan, L. (2022) · Applied Sciences · doi:10.3390/app12042008 · Cited by 28
“For this review article, a narrative [13] review was performed using a comprehensive literature search using the Scopus database to identify any studies for nanoparticle use in various areas of dentistry.”
MethodEffect of Attentional Focus on Sprint Performance: A Meta-Analysis
Li, D., Zhang, L., Yue, X., Memmert, D., & Zhang, Y. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19106254 · Cited by 31
“As we know, a systematic review is an opportunity to narrow the evidence–practice gap [25], and a meta-analytic review is characterized by its comprehensiveness, quantified evidence, and guidance for practice [26].”
BackgroundZhao, J., Deng, H., Xun, C., Chen, C., Hu, Z., Ge, L., & Jiang, Z. (2023) · Stem Cell Research & Therapy · doi:10.1186/s13287-023-03270-2 · Cited by 22
“A systematic review consists of gathering, selecting, analyzing, and synthesizing all the relevant evidence to address a particular research question [16, 17].”
BackgroundBhardwaj, A., Budaraju, D., Venkatesh, P., Chowdhury, D., Kumar, R. P., Pal, K., Sivaraman, J., & Neelapu, B. C. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09935-8 · Cited by 21
Trends and gaps in the study of fire effects on plant–animal interactions in Brazilian ecosystems
Ballarin, C. S., Mores, G. J., de Goés, G. A., Fidélis, A., & Cornelissen, T. (2023) · Austral Ecology · doi:10.1111/aec.13420 · Cited by 23
Is deterrence approach effective in combating tax evasion? A meta-analysis
Dularif, M., Sutrisno, T., Nurkholis, N., & Saraswati, E. (2019) · Problems and Perspectives in Management · doi:10.21511/ppm.17(2).2019.07 · Cited by 32
What's hot and what's not – Identifying publication trends in insect ecology
Andrew, N. R., Evans, M. J., Svejcar, L. N., Prendegast, K., Mata, L., Gibb, H., Stone, M. J., & Barton, P. S. (2021) · Austral Ecology · doi:10.1111/aec.13052 · Cited by 21
A hierarchical, multivariate meta‐analysis approach to synthesising global change experiments
Ogle, K., Liu, Y., Vicca, S., & Bahn, M. (2021) · New Phytologist · doi:10.1111/nph.17562 · Cited by 21
Kelley, G. A., Kelley, K. S., & Stauffer, B. L. (2021) · Blood Pressure · doi:10.1080/08037051.2021.1940837 · Cited by 24
“to be the ‘grandmother’ of big data [69], to examine”
BackgroundCo-culture of rice and aquatic animals mitigates greenhouse gas emissions from rice paddies
Huang, M. H., Zhou, Y., Guo, J., Dong, X., An, D., Shi, C., Li, L., Dong, Y., & Gao, Q. (2023) · Aquaculture International · doi:10.1007/s10499-023-01243-z · Cited by 19
“Thus, we conducted a meta-analysis, a useful tool for establishing evidence-based practices and resolving seemingly contradictory research results (Gurevitch et al. 2018), to better understand the effects of CSRAA on CH 4 and N 2 O emissions from rice paddies.”
Hao, S., Tian, Y., Lin, Z., Xie, L., Zhou, X., & Bañuelos, G. S. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1327649 · Cited by 19
“It is a dynamic process that requires continuous evaluation and consideration of new evidence (Gurevitch et al., 2018).”
BackgroundCao, H., Ding, R., Du, T., Kang, S., Tong, L., Chen, J., & Gao, J. (2024) · Physiologia Plantarum · doi:10.1111/ppl.14282 · Cited by 20
Global Ecosystem Nitrogen Cycling Reciprocates Between Land‐Use Conversion and Its Reversal
Uwiragiye, Y., Wang, J., Huang, Y., Wu, L., Zhou, J., Zhang, Y., Chen, M., Jing, H., Qian, Y., Elrys, A. S., et al. (2024) · Global Change Biology · doi:10.1111/gcb.17537 · Cited by 22
“The land-use response was then calculated for each individual observation regarding N flux and N pool using both the metric (Van Groenigen, Osenberg, and Hungate 2011) and the land-use effect through the response ratio (Gurevitch et al. 2018).”
Yang, N., Zohner, C. M., Crowther, T. W., Feng, J., Wu, J., Chen, X., Han, W., Stocker, B. D., Hui, D., Augusto, L., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-60633-4 · Cited by 26
“This method for determining weight has been widely used in previous meta-analyses 61,65 .”
Ahmad, H., Miranda, L. E., Dunn, C. G., Boudreau, M. R., & Colvin, M. E. (2025) · Hydrological Processes · doi:10.1002/hyp.70260 · Cited by 22
“…Systematic Reviews and Meta-analysis—PRISMA framework (Page et al. 2021), a method that analyses and synthesises literature to discover key ideas, research gaps, and forms of evidence that inform future research priorities and influence policy and practice (Gurevitch et al. 2018; Page et al. 2021).”
Coles, N. A., Larsen, J. T., Kuribayashi, J., & Kuelz, A. (2019) · Emotion Review · doi:10.1177/1754073919868762 · Cited by 24
Secretive marsh bird habitat associations in the Mississippi Flyway: a meta-analysis
Malone, K. M., Webb, E. B., Mengel, D. C., Kearns, L. J., Matteson, S. W., & McKellar, A. E. (2021) · Avian Conservation and Ecology · doi:10.5751/ace-01924-160212 · Cited by 23
“A meta-analysis is a quantitative and robust method to synthesize research findings across studies by enabling evaluation of the strength and heterogeneity in ecological patterns (Koricheva et al. 2013, Gurevitch et al. 2018).”
MethodResultFeng, C., Huang, W., Xu, K., Stewart, J. L., Camilleri, J. A., Yang, X., Wei, P., Gu, R., Luo, W., & Eickhoff, S. B. (2022) · Clinical Psychology Review · doi:10.1016/j.cpr.2022.102189 · Cited by 20
Estrogens in plants and emerging risks to human health
Adeel, M., Zain, M., Shakoor, N., Azeem, I., Hussain, M., Ahmad, M. A., Chaudhary, S., Zaheer, U., Aziz, M. A., Ahmar, S., et al. (2023) · Environment International · doi:10.1016/j.envint.2023.107985 · Cited by 18
The Treatment of Depersonalization-Derealization Disorder: A Systematic Review
Wang, S., Zheng, S., Zhang, X., Ma, R., Feng, S., Song, M., Zhu, H., & Jia, H. (2023) · Journal of Trauma & Dissociation · doi:10.1080/15299732.2023.2231920 · Cited by 25
“To robustly review different types of clinical interventions (Gurevitch et al., 2018), we gathered as much evidence as possible for this systematical review and provide clear descriptions of the interventions (e.g., dose, add-on treatment, efficacy).”
MethodYang, H., Li, R., Li, J., Guo, Y., Gao, T., Guo, D., & Zhang, Q. (2023) · Environmental Pollution · doi:10.1016/j.envpol.2023.122442 · Cited by 18
Zema, D. A., & Lucas‐Borja, M. E. (2023) · Hydrological Processes · doi:10.1002/hyp.14957 · Cited by 19
Navigating contemporary challenges and future prospects in digital industry evolution
Gallab, M., Nardo, M. D., & Naciri, L. (2024) · Discover Applied Sciences · doi:10.1007/s42452-024-05913-2 · Cited by 27
“Accurate drafting of literature analysis is based on compliance with a rigorous standardised work protocol developed by a group of experts in 2005 and known by the acronym of PRISMA (Preferred Reporting Items for Systematic reviews and Me-ta-Analyses) Statement described by [8, 73].”
Navigating heterogeneity in meta-analysis: methods for identification and management
Ilmawan, M. (2024) · Deka in Medicine · doi:10.69863/dim.2024.e269 · Cited by 34
Protocols for meta-analysis of intervention safety seldom specified methods to deal with rare events
Zhou, Y., Zhu, B., Lin, L., Kwong, J. S., & Xu, C. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.09.023 · Cited by 19
Rosenberger, K. J., Xu, C., & Lin, L. (2021) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13578 · Cited by 18
“Systematic reviews and meta-analyses are considered of the highest quality and can be viewed as a lens through which evidence is evaluated.(14-17) However, systematic reviews and meta-analyses do not always provide a rationale for their methodology; if not used properly, they could produce ambiguous results and exacer…”
BackgroundZizka, A., Barratt, C. D., Ritter, C. D., Jörger‐Hickfang, T., & Zizka, V. M. A. (2021) · Ecography · doi:10.1111/ecog.05557 · Cited by 20
Kelley, G. A., Kelley, K. S., & Pate, R. R. (2020) · Childhood Obesity · doi:10.1089/chi.2020.0056 · Cited by 21
“…of the investigators’ knowledge, this is the first study to use the meta-analytic approach, considered by some to be the ‘‘grandmother’’ of big data, 27 to use SD data for examining inter-individual response variance as it pertains to the effects of aerobic exercise training on fat mass and percent…”
Peng, B., Liao, P., & Jiang, Y. (2024) · Langmuir · doi:10.1021/acs.langmuir.4c00274 · Cited by 22
Incorporating Genetic Diversity to Optimize the Plant Conservation Network in the Third Pole
Wambulwa, M. C., Zhu, G., Luo, Y., Wu, Z., Provan, J., Cadotte, M. W., Jump, A. S., Wachira, F. N., Gao, L., Yi, T., et al. (2025) · Global Change Biology · doi:10.1111/gcb.70122 · Cited by 19
“We acknowledge that the disproportionate distribution of genetic data in the study area may impact the reliability of our results, as is often the case in meta-analyses (Gurevitch et al. 2018).”
Dar, S. A., & Gani, K. M. (2025) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2025.179304 · Cited by 18
A global meta-analysis reveals the toxicity of plastics on insect health
Abbas, M., Jafir, M., Nazir, T., Hussain, S., Sarwar, N., Song, L., & Wan, X. (2025) · Ecotoxicology and Environmental Safety · doi:10.1016/j.ecoenv.2025.118611 · Cited by 21
Scotti, K. L., Young, S. M. N., Gainey, M. A., & Lan, H. (2025) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.70046 · Cited by 37
“Challenges in replicating research findings across disciplines has underscored the need for rigorous evidence synthesis methods as a mechanism for identifying sources of variability in results and promoting the reliability of research [5, 16].”
Harabiš, F., & Hronková, J. (2020) · European Journal of Entomology · doi:10.14411/eje.2020.035 · Cited by 18
“We have already mentioned meta-analytical methods, which many scientists perceive to be very powerful tools for solving very complex issues (Gurevitch et al., 2018).”
BackgroundNovel robust time series analysis for long-term and short-term prediction
Okamura, H., Osada, Y., Nishijima, S., & Eguchi, S. (2021) · Scientific Reports · doi:10.1038/s41598-021-91327-8 · Cited by 21
Lee, S., Yang, H., Lee, H.-J., Park, D. J., Kong, S., & Park, S. K. (2022) · Frontiers in Genetics · doi:10.3389/fgene.2022.928783 · Cited by 18
“A meta-analysis was performed to synthesize a total of 522 SNPs associated with GC that were included in 12 eligible studies as follows (Gurevitch et al., 2018):”
MethodTeaching Quality and Student Outcomes in TIMSS and PISA
Klieme, E., & Nilsen, T. (2022) · Springer international handbooks of education · doi:10.1007/978-3-030-88178-8_37 · Cited by 22
Boleydei, H., & Vaneeckhaute, C. (2024) · Journal of Water Process Engineering · doi:10.1016/j.jwpe.2024.105523 · Cited by 17
Papaefthimiou, M., Kontou, P. I., Bagos, P. G., & Braliou, G. G. (2024) · Antioxidants · doi:10.3390/antiox13060692 · Cited by 24
“Moreover, it can identify sources of heterogeneity in outcomes and, thus, shed light on the overall phenomenon and help examine factors that modify outcomes [56].”
BackgroundMethodRefinement on the Way Towards Replacement: Are We Doing What We Can?
Herrmann, K. (2019) · BRILL eBooks · doi:10.1163/9789004391192_002 · Cited by 41
An Overview of Systematic Reviews of Randomized Controlled Trials on Acupuncture Treating Migraine
Zhang, X., Li, X., Zhao, C., Hu, Y., Lin, Y., Chen, H., Shi, Z., Zhang, X., Shang, H., & Tian, G. (2019) · Pain Research and Management · doi:10.1155/2019/5930627 · Cited by 17
“Systematic reviews can provide a robust review of the effectiveness of clinical interventions [9]; according to the literature search, clinical researchers have completed a number of systematic reviews of acupuncture treating migraine.”
BackgroundUsing digital devices to help people lose weight: a systematic review
Besson, M., Gurviez, P., & Carins, J. (2020) · Journal of Social Marketing · doi:10.1108/jsocm-07-2019-0115 · Cited by 16
Questionable Research Practices
O’Boyle, E. H., & Götz, M. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780190938550.003.0010 · Cited by 28
Xu, C., Doi, S. A. R., Zhou, X., Lin, L., Furuya‐Kanamori, L., & Tao, F. (2022) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2022.101708 · Cited by 20
Krüeger, F., Bellucci, G., Xu, P., & Feng, C. (2020) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2020.00176 · Cited by 19
“…metaanalyses (Feng et al., 2015; Bellucci et al., 2017a)—in combination with task-based and task-free functional connectivity analyses (Gurevitch et al., 2018)—has revealed the right anterior insula (R AI) as a candidate brain region for detection of norm deviations in trusting…”
BackgroundResultGrossule, V., Henjak, M., Beggio, G., & Tomberlin, J. K. (2024) · Journal of Environmental Management · doi:10.1016/j.jenvman.2024.123605 · Cited by 21
Detection, assessment, and control strategies for managing Vibrio parahaemolyticus risk in seafood
Liu, J., Xiong, R., Wu, Q., Xu, T., Caridad, O. O., Zhu, Y., Pan, Y., Malakar, P. K., Zhao, Y., & Zhang, Z. (2025) · Food Quality and Safety · doi:10.1093/fqsafe/fyae051 · Cited by 25
Li, B., Chen, C., Zhou, X., Liu, H., Zhou, Z., Wang, X., Liang, J., Guo, Y., & Liang, S. (2025) · Antioxidants · doi:10.3390/antiox14050609 · Cited by 18
“Meta-analysis, a statistical analysis technique that synthesizes results from a group of independent studies related to a specific topic and design, enables comprehensive understanding of research questions and identification of sources of variability in outcomes [24].”
Urban morphology and residential building net-zero energy performance: A global meta-analysis
Li, H., Zheng, J., Zhang, Y., & Zhang, R. (2025) · Energy and Buildings · doi:10.1016/j.enbuild.2025.116142 · Cited by 19
Crystal‐Ornelas, R., Brown, J. A., Valentin, R. E., Beardsley, C., & Lockwood, J. L. (2021) · Ornithological Applications · doi:10.1093/ornithapp/duab040 · Cited by 19
“However, robust meta-analyses can only incorporate existing evidence that fits inclusion criteria (Gurevitch et al. 2018).”
Zhao, K., Wen, Y., Bunpetch, V., Lin, J., Hu, Y., Zhang, X., Xie, Y., Zhang, S., & Hongwei, O. (2022) · Journal of Orthopaedic Translation · doi:10.1016/j.jot.2021.11.006 · Cited by 16
“Meta-analytic approach is a largescale effort to collect data across different studies to form a generalizable system and make evidence-based decisions [53].”
BackgroundGrames, E. M., Montgomery, G. A., Boyes, D., Dicks, L. V., Forister, M. L., Matson, T. A., Nakagawa, S., Prendergast, K. S., Taylor, N. G., Tingley, M. W., et al. (2022) · Conservation Science and Practice · doi:10.1111/csp2.12687 · Cited by 25
“Rigorous methods of evidence synthesis were first introduced to conservation biology over two decades ago (Pullin & Knight, 2001) and have existed for nearly half a century in fields such as medicine and public health (Glass, 1976; Gurevitch et al., 2018).”
MethodSelf-citationWang, C., & Kang, J. (2022) · Building Simulation · doi:10.1007/s12273-022-0901-4 · Cited by 17
“It is widely used in medicine, astronomy, and other fields (Hedges 1992; Gurevitch et al. 2018).”
MethodKnogler, M., Hetmanek, A., & Seidel, T. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.873995 · Cited by 27
“field of practice, but rather for synthesizing findings in a particular research area for theory-building and for reaching broad generalizations (Gurevitch et al., 2018).”
BackgroundFinestack, L. H., Elmquist, M., Kuchler, K., Ford, A. L. B., Cakir-Dilek, B., Riegelman, A., Brown, S. J., & Marsalis, S. E. (2022) · Journal of Speech Language and Hearing Research · doi:10.1044/2022_jslhr-21-00543 · Cited by 22
Ziemski, M., Adamov, A., Kim, L., Flörl, L., & Bokulich, N. A. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac639 · Cited by 24
“Meta-analysis allows the synthesis of findings from individual studies to reach a more complete understanding: identifying consistent and reproducible signatures across studies, and resolving causes of variation among study results (Gurevitch et al., 2018).”
BackgroundPereira, F., Fernández‐Barral, A., Larriba, M. J., Barbáchano, A., & González‐Sancho, J. M. (2023) · FEBS Journal · doi:10.1111/febs.16955 · Cited by 19
“Although RCTs provide high-level evidence to establish causality, systematic reviews and meta-analysis are of great value in evaluating and synthesizing the data to reach broad generalizations across a large number of study outcomes and to give a more comprehensive picture [294].”
BackgroundSantos, C. C., Rosa, D., Gonçalves, J. M. S., & Coelho, R. (2023) · Aquatic Conservation Marine and Freshwater Ecosystems · doi:10.1002/aqc.4027 · Cited by 29
“To address this challenge, statistical methods such as meta-analysis have been used to summarize the existing research outcomes, helping scientists to reach conclusions on the effectiveness of a specific intervention (e.g. Koricheva, Gurevitch & Mengersen, 2013; Gurevitch et al., 2018).”
MethodShifting forward: Urban ecology in perspective
Pickett, S. T. A., Frantzeskaki, N., Andersson, E., Barau, A. A., Childers, D. L., Hoover, F.-A., Lugo, A. E., McPhearson, T., Nagendra, H., Schepers, S., et al. (2024) · AMBIO · doi:10.1007/s13280-024-02007-6 · Cited by 20
“…representing distinct disciplines or scales (Carpenter et al. 2009; Palmer et al. 2016); (2) bringing complementary data together in new ways (Gurevitch et al. 2018); (3) identifying neglected perspectives such as Indigenous ecological knowledge (Gewin 2021); and (4) conducting horizon…”
Wu, H. B., Jiao, X., Wang, Y., Wang, M., Zhou, J., Wu, K., & Chen, D. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.172991 · Cited by 22
Leclercq, V., Beaudart, C., Ajamieh, S., Tirelli, E., & Bruyère, O. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-036349 · Cited by 13
Solmi, M., Bodini, L., Cocozza, S., Seeman, M. V., Vieta, E., Dragioti, E., Carvalho, A. F., & Fusar‐Poli, P. (2020) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2020.09.635 · Cited by 17
van der Heijden, J. (2020) · Regulation & Governance · doi:10.1111/rego.12378 · Cited by 19
Mantia, D. L., Bernardini, C., Zannoni, A., Salaroli, R., Wang, C., Bencivenni, S., & Forni, M. (2022) · Animals · doi:10.3390/ani12060749 · Cited by 16
“Publication bias and search bias are potential problems in all meta-analyses [97]; this arises from the fact that unpublished studies may contradict the results due to the tendency not to publish negative studies, thus leading to the over-representation of “positive” ones [98].”
BackgroundResultHow failure to falsify in high-volume science contributes to the replication crisis
Rajtmajer, S. M., Errington, T. M., & Hillary, F. G. (2022) · eLife · doi:10.7554/elife.78830 · Cited by 25
Dyer, L. A., & Forister, M. L. (2018) · Current Opinion in Insect Science · doi:10.1016/j.cois.2018.11.008 · Cited by 16
Going beyond mean effect size: Presenting prediction intervals for on-farm network trial analyses
Laurent, A., Miguez, F. E., Kyveryga, P., & Makowski, D. (2020) · European Journal of Agronomy · doi:10.1016/j.eja.2020.126127 · Cited by 21
Scientific Integrity and Misconduct—Yet Again
Kharasch, E. D. (2021) · Anesthesiology · doi:10.1097/aln.0000000000003916 · Cited by 19
Methods to Address Confounding and Other Biases in Meta-Analyses: Review and RecommendationsPreprint
Mathur, M., & VanderWeele, T. (2021) · Annual Review of Public Health · doi:10.31219/osf.io/v7dtq · Cited by 81
“Meta-analyses contribute critically to cumulative science (5, 14), but they can produce biased estimates and misleading conclusions if their constituent primary studies are themselves biased.”
Background
Metadata
- Authors
- Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G.
- Affiliations
- Stony Brook University (US)
- Royal Holloway University of London (GB)
- Garvan Institute of Medical Research (AU)
- Newcastle University (GB)
- Corresponding author
- Gurevitch, Jessica
- Koricheva, Julia
- Nakagawa, Shinichi
- Stewart, Gavin
- Venue
- Nature
- Published
- 2018-03
- Volume
- 555
- Issue
- 7695
- Pages
- 175-182
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2791681873
- Semantic Scholar
- ac5ad1932e31c3ea141366e8e073d6d95869c4ef
- Cited by
- 1,709
- References
- 101
- Citations by year
- 2026: 177 · 2025: 230 · 2024: 231 · 2023: 199 · 2022: 234 · 2021: 235 · 2020: 204 · 2019: 130 · 2018: 64
- FWCI
- 31.20
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- UNSWorks (UNSW Sydney) (submittedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- Newcastle University ePrints (Newcastle Univesity)
- Funders
- National Science Foundation (1262402)
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- meta-analysis, evidence synthesis, history of meta-analysis, evolutionary biology, conservation biology, artificial intelligence, evidence-based practice
- MeSH terms
- Animals, Biological Science Disciplines, Ecology, Biological Evolution, Research, Meta-Analysis as Topic, History, 20th Century, History, 21st Century
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.