Meta-analysis and the science of research synthesis
Open accessThe DOI matches its Crossref record; 102 of the 102 references it cites were found in a registry or an index; 9 of the 69 sources its claims cite were checked for support.
Abstract
Meta-analysis is the quantitative, scientific synthesis of research results. Since the term and modern approaches to research synthesis were first introduced in the 1970s, meta-analysis has had a revolutionary effect in many scientific fields, helping to establish evidence-based practice and to resolve seemingly contradictory research outcomes. At the same time, its implementation has engendered criticism and controversy, in some cases general and others specific to particular disciplines. Here we take the opportunity provided by the recent fortieth anniversary of meta-analysis to reflect on the accomplishments, limitations, recent advances and directions for future developments in the field of research synthesis.
Overview
CoChat Summary
TL;DR
This review reflects on meta-analysis as quantitative research synthesis since the 1970s, its contributions to evidence-based practice, and its methodological development across medicine, social sciences, and ecology/evolution/conservation. It argues that meta-analysis has resolved contradictory evidence and enabled broad generalizations, but faces a “midlife crisis” from poor-quality applications, publication/research bias, and misuse of the term.
Key findings
- Meta-analysis was first introduced in the 1970s; the term was coined by Gene Glass in 1976, and the paper marks its 40th anniversary. It has helped establish evidence-based practice, resolve seemingly contradictory results, and spread across many scientific fields.
- In medicine, a meta-analysis of 20 clinical studies found maternal obesity associated with increased neural tube defect risk: a 1.7-fold rise for moderately obese women and over a 3-fold risk for severely obese women. In social science, a meta-analysis of 35 studies found no significant differences between multisystemic therapy and conventional social services for serious juvenile offenders.
- In ecology, evolution and conservation, meta-analysis was adopted in the early 1990s. Parmesan and Yohe used data on over 1,700 species to show significant range shifts and advancement of spring events; Benayas and colleagues found ecological restoration increased biodiversity and ecosystem services, though values remained lower in restored versus intact reference ecosystems.
- Methodological advances include fixed- and random-effects models, meta-regression, heterogeneity quantification (τ and I²), publication-bias tools, network meta-analysis, phylogenetic meta-analysis, and software such as MetaWin and OpenMEE. PRISMA includes a 27-item checklist for systematic review reporting.
Methods
- Design: narrative review/commentary on meta-analysis and research synthesis, not a new meta-analysis. The authors state they highlight principles, limitations, utility, and future developments.
- Data or sample: draws on published literature and illustrative examples across medicine, social sciences, and ecology/evolution/conservation; includes Box 1 on meta-analytic models and Table 1 on EEC milestones.
- Main analysis: conceptual and methodological synthesis; no primary data collection, systematic search protocol, or quantitative pooling is reported in the paper.
Limitations
- The paper states that meta-analysis and systematic reviews can highlight evidence deficiencies but cannot overcome them; publication bias and research bias may be suspected and sometimes estimated but cannot truly be corrected; selective or incomplete reporting can constrain synthesis.
- The paper states that too many published meta-analyses are of low quality, the term “meta-analysis” is frequently misapplied, and vote-counting persists despite being discredited; it also notes concerns about academic inequality and the “research parasites” controversy.
- The text makes evident that, as a narrative review, it does not provide a systematic search strategy, formal inclusion criteria, or a quantitative synthesis; its examples are illustrative rather than exhaustive.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 4 Not valid · 37 Not checked · 33 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Similarly, a synthesis may be constrained by either selective or incomplete reporting 66,67 .”
What the sources say
- Not checked yetcites [66] Stewart, L. A . et al. Preferred reporting items for a systematic review and meta-analysis of individual participant data: the PRISMA-IPD statement. J. Am. Med. Assoc. 313, 1657–1665 (2015) source
- Partly supportedcites [67] Saldanha, I. J . et al. Evaluating Data Abstraction Assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial. Syst. Rev. 5, 196 (2016) sourceExperimental — accuracy not yet measured
The Background (page unavailable) reports that omissions and errors during data abstraction changed summary meta-analytic results, supporting the general point that a synthesis can be affected by missing information. It describes information present in articles but omitted by abstractors, not selective or incomplete reporting by the studies themselves; that specific mechanism is unsupported. (LLM, conf=0.78)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“When the number of studies is an order of magnitude greater than that, the inability of narrative processes to adequately summarize the outcomes increases exponentially 2,3 .”
What the sources say
- Not checked yetcites [2] Roberts, P. D ., Stewart, G. B. & Pullin, A. S. Are review articles a reliable source of evidence to support conservation and environmental management? A comparison with medicine. Biol. Conserv. 132, 409–423 (2006) source
- Partly supportedcites [3] Bastian, H ., Glasziou, P . & Chalmers, I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 7, e1000326 (2010) sourceExperimental — accuracy not yet measured
Our copy supports the qualitative half of the claim: the 'How Close Are We to Archie Cochrane's Goal?' section cites Goldschmidt and Mulrow for 'how great the potential is for error in reviews of health literature that were not conducted systematically' and states that 'the staple of medical literature synthesis remains the non-systematic narrative review', while the same section (Figure 4 discussion) and 'Where to Now?' document the growth from ~14 trials a day in 1979 to 75 trials and 11 systematic reviews a day, with 'the task keeps increasing in size and complexity' ('Where Are We Now?'). What the source does NOT support is the quantitative assertion attributed to it: nowhere does our copy say the inability of narrative processes to summarise outcomes 'increases exponentially', nor tie it to study counts an order of magnitude beyond a dozen — the only dozen-scale figure in the paper is the median number of trials per systematic review (six to 16, Cochrane reviews averaging over 12), which is offered as a measure of reviewing workload and coverage, not of narrative-review failure. The exponential scaling is therefore the citing authors' characterisation, not this paper's. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, as the number of publications and research results has mushroomed across scientific fields, the difficulties encountered in carrying out narrative reviews to identify and summarize evidence in a transparent manner has become increasingly apparent 1 .”
What the sources say
- Not checked yetcites [1] Jennions, M. D ., Lortie, C. J. & Koricheva, J. in The Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J . et al.) Ch. 23, 364–380 (Princeton Univ. Press, 2013) source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, in many cases of the resolution of scientific problems, when the results of an individual study are combined with those of other studies, the resulting synthesis provides a more general and complete picture of the questions being addressed.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A statement the paper makes itself — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Sometimes, the striking results of initial studies are not confirmed by those of subsequent studies or by syntheses of a body of research.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
15 more claims
Create a free account to see all 78 claims and to verify the 60 references still open.
- It may state two separate claims as one. 32
- Probably not a substantial claim. 48
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The trade-off with our method is that it does not allow for determining the magnitude of the effect of the practices33–36 but only the proportion of impacts33,37 as positive, negative or neutral.”
From Farming practices to enhance biodiversity across biomes: a systematic review (2024) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Besides, publication biases that have been documented in several scientific fields90,91 can also affect ML-based science.92,93”
From Leakage and the reproducibility crisis in machine-learning-based science (2023) · cites this paper as [91]
Experimental — accuracy not yet measuredIn the citing paper
“Moreover, since a meta-analysis can be conducted only when sufficient and appropriate quantitative data from original studies are available, the number of published meta-analyses about a certain topic is an indicator of the overall understanding of the subject, allowing us to reveal research gaps (25, 26).”
From Agricultural diversification promotes multiple ecosystem services without compromising yield (2020) · cites this paper as [25]
In the citing paper
“Meta-analysis optimises statistical power for detecting treatment effects by combining data from multiple similar studies [22, 23].”
From Does the growing of Bt maize change abundance or ecological function of non-target animals compared to the growing of non-GM maize? A systematic review (2022) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Box 1: Meta-analytic models and heterogeneity
“𝜀 is the within-study sampling error (solid black lines), the deviation from the study-specific mean.”
§ Box 1: Meta-analytic models and heterogeneity
“the solid black lines are deviations from the overall mean (panels a,b) or slope (panels c,d)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Box 1: Meta-analytic models and heterogeneity
“The assumption of the fixed-effect meta-regression is that a moderator(s) can explain all between-study heterogeneity”
§ Box 1: Meta-analytic models and heterogeneity
“the blue distributions the between-study variance (panels a-d).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Box 1: Meta-analytic models and heterogeneity
“Two basic models of meta-analysis are the fixed-effect model (Fig 1a) and the random-effects model (Fig 1b) 6 .”
§ Box 1: Meta-analytic models and heterogeneity
“Figure 1. A variety of charts and plots common in meta-analysis. a. PRISMA diagram 20 , b. a forest plot showing overall means based on a fixed-effect model and random-effects model”
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 1×
“Procedural protocols for systematic reviews are well established especially in the medical and social sciences (Preferred Reporting Items for Systematic Reviews, or PRISMA; www.prisma-statement.org) 20 .”
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · cited 1×
“…the use of machine learning to screen studies 82 for inclusion in systematic reviews and meta-analyses for large reviews, increasingly sophisticated software and models for complex meta-regression 53,83 , and integration of meta-analysis with decision support in medicine and other domains 84 .”
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Research works printed but not linked: 3
- Hedges, L. & Olkin, I. Statistical Methods for Meta-analysis (Academic Press, 1985)
- Higgins, J. P. T . & Green, S. (eds) Cochrane Handbook for Systematic Reviews of Interventions: Version 5.1.0 (Wiley, 2011).This large collaborative work provides definitive guidance for the production of systematic reviews in medicine and is of broad interest for methods development outside the medical field.
- Cooper, H. M ., Hedges, L. V . & Valentine, J. C. (eds) The Handbook of Research Synthesis and Meta-analysis 2nd edn (Russell Sage Foundation, 2009).This book is an important compilation that builds on the ground-breaking first edition to set the standard for best practice in meta-analysis, primarily in the social sciences but with applications to medicine and other fields.
Cited by
Items 101–200 of 1,000
Bat responses to climate change: a systematic review
Festa, F., Ancillotto, L., Santini, L., Pacifici, M., Rocha, R., Toshkova, N., Amorim, F., Benítez‐López, A., Domer, A., Hamidović, D., et al. (2022) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/brv.12893 · Cited by 164
“Hence, our studies were not homogenous and sample sizes for each response were too low to allow systematic extraction of quantitative data that could be summarised as standardised effect sizes for a meta-analysis (Gurevitch et al., 2018).”
BackgroundSoil organic carbon in irrigated agricultural systems: A meta‐analysis
Emde, D., Hannam, K. D., Most, I., Nelson, L. M., & Jones, M. D. (2021) · Global Change Biology · doi:10.1111/gcb.15680 · Cited by 169
Tillage Intensity Effects on Soil Structure Indicators—A US Meta-Analysis
Nunes, M. R., Karlen, D. L., & Moorman, T. B. (2020) · Sustainability · doi:10.3390/su12052071 · Cited by 171
“Effect size is commonly used in meta-analyses to standardize results by providing a summary of the magnitude and direction of treatment effects [30].”
MethodAn Analysis of Mesenchymal Stem Cell-Derived Extracellular Vesicles for Preclinical Use
Tieu, A., Lalu, M. M., Slobodian, M., Gnyra, C., Fergusson, D. A., Montroy, J. R., Burger, D. E., Stewart, D. J., & Allan, D. (2020) · ACS Nano · doi:10.1021/acsnano.0c01363 · Cited by 153
García‐Robles, P., Cortés‐Pérez, I., Nieto-Escámez, F. A., García‐López, H., Obrero‐Gaitán, E., & Osuna‐Pérez, M. C. (2024) · Anatomical Sciences Education · doi:10.1002/ase.2397 · Cited by 191
“101,102 Publication bias was explored in the meta-analyses of independent groups but was not recommended for proportional meta-analyses.”
BackgroundAzeem, I., Adeel, M., Ahmad, M. A., Shakoor, N., Zain, M., Youssef, N. A., Yinghai, Z., Azeem, K., Zhou, P., White, J. C., et al. (2022) · Environmental Science & Technology Letters · doi:10.1021/acs.estlett.2c00107 · Cited by 141
Do cultural differences affect users’ e‐learning adoption? A meta‐analysis
Zhao, Y., Wang, N., Li, Y., Zhou, R., & Li, S. (2021) · British Journal of Educational Technology · doi:10.1111/bjet.13002 · Cited by 132
Chen, L., Thorup, V. M., Kudahl, A. B., & Østergaard, S. (2023) · Journal of Dairy Science · doi:10.3168/jds.2023-24059 · Cited by 188
“Traditionally, weighting studies based on inversed variance can incorporate information on the quality of included studies and decrease the influence of small studies (Gurevitch et al., 2018).”
BackgroundA declaração PRISMA 2020: diretriz atualizada para relatar revisões sistemáticas
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T., Mulrow, C. D., Shamseer, L., Tetzlaff, J., Akl, E. A., Brennan, S., et al. (2022) · Epidemiologia e Serviços de Saúde · doi:10.5123/s1679-49742022000200033 · Cited by 387
Effects of nitrogen enrichment on tree carbon allocation: A global synthesis
Li, W., Zhang, H., Huang, G., Liu, R., Wu, H., Zhao, C., & McDowell, N. G. (2019) · Global Ecology and Biogeography · doi:10.1111/geb.13042 · Cited by 148
A systematic review and meta-analysis of the relationship between flow states and performance
Harris, D., Allen, K., Vine, S. J., & Wilson, M. (2021) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1929402 · Cited by 162
“As systematic reviews serve to highlight research gaps and suggest directions for future empirical work (Gurevitch et al., 2018; Petticrew & Roberts, 2008) a review of the fl ow – performance relationship could be highly valuable for a theoretical understanding of fl ow, as well as for designing e ff…”
Three-bestseller pesticides in Brazil: Freshwater concentrations and potential environmental risks.
Brovini, E. M., de Deus, B. C. T., Vilas-Boas, J., Quadra, G., Carvalho, L., Mendonça, R., Pereira, R. E. L. D. O., & Cardoso, S. (2021) · Science of the Total Environment · doi:10.1016/j.scitotenv.2020.144754 · Cited by 103
Microbial Responses to Biochar Soil Amendment and Influential Factors: A Three-Level Meta-Analysis
Kerner, P., Struhs, E., Mirkouei, A., Aho, K. A., Lohse, K. A., Dungan, R. S., & You, Y. (2023) · Environmental Science & Technology · doi:10.1021/acs.est.3c04201 · Cited by 149
Zhou, G., Luo, Q., Chen, Y., He, M., Zhou, L., Frank, D. A., He, Y., Fu, Y., Zhang, B., & Zhou, X. (2018) · Global Change Biology · doi:10.1111/gcb.14533 · Cited by 134
Gu, R., Huang, W., Camilleri, J., Xu, P., Wei, P., Eickhoff, S. B., & Feng, C. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.02.017 · Cited by 124
“Meta-analysis is an increasingly popular method to overcome the heterogeneity and divergence of previous results and assess the consistency of previous findings (Gurevitch et al., 2018; Kotov et al., 2010; Müller et al., 2018).”
Dastogeer, K. M. G., Zahan, M. I., Tahjib‐Ul‐Arif, M., Akter, M. A., & Okazaki, S. (2020) · Frontiers in Plant Science · doi:10.3389/fpls.2020.588550 · Cited by 148
“Meta-analyses have been used broadly in various disciplines and have become increasingly common in plant ecology and evolutionary biology (Lau et al., 2013; Koricheva and Gurevitch, 2014; Gerstner et al., 2017; Gurevitch et al., 2018).”
MethodP-hacking and publication bias interact to distort meta-analytic effect size estimates
Friese, M., & Frankenbach, J. (2020) · Psychological Methods · doi:10.1037/met0000246 · Cited by 122
Guideline on allergen immunotherapy in IgE-mediated allergic diseases
Pfaar, O., Ankermann, T., Augustin, M., Bubel, P., Böing, S., Brehler, R., Eng, P. A., Fischer, P. J., Gerstlauer, M., Hamelmann, E. H., et al. (2022) · Allergologie select · doi:10.5414/alx02331e · Cited by 136
“The term meta-analysis ought to be only used when established statistical methods such as appropriate calculation of effect size, weighting, and analysis of heterogeneity as well as statistical models taking into account the different hierarchical structure of meta-analytic data have been followed [79, 80, 81].”
BackgroundMethodNakagawa, S., Noble, D. W. A., Lagisz, M., Spake, R., Viechtbauer, W., & Senior, A. M. (2022) · Ecology Letters · doi:10.1111/ele.14144 · Cited by 143
Self-citationDams have varying impacts on fish communities across latitudes: a quantitative synthesis
Turgeon, K., Turpin, C., & Gregory‐Eaves, I. (2019) · Ecology Letters · doi:10.1111/ele.13283 · Cited by 141
Zaboski, B. A., Kranzler, J. H., & Gage, N. A. (2018) · Journal of School Psychology · doi:10.1016/j.jsp.2018.10.001 · Cited by 142
Effect of salinity stress on rice yield and grain quality: A meta-analysis
Zheng, C., Liu, C., Liu, L., Tan, Y., Sheng, X., Yu, D., Sun, Z., Xuewu, S., Chen, J., Yuan, D., et al. (2023) · European Journal of Agronomy · doi:10.1016/j.eja.2023.126765 · Cited by 150
Zhao, J., Xie, X., Jiang, Y., Li, J., Fu, Q., Qiu, Y., Fu, X., Yao, Z., Dai, Z., Qiu, Y., et al. (2023) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2023.168793 · Cited by 136
Effect of microplastics on aquatic biota: A hormetic perspective
Sun, T., Zhan, J., Li, F., Ji, C., & Wu, H. (2021) · Environmental Pollution · doi:10.1016/j.envpol.2021.117206 · Cited by 124
Response of soil organic carbon content to crop rotation and its controls: A global synthesis
Liu, X., Tan, S., Song, X., Wu, X., Zhao, G., Li, S., & Liang, G. (2022) · Agriculture Ecosystems & Environment · doi:10.1016/j.agee.2022.108017 · Cited by 129
Nair, U., Armfield, N. R., Chatfield, M. D., & Edirippulige, S. (2018) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x18794332 · Cited by 111
EviAtlas: a tool for visualising evidence synthesis databases
Haddaway, N., Feierman, A., Grainger, M., Gray, C. T., Tanriver-Ayder, E., Dhaubanjar, S., & Westgate, M. J. (2019) · Environmental Evidence · doi:10.1186/s13750-019-0167-1 · Cited by 141
Mori, A., Cornelissen, J. H. C., Fujii, S., Okada, K.-I., & Isbell, F. (2020) · Nature Communications · doi:10.1038/s41467-020-18296-w · Cited by 124
Calculation of external climate costs for food highlights inadequate pricing of animal products
Pieper, M. M., Michalke, A., & Gaugler, T. (2020) · Nature Communications · doi:10.1038/s41467-020-19474-6 · Cited by 135
Revisiting and expanding the meta‐analysis of variation: The log coefficient of variation ratio
Senior, A. M., Viechtbauer, W., & Nakagawa, S. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1423 · Cited by 110
Self-citationEating‐disorder psychopathology in female athletes and non‐athletes: A meta‐analysis
Chapa, D. A. N., Johnson, S. N., Richson, B. N., Bjorlie, K., Won, Y. Q., Nelson, S. V., Ayres, J., Jun, D., Forbush, K. T., Christensen, K. A., et al. (2022) · International Journal of Eating Disorders · doi:10.1002/eat.23748 · Cited by 107
Plant diversity loss reduces soil respiration across terrestrial ecosystems
Chen, X., & Chen, H. Y. H. (2019) · Global Change Biology · doi:10.1111/gcb.14567 · Cited by 112
Elevated CO 2 shifts soil microbial communities from K ‐ to r ‐strategists
Sun, Y., Wang, C., Yang, J., Liao, J., Chen, H. Y. H., & Ruan, H. (2021) · Global Ecology and Biogeography · doi:10.1111/geb.13281 · Cited by 115
“A quantitative synthesis of results across various studies could assist in determining the overall effects of elevated CO2 on microbial communities and identify sources of variations (Gurevitch et al., 2018).”
BackgroundEffect of biochar addition on crop yield, water and nitrogen use efficiency: A meta-analysis
Han, M., Zhang, J., Zhang, L., & Wang, Z. (2023) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2023.138425 · Cited by 123
Accumulating evidence in ecology: Once is not enough
Nichols, J. D., Kendall, W. L., & Boomer, G. S. (2019) · Ecology and Evolution · doi:10.1002/ece3.5836 · Cited by 118
Significant response of microbial community to increased salinity across wetland ecosystems
Chen, H., Ma, K., Huang, Y., Fu, Q., Qiu, Y., & Yao, Z. (2022) · Geoderma · doi:10.1016/j.geoderma.2022.115778 · Cited by 123
Wu, J., Zhang, H., Pan, Y., Cheng, X., Zhang, K., & Liu, G. (2023) · Soil and Tillage Research · doi:10.1016/j.still.2023.105770 · Cited by 115
Savilaakso, S., Johansson, A., Häkkilä, M., Uusitalo, A., Sandgren, T., Mönkkönen, M., & Puttonen, P. (2021) · Environmental Evidence · doi:10.1186/s13750-020-00215-7 · Cited by 129
Brunauer, M., Roch, F., & Conrady, B. (2021) · Animals · doi:10.3390/ani11041014 · Cited by 138
“Thus, results of the meta-analysis are more powerful and less bi-Animals ased than conventional statistical methods and/or results of an individual study regarding NCD prevalences [13,38,85].”
Wassenaar, T. M., Yaffe, K. S., van der Werf, Y. D., & Sexton, C. E. (2019) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2019.04.006 · Cited by 122
“Meta-analyses could further help synthesize findings (Gurevitch et al., 2018) to inform public health discourse and give direction to research.”
BackgroundA global meta‐analysis of nitrous oxide emission from drip‐irrigated cropping system
Kuang, W., Gao, X., Tenuta, M., & Zeng, F. (2021) · Global Change Biology · doi:10.1111/gcb.15636 · Cited by 116
“In cases where the Funnel plots and Egger's regression test showed publication bias, the trim and fill method was applied to estimate the missing studies and their effect on the mean effect size or EF (Duval & Tweedie, 2000; Gurevitch et al., 2018).”
MethodTemporal Instability of Evidence Base: A Threat to Policy Making?
Koricheva, J., & Kulinskaya, E. (2019) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2019.05.006 · Cited by 104
Self-citationReplication in field ecology: Identifying challenges and proposing solutions
Filazzola, A., & Cahill, J. F. (2021) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13657 · Cited by 137
International treaties have mostly failed to produce their intended effects
Hoffman, S. J., Baral, P., Katwyk, S. R. V., Sritharan, L., Hughsam, M., Randhawa, H., Lin, G. O., Campbell, S., Campus, B., Dantas, M., et al. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2122854119 · Cited by 147
“Integrating these techniques (40) allows us to draw inferences from all 224 studies that together represent the most comprehensive collection of evidence yet compiled on treaties’ effects.”
BackgroundLi, K., Chen, Y., & Zhang, L. (2020) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2020.102107 · Cited by 135
Liu, Y., Xiao, Z., Chen, F., Yue, L., Zou, H., Lyu, J., & Wang, Z. (2021) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.146578 · Cited by 101
Cinar, O., Nakagawa, S., & Viechtbauer, W. (2021) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13760 · Cited by 124
Self-citationTwenty years of network meta‐analysis: Continuing controversies and recent developments
Ades, A. E., Welton, N. J., Dias, S., Phillippo, D., & Caldwell, D. M. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1700 · Cited by 127
“” 33 First, the estimation of specific treatment effects in specific populations, and second the broad generalization often based on large numbers of studies, accompanied by attempts to identify the reasons for heterogeneity.”
A Meta‐Analysis on Nitrogen Retention by Buffer Zones
Valkama, E., Usva, K., Saarinen, M., & Uusi‐Kämppä, J. (2018) · Journal of Environmental Quality · doi:10.2134/jeq2018.03.0120 · Cited by 100
King, A. E., Congreves, K. A., Deen, B., Dunfield, K. E., Voroney, R. P., & Wagner‐Riddle, C. (2019) · Soil Biology and Biochemistry · doi:10.1016/j.soilbio.2019.04.019 · Cited by 112
Meta-analysis reveals global variations in plant diversity effects on productivity
Chen, C., Xiao, W., & Chen, H. Y. H. (2025) · Nature · doi:10.1038/s41586-024-08407-8 · Cited by 126
“The weight for each observation is usually calculated by the inverse of sampling variance when it was available 68 .”
Lee, S. W., & Koo, M. J. (2022) · Life Cycle · doi:10.54724/lc.2022.e9 · Cited by 99
“The term “meta-analysis” was first coined in 1976, and the research in this field has grown explosively, especially in the medical field, since the 1990s.[1] Meta-analysis is of particular importance because a myriad of studies has been conducted by different groups in recent years.”
BackgroundPlastic pollution in aquatic systems in Bangladesh: A review of current knowledge
Chowdhury, G. W., Koldewey, H. J., Duncan, E. M., Napper, I. E., Niloy, M. N. H., Nelms, S. E., Sarker, S., Bhola, S., & Nishat, B. (2020) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2020.143285 · Cited by 110
Chen, J., Xiao, H., Li, Z., Liu, C., Ning, K., & Tang, C. (2020) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2020.139517 · Cited by 98
Effects of biochar on soil chemical properties: A global meta-analysis of agricultural soil
Sun, Z., Hu, Y., Shi, L., Li, G., Pang, Z., Liu, S., Chen, Y., & Jia, B. (2022) · Plant Soil and Environment · doi:10.17221/522/2021-pse · Cited by 124
Oser, J. L., Grinson, A., Boulianne, S., & Halperin, E. (2022) · Political Communication · doi:10.1080/10584609.2022.2086329 · Cited by 153
“We used Peterson and Brown’s (2005) approach to convert the standardized coefficients to Pearson’s correlation coefficients (r), and then to Fisher’s z scale with a corresponding standard error (Borenstein et al., 2009; Gurevitch et al., 2018; Lipsey & Wilson, 2001).”
MethodFarming practices to enhance biodiversity across biomes: a systematic review
Cozim-Melges, F., Ripoll-Bosch, R., (Ciska) Veen, G. F., Oggiano, P., Bianchi, F. J. J. A., van der Putten, W. H., & van Zanten, H. H. E. (2024) · npj Biodiversity · doi:10.1038/s44185-023-00034-2 · Cited by 97
“The trade-off with our method is that it does not allow for determining the magnitude of the effect of the practices 33 – 36 but only the proportion of impacts 33,37 as positive, negative or neutral.”
Wassenaar, T. M., Williamson, W., Johansen-Berg, H., Dawes, H., Roberts, N., Foster, C., & Sexton, C. E. (2020) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-020-00959-y · Cited by 97
“Although meta-analyses are a useful statistical tool to quantitatively synthesise findings [54, 55], care should be taken in their implementation.”
MethodMeta-analysis challenges a textbook example of status signalling and demonstrates publication bias
Sánchez‐Tójar, A., Nakagawa, S., Sánchez-Fortún, M., Martin, D. A., Ramani, S., Girndt, A., Bókony, V., Kempenaers, B., Liker, A., Westneat, D. F., et al. (2018) · eLife · doi:10.7554/elife.37385 · Cited by 108
“Meta-analyses are therefore able to provide more robust conclusions than single studies and are increasingly used in evolutionary ecology (Gurevitch et al., 2018; Nakagawa and Poulin, 2012a; Nakagawa and Santos, 2012b; Senior et al., 2016).”
BackgroundSelf-citationMeta-Analysis and Meta-Synthesis Methodologies: Rigorously Piecing Together Research
Leary, H., & Walker, A. E. (2018) · TechTrends · doi:10.1007/s11528-018-0312-7 · Cited by 136
Studying human–nature relationships through a network lens: A systematic review
Kluger, L. C., Gorris, P., Kochalski, S., Mueller, M. S., & Romagnoni, G. (2020) · People and Nature · doi:10.1002/pan3.10136 · Cited by 105
Updating the consensus on fishway efficiency: A meta‐analysis
Hershey, H. J. (2021) · Fish and Fisheries · doi:10.1111/faf.12547 · Cited by 109
“Meta- analysis is an excellent tool for synthesizing studies with variation in sample sizes and effect sizes (i.e. efficiency estimates), because it is a rigid framework with well- documented methodologies for identifying the sources of the variation (Gurevitch et al., 2018).”
MethodThe COVID-NMA Project: Building an Evidence Ecosystem for the COVID-19 Pandemic
Boutron, I., Chaimani, A., Meerpohl, J. J., Hróbjartsson, A., Devane, D., Rada, G., Tovey, D., Grasselli, G., & Ravaud, P. (2020) · Annals of Internal Medicine · doi:10.7326/m20-5261 · Cited by 95
“E before the coronavirus disease 2019 (COVID19) pandemic, the ability of the evidence synthesis model to meet the needs of stakeholders was challenged (1, 2).”
BackgroundSun, T., Wang, S., Ji, C., Li, F., & Wu, H. (2021) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2021.127533 · Cited by 103
Burton, A. M., Cowburn, I. H. J., Thompson, F., Eisenmann, J. C., Nicholson, B., & Till, K. (2023) · Sports Medicine · doi:10.1007/s40279-023-01886-1 · Cited by 175
Liu, C., Li, W., Xu, J., Wei, W., Xue, P., & Yan, H. (2020) · Soil and Tillage Research · doi:10.1016/j.still.2020.104822 · Cited by 97
Microplastic pollution in Bangladesh: Research and management needs
Islam, T., Li, Y., Rob, M. M., & Cheng, H. (2022) · Environmental Pollution · doi:10.1016/j.envpol.2022.119697 · Cited by 101
Korous, K. M., Causadías, J. M., Bradley, R. H., Luthar, S. S., & Levy, R. (2020) · Psychological Reports · doi:10.1177/0033294120984127 · Cited by 123
“Because one of the goals of metaanalysis is to provide more precise estimates of effect sizes (Gurevitch et al., 2018), identifying key moderators that narrow CIs is important for advancing meta-analyses on SES.”
BackgroundSantos, M. J., Smith, A. B., Dekker, S. C., Eppinga, M. B., Leitão, P. J., Moreno‐Mateos, D., Morueta‐Holme, N., & Ruggeri, M. (2021) · Landscape Ecology · doi:10.1007/s10980-021-01276-w · Cited by 100
Spake, R., Bowler, D. E., Callaghan, C. T., Blowes, S. A., Doncaster, C. P., Antão, L. H., Nakagawa, S., McElreath, R. L., & Chase, J. M. (2023) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/brv.12939 · Cited by 108
“Meta-analysis is a widely used approach in ecology (Gurevitch et al ., 2018; Anderson et al ., 2021), which often explicitly sets out to test for and explain context dependence in ecological effects (e.g. Leal & Peixoto, 2017; Marino, Romero & Farjalla, 2018; Albertson et al ., 2021).”
Self-citationThe global fate of inorganic phosphorus fertilizers added to terrestrial ecosystems
Luo, X., Elrys, A. S., Zhang, L., Ibrahim, M. M., Liu, Y., Fu, S., Yan, J., Ye, Q., Wen, D., & Hou, E. (2024) · One Earth · doi:10.1016/j.oneear.2024.07.002 · Cited by 102
Review of carbon leakage under regionally differentiated climate policies
Yu, B., Zhao, Q., & Wei, Y. (2021) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.146765 · Cited by 107
Yu, L., Gao, X., & Zhao, X. (2019) · Agricultural Systems · doi:10.1016/j.agsy.2019.102723 · Cited by 98
Impacts of landscape changes on local and regional climate: a systematic review
Cao, Q., Liu, Y., Georgescu, M., & Wu, J. (2020) · Landscape Ecology · doi:10.1007/s10980-020-01015-7 · Cited by 101
Meta-Analysis of Present-Bias Estimation using Convex Time Budgets
Imai, T., Rutter, T. A., & Camerer, C. F. (2021) · The Economic Journal · doi:10.1093/ej/ueaa115 · Cited by 86
Meta-analyses of flipped classroom studies: A review of methodology
Hew, K. F., Bai, S., Dawson, P., & Lo, C. K. (2021) · Educational Research Review · doi:10.1016/j.edurev.2021.100393 · Cited by 138
Responses of soil pH to no‐till and the factors affecting it: A global meta‐analysis
Zhao, X., He, C., Liu, W., Liu, W., Liu, Q., Bai, W., Li, L., Lal, R., & Zhang, H. (2021) · Global Change Biology · doi:10.1111/gcb.15930 · Cited by 99
Nicholson, B., Dinsdale, A., Jones, B. L., & Till, K. (2020) · Sports Medicine · doi:10.1007/s40279-020-01372-y · Cited by 126
“, football code, sex, age, playing standard, stage of the season) that may impact upon short-sprint training outcome [62].”
BackgroundA better measure of research from the global south
Lebel, J., & McLean, R. K. D. (2018) · Nature · doi:10.1038/d41586-018-05581-4 · Cited by 90
Ferraro, S., Klugah‐Brown, B., Tench, C. R., Bazinet, V., Bore, M. C., Nigri, A., Demichelis, G., Bruzzone, M. G., Palermo, S., Zhao, W., et al. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104915 · Cited by 94
Sun, T., Zhan, J., Li, F., Ji, C., & Wu, H. (2021) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2021.125581 · Cited by 89
“Promisingly, meta-analysis is particularly powerful in integrating the conflicting findings with the same topic to determine a general trend and obtain a reliable estimator (Gurevitch et al., 2018).”
BackgroundDo Bioinoculants Affect Resident Microbial Communities? A Meta-Analysis
Cornell, C., Kokkoris, V., Richards, A., Horst, C., Rosa, D. J., Bennett, J. A., & Hart, M. M. (2021) · Frontiers in Agronomy · doi:10.3389/fagro.2021.753474 · Cited by 108
“Our approach was based on standard protocols described in Moher et al. (2015) and Gurevitch et al. (2018).”
MethodMethods to Address Confounding and Other Biases in Meta-Analyses: Review and Recommendations
Mathur, M. B., & VanderWeele, T. J. (2022) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-051920-114020 · Cited by 85
The role of reporting standards in producing robust literature reviews
Haddaway, N., & Macura, B. (2018) · Nature Climate Change · doi:10.1038/s41558-018-0180-3 · Cited by 96
Vasseghian, Y., Rad, S. S., Vilas–Boas, J. A., & Khataee, A. (2020) · Chemosphere · doi:10.1016/j.chemosphere.2020.128904 · Cited by 85
Berlino, M., Mangano, M. C., Vittor, C. D., & Sarà, G. (2021) · Environmental Pollution · doi:10.1016/j.envpol.2021.117174 · Cited by 90
Biodiversity and yield trade‐offs for organic farming
Gong, S., Hodgson, J. A., Tscharntke, T., Liu, Y., van der Werf, W., Batáry, P., Knops, J. M. H., & Zou, Y. (2022) · Ecology Letters · doi:10.1111/ele.14017 · Cited by 110
Gao, R., Zhuo, L., Duan, Y., Yan, C., Yue, Z., Zhao, Z., & Wu, P. (2024) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2024.110075 · Cited by 112
Cheng, B., Jiang, Y., & Cao, C. (2021) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2021.127793 · Cited by 88
Sensitivity analysis with iterative outlier detection for systematic reviews and meta‐analyses
Meng, Z., Wang, J., Lin, L., & Wu, C. (2024) · Statistics in Medicine · doi:10.1002/sim.10008 · Cited by 107
Reflections on the history of systematic reviews
Clarke, M., & Chalmers, I. (2018) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2018-110968 · Cited by 81
Why and how we should join the shift from significance testing to estimation
Berner, D., & Amrhein, V. (2022) · Journal of Evolutionary Biology · doi:10.1111/jeb.14009 · Cited by 97
“Such meta- analyses summarize data or effect estimates and their precision, not the number of claims about hypotheses such as ‘we confirmed there is a relationship (p < 0.05)’ (Gurevitch et al., 2018; Halsey, 2019).”
BackgroundInfluentialYang, Y., Sánchez‐Tójar, A., O’Dea, R. E., Noble, D. W. A., Koricheva, J., Jennions, M. D., Parker, T. H., Lagisz, M., & Nakagawa, S. (2023) · BMC Biology · doi:10.1186/s12915-022-01485-y · Cited by 120
“evolutionary biology, and, if attributed to actual publication bias, their impacts on estimates of effect size, statistical power, and Type M and S errors [30].”
BackgroundMethodSelf-citationLand Use Change Alters Soil Organic Carbon: Constrained Global Patterns and Predictors
Huang, X., Ibrahim, M. M., Luo, Y., Jiang, L., Chen, J., & Hou, E. (2024) · Earth s Future · doi:10.1029/2023ef004254 · Cited by 95
“The effect size of the seven LUC types was weighed using the natural log‐transformed response ratio (Ln(RR)) (Gurevitch et al., 2018).”
MethodDavidson, I., Cott, G. M., Devaney, J. L., & Simkanin, C. (2018) · Global Change Biology · doi:10.1111/gcb.14426 · Cited by 88
Gieseler, K., Loschelder, D. D., & Friese, M. (2019) · Springer eBooks · doi:10.1007/978-3-030-13788-5_1 · Cited by 86
Consequences of multiple imputation of missing standard deviations and sample sizes in meta‐analysis
Kambach, S., Bruelheide, H., Gerstner, K., Gurevitch, J., Beckmann, M., & Seppelt, R. (2020) · Ecology and Evolution · doi:10.1002/ece3.6806 · Cited by 94
“…variance measures can be expected to become a standard feature to increase the quality and trustworthiness of future meta-analyses, as advocated by Gurevitch et al. (2018) and Nakagawa et al. (2017) Our results clearly show that complete-case and unweighted analyses, although frequently applied,…”
BackgroundResultSelf-citationHülsmann, L., Chisholm, R. A., & Hartig, F. (2020) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2020.10.003 · Cited by 83
Pathy, A., Pokharel, P., Chen, X., Balasubramanian, P., & Chang, S. X. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2022.161252 · Cited by 93
Toxic effects of microplastics and nanoplastics on plants: A global meta-analysis
Wang, C., Luo, Q., Zhang, J., Zhang, X., Yang, N., & Feng, L. (2023) · Environmental Pollution · doi:10.1016/j.envpol.2023.122593 · Cited by 83
Metadata
- Authors
- Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G.
- Affiliations
- Stony Brook University (US)
- Royal Holloway University of London (GB)
- Garvan Institute of Medical Research (AU)
- Newcastle University (GB)
- Corresponding author
- Gurevitch, Jessica
- Koricheva, Julia
- Nakagawa, Shinichi
- Stewart, Gavin
- Venue
- Nature
- Published
- 2018-03
- Volume
- 555
- Issue
- 7695
- Pages
- 175-182
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2791681873
- Semantic Scholar
- ac5ad1932e31c3ea141366e8e073d6d95869c4ef
- Cited by
- 1,709
- References
- 101
- Citations by year
- 2026: 177 · 2025: 230 · 2024: 231 · 2023: 199 · 2022: 234 · 2021: 235 · 2020: 204 · 2019: 130 · 2018: 64
- FWCI
- 31.20
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- UNSWorks (UNSW Sydney) (submittedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- Newcastle University ePrints (Newcastle Univesity)
- Funders
- National Science Foundation (1262402)
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- meta-analysis, evidence synthesis, history of meta-analysis, evolutionary biology, conservation biology, artificial intelligence, evidence-based practice
- MeSH terms
- Animals, Biological Science Disciplines, Ecology, Biological Evolution, Research, Meta-Analysis as Topic, History, 20th Century, History, 21st Century
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.